Building a Document Q&A System with On-Device AI

The useful part

Even better, I completely forget to even show the demo in the presentation. A few weeks ago I made my first presentation at RenderATL and gave my talk on Chrome's on-device AI technology. I've been blogging about this particular set of APIs for a few years now and I've given talks on it at numerous conferences.

How it works

• While these APIs are still evolving, many of them (and the ones I'm using today in particular) are now fully GA for Chrome.
• For this demo, I put together a few things I've done before into one cohesive tool.
• a completely client-side application that lets you pick a file, get a summary, and then chat with it.
• First, I'm making use of the incredibly excellent officeParser library by Harsh Ankur.
• This library can take any Office document, or PDF, or ODT, or CSV, and so on, and creates a parsed version you can do nearly anything with.

What to take from it

For me, the most important feature is the ability to convert the file into Markdown which can then be used with AI. I used my AI agent to set up the application and create the design and there was quite a bit o...

Example or evidence

• Every now and then I build a demo for a presentation and think to myself.

Details worth keeping

Next, it makes use of Chrome's Summarizer API which. The web app itself is a simple Vite app using vanilla JavaScript.