Last updated
1/16/2025
Share
Get started
Document
Q&A

Query structured documents using a lightweight LLM workflow

This Blueprint demonstrates how to use open-source models and a simple LLM workflow to answer questions based on structured documents.

It is designed to showcase a simpler alternative to more complex and/or resource demanding alternatives, such as RAG systems that rely on vectorDBs and/or long-context models with large token windows.

If you encounter any issues with the hosted demo below, try the Blueprint in the GPU-enabled Google Colab Notebook available here.

Contributors
Tags
Text-to-Text
Local AI
License
Hosted Demo
Tools used to create

Trusted open source tools used for this Blueprint

PyMuPDF

Use pymupdf4llm to convert the document into markdown and then split into sections

Llama.cpp

Use llama.cpp to load GGUF-type models, enabling efficient question answering using text-to-text model.

Streamlit

Use Streamlit to build an interactive app to query your strcutured documents.

Step by step walkthrough
Use Cases

Explore how the Blueprint configuration parameters have been adjusted to create solutions that fit your specific needs.

No items found.
Extensions

Explore how the Blueprint components have been extended to expand its scope and unlock new capabilities

No items found.
Choices

Insights into our motivations and key technical decisions throughout the development process.

No items found.