"Can the chatbot use our PDFs?" is the single most common technical question we get. The short answer is yes — but the interesting question is how, and what that means for accuracy and maintenance.
Here's the pipeline, in plain English.
Step 1 — Ingest
We pull content from the source: a folder of PDFs, a website sitemap, a Confluence space, a Notion workspace, a Google Drive folder, a Zendesk help centre, a CRM export.
For unstructured formats (PDFs, Word docs), we extract clean text — dropping headers, footers, and page numbers that would confuse retrieval.
Step 2 — Chunk
Documents get split into smaller passages ("chunks") of a few hundred tokens each. Chunk boundaries matter — cut in the wrong place and you split a sentence away from its context.
Good chunking follows document structure: sections, headings, paragraphs. Bad chunking splits arbitrarily every N characters.
Step 3 — Embed
Each chunk gets converted into an embedding — a numeric vector that captures the meaning of the text. Semantically similar chunks end up close in vector space.
The embeddings live in a vector database (pgvector, Weaviate, Pinecone, Qdrant) so we can search them fast.
Step 4 — Retrieve
When a user asks a question, we embed the question the same way. The vector DB returns the closest-matching chunks.
We usually combine vector search with keyword search (hybrid retrieval) and then apply a reranker to pick the truly best matches. This gets us high recall without drowning in noise.
Step 5 — Filter by permissions
For anything involving user-specific content, we filter retrieved chunks by the current user's permissions before showing anything to the model. A support agent shouldn't be able to see leadership-only documents just because they asked the right question.
Step 6 — Generate
The best matching chunks get handed to the language model along with the user's question and a carefully-written system prompt: "answer using only the provided passages; if you can't, say so."
The model writes the answer, and we display it with citations back to the source chunks.
Step 7 — Evaluate and improve
Real conversations are logged. We measure groundedness (did the answer come from the passages?), correctness (was it right?), and helpfulness (did the user say thanks or ask a follow-up?).
Failing categories drive the next round of content improvement — sometimes the docs are wrong; sometimes chunking needs a tweak.
Frequently asked questions
Conclusion
A well-built RAG chatbot turns your documents into an answerable knowledge base. The magic isn't in any one step — it's in doing all seven of them competently, and evaluating quality continuously.