How RAG-over-PDF works

Text-only extraction. We do not need rasterised pages, layout reconstruction, or font metrics — only the readable text the user expects to ask questions about.

pdf-parse walks the PDF's text objects and concatenates them in reading order. It runs in pure JavaScript with no native bindings, which means it works on every Node host including Vercel's serverless runtime. Scanned PDFs return empty text — that is a deliberate non-goal handled with a 400 response.

Whole documents are too big to send to an LLM. Whole pages are too coarse for retrieval. Sentences are too narrow to carry context. Paragraph-sized chunks with overlap is the documented sweet spot.

The chunker walks the text in 1,000-character windows, sliding by 800 characters each step (so each pair of consecutive chunks shares 200 characters of overlap). A sentence that spans two chunks is therefore findable in either chunk. Configurable via CHUNK_SIZE and CHUNK_OVERLAP env vars.

Cosine similarity over high-dimensional vectors is the cheapest reliable way to surface "this chunk is about the same thing as this question". OpenAI's text-embedding-3-small is best-in-class for the price.

Each chunk is sent to OpenAI as one batch request, returning 1,536-dimensional Float32 vectors. The same model embeds the user question at retrieval time so the two vectors live in the same space. £0.000016 per 1k tokens means a 500-page PDF indexes for about 2 pence.

For corpora under ten thousand chunks, the round-trip to a vector database is slower than the cosine math itself. In-memory wins on latency and zero infrastructure.

The store is a JavaScript array of { content, embedding, source }. Search computes cosine similarity against every chunk in O(N · 1536) and sorts. At 1,000 chunks the search completes in 12ms. At 10,000 chunks, 60ms. Beyond that, switch to pgvector.

The LLM is a powerful reader, not a powerful researcher. The prompt forces it to ground answers in retrieved chunks and to refuse rather than hallucinate when the chunks do not cover the question.

The system prompt pins the refusal instruction. The retrieved chunks are inserted as numbered passages so the model can reference them. The user question follows. We deliberately do not include conversation history in the demo — single-shot question answering is the default.

Time to first token dominates perceived latency. Streaming the answer through the App Router's ReadableStream means users see characters appear within 600 to 900 milliseconds of pressing send.

The OpenAI SDK's stream:true option yields delta chunks. Each non-empty content delta is encoded and pushed into a Response stream. The client reads the body via the Fetch streams API and renders tokens as they arrive. No SSE library, no WebSocket setup.

Streaming responses, file-based API routes, and edge-ready deployment in one framework. The whole backend is two route files.

Express + Vite frontend — two repos, two deployments, two sets of CORS to configure. Zero benefit for this workload.

Vector dimensions, embedding shapes, and OpenAI response types are all checked at compile time. Refactors are safe; provider API changes break the build, not your users.

JavaScript — undefined property access at runtime is the most common failure mode in RAG systems. TypeScript catches that class entirely.

Pure JavaScript, no native deps, single dependency, handles 95% of real PDFs by extracting the text layer.

unstructured — Python dependency awkward on Node serverless. pdf2pic — rasterises every page wastefully when you only need text.

Cheapest credible embedding model on the market, 1,536 dims, faster than ada-002, scores higher on MTEB.

text-embedding-3-large — 3,072 dims is overkill for most corpora and doubles storage cost without proportionally better retrieval.

Zero infrastructure, zero network hops, sub-25ms search up to ten thousand chunks. Migration to pgvector is a single-file change when you outgrow it.

Pinecone — network round trip, billing surface, vendor lock-in. Worth it at a million chunks. Wasteful at a thousand.

Cheap, fast, smart enough to read context and answer accurately. £0.15 per million input tokens makes per-question cost trivial.

gpt-4o — 6x more expensive without measurably better answers when the context is already pre-filtered by retrieval.

Push to GitHub, live in 60 seconds. Streams work natively. The only secret is OPENAI_API_KEY.

AWS / GCP — 50x more configuration for the same result. Lambda + API Gateway + CloudFront for what Vercel does in one button click.