Ask a team running retrieval-augmented generation in production where their reliability problem lives, and most answers land on the model. Hallucination is the comfortable diagnosis, because it puts the fault in something a vendor is already working on. The uncomfortable diagnosis sits a layer below. The index was built from a corpus that moved three weeks ago. The retriever returned exactly k chunks whether or not any of them were relevant. Nobody recorded which version of the index produced the answer, and the citation attached to the claim was never checked against the claim. The model did not invent an answer out of nothing, it composed one out of stale, unversioned, untraceable evidence, and every layer above the data did exactly what it was told.
Your RAG system’s biggest lie is not the hallucination. It is the stale data it hallucinated from, confidently.
Inside one tight publishing window in mid-2026, five components of the fix shipped in public, most of them with working code or a reproduction repository. None of the publishers shipped the other four. They also came from fewer teams than five pieces suggests: three of the five trace back to LanceDB or its engineers, one to AWS, one to Zep. That is worth saying out loud in an article whose closing argument is that a number’s publisher is its most important attribute. That is the honest state of production RAG right now: the complete stack exists, most of it is open source, one piece of it is a paid managed API, it is adoptable this quarter, and no vendor sells the assembly. This article assembles it from the primary sources, then closes on the thing none of the five pieces mentions, which is where the assembled stack stops working.
Figure 1 - Five components, one ceiling: Freshness keeps the index true, the evidence layer makes an answer citable, versioning makes it reproducible, retrieval planning handles questions one embedding cannot represent, and the provenance graph traces facts the model composed rather than copied. Every one of the five makes the pipeline retrieve more and rank harder. The line above them is the part nobody selling a component mentions.
Five Pieces, Nobody Selling the Whole Thing
The five components, in the order a team should adopt them:
- Freshness. Keep the vector data true as sources, transformation logic, schema, and target all change independently of one another.
- Evidence layer. Turn messy documents into a citable, gradeable store where every retrieved fragment carries the identity of the page it came from.
- Dataset versioning. Version the vector dataset the way a team versions code, so an experiment cannot corrupt production and an audit can pin the exact state that produced an answer.
- Retrieval planning. Decompose a multi-part question, retrieve per sub-query, judge whether the evidence is sufficient, and iterate under a bound.
- Provenance for synthesized facts. Trace a fact the model composed from several sources back to all of them, and keep that trace correct through entity merges and deletions.
Every benchmark number in this article is author-reported by the team that built the thing being measured, on hardware and datasets of their own choosing, with no independent audit. We name the publisher inline every time a number appears, because that provenance is the number’s most important attribute.
Component 1: Freshness Has Four Axes, Not One
Most teams treat freshness as a single question: did the source data change. CocoIndex, an embedded pipeline runtime that sits between raw sources and a target store, names four independent axes of staleness instead, each with its own detection mechanism [1]:
- New records or files are added. The source changed, and the target does not know the new object exists.
- Existing records are modified or deleted. The source changed, and outdated fields, stale embeddings, or deleted objects keep answering queries.
- Transformation logic changed. The source is untouched, and parsing, normalization, or embedding code now means something different.
- Target schema changed. The source is untouched, and the record shape the target should hold has evolved, usually because the logic now produces new fields.
Figure 2 - Four axes, four fingerprints: The design point is separability. Treating “did anything change” as one undifferentiated re-run trigger is what forces a full reindex every time an engineer edits a chunking function. Give each axis its own fingerprint and a change on one stops dragging the other three along with it.
The mechanism that makes the separation real is function-level memoization. A function marked @coco.fn(memo=True) reuses its cached output unless its input or its own code fingerprint changes [1]. Renaming a source file without touching its content does not trigger a re-embed, since the content fingerprint is unchanged. Swapping the embedding model does trigger a recompute, even though not one byte of source data moved. Model identity is answered by a dunder method on the embedder class, so the engine has something stable to compare run over run, and the text and image embedders sit behind separate keys, so changing one recomputes that modality’s vectors while the other stays untouched.
Schema evolution falls out of the same design. Adding a nullable column and re-running reprocesses every row with no migration step, and the reprocessed-row count looks alarming while almost nothing expensive actually reruns, since the existing embeddings come straight from the memo cache [1]. The author reports roughly 12 seconds to ingest 1,000 product listings with metadata and images, and under one second to react to a single file edit in live-watch mode, on his own laptop against a 1,000-row slice of a much larger dataset [1]. Those are illustrative timings from a personal machine rather than a throughput guarantee, and the shape is the point: one bulk load, then near-instant reaction to individual changes.
Figure 3 - What actually reruns: A rename changes the filename and not the content fingerprint, so the cached embedding is reused. A new embedding model changes the model-identity fingerprint, so that modality recomputes and the other one does not.
The storage layer underneath adds a second freshness technique. Lance, the columnar format LanceDB is built on, supports zero-copy schema evolution: adding a column writes only that column’s data and does not rewrite the existing table [2]. Re-embedding a corpus with a newer model therefore becomes an additive column operation on the live table rather than a full reindex-and-swap into a shadow table. Every schema evolution and backfill also commits a new table version, and LanceDB reports one demo table reaching 26 versions through routine backfills [2]. A specific version is addressable, which turns “what did the index contain when it produced that answer” from a forensic reconstruction into a query. That is the freshness layer quietly doing the evidence layer’s work, which is why versioning arrives as Component 3 rather than as a separate concern.
Component 2: The Evidence Layer Is a Schema Decision, Not a Prompt
Flattening a document into a bag of chunks at ingestion destroys the evidence trail before any model is involved. Once page identity is gone, a wrong answer cannot be attributed to a bad parse, a broken chunking rule, or a retriever that found the wrong page, since all three produce the same symptom. The fix in LanceDB and LlamaIndex’s joint reference build is a schema decision made before the first embedding is computed [3]: every record, at every granularity, is stamped with the page it came from.
page_id = f"{doc_id}:p{page_num}" # nvidia_fy2024:p2chunk_id = f"{page_id}:c{chunk_index}" # nvidia_fy2024:p2:c0asset_id = f"{page_id}:asset:{name}" # nvidia_fy2024:p13:asset:image_p13_0Evidence lives at three granularities in three tables: pages (full page text plus a screenshot), chunks (page-bounded text slices that never straddle a page boundary), and assets (extracted figures and page screenshots). Every record carries page_id, so each table is searched independently and the hits merge, in application code, into one result per page carrying that page’s full evidence. Large binaries stay out of the way: page screenshots are marked for out-of-line blob storage, so a search scans only lightweight position-and-size descriptors and the image bytes are fetched on demand for the few rows actually retrieved [3].
Figure 4 - One key, three granularities: The page identity is the join key that lets a chunk hit, a page hit, and a figure hit collapse into a single result carrying all of that page’s evidence. Nothing here requires a better embedding model. It is a naming convention with consequences.
The retrieval mode built on that schema runs the chunk, page, and figure searches concurrently and fuses them into one page-ranked list keyed by page_id, giving each page several independent chances to rank. The authors flag its limitation themselves, and the caveat travels with every number below: the fusion merges purely by vector distance, so it is recall-oriented fusion rather than true reranking, and it does not sort the survivors by relevance [3].
Now the numbers, with their provenance attached. The evaluation ran on a 50-question, 6-report subset of Climate Finance Bench, an open academic benchmark whose full version covers 33 sustainability reports and 330 expert-validated question-answer pairs [4]. The subset is the detail that matters when reading the results, since a 50-question run is a very different claim than a 330-question one. On that subset, LanceDB’s own authors report the fused mode reaching 82% any-page-hit@5, against 76% for page-only search, 72% for chunk-only search, and 38% for figure-only search, all measured on their own machine and not independently replicated [3].
Then they ran an agent on top of it, and the gap is the interesting result. The same authors report the agent answering 74% of the questions correctly as judged by a model, against that 82% retrieval ceiling [3]. Their framing of the 8-point gap is the part worth carrying into your own postmortems: “[g]etting the right page in front of the model is the retrieval layer’s job… turning the retrieved context into a correct answer is the agent’s job” [3]. Most RAG postmortems conflate those two into one end-to-end accuracy number and then argue about the model.
Figure 5 - The ceiling and the floor are different metrics: The retrieval layer’s ceiling is measurable with no model in the loop at all, and everything below it belongs to the agent. Both figures are LanceDB-published, author-reported, and drawn from a 50-question subset of Climate Finance Bench rather than the full 330-question benchmark.
A second, separately authored evaluation from the same two companies pushes on the other side of the same wall [5]. Against a two-page medication factsheet with deliberately awkward structure, an agent given a vector-search tool and an image-fetch tool scored 84.4% overall across 20 questions, with perfect scores on cross-category reasoning and 33.3% on aggregation and counting, again author-reported [5]. The worst failures trace to one cause the authors name directly: vector similarity search is built for relevance ranking, not exhaustive coverage, so “how many unique side effects are listed in total” is structurally the wrong shape for it [5]. Their fix is a second tool, a structured query over a normalized schema derived from the same parsed output, for questions that need completeness rather than top-k relevance.
One small piece finishes the layer. A store that knows where a fact came from still needs a wire format to hand that lineage to whatever renders the answer, and AWS documents a minimal one on its Bedrock web-search output: a citation object carrying a type, a title, a URL, and a start and end index, where “start_index and end_index are character offsets into output_text, letting you render inline footnotes or highlight the exact span each citation supports” [6]. Character offsets are implementation-agnostic, so they work whether the lineage came from a page-identity lookup, a table-version lookup, or the graph walk in Component 5.
KEY INSIGHT: Measure your retrieval ceiling separately from your agent’s accuracy. One is a schema and indexing problem you can fix without touching a model, the other is a reasoning problem you cannot. A single end-to-end accuracy number hides which one you actually have.
We made the broader case for treating verification as a design problem rather than a prompting problem in The Verification Layer for Knowledge Agents [16]. The page-identity schema is that argument applied before any model is in the loop.
Component 3: Version the Dataset the Way You Version the Code
Every write into a Lance-backed target already produces a new dataset version. Component 3 is the layer above that: branching, tagging, and cloning those versions so an experiment cannot damage production and an auditor can pin the exact state that produced an answer [7].
The Lance authors’ own comparison, worth reading with the note that the post’s author designed Iceberg’s branching himself, is that two prior approaches got most of the way there and stopped [7]. Apache Iceberg supports branches and tags inside a table, with all named references living in one root metadata file, and that shared file is the problem: every operation on any branch updates it, so high-frequency experimental writes conflict with production commits and invalidate production read caches [7]. Branches also share the table directory, which leaves no physical isolation for access control and no way for audit logs to separate production access from experimental access. Delta Lake’s shallow clone sidesteps all three by making the clone an entirely separate table, buying isolation and clean cost attribution at the price of the thing that made branching valuable: the clone’s relationship to its source is no longer tracked automatically, and lifecycle policies no longer span the pair [7].
Lance’s answer is to track branches by root rather than by head. A branch is a shallow clone living inside the source dataset’s own directory structure, plus a small reference recording where it forked from. The branch gets its own manifest directory, version history, and data files, and its commits never touch the source dataset’s root manifest [7].
{dataset_root}/ _refs/ branches/ feature-a.json # parent branch + fork version only tree/ feature-a/ _versions/ # the branch's own version history data/ # the branch's own data filesThe three Iceberg problems disappear together. Writes to a branch never touch main’s metadata, so there are no commit conflicts and no production cache invalidation. Branch data lives in its own directory, so storage-level access control can enforce read-only on main, and audit logs and storage costs are attributable per branch. By the same authors’ account, in Iceberg only main keeps a snapshot lineage, while in Lance every branch keeps a complete version history, so time travel works on any branch [7].
ds.tags.create("baseline", 1)ds.tags.create("training-v1", ds.version)
experiment = ds.create_branch("feature-experiment")variant_a = experiment.create_branch("variant-a")
clone = ds.shallow_clone("s3://experiments/clone-baseline", "training-v1")Tags are the part most teams use first and appreciate later. They are immutable named pointers stored outside the version timeline, invariant under rollback, and exempt from garbage collection [7]. Tag the dataset version a regulated answer was served from and the reproduction question closes permanently, which is the same instinct we argued for in The Log Is the Agent [17]: the durable record is the thing worth owning, and everything else is a view derived from it.
Figure 6 - Shared metadata versus isolated roots: On the left, every branch operation rewrites one shared metadata file, so experimental writes collide with production commits. On the right, each branch owns its manifest directory and version history, and production never notices the experiment. This is a design post from the format’s authors with no published production throughput figures attached, so adopt it on the architecture, not on a benchmark.
Component 4: One Query Embedding Cannot Represent Two Questions
Single-shot retrieval fails on multi-part questions for a structural reason rather than a tuning reason. A single query embedding for “compare how the company talked about hiring, long-term investment, and customer obsession in 2020 versus 2023” is an average of competing intents, and the top-k results either scatter loosely across all of them or cluster on whichever intent has the strongest signal [8]. No amount of reranking fixes an averaged query vector, since the right documents were never candidates in the first place.
Amazon’s AgenticRetrieveStream API runs the fix as a managed planning loop inside one streamed call [8][9]. The loop decomposes the question into sub-queries, retrieves per sub-query, judges whether the collected evidence is sufficient, iterates under a hard bound, deduplicates, and synthesizes. Each step streams as an ordered trace event, so the evidence layer from Component 2 extends naturally from “cite the page” to “show the retrieval reasoning that found the page.” AWS’s blog post names the individual steps in its walkthrough, and it is worth knowing that the live API reference types the step attribute as a generic string with no inline enumeration, so those literal step names rest on the blog’s prose rather than on the schema [8][9].
Figure 7 - The loop a human analyst already runs: Decompose, retrieve per part, ask whether that is enough, go again if it is not, and stop at a bound you set. The iteration bound keeps this from becoming an unbounded spend, and the streamed trace is what makes it auditable rather than magical.
The tuning surface is deliberately small. An iteration ceiling caps planning-plus-retrieval rounds, and AWS’s own guidance is 3 for a single knowledge base and 4 to 5 for multi-source or comparative queries, then measure rather than guess [8]. Up to 5 knowledge-base retrievers can be registered per request, each carrying a natural-language description the planner reads to route each sub-query, and AWS advises writing those descriptions like product marketing because vague descriptions produce vague routing [8].
On the benchmark, AWS reports roughly a 20 percentage-point absolute recall improvement over single-shot retrieval on MuSiQue, a public multi-hop question-answering benchmark, with gains scaling by difficulty: 22.8 points at 2 hops, 31.9 points at 3 hops, and 37.3 points at 4 hops [8]. AWS also reports that the planner’s own average hop count tracks the benchmark’s annotated chain length closely, at 1.98, 3.50, and 4.78 respectively, which says the planner is not wandering [8]. Every one of those numbers is AWS evaluating its own product, with no third-party replication.
The cost model is stated plainly and belongs in front of a client before anyone enables this by default. AWS prices it at $4 per 1,000 agentic retrieval calls using the managed planner model, plus $1 per 1,000 underlying retrieval calls the planner issues internally, with cost and latency scaling by iteration count rather than token count [8]. The practical split is easy to hold: plain retrieval for short, well-scoped, single-intent lookups, and the planning loop only for questions that are genuinely multi-part, comparative, exploratory, or spread across several sources.
Component 5: Provenance for the Facts Nobody Actually Wrote Down
Components 2 and 4 answer “which page did this text come from” for text that is the page. Neither answers the harder and more common production question: which sources fed the fact the model just wrote in its own words. Daniel Chalef of Zep opens on exactly that gap: “Synthesis often destroys the paper trail of how these outputs were originated” [10]. A clean, confident fact may have been composed from a formal record, a scanned report, and something a user typed into a chat box, three sources of wildly different reliability, and appear verbatim in none of them.
The naive fix is a source-ID column on the fact, and Chalef rules it out first. It works in a data warehouse where a pipeline outputs one value copied or mutated deterministically. It breaks in an LLM-driven pipeline for three reasons: one fact routinely has several parent sources, entity resolution merges two identities whose facts came from different places, and new data invalidates old facts so the store keeps changing underneath a static pointer [10]. His conclusion is that lineage has to be an evolving set that survives mutation, which a scalar column cannot be and a graph relationship can.
The pattern models provenance as ordinary graph structure. Source data is ingested as episodes, entities are extracted from episodes, facts are edges between entities, and the relationship between a fact and the episodes it came from is itself an edge. Tracing a fact to its source becomes a graph walk rather than a separate audit subsystem bolted on afterward [10]. The framework implementing it, Graphiti, is open source under Apache 2.0 [11].
Figure 8 - Lineage as edges, not as a column: Three episodes feed the entities, the fact is the edge between two entities, and every derivation is itself an edge you can walk. The merge on the right is where a source-ID column silently fails and this design does not.
Three properties fall out, and each maps onto something a regulated client eventually asks for.
Entity merges keep both parents’ source links. Chalef’s warning is specific: “[w]hen two entities merge, the merged entity needs to keep all source links from both, otherwise we silently drop a source and we lose lineage” [10]. This is the bug that makes a pointer-based design quietly wrong rather than loudly broken. Deduplicating two spellings of the same name is a routine pipeline step, and it is exactly the step that destroys half a fact’s provenance while leaving the fact looking fully sourced.
Verification propagates by tag. Episodes can be tagged at ingestion, and every entity and fact derived from a tagged episode inherits the tag, so an agent that should only act on verified sources filters for the tag while walking the graph [10]. The interesting case is a fact with several parents of mixed status, where the correct rule depends on the stakes and not on the graph. For a safety-critical fact, any unverified parent should block the agent from trusting it. For a compliance fact like consent being on file, every parent must be verified. Chalef is explicit about where that decision lives: “the graph or the underlying store exposes that choice… but your agent needs to execute or apply your business rules. That’s not necessarily something we bake into the graph. It’s situational” [10]. The framework supplies the lineage and the propagation mechanism, not the policy. The healthcare examples in the talk are the speaker’s own illustrative constructions rather than an audited deployment, and no accuracy, latency, or cost benchmark for the framework appears anywhere in it.
Deletion becomes computable instead of manual. A retention or right-to-be-forgotten request targeting one source out of three has an obviously correct answer and no obvious implementation in a flat store. Here the rule is one sentence: “a fact is only deleted if no remaining episodes support it” [10]. Delete the chat-intake episode and a fact still supported by two other episodes survives, while a fact derived solely from that episode is removed. That differs meaningfully from a blanket cascade delete, which over-deletes, and from leaving derived facts untouched, which under-deletes.
Figure 9 - Delete the source, keep what still stands up: One deleted episode, two different outcomes. The fact with two surviving parents stays, the fact with none goes, and the same edge that made provenance traceable is what makes partial deletion computable.
None of this is free, and the source says so. Chalef notes plainly that lineage and provenance are expensive and that graph construction in this style is expensive, without attaching a figure to either claim [10]. He is equally direct about the alternative most teams reach for first: “markdown suffers from provenance. File-based memory starts to break down with provenance. It’s very difficult when you mutate lines in a file to understand the lineage or the provenance of why those changes occurred” [10]. The two positions reconcile once scoped. A file-based working memory suits a single operator whose audit requirement is low. A lineage graph is for the multi-source, regulated case where “why does the system believe this” has to be answerable as a query.
KEY INSIGHT: Provenance is invisible until a regulator asks for it, and it is usually not retrofittable, because lineage that was never captured for a fact the model synthesized often cannot be reconstructed at all. Decide before ingestion, not after the audit request.
The Cheapest Evidence Control You Can Ship This Week
All five components are architecture. There is one control small enough to add to an existing pipeline in an afternoon, and it belongs here precisely because it is unglamorous. AWS’s reference architecture for document-heavy due diligence includes a citation-check evaluator, a small function that examines every factual claim in a generated response and flags assertions lacking a supporting source citation, with, in AWS’s words, an exit code that “is 0 on pass and 1 on fail, making it composable in CI pipelines” [12]. A generation step that fails the check fails the build.
Lead with its limitation rather than its convenience, because the limitation is the more durable lesson. A check of this shape compares a claim against the text of its attached citation. That reliably catches a fabricated citation, where the cited text shares nothing meaningful with the claim, since there is nothing to match against. It does not catch a misattributed citation, where a real source is topically adjacent and shares the claim’s vocabulary without supporting its substance. Treat it as the floor under an evidence layer rather than its ceiling, and ship it anyway: it is cheap enough to run on every generation, needs no second model call, and catches the crudest failure an agent produces.
Figure 10 - What the cheap gate catches, and what walks past it: A fabricated citation has nothing in common with the claim, so an automated comparison flags it immediately. A misattributed one shares the claim’s vocabulary and passes clean. Ship the gate, and do not describe it as citation verification solved.
The Ceiling: Where All Five Components Stop Working
Here is the part none of the five sources mentions, and it matters most to anyone who has already built two or three of the components above.
Every component in this stack makes the pipeline retrieve more and rank harder. Freshness widens the corpus. The evidence layer multiplies the retrievable granularities. Retrieval planning fans one question into several. Each of those quietly increases the number of candidate documents reaching the reranker, and the assumption underneath the whole design is that a reranker handed more candidates produces better results. That assumption is measurably wrong past a certain point.
Mathew Jacob and colleagues, in “Drowning in Documents: Consequences of Scaling Reranker Inference,” report that “the best existing rerankers provide initial improvements when scoring progressively more documents, but their effectiveness gradually declines and can even degrade quality beyond a certain limit” [13]. In a later interview about that work, Jacob describes the shape of the collapse in his own words: “after you got around like 100 documents your quality recall at 10 to start of plummeting” [14]. That threshold is conversational, not a measured constant, so treat it as an order of magnitude rather than a number to configure. It is also pair-specific. Where your own retriever and reranker hit their ceiling is an empirical question about your stack, not a value to copy out of a paper.
Figure 11 - More candidates is not a free precision lever: Recall improves as the candidate set grows, peaks, and then falls. The location of that peak depends on the retriever and reranker pair, so the curve is the lesson and the number on the axis is not.
The mechanism explains why the failure is structural rather than a tuning miss. Most practitioners carry a mental model of a cascade of increasing strength: a fast, weak first-stage retriever followed by a stronger, slower cross-encoder that independently judges relevance. Jacob’s framing overturns that. Cross-encoder rerankers behave more like boosting-style error-correctors for the retriever than like independently strong rankers, since they are trained on negatives drawn from that retriever’s own error patterns. In his words, “they’re just very good at fitting on the errors that the retriever gives you, in these common settings. And so once you start blowing up the K size, it starts doing much poorly because it’s not things they would have seen in training” [14]. Past the training distribution’s shape, the reranker inherits the retriever’s shape instead of correcting it. In one full-scoring experiment on a downsampled corpus, plain BM25 outperformed the cross-encoders outright on one dataset [14].
The visible symptom is the dangerous part. Jacob calls it a phantom hit, which is his own explanatory vocabulary for his research in that interview rather than established terminology from the retrieval literature, and it means a confidently high score assigned to a plainly irrelevant document. His example is a query about a disease affecting children in Gabon, where the reranker placed a document about dishwashers and a cabinet at rank three [14]. A retriever would have discarded it outright. The reranker promoted it with confidence. Nothing downstream flags a phantom hit, because the score does not look wrong.
Figure 12 - A confident false positive is worse than a low score: The failure is not that the reranker was unsure. The failure is that it was sure, about a document with no relationship to the query, and every downstream threshold that trusts the score passes it through.
Two consequences land directly on the stack assembled above. First, the retrieval-planning component is the most efficient way to inflate the candidate set without the operator noticing, since fanning one question into four sub-queries multiplies candidates while the configuration file still says the same k. Second, the evidence layer’s fused retrieval mode is recall-oriented fusion that does not rank its survivors by relevance, as its own authors state [3], which makes its output exactly the kind of large, unordered candidate pool this failure targets. Both components are good. Together, unbounded, they walk the pipeline straight into the collapse region.
The mitigations are known and none of them is free. Listwise and setwise rerankers, which make relative judgments across a set rather than assigning each document an isolated calibrated score, held up considerably better under the same scaling in Jacob’s testing, at the cost of higher latency from sequential passes [14]. Related work attacks the configuration problem directly: “Natural Language Query to Configuration for Retrieval Agents,” from UC Berkeley, the University of Washington, and Microsoft Azure Research, converts each query into workload characteristics and selects the retrieval configuration predicted to maximize correctness against cost [15]. That is the right long-term shape, and it is research rather than something to deploy on Monday.
The practical instruction is simpler. Establish where your own pipeline’s recall ceiling sits before you raise k, and put that measurement in an evaluation harness so a regression shows up in CI rather than in production, which is the same argument we made about evaluations generally in Evals in Practice [18].
KEY INSIGHT: Treat candidate-set size as a tuned parameter with a measured ceiling, never as a dial that only goes up. Every component that makes retrieval smarter also makes the candidate set bigger, and the failure it eventually produces reports itself as a high confidence score.
Conclusion
The complete production RAG stack is not a product anyone is selling, and that is the useful finding rather than a complaint. Five components shipped in public within months of each other, from three organizations, most with working code, each solving a real failure mode the other four leave open. Freshness with four separable axes keeps the index true. Page-identity fusion makes an answer citable and, more importantly, makes retrieval quality measurable without a model in the loop. Branch-by-root versioning makes an answer reproducible. A bounded planning loop handles the questions one embedding cannot represent. A provenance graph traces the facts the model composed rather than copied.
Adopt them in that order, because each one makes the next one measurable. An index nobody trusts makes every downstream measurement meaningless, page identity cannot be retrofitted cheaply once a corpus has been chunked, versioning is close to free when the storage layer already commits a version per write and is what lets the last two ship as experiments rather than migrations, and the provenance graph earns its ingestion cost only where facts get synthesized across sources and someone will eventually ask where one came from.
What none of the five publishes is where the assembly stops working, which is why the ceiling belongs in the same article as the components. Every one of them increases the number of candidates reaching the reranker, and past a point specific to each retriever and reranker pair, more candidates make recall worse rather than better, while the resulting failures announce themselves as confident scores on irrelevant documents. That is an argument for measuring your own ceiling before something else finds it for you, not an argument against building the stack.
One exercise tells you whether a deployment is actually production-grade. Pick an answer your system produced last month and ask five questions about it. Which version of the index produced it. Which pages were the evidence. Which sources fed each synthesized fact. How many candidates did the reranker see. Would any of that still be answerable if one of the source documents had since been deleted. A system that cannot answer those five is not unreliable because of the model, and that gap is the audit worth running before the next model upgrade gets blamed for it.
Every benchmark figure quoted above was published by the team that built the thing being measured, none of it has been independently audited, several numbers come from single runs on personal machines, and one widely quoted retrieval result comes from a 50-question subset of a 330-question benchmark. The components are still the right components. Bring your own measurements, and keep the provenance attached to every number you inherit, including these.
References
[1] P. Rao, “Incremental multimodal data pipelines with CocoIndex and LanceDB,” The Data Quarry, Jul 2026. https://thedataquarry.com/blog/incremental-multimodal-data-pipelines-with-cocoindex-and-lancedb
[2] A. Chaurasia, “One table to train your robot: LanceDB as the data layer for lerobot,” LanceDB Blog, Jul 2026. https://www.lancedb.com/blog/one-table-to-train-your-robot-lancedb-as-the-data-layer-for-lerobot
[3] P. Rao and C. A. Bertelli, “From Messy PDFs to Verifiable Answers with LiteParse and LanceDB,” LanceDB Blog, Jul 2026. https://www.lancedb.com/blog/from-messy-pdfs-to-verifiable-answers-with-liteparse-and-lancedb
[4] R. Mankour, Y. Chafai, H. Saleh, G. Ben Hassine, T. Barreau, and P. Tankov, “Climate Finance Bench,” arXiv:2505.22752, May 2025. https://arxiv.org/abs/2505.22752
[5] C. A. Bertelli and P. Rao, “Smart Parsing Meets Sharp Retrieval: Combining LiteParse and LanceDB,” LanceDB Blog, Apr 2026. https://www.lancedb.com/blog/smart-parsing-meets-sharp-retrieval-combining-liteparse-and-lancedb
[6] AWS, “Introducing Web Search on Amazon Bedrock for foundation model grounding,” AWS Machine Learning Blog, Aug 2026. https://aws.amazon.com/blogs/machine-learning/introducing-web-search-on-amazon-bedrock-for-foundation-model-grounding/
[7] J. Ye, “Branching and Shallow Cloning in Lance: Towards a ‘Git for AI Data’,” LanceDB Blog, Jul 2026. https://www.lancedb.com/blog/branching-and-shallow-clone
[8] AWS, “Agentic retrieval for Amazon Bedrock Managed Knowledge Base,” AWS Machine Learning Blog, Jul 2026. https://aws.amazon.com/blogs/machine-learning/agentic-retrieval-for-amazon-bedrock-managed-knowledge-base/
[9] AWS, “AgenticRetrieveStream,” Amazon Bedrock Agent Runtime API Reference. https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_AgenticRetrieveStream.html
[10] D. Chalef, “Citation Needed: Provenance for LLM-Built Knowledge Graphs,” Zep AI, AI Engineer, YouTube. https://www.youtube.com/watch?v=H7puB0RwJMM
[11] Zep AI, “Graphiti,” GitHub (Apache-2.0). https://github.com/getzep/graphiti
[12] AWS, “Accelerating M&A due diligence with Amazon Bedrock AgentCore,” AWS Machine Learning Blog, Aug 2026. https://aws.amazon.com/blogs/machine-learning/accelerating-ma-due-diligence-with-amazon-bedrock-agentcore/
[13] M. Jacob, E. Lindgren, M. Zaharia, M. Carbin, O. Khattab, and A. Drozdov, “Drowning in Documents: Consequences of Scaling Reranker Inference,” arXiv:2411.11767, ReNeuIR 2025 Workshop at SIGIR 2025. https://arxiv.org/abs/2411.11767
[14] “Drowning in Documents with Mathew Jacob,” Weaviate Podcast #141, YouTube. https://www.youtube.com/watch?v=fWuavBcoTzk
[15] M. Z. Pan, N. Arabzadeh, M. Jacob, F. Kazhamiaka, E. Choukse, and M. Zaharia, “Natural Language Query to Configuration for Retrieval Agents,” arXiv:2605.27361. https://arxiv.org/abs/2605.27361
[16] G. Dotzlaw, K. Dotzlaw, and R. Dotzlaw, “The Verification Layer for Knowledge Agents,” 2026. /insights/ai-15-verification-layer-knowledge-agents/
[17] G. Dotzlaw, K. Dotzlaw, and R. Dotzlaw, “The Log Is the Agent: Why Whoever Hosts It Owns What You Built,” 2026. /insights/ai-24-log-is-the-agent/
[18] G. Dotzlaw, K. Dotzlaw, and R. Dotzlaw, “Evals in Practice,” 2026. /insights/ai-12-evals-in-practice/
Building production AI, or modernizing a legacy system?
That is the kind of work we do at Dotzlaw Consulting. Book a free 20-minute intro call and tell us what you are trying to build, or what is slowing you down.