RAG That Passes a Compliance Audit: Citations, Data Residency & Evaluation

Retrieval-augmented generation has become the default architecture for grounding large language models in an organisation's own knowledge. Chunk your documents, embed them, drop the vectors in a store, retrieve the top matches, stuff them into a prompt. It is a pattern any competent engineer can stand up in a day, and for a consumer chatbot it is often good enough.

In financial services, legal, insurance, and healthcare it is not good enough, and the reasons why are precisely the reasons these systems fail audits. A RAG system in a regulated vertical is not judged on whether it usually gives a good answer. It is judged on whether every answer can be traced, defended, and reproduced. That is a fundamentally different engineering problem.

Why standard chunk-and-embed RAG fails in regulated settings

The naive pipeline has four failure modes that are invisible in a demo and fatal in an audit.

It cannot prove where an answer came from. When you concatenate several retrieved chunks into a prompt and let the model synthesise, the resulting text is an amalgam. If a compliance officer asks "which source justifies this specific claim?", the honest answer is often "we don't know exactly." That is disqualifying when the output informs a lending decision or a clinical summary.

It ignores data residency and access control. A flat vector store treats all documents as equally retrievable by all users. But in a regulated firm, document access is governed by role, jurisdiction, and data-residency law. A UK client's data may be legally required to stay in a UK region; one user may be entitled to see a document another is not. Retrieval that ignores this is not a bug. It is a breach.

It fragments meaning. Fixed-size chunking splits a clause from its qualifying condition, a figure from its footnote, a diagnosis from its caveat. The model then confidently answers from a fragment that, in isolation, means the opposite of the full document.

It has no notion of correctness. The naive pipeline has no evaluation. It cannot tell you whether last week's index update degraded accuracy, or whether swapping the embedding model helped or hurt. In a regulated context, "we think it's fine" is not an answer you can give a regulator.

The five properties of audit-ready RAG

Building retrieval that survives scrutiny means designing for five properties from the outset.

1. Citation traceability

Every factual claim in an answer must map to a specific, retrievable source, ideally to a span within a source document, not just the document as a whole. In practice this means:

• Preserving document identity, version, page, and section through the entire pipeline, from ingestion to answer.
• Prompting and constraining the model to attribute each claim to a retrieved passage, and rejecting or flagging unattributed claims.
• Surfacing citations in the interface so a human can click straight through to the exact source span.

The test is simple: can a reviewer, looking only at the answer, verify every claim against the cited source in seconds? If not, it will not pass.

2. Data residency

Where data physically lives and where it is processed must be controllable and provable. On AWS this means pinning storage, embedding, and inference to specific regions; ensuring the vector store and the model endpoint honour those boundaries; and being able to demonstrate, with configuration and logs, that a UK client's data never left its permitted jurisdiction. This is an architectural decision made on day one, not a setting toggled at the end.

3. Access control at retrieval time

Retrieval must respect the same permissions as the underlying documents. That means filtering the candidate set by the requesting user's entitlements before the model ever sees a passage, not relying on the model to "know" it shouldn't reveal something. We attach access metadata to every indexed chunk and enforce it as a hard filter in the retrieval query, so a document a user cannot open is a document the agent cannot cite to them.

4. Multi-hop reasoning

Real regulated questions rarely map to a single passage. "Does this policy cover this scenario, given the endorsement added in March?" requires retrieving the base policy, the endorsement, and the definitions section, then reasoning across them. Naive top-k retrieval returns the most similar chunks, which are often near-duplicates of each other, not the complementary passages the question actually needs.

Audit-ready systems use techniques such as query decomposition, structured retrieval that follows document relationships, and re-ranking that rewards coverage over raw similarity, so the model reasons from a complete picture rather than a lopsided one.

5. An evaluation harness

You cannot claim a RAG system is correct without measuring it. A proper harness includes:

• A curated, versioned set of representative and adversarial question-answer pairs, with the correct source spans labelled.
• Programmatic scoring of both answer quality and, crucially, citation accuracy: did the system attribute claims to the right sources?
• Regression gating, so no change to chunking, embeddings, retrieval, or prompts reaches production without being measured against the harness.

This harness is also the artefact you show an auditor. It is concrete evidence that the system performs within a stated tolerance and that changes are controlled.

Chunking is a domain decision, not a default

One practical point deserves emphasis. The instinct to split documents into uniform 500-token chunks is the root of a surprising number of correctness failures. In regulated documents, the meaningful unit is usually structural, such as a clause, a definition, a table row, or a numbered obligation, not a fixed character count.

We chunk along the document's own structure, keep each chunk anchored to its heading and context, and preserve enough metadata that a fragment can always be traced back to its place in the whole. It is slower to build and dramatically more reliable to operate.

What we build

For regulated clients we build production RAG on AWS-native infrastructure: region-pinned storage and inference for data residency, retrieval that enforces access control as a hard filter, structure-aware ingestion that preserves citation lineage, and an evaluation harness that gates every release. The result is a system where every answer carries its sources, every source respects its permissions, and every change is measured before it ships.

That is the difference between a RAG demo and a RAG system a compliance function will sign off. You can see how this plays out on real engagements in our case studies, and explore the broader pattern in our use cases.

The bottom line

Standard RAG answers the question "can the model find relevant text?" Audit-ready RAG answers a harder set of questions: can you prove where the answer came from, that the data stayed where it was legally required to, that the user was entitled to see it, and that the whole system performs within a measured tolerance? In regulated verticals, only the second set of questions matters, because those are the questions an auditor will ask.

---

Building a knowledge system that has to pass a compliance review? Talk to our team about RAG that's designed to be audited, not just demoed.