There is a hard line in compliance AI: output derived from your actual evidence, and output derived from the model's general knowledge. The first is a productivity gain. The second is a liability, because a claim you cannot trace to a document is exactly what an auditor will ask you to prove.
What goes wrong without grounding
Ask an ungrounded model whether you meet an ISO 27001 control and it will produce a confident, well-structured, plausible answer describing what a typical organisation does. It reads well. It is not about you. Three specific failures follow:
- Fabricated specifics. Policy names, dates and review frequencies that sound right and do not exist.
- Generic narratives. Control descriptions that describe best practice rather than your implementation - which an auditor spots immediately and which undermines the rest of your submission.
- Unfalsifiable claims. Statements nobody can verify because there is no source to check.
The danger is not that AI produces obvious nonsense. It is that it produces something entirely reasonable that happens not to be true of your organisation - and reads more polished than the honest answer.
How grounding works
Retrieval-augmented generation inverts the process. Rather than asking the model what it knows, you:
- Index your evidence - policies, configurations, reports, prior answers - as searchable chunks.
- Retrieve the passages most relevant to the question.
- Constrain the model to answer from those passages only.
- Return citations so every claim points to a document.
- Say "not found" when retrieval turns up nothing - which is the most valuable behaviour of all.
That last point is the real test. A grounded system tells you a control is unevidenced. An ungrounded one invents a plausible description of it.
AI that cites its sources
GRC Copilot indexes your evidence and grounds every suggestion in the documents it retrieved - so each control narrative, gap finding and questionnaire answer traces back to a real artefact.
Try GRC Copilot free Generate an AI-powered assessment Download checklist Book a demo
Why retrieval quality beats model size
For compliance work, the intelligence sits in the retrieval, not the generation. The tasks are narrow: find the clause addressing this control, compare these two control texts, draft a narrative from these documents. A modest model with excellent retrieval outperforms a frontier model with poor retrieval - and it is cheaper, faster, and easier to run inside your own boundary if residency matters.
Evaluating whether a tool is genuinely grounded
Vendors all claim grounding. Test it:
- Ask about a control you have no evidence for. A grounded tool says so. An ungrounded one writes you a paragraph.
- Click the citation. Does it open the actual source passage, or just name a document?
- Remove a document and re-ask. The answer should change or degrade to "not found".
- Ask something outside your evidence entirely - it should decline rather than improvise.
- Check the provenance log. Which model, which prompt, which sources - you need this for audit.
Grounding is necessary, not sufficient
A grounded answer can still be wrong: retrieval can surface an outdated policy, or the document itself may not say what the control requires. Grounding makes the output checkable, which is the property that matters. Keep a human reviewing and approving before anything becomes a record, and treat citations as the thing you verify rather than the thing you trust.
Frequently asked questions
Will auditors accept AI-assisted narratives?
They assess evidence and controls, not the tool that helped assemble them. A narrative supported by real, dated artefacts is fine. One with no underlying evidence is a finding regardless of how it was written.
Does grounding eliminate hallucination?
It reduces it substantially and, more importantly, makes remaining errors detectable because every claim is checkable against a cited source.
What if our evidence is poorly organised?
Retrieval quality follows evidence quality. Grounded AI will surface that gap quickly - which is uncomfortable but useful, since the gap was already there.
Can this run on a self-hosted model?
Yes, and for sensitive compliance data it often should. Since retrieval carries most of the workload, self-hosted models perform well on these tasks.
Key takeaways
- Ungrounded compliance AI produces plausible claims about a generic organisation.
- The key behaviour is saying "not found" rather than inventing.
- Retrieval quality matters more than model size for GRC tasks.
- Grounding makes output checkable - a human still approves.