AI in regulated audit: the model drafts, the gates decide

Internal audit and quality audit look like ideal work for AI. A large part of the job is reading a regulation, matching it to a control, writing test steps, and drafting a finding in a fixed format. Models are good at all of that. They are also capable of producing a confident finding that cites a clause that does not exist.

In a regulated manufacturer, a food business or a pharmaceutical plant, that is not a small mistake. An audit conclusion is evidence. Someone may have to defend it to a regulator years later, and “the model said so” is not a defence.

We have been building an audit and controls platform for regulated manufacturing to work through this problem in code rather than in slides. The main lesson so far is simple. Give the model a smaller job than you first planned, and put most of the engineering around it.

The model drafts. The gates decide.

The platform runs an audit as a chain of steps: scope the risks, build the risk and control matrix, test the controls, write up the gaps, and produce the report. A model can draft at each step. It never has the last word on any of them.

Every output passes through the same runtime checks, whether a model wrote it or a set of plain rules did:

• Library only. It can only select controls that exist in the control library. It cannot invent new ones.
• Citations it was given. It can only cite regulatory clauses from the material retrieved for that task. A citation to anything else is blocked.
• Evidence before conclusions. A control cannot be called effective when exceptions were recorded, or when there is no evidence reference. The conclusion is downgraded, and the reviewer can see why.
• Three-way citation. Every finding must point to a clause, the control it tests, and the evidence behind it. If any of the three is missing, the finding cannot be signed.
• People release. Nothing reaches a report until a reviewer and an approver have signed it off.

None of these checks is clever. They are the rules an experienced audit manager applies when reviewing a junior's work papers. The difference is that they run on every output, every time, and every decision they make is recorded.

It has to work without the model

Every step has a deterministic path that needs no model at all. That sounds like a step backwards, but it does three useful things.

It gives you something to compare against. If the model's draft and the rules disagree, that is worth a reviewer's attention. It gives you a fallback. When a call fails or the model declines, the work carries on and the trail notes what happened. And it keeps the platform independent of any one vendor. The model sits behind a narrow interface, so the client can choose the provider their risk and IT teams already approve.

Make the record impossible to quietly edit

Every step records a bundle covering what it was given, what it retrieved, what it produced, which checks fired, which versions were running, and who acted. Each bundle is sealed with a hash that includes the hash before it, so editing any past record breaks the chain from that point on. Anyone can re-run the check.

That matters more than any single drafting feature. When someone later asks why a finding says what it says, the answer is a record, not somebody's memory of the prompt.

Content is data, and it needs an expert

The regulations, controls and test steps for each sector live in content packs, not in code. There is a common pack for finance, IT general controls and governance, plus packs for food safety and for pharmaceutical good manufacturing practice. Adding a sector means writing a new pack. It is checked against a schema when it loads, and a control that cites an unknown clause is rejected.

That keeps the engine small and portable, but it moves the hard work into the content. The clause summaries in our packs are paraphrased drafts, clearly marked as such. They do not become usable until someone who has run quality in a plant reviews them and signs them off. A model can help write that content. It cannot vouch for it.

Treat the model like any other configuration

In regulated industries, software that supports quality decisions has to be validated. A model and its prompts are part of that software. We treat the model version, the prompts, the checks and the content packs as configuration items. Changing any of them bumps a version and re-runs a set of expert-labelled evaluation cases. A drop in results blocks the release. Requirements are traced to tests automatically, so the traceability matrix cannot drift away from the code.

It is the same discipline a validated system has always needed. It now applies to the model as well.

The hard part is not the AI

The biggest surprise was where the remaining effort goes. Getting a model to draft a decent finding is now the easy part. Making the platform robust is mostly unglamorous work: a database that enforces append-only records, transactions and conflict detection when two people edit the same engagement, proper single sign-on, and electronic signatures that ask for authentication again at signing and are tied to exactly what was signed.

That is where we are heading next. None of it is new technology. All of it decides whether an auditor, a quality head or a regulator can trust what the platform produces.

If you are evaluating AI for audit

Ask the vendor, or your own team, four questions. What stops the model citing something it was not given? What happens to a conclusion that contradicts the evidence? Can you show me, months later, exactly what the model saw and decided? And what still works when the model is switched off?

If the answers are about the model, keep asking. The answers that matter are about everything around it.