Deloitte Australia had to return part of a $290,000 government consulting fee after a report it delivered, produced with help from an Azure OpenAI agent, cited court cases that do not exist and quoted people who never said the words attributed to them. That is not, at its core, a story about an AI model getting things wrong. Generative models do that. It is a story about a verification control that should have existed and did not.
The specific failure traced back to something almost mundane: there was no two-person verification requirement for legal citations in the report, and no mandatory human review of numerical or citation claims in AI-assisted deliverables before the document went to the client. That is a process gap, not a technology one, and it is a gap that almost certainly exists inside plenty of other organisations that have adopted AI-assisted drafting without changing how they check what comes out the other end.
The consequences of that kind of gap are not limited to consulting reports. In the United States, Nippon Life Insurance Company of America sued OpenAI in March 2026, alleging that ChatGPT had effectively engaged in the unlicensed practice of law after a pro se litigant used it for legal advice and, on ChatGPT's recommendation, reopened a settled lawsuit. Different jurisdiction, different specifics, same underlying pattern: an AI system producing professional grade output with nobody positioned to catch it before real consequences followed.
What Actually Went Wrong at Deloitte
The report itself was, by most accounts, well formatted and confidently written, which is exactly the problem. Fabricated citations and invented quotes in a document that reads as authoritative are harder to catch than an obviously broken output, because nothing about the presentation signals that anything is wrong. The Azure OpenAI agent used to help produce the report did what generative systems do when asked to support a claim: it produced something plausible. Nobody in the review chain was specifically tasked with checking whether that plausible citation actually existed.
Why This Is a Governance Failure, Not an AI Failure
It is tempting to read this story as evidence that AI models still hallucinate too often to be trusted in professional work. That is true, but it is not really the point. The point is that professional services firms have quality control processes built up over decades specifically because human-drafted work also contains errors, and those processes, partner review, technical sign off, citation checking on anything going to a regulator or a court, were built around the error patterns of human drafting. AI-assisted drafting produces a different error pattern: confident, well formatted fabrication rather than a typo or an obviously weak argument. Applying the old review process to the new error pattern is where the gap opens up.
In practice, this tends to get missed precisely because the output looks so finished. A reviewer skimming a polished, professionally formatted report is looking for the kinds of errors a human makes under time pressure. Fabricated case citations dressed up in perfect formatting do not trigger the same instinctive scepticism.
The Vendor Accountability Question No One Has Answered
This connects directly to a question Australia's own procurement environment is starting to force. The Digital Transformation Agency's guidance for government suppliers already requires organisations delivering services to the Commonwealth to disclose their use of AI and accept accountability for it under contract. Extend that logic to any organisation that engages a consulting firm, a law firm, an advisory practice, and the question becomes unavoidable: when an AI-assisted deliverable turns out to be wrong, whose governance failure is that, the vendor's or the client's for not asking how the work was checked?
Most organisations engaging professional services firms have not asked that question yet. The Deloitte case is a reasonable indication that they should start.
What a Working Verification Control Actually Looks Like
The fix is not complicated in principle, which is part of why its absence is so avoidable. A defined two-person verification requirement specifically for citations, quotes and numerical claims in any AI-assisted deliverable, applied before the document leaves the organisation. A documented workflow that distinguishes AI-assisted content from human-drafted content, so reviewers know where to apply the heightened scrutiny. And disclosure to the client about where AI was used in producing the work, so the client can make its own judgement about what additional checking it wants to apply.
What This Means for Organisations That Rely on Vendor Deliverables
If your organisation engages consultants, advisers or any professional services firm and has not asked how that firm verifies AI-assisted work, the Deloitte case is the prompt to ask. Assuming that a firm's decades old quality control processes have automatically kept pace with its AI adoption is, on the evidence so far, not a safe assumption.
What This Means for Your Organisation
What we see across engagements is that most organisations, on both sides of a consulting relationship, are still applying pre-AI verification habits to AI-assisted work. The Deloitte case is not really about one firm's error. It is about how widespread that mismatch between old verification habits and new production methods still is, on both the vendor side producing the work and the client side receiving it.
Key Takeaways
- Deloitte Australia had to refund part of a $290,000 government consulting fee after an Azure OpenAI-assisted report contained fabricated court citations and invented quotes.
- The specific failure was the absence of two-person verification for legal citations and no mandatory human review of numerical or citation claims in AI-assisted deliverables, a process gap rather than a technology one.
- Professional services quality control processes were built around human error patterns and have not consistently been updated for the different error pattern AI-assisted drafting produces, confident, well formatted fabrication.
- Australia's DTA procurement guidance already requires government suppliers to disclose AI use and accept accountability for it, a standard that logically extends to any organisation engaging professional services firms more broadly.
- A workable fix requires a defined two-person verification step for citations and figures, a documented AI-assisted content workflow and disclosure to clients about where AI was used.
How Trusenta Can Help
AI Governance registers where AI is used in producing client deliverables and assigns accountable owners for verifying that output before it leaves the organisation.
Service Delivery codifies the two-person verification and review workflow this post describes directly into how client engagements and deliverables are managed.
AI Governance Foundations builds the AI-assisted content verification structure quickly for organisations that have not yet updated their quality control processes for AI adoption.
Conclusion
The uncomfortable lesson in the Deloitte case is not that AI hallucinates. Everyone building AI governance in 2026 already knows that. The lesson is that a decades old quality control process built for a different kind of error will not automatically catch a new one, and the organisations still relying on those old habits, whether they are producing the AI-assisted work or receiving it from a vendor, are the ones most likely to be next.
