AI for legal and compliance: useful, until it's confidently wrong about the law

By now you've heard the cautionary tale, probably more than once. A lawyer files a brief, the brief cites a handful of perfect-sounding precedents, and the precedents turn out not to exist. The model wrote them. Names, citations, holdings, all fabricated, all delivered with the same calm authority as the real ones. The lawyer gets sanctioned, the story makes the rounds, and everyone in legal gets a little more nervous about this whole thing.

That story is worth holding onto. It tells you exactly what AI is and isn't good for in a legal or compliance setting. The model is a fluent language engine with no built-in commitment to truth and no actual knowledge of the law as it stands today. Point it at the right task and it does real, valuable work. Ask it to be a lawyer and it will happily improvise one, fake citations and all.

What it's actually good at here

The legitimate uses cluster around one idea: working with documents you give it, not pronouncing on law it claims to know.

Contract review is the obvious one. Hand it an agreement and your standard playbook, and it'll flag where the contract deviates: the indemnity that's missing, the liability cap that's unusually low, the auto-renewal buried in clause 19, the governing-law line that isn't where you'd want it. A first pass like that across a stack of vendor contracts saves a junior associate an afternoon, and it catches the odd thing a tired human skims past.

Clause extraction and comparison is more of the same: pull every termination provision across forty NDAs into a table, find the three that don't match your template, surface the outliers. It's tedious, mechanical, high-volume work. Precisely the kind of thing worth automating.

Policy and regulation Q&A works if you ground it: point a retrieval system at your own policies, your own regulatory filings, your own internal guidance, and let people ask it questions with answers cited back to the source paragraph. Due-diligence triage across a giant document room, drafting from your own approved templates, summarizing a long contract for a non-lawyer stakeholder. All real, all within reach.

Notice what every one of those has in common. The source of truth is a document you supplied — the model is reading and reorganizing, not recalling.

Where it goes wrong, and why the stakes are different

The fabricated-cases problem isn't a quirk you can prompt away. When the model doesn't have the answer, it fills the gap with something shaped like an answer, and in law, "shaped like an answer" means a plausible citation to a case that was never decided. It doesn't know it's guessing. Neither do you, unless you check every reference, which you must.

Three things make legal higher-stakes than the average AI use case. First, the model's training is frozen, and the law isn't. A statute amended last quarter or a ruling from last month simply isn't in there, and it will answer about the old world with full confidence. Second, jurisdiction matters enormously, and a model blends jurisdictions the way it blends everything else, giving you a confident synthesis of rules that don't all apply where you are. Third, and this is the one that matters most: somebody acts on a legal answer. A wrong summary of a marketing email is an annoyance. A wrong answer about whether a clause is enforceable is a liability, and it's your liability, not the model's.

There's a confidentiality dimension too. The documents you'd feed a legal AI are often privileged or sensitive, so where the data goes is a real question, not a checkbox. The governance and data-handling discipline that applies to any sensitive corpus applies double here, and for some matters the answer is on-prem or nothing.

How to use it without getting burned

The pattern that works is narrow and unglamorous. Ground everything in documents you control, and make the model cite the specific clause or paragraph it's drawing from, so every claim is traceable to a source a human can open and read. Keep a qualified person in the loop on anything that leaves the building or informs a decision. Use the tool to find and to draft, never to decide or advise. Verify every citation, every time. The one you don't check is the one that's invented.

Treat it like an expensive AI standing in for a very fast paralegal, and you've got the right mental model. A paralegal reads the contract, pulls the relevant clauses, drafts from the template, flags the weird stuff, and hands it to the lawyer who actually signs off. A paralegal doesn't give legal advice or get cited as authority. Neither should the model. Kept inside that boundary, AI is a real force multiplier for legal and compliance work. Pushed past it, it's a malpractice generator with excellent grammar.