Ep. 5: Did Your AI Actually Read That?

Disclosure: this episode’s hosts, Mia Hale and Mark Dawson, are AI-generated voices. The script is written and fact-checked by the AI Tools For Law team. Nothing in this episode is sponsored.

Download the MP3 · 9:05

Stories and sources

News from Nov. 6–10, 2026. Vendor claims are attributed to the vendor; we note what each announcement did not disclose.

Did your AI actually read that?

Top story

Midpage said Nov. 6 that it ran the same AI model, Anthropic's Opus 5.5, on 100 legal research questions twice. With ordinary web search, the AI had opened only 33% of the cases its answers relied on, compared with 82% using Midpage's case law tools; an AI judge preferred the Midpage answer on 89 of 100. This is the company's own test, graded by an AI model, not an independent study. Whatever tool you use, open and read every case an AI answer cites before you rely on it.

Review tables that keep up with new files

Tools

Harvey said Nov. 9 that its Review Tables can stay connected to the folder (a Vault) the documents came from; when new files land, one Sync click adds them and runs the same questions while keeping rows already reviewed. On Nov. 5 Harvey also introduced a Fact Graph that, it says, links each fact to its source text; early access opens soon, per Harvey. Someone still has to check the new answers, and it's worth asking any vendor whether its tool links each fact to the exact line it came from.

How to read a vendor's security report

Security

Aderant said Nov. 9 that an outside accounting (CPA) firm finished a SOC 2 Type 2 exam of five cloud products (Expert Sierra, vi by Aderant, iTimekeep, Onyx and BillBlast), covering security, availability and confidentiality. The review periods all ended Sept. 30, 2026; BillBlast's started only July 1, 2026. Aderant says firms can request the report through its trust center. Ask any vendor that holds client data for its SOC 2 Type 2 report, the period it covers, and the exceptions.

The engine under your software

Industry

Anthropic released Fable 5.2 and Mythos 5.2 on Nov. 9, the same underlying model under two names: Fable is open to all users, and Mythos is for organizations approved through its cybersecurity and life sciences access programs. Anthropic says they answer up to 30% faster than the 5.1 versions and cost less, and it recommends starting most coding and knowledge tasks on Opus 5.5. If a vendor says it upgraded its AI, ask which model it now uses and whether your price changes.

Legora: two firm rollouts and a contract-system link

The Short Docket

Legora said Nov. 5 that Philadelphia-headquartered Obermayer Rebmann Maxwell & Hippel LLP selected it as its legal AI platform, and that Fieldfisher is rolling it out to about 1,400 lawyers and business-services staff after a six-week pilot with 137 users (company figures). On Nov. 9 Legora and Agiloft, a contract management system, announced a phased integration aimed at in-house legal teams; it matters to most small firms only if a client runs both tools.

Terms explained in this episode

AI model: The engine that reads and writes text inside an AI tool.
Review table: A tool that pulls the same answers out of many documents at once, one row per document.
Deal data room: The online folder where a seller shares records with a buyer.
Document management system: The system a firm stores its files in.
Due diligence: Checking a company's records before a purchase.
Fact graph: A map of how facts in a matter connect, each linked to its source.
SOC 2: A standard audit of a vendor's controls, the rules and safeguards it says it follows.
Type 2: A SOC 2 report where the auditor checked that the controls actually worked over a period of time.
Exceptions: Places in an audit report where a control didn't work as described.
Availability: Whether a system is up when you need it.
Contract management system: Software that stores contracts and tracks their key dates and obligations.

Transcript

Cold open

Mia: Quick question for every lawyer listening. When an AI tool hands you a case, did it actually read the case?

Mark: One company tested exactly that. Sometimes the answer was no.

Mia: That's the confidence of someone who skimmed the syllabus.

Mark: Stay with us. We'll keep it short.

Mia: This is Legal AI This Week, from the AI Tools For Law team. I'm Mia Hale.

Mark: And I'm Mark Dawson.

Mia: Coming up: an AI research test with a lesson for every firm, document review that keeps up with new files, how to read a vendor's security report, and the AI engine under your software getting an upgrade.

Mia: First up. Midpage, which makes legal research tools, said on November sixth that it ran a test. Same AI model, Anthropic's Opus five point five. One hundred legal research questions. It ran each question twice.

Mark: Once with Midpage's own case law tools, and once with ordinary web search.

Mia: And here's the number. In the web search version, the AI had opened only thirty-three percent of the cases its answers relied on. With Midpage's tools, eighty-two percent.

Mark: Let me put that plainly. With web search, about two out of three cases the answers relied on were never opened by the AI.

Mia: Citing a case you haven't read. Bold.

Mark: Bold is one word for it.

Mia: Okay, I'll be the skeptic. It's a company testing its own product. Of course it came out ahead.

Mark: Fair, and worth saying. This is Midpage's own test, not an independent study. And the grading was done by an AI model too. An AI judge preferred the Midpage answer on eighty-nine of the hundred questions.

Mia: An AI grading AI. Very efficient.

Mark: Efficient, yes. Independent, no. So don't take away "buy this tool." Take away the habit.

Mia: Which is?

Mark: Whatever tool you use, open and read every case an AI answer cites before you rely on it. Not the summary. The case.

Mia: Even if the tool is the expensive one with the nice logo?

Mark: Especially then. The test shows how an answer can sound complete while resting on things nobody read. Your name goes on the filing, not the tool's.

Mia: So the rule hasn't changed since law school.

Mark: It never does. The tool finds. You read. You decide.

Mia: Next, documents that keep showing up. Harvey, which makes AI tools for law firms, said on November ninth that its Review Tables can now stay connected to the folder the documents came from.

Mark: Quick explanation. A review table pulls the same answers out of many documents at once. One row per document. Think of a spreadsheet that fills itself in.

Mia: And Harvey calls that folder a Vault.

Mark: Right. So when new files land in that folder, say from a deal data room or a document management system, the table shows it's out of date. One Sync click adds the new files and runs the same questions on them.

Mia: Deal data room?

Mark: The online folder where a seller shares records with a buyer. And document management is the system a firm stores its files in.

Mia: And the rows already reviewed? Please tell me they don't start over.

Mark: Harvey says they stay put. Its examples include deal due diligence, which is checking a company's records before a purchase, and group claims that come in waves.

Mia: So no more comparing the folder to the spreadsheet by hand at eleven at night.

Mark: That's the manual step it cuts. But someone still has to check the new answers.

Mia: Harvey also announced something called a Fact Graph, for disputes and investigations. Harvey says it finds and connects facts across large sets of documents, and links each fact to the source text it came from.

Mark: A graph here just means a map of how facts connect.

Mia: And it grows the record as new documents arrive, Harvey says. Early access opens soon.

Mia: Most small firms aren't Harvey customers. Why should they care?

Mark: Because the useful idea travels. When any tool gives you a fact, you want a link to the exact line it came from. Ask your vendor whether its tool shows that.

Mia: Now, security reports. Aderant, which makes law firm billing, timekeeping and business software, said on November ninth that an outside accounting firm finished a SOC two Type two exam of five of its cloud products.

Mark: Decoding time. SOC two is a standard audit of a vendor's controls, the rules and safeguards it says it follows. Type two means the auditor checked that the controls actually worked over a period of time, not just that they looked good on paper.

Mia: And this one covered security, availability and confidentiality.

Mark: Availability meaning the system is up when you need it.

Mia: There's a detail I liked. The review periods all ended September thirtieth. But for one product, BillBlast, the period only started in July.

Mark: That matters. A shorter period means less time was observed. It's a real exam, just a shorter look.

Mia: So how does a five-lawyer firm use this? Nobody's reading a security audit for fun.

Mark: Not for fun. But here's the trick. Aderant says firms can request the report through its trust center. Ask for it, and go straight to the exceptions.

Mia: Exceptions?

Mark: The places where the auditor found a control didn't work as described. That's where the real information is, not the press release.

Mia: And for vendors who aren't Aderant?

Mark: Same question for any vendor that holds client data. Do you have a SOC two Type two report, what period does it cover, and can I see the exceptions?

Mia: Last story. Anthropic, an AI model developer, released two new models on November ninth. Fable five point two and Mythos five point two.

Mark: An AI model is the engine that reads and writes text inside an AI tool. Two names, but Anthropic says they're the same underlying model.

Mia: Fable is open to all users. Mythos is for organizations approved through its cybersecurity and life sciences programs. Anthropic says they answer up to thirty percent faster than the five point one versions, and cost less.

Mark: And Anthropic itself recommends starting most coding and knowledge tasks on a different model, Opus five point five, and moving up only for harder jobs.

Mia: A company telling you not to start with its newest one. Write that down.

Mark: Noted. Here's why it's on a law firm show. Many legal AI products are built on models like these. The vendor you pay may not have built the engine.

Mia: So when a vendor emails, "We've upgraded our AI"...

Mark: Ask two things. Which model do you use now? And does my price change?

Mia: Quick hits. Legora, another legal AI company, says a Philadelphia full-service firm, Obermayer, picked it as its AI platform. And it says Fieldfisher is rolling it out to about fourteen hundred people after a six-week pilot with a hundred thirty-seven users.

Mark: Those are company figures. Big firms, mostly a sign of where the market is heading.

Mia: Legora also announced a partnership with Agiloft, a contract management system, so shared customers can search their contracts from inside Legora. It's rolling out in phases.

Mark: That's aimed at in-house legal teams. It matters to a small firm mainly if a client runs both tools.

Mia: So, Mark. Three moves for Monday morning.

Mark: One: open and read every case an AI answer cites before you rely on it.

Mia: Two?

Mark: Ask each vendor that holds client data for its SOC two Type two report, and read the exceptions.

Mia: And three?

Mark: Ask which AI model sits under your tools, and whether an upgrade changes your price.

Mia: And check your state's ethics rules on all of it. This isn't legal advice.

Mark: That's Legal AI This Week, from the AI Tools For Law team. New episode every Friday at aitoolsforlaw dot A I. Sources are in the show notes.

Mia: Spotted an error? Email hello at aitoolsforlaw dot A I.

About the show

Not legal advice. Corrections: hello@aitoolsforlaw.ai.

Listen on Spotify · Apple Podcasts · RSS. Written version: the newsletter.

All episodes