AI Can Review Contracts—But It Can’t Own the Mistakes

The widely cited 94% versus 85% accuracy figure comes from a 2018 LawGeex whitepaper that tested its own AI on five NDAs with twenty lawyers, where the answer key was partly derived from the AI’s own responses, the study was never peer‑reviewed or replicated, and the page is now offline. Independent benchmarks such as the Vals Legal AI Report show AI outperforming lawyers in question answering, summarization and data extraction, but lagging in redlining and nuanced clause detection, with lawyers scoring 79.7 versus the best AI’s 65 on redlining tasks. Randomized trials from Minnesota found no significant quality improvement in contract drafting, though speed gains were observed. Vendor terms of service explicitly disclaim accuracy and cap liability, typically to fees paid in the prior year, with examples like Harvey’s $250,000 limit for wrong output. No court cases have yet adjudicated AI‑reviewed contract errors, but professional liability insurers report rising AI‑related claims, especially in transactional work, indicating a shift from theoretical to real risk. Ultimately, legal responsibility remains with the lawyer or user who signs the contract, as AI tools assist but do not replace professional judgment.

(Source：Hackernoon)