AI Tutors Beat Law Professors in Stanford Blind Study, Exposing Bias Risk

In a blind evaluation conducted at Stanford Law School, 16 contracts‑law professors compared AI‑generated answers to student questions with those written by peers. The AI, primarily Google’s Gemini 2.5 Pro and NotebookLM, won 75 % of pairwise matchups and matched the performance of the best human instructor. Professors flagged AI responses as pedagogically harmful only 3.5 % of the time, versus 12 % for human answers, indicating that AI is rarely counterproductive. The study underscores that legal reasoning—requiring synthesis, doctrine application, and judgment—can be effectively supported by current AI models. However, the same research team uncovered systematic bias when AI models, including GPT‑4, delivered worse outcomes to users with names associated with racial minorities or women. Using a social‑science audit design, they found disparate treatment across 42 prompt templates and multiple models, and later demonstrated that bias‑driving neurons can be pruned, though the effect is context‑specific. These findings raise questions about liability for foundation model developers versus deployers, and highlight the need for bias auditing in AI‑enabled legal tools. Meanwhile, law schools remain divided: the University of Chicago has banned laptops in first‑year core courses to protect independent reasoning, while the Stanford data suggests that blanket skepticism toward AI tutors may be unwarranted. The liftlab team also explores AI as a research assistant, developing tools for contract drafting risk assessment and immigration intake, aiming to extend high‑quality legal guidance to underserved populations while addressing bias and access‑to‑justice concerns.

(Source：Techtimes)