Supreme Court justice Kagan praises Claude AI for “exceptional” legal analysis

Supreme Court Justice Elena Kagan recently praised Anthropic’s Claude chatbot for providing “exceptional” analysis of a complex Constitutional dispute involving the Confrontation Clause. Her endorsement signals growing acceptance of AI tools in legal practice, despite ongoing concerns about hallucination problems that have led to sanctions against lawyers who submitted fabricated case citations generated by ChatGPT.

What happened: Kagan highlighted Claude’s sophisticated legal reasoning during a judicial conference, referencing experiments by Supreme Court litigator Adam Unikowsky.

• Unikowsky used Claude 3.5 Sonnet to analyze the Court’s majority and dissenting opinions in Smith v. Arizona, a Confrontation Clause case where Kagan authored the majority opinion.
• “Claude is more insightful about the Confrontation Clause than any mortal,” Unikowsky wrote in his blog post describing the experiment.
• The Confrontation Clause, part of the Sixth Amendment, guarantees defendants the right to cross-examine witnesses testifying against them.

The legal profession’s AI struggle: While Kagan’s praise represents a high point, the legal field continues grappling with AI’s reliability issues.

• Multiple lawyers have faced sanctions for submitting ChatGPT-generated briefs containing fabricated legal cases and citations.
• Last month, a federal judge sanctioned three Alabama lawyers for including fictitious cases generated by ChatGPT in a filing defending the state’s prison system.
• No formal rules currently bar lawyers from using AI, though several legal bodies have issued ethics guidelines and best practices.

Why this matters: Kagan’s endorsement could encourage broader AI adoption among legal professionals, but significant risks remain.

• Chief Justice John Roberts noted in a 2023 report that AI legal advisors could potentially help those unable to afford human lawyers, while expressing confidence that judges won’t become obsolete.
• A Microsoft report placed “lawyers, judges, and related workers” in the middle tier of jobs likely to be replaced by AI.
• Kagan admitted she didn’t “have the foggiest idea” how AI will ultimately reshape the legal field.

The bigger picture: AI’s pattern recognition capabilities align well with legal analysis, but hallucination problems prevent widespread adoption.

• Chatbots like Claude and ChatGPT excel at detecting subtle patterns across large data sets, similar to what human lawyers do but at greater scale.
• The ongoing reality of AI hallucination means meaningful integration into legal workflows remains challenging.
• Without federal regulation, individual discretion governs AI use in legal practice, raising concerns given the high stakes involved.