After lawyers fell into the trap of artificial intelligence errors, judges themselves have begun experimenting with the technology. They are confident it can speed up the judicial system, but errors are already appearing, raising a crucial question: who holds the judge accountable when their AI makes a mistake?
Article contents:
- Introduction: AI enters the courtroom
- Drawing the line: between assistance and ruling
- The slippery slope: is summarization a safe task?
- Errors are already happening
- The accountability crisis: when the judge makes a mistake
- “A crisis waiting to happen”
- The white page problem: the core of the judge’s work
- Conclusion: a delicate balance between efficiency and justice
Introduction: AI enters the courtroom
The tendency of artificial intelligence systems to make errors and human failures to detect those errors has been starkly evident in the US legal system recently. The blunders began when lawyers submitted documents citing non-existent cases. Now, judges are also beginning to experiment with generative AI. Some are confident that with proper precautions, the technology can accelerate legal research, summarize cases, draft routine orders, and generally help speed up an overburdened court system.
Drawing the line: between assistance and ruling
Federal Judge Xavier Rodriguez uses generative AI tools to summarize cases. He will ask the AI to identify the key players involved and then have it generate a timeline of major events. Before certain hearings, he will also ask it to generate questions for lawyers based on the materials they submit. These tasks, for him, do not rely on human judgment. He says, “It’s not any final decision being made, and therefore it’s relatively risk-free.” But he believes using AI to predict whether someone should be eligible for bail, for example, goes too far in the direction of judgment and discretion.
The slippery slope: is summarization a safe task?
Erin Solovey, a professor and researcher in human-AI interaction, says the line between what is appropriate for a human judge and what is appropriate for AI tools varies from judge to judge and scenario to scenario. Even so, some of these tasks simply do not match what AI is good at. Asking to summarize a large document may yield radically different results depending on whether the model was trained to summarize for a general audience or a legal audience. AI also struggles with logic-based tasks like ordering the events of a case. Solovey says: “A timeline that looks very plausible might be factually incorrect.”
Errors are already happening
This summer, we have already seen AI-generated errors that went undetected and were cited by judges. A federal judge in New Jersey had to reissue an order full of errors that may have been caused by AI, and a judge in Mississippi refused to explain why his order also contained errors that looked like AI hallucinations. In June, an appellate judge in Georgia issued an order that relied in part on fictitious cases submitted by a party, an error that went undetected.
The accountability crisis: when the judge makes a mistake
The risks of early adoption have alarmed Judge Scott Schlegel, who warned that AI-generated errors in judicial rulings point to “a crisis waiting to happen,” one that would go beyond the problem of lawyers filing briefs with fake cases. Lawyers who make mistakes can be sanctioned, have their motions denied, or lose their cases when the opposing party discovers and reports the errors. Schlegel says: “When a judge makes a mistake, that is the law. I cannot go back a month or two later and say, ‘Oops, I’m so sorry,’ and take back my decision. That’s not how things work.”
“A crisis waiting to happen”
Unlike lawyers, whom the court can order to explain why there are errors in their filings, judges do not have to show much transparency. On August 4, a federal judge in Mississippi was forced to issue a new ruling in a civil rights case after it was revealed that the original contained incorrect names and gross errors. The judge did not fully explain what led to the errors even after the state asked him to do so. The judge wrote: “There is no justification for further clarification.”
The white page problem: the core of the judge’s work
Schlegel says most of a judge’s job is dealing with what he calls the “white page problem”: you are presiding over a complex case with a blank page in front of you, forced to make difficult decisions. Thinking through those decisions is already the judge’s job. Getting help with a first draft from AI undermines that purpose.
He adds: “If you’re making a decision on who gets the kids this weekend and someone finds out you used Grok and you should have used Gemini or ChatGPT — you know, that’s not the justice system.”
Conclusion: a delicate balance between efficiency and justice
The temptation to increase efficiency using AI in the judicial system is understandable. However, the stakes are too high. The more the line between assistance and ruling becomes blurred, the greater the risk of eroding public trust in the legitimacy of the courts. Before AI becomes an integral part of the judiciary, clear frameworks for accountability and transparency must be established to ensure that justice ultimately remains human.
