AI Assisted in Writing the California Bar Exam, Sparking Outrage in the Legal Community.

You’ve heard of AI models taking the bar exam, but in this instance, AI also played a role in creating the questions. The State Bar of California disclosed on Monday that it utilized AI to develop part of its exam questions, as reported by the LA Times. The AI-generated exam questions were crafted by an independent psychometrician named ACS Ventures, hired by the State Bar. According to the statement from the State Bar, the questions were “developed with the assistance of AI and subsequently reviewed by content validation panels and a subject matter expert in advance of the exam.” The LA Times indicated that 23 out of the 171 multiple-choice questions were created by ACS Ventures, while the majority of the questions were developed by Kaplan, with a “small subset” taken from the First-Year Law Students’ Exam. This past year, the bar exam was administered remotely to California-based test takers, leading to frustration as students and educators experienced issues with the remote platform, which was plagued by glitches. The revelation that some exam questions were generated by AI has intensified this frustration.

“I’m almost speechless. Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable,” said Mary Basick, assistant dean of academic skills at UC Irvine School of Law, as quoted by the Times. “It’s a staggering admission,” noted Katie Moran, an associate professor at the University of San Francisco School of Law.

Moran also highlighted that ACS Ventures, the firm responsible for crafting the AI-generated questions, was also the one to approve them. Alex Chan, chair of the State Bar’s Committee of Bar Examiners, mentioned that the California Supreme Court had urged the State Bar to explore “new technologies, such as artificial intelligence” to enhance reliability and cost-efficiency.

The adoption of AI for automating tasks has surged since the rise of generative AI, affecting not only simple tasks and low-stakes work but also critical jobs with significant implications. There are concerns that the formula used to determine the Trump Administration’s tariff rates may have been developed by AI technologies. In 2023, two New York lawyers faced sanctions for using AI in a legal brief that cited fabricated cases. Academic journals are now filled with papers containing AI-generated content, illustrating the widespread use and potential issues that arise from this practice. The rapid ability of generative AI to write, summarize, and source information presents an appealing way for workers to save time and effort, but it also comes with inherent hallucination problems and ethical dilemmas—especially for law students whose entire careers hinge on passing the bar exam.

Topics
Artificial Intelligence