{"id":53060,"date":"2026-05-27T19:39:19","date_gmt":"2026-05-27T19:39:19","guid":{"rendered":"https:\/\/www.europesays.com\/ai\/53060\/"},"modified":"2026-05-27T19:39:19","modified_gmt":"2026-05-27T19:39:19","slug":"artificial-intelligence-can-be-used-for-grading-law-school-exams-but-should-it-be","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/ai\/53060\/","title":{"rendered":"Artificial intelligence can be used for grading law school exams, but should it be?"},"content":{"rendered":"

\t\t\t\t\tHome<\/a>
\n\t\t\t\t\tWeb First<\/a>
\n\t\t\t\t\tArtificial intelligence can be used for grading\u2026<\/p>\n

Legal Education<\/p>\n

\t\t\t\tArtificial intelligence can be used for grading law school exams, but should it be?<\/p>\n

By Julianne Hill<\/a><\/p>\n

May 27, 2026, 2:08 pm CDT<\/p>\n

\"AI<\/p>\n

Artificial intelligence is being put to the test, literally, as it is being used to grade law students\u2019 exams. But should it be? (Image from Shutterstock)<\/p>\n

Artificial intelligence is being put to the test, literally, as it is being used to grade law students\u2019 exams. But should it be?<\/p>\n

Jack Graves, a professor at the Syracuse University College of Law in New York, says yes.<\/p>\n

\u201cAI doesn\u2019t get tired; it doesn\u2019t get distracted; it doesn\u2019t get frustrated; it doesn\u2019t say, \u2018Why didn\u2019t you understand this?\u2019\u201d Graves says. \u201cIf I use it correctly, I am absolutely certain it is more consistent in scoring essays than I am, and when you\u2019re scoring on a curve, consistency is the No. 1 concern.\u201d<\/p>\n

Graves, who actively used ChatGPT to grade student work last semester but is considering shifting to Claude<\/a>, says he finds no meaningful variances statistically between the scores given by AI and the grades that he\u2019s given.<\/p>\n

A recent paper<\/a> published in the Journal of Law and Empirical Analysis backs that up.<\/p>\n

The paper\u2019s six authors, all law professors at different schools, compared OpenAI\u2019s GPT-5\u2019s ability to grade final written exams from four different subjects at top 30 law schools to the grading by human professors.<\/p>\n

\"Jack
\nJack Graves of the Syracuse University College of Law in New York: \u201cAI doesn\u2019t get tired; it doesn\u2019t get distracted; it doesn\u2019t get frustrated; it doesn\u2019t say, \u2018Why didn\u2019t you understand this?\u2019\u201d<\/p>\n

\u201cThe professors took old law school exams they\u2019d actually given to law students and had graded themselves and fed them to AI along with the actual rubrics. Then they tried various prompting methods to grade the same exams to see how they compared the actual grades given by professors,\u201d says Daniel Schwarcz, a study co-author and a professor at the University of Minnesota Law School.<\/p>\n

\u201cWhen provided with a detailed rubric, the [large language model] grades correlate with the human grader at Pearson correlation coefficients of up to 0.93,\u201d according to the paper, meaning that there was a significant correlation between how a human grades and how AI grades an assignment.<\/p>\n

For Schwarcz, the question is not whether professors can use AI but how.<\/p>\n

\u201cYou should first grade on your own without an AI,\u201d Schwarcz says. \u201cAnd then you may want to use an AI to sort of see if there are any discrepancies and then double-check.\u201d<\/p>\n

Graves disagrees and lets AI take the lead in grading.<\/p>\n

\u201cEarly on, I pretty much redid its grading. Now, I\u2019m very comfortable spot-checking,\u201d he says. \u201cI don\u2019t vet every single interaction.\u201d<\/p>\n

It\u2019s all how you train the AI, which involves a front-end investment of time and money, Graves says. He has been using ChatGPT but is considering shifting to Claude Cowork.<\/p>\n

His method involves inputting assignments and texts, collaborating with the tool to create rubrics, and providing it with model answers and sample assignments that he\u2019s already graded. He carefully provides prompts to avoid sites such as Reddit and works to make the likelihood of hallucinations low.<\/p>\n

\u201cSo far, in one semester of evidence and one semester of contracts and over 1,000 interactions, I have evidence of one outright hallucination,\u201d Graves says. \u201cIt gets to know you and your system. It applies your input on a very nuanced and a very consistent basis. That\u2019s much the way we historically train people to score the essays on a bar exam.\u201d<\/p>\n

But Schwarcz says he refuses to use AI as the first method of grading for \u201ca procedural fairness reason.\u201d<\/p>\n

\u201cYour students have an expectation that their exams will be graded by a human. It\u2019s not fair to deprive them of that opportunity without really getting their buy-in,\u201d Schwarcz adds.<\/p>\n

\"Daniel
\nDaniel W. Linna Jr. of the Northwestern University Pritzker School of Law: \u201cWe need to move forward and get past this idea of, \u2018We\u2019re going to prohibit; we\u2019re going to disclose.\u2019 The tools are here. They\u2019re integrated in more and more places.\u201d<\/p>\n

Just as many universities require students to disclose whether they\u2019re using an AI to write a paper, Schwarcz adds, professors have to disclose their use.<\/p>\n

\u201cThe power dynamic is such that, what is a student supposed to do, right?\u201d Schwarcz says.<\/p>\n

Daniel W. Linna Jr., a senior lecturer and the director of law and technology initiatives at the Northwestern University Pritzker School of Law in Illinois, disagrees.<\/p>\n

\u201cWe need to move forward and get past this idea of, \u2018We\u2019re going to prohibit; we\u2019re going to disclose,\u2019\u201d says Linna, a 2018 ABA Journal Legal Rebel<\/a>, noting that Northwestern University provides Microsoft Copilot and Microsoft 365 Copilot (formerly Copilot Enterprise). \u201cThe tools are here. They\u2019re integrated in more and more places.\u201d<\/p>\n

While Linna does not use AI to grade, he does use Perusall, a collaborative reading and annotation platform, to measure engagement with readings.<\/p>\n

Making the grade<\/p>\n

While grading with AI, Graves makes sure to protect students\u2019 identities. He carefully assigns each student\u2019s work a number to eliminate privacy concerns and creates a project that is only facing him, so that the test is not available to others.<\/p>\n

Grading with AI allows the professor to interrogate it to offer insights into the professor\u2019s grading and teaching, Graves adds.<\/p>\n

\u201cI can ask it, \u2018What are the 10 issues my students struggled with the most on this exam?\u2019\u201d he says. \u201cIt\u2019s much better at remembering everything it did in the process of scoring than I am.\u201d<\/p>\n

That information can change how he handles the topic in the next semester\u2019s class, Graves says.<\/p>\n

Graves adds that AI bots could also administer oral exams, including tests during the semester for students to demonstrate progress and understanding of the material.<\/p>\n

\u201cI can see saying to a student, \u2018We\u2019ve reached week three, and we\u2019ve covered this set of materials. You now have to go on this bot, take the test and return it to me a passing score,\u2019\u201d he says, adding that he would allow students to take it as many times as possible and to file protests if they think that the AI has treated them unfairly. \u201cIf you bomb it the first time, I don\u2019t see it. Do it again and again. But to continue with the course, you have to submit to me a link that shows me that the bot said you pass.\u201d<\/p>\n

The paper\u2019s authors agree.<\/p>\n

\u201cEven if they do not fully replace humans in the near future,\u201d the paper concluded, \u201cLLMs could soon be put to valuable tasks by law school professors, such as reviewing and validating professor grading, providing substantive feedback on ungraded midterms, and providing students feedback on self-administered practice exams.\u201d<\/p>\n

\t\t\t
\n Write a letter to the editor, share a story tip or update, or report an error.
\n<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"Home Web First Artificial intelligence can be used for grading\u2026 Legal Education Artificial intelligence can be used for…\n","protected":false},"author":2,"featured_media":53061,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[5190,24,25,5203,31237,5198,2105,31234,1528,597,120,2594,31235,28734,31236,564,5191,2348,3423,31238,5200,5192,134],"class_list":["post-53060","post","type-post","status-publish","format-standard","has-post-thumbnail","category-ai","tag-aba-journal","tag-ai","tag-artificial-intelligence","tag-artificial-intelligence-robotics","tag-bar-exam","tag-career-practice","tag-careers","tag-education-law","tag-ethics","tag-government","tag-illinois","tag-law","tag-law-professors","tag-law-schools","tag-law-students","tag-legal-education","tag-legal-news","tag-minnesota","tag-new-york","tag-privacy-law","tag-public-interest","tag-states","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/53060","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=53060"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/53060\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/53061"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=53060"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=53060"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=53060"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}