The prowess of ChatGPT, a chatbot that runs on massive amounts of data from the Internet and is able to write texts in response to simple questions, has been admired, but also feared, by many Internet users since its launch late last year. Some of the results were so compelling that professors at several universities became concerned about the risk of widespread fraud and the end of traditional classroom methods.
“ChatGPT has demonstrated that it has mastered the basic legal norms”
Jonathan Choi, a professor at the University of Minnesota School of Law, gave ChatGPT the same test students take at graduation: 95 multiple-choice questions and 12 quizzes across four disciplines. In a scientific paper published on Monday, January 23, 2023, Mr. Choi and his co-authors indicated that the robot received an overall C+ grade. Although this score is enough for admission, in most subjects the robot took the last place in the class, scientists say. “During the writing of his essays, ChatGPT has demonstrated that it has internalized the basic legal norms and that its organization and structure are consistently sound,” the authors write.
Perfect and Repetitive ChatGPT Grammar
However, the chatbot “often struggled to spot problems when asked an open-ended question, which is an important skill in law school exams,” they note. Authorities in New York and other jurisdictions have banned the use of ChatGPT in schools, but Choi believes the robot could be a great teaching aid. “Overall, ChatGPT was not a very good law student when acting alone,” he tweeted. “But we believe that when collaborating with people, language models like ChatGPT can be very useful for law students taking exams and legal practitioners,” he added.
To allay fears of fraud, Choi also said that two out of three proofreaders noticed the assignment written by the chatbot. “[Ils] he had a hunch and it turned out to be correct because ChatGPT had perfect grammar and was somewhat repetitive,” he wrote.