Description
A chatbot powered by tons of data from the internet passed exams at a US law school after writing essays on topics ranging from constitutional law to taxes and torts. OpenAI's ChatGPT, an American company that received a massive cash injection from Microsoft this week, uses artificial intelligence (AI) to generate text streams from simple prompts. The results have been so good that educators have warned that it could lead to widespread fraud and even signal the end of traditional classroom teaching methods.
Jonathan Choi, a professor at the University of Minnesota Law School, gave ChatGPT the same test the students faced, consisting of 95 multiple-choice and 12 essay questions.
In a white paper titled "ChatGPT Goes to Law School" published Monday, he and his co-authors reported that the bot earned a C overall.
While that was enough to pass, the bot was near the bottom of the class in most subjects and was "bombarded" with math-related multiple-choice questions.
- 'Not a good student' -
"When writing essays, ChatGPT displayed a strong understanding of basic legal norms and had consistently strong organization and composition," the authors wrote.
But the bot "often had trouble detecting problems when given an open prompt, an essential skill in law school exams."
Officials in New York and other jurisdictions have banned the use of ChatGPT in schools, but Choi suggested it could be a great teaching aid.
“Overall ChatGPT was not a great law student acting alone,” he wrote on Twitter.
"But we hope that collaboration with human, language models like ChatGPT will be very useful for law students taking exams and practicing law."
And downplaying the possibility of cheating, he wrote in response to another Twitter user that two of the three markers had seen the paper written by the bot.
"(They) had a hunch and their hunch was correct as ChatGPT had perfect grammar and was somewhat repetitive," Choi wrote.
