ChatGPT Editing Adding Hallucinations to Otherwise Accurate Material?

From Friday’s Connecticut Supreme Court decision in Tov Realty LLC v. Suarez:

The record and the representations to this court at the show cause hearing explain the facts concerning the relevant filings.

Attorney Gottlieb initially used LexisNexis to conduct the legal research necessary to draft his public interest appeal applications and the briefs that followed. He verified the accuracy of the quotations and citations in the initial rough drafts using the Shepard’s Citations Service on LexisNexis.

He then copied and pasted those drafts into ChatGPT, a generative AI program, to make his brief writing process more efficient and to improve the quality of the final written product. Unbeknownst to Attorney Gottlieb, the drafts produced by ChatGPT added new case citations or altered existing case citations, making the final documents filed in this court inaccurate and misleading.

Because the hallucinations created by ChatGPT did not affect the substantive accuracy of the legal propositions contained within the filings, which remained well established and correct, the AI generated changes did not prompt Attorney Gottlieb or his law firm colleagues who reviewed the documents to verify the supporting citations a second time.

Attorney Gottlieb became aware of the erroneous citations only when various amici curiae, represented by the Jerome N. Frank Legal Services Organization, apprised this court and counsel of what appeared to be “hallucinated” case citations in the briefs. Attorney Gottlieb subsequently filed errata sheets and corrected the briefs in both cases.

We then issued a sua sponte order requiring Attorney Gottlieb and the law firm to explain the cause of the erroneous material. In the memorandum filed in response to that order, Attorney Gottlieb and the law firm expressed their deep regret for the errors. They explained that the errors were caused when Attorney Gottlieb, whose original legal research and citations had been verified, used ChatGPT to improve the organization and quality of his rough drafts and that he was unaware that ChatGPT would make substantive changes to that material.

I’ve heard about this happening in other cases, and have no reason to doubt the factual account. The moral: If you want to use AI tools for editing (I’ve heard from people that they often give good editing advice), (1) re-cite-check as well as proofreading everything after the AI editing pass, or (2) ask the AI to offer editing suggestions that you can then manually decide to enter or not.