Should the Bar 'Bar' AI? Use in Legal Profession Has Risks and Rewards

DarrowEverett LLP

DarrowEverett LLP

ChatGPT, a platform from OpenAI, has been leading the charge for general use Artificial Intelligence (“AI”), with serious implications for businesses. AI is, at its core, “… the overarching description for technologies that use computers and software to create intelligent, humanlike behavior.”[1] While the legal profession can be slow to adopt new technologies, the industry has seen an impressive, if, albeit dramatic, push to adopt AI technology — and, legal clients may be expecting their counsel to do the same. According to a Thompson Reuters Institute report, more than 80% of law firm leaders believe AI can be applied to legal work now and more than 50% believe it should be applied to legal work.[2] AI is currently being used in reviewing contracts, the discovery process, legal research, and even litigation strategy.[3] However, before legal professionals can truly evolve with AI, practitioners (and their clients) need to understand the risks and rewards that come with adopting these new technologies.

Risks of AI Use

In February 2023, a Defendant was planning on using AI to try to battle a traffic ticket issued in California.[4] DoNotPay, a New York-based company, had been using ChatGPT and other AI sources to provide written defenses to traffic tickets on behalf of clients. The Defendant intended to use smart glasses in court and then respond to any questions with AI generated arguments through an earpiece. Once word got out of the plan, state bars sent DoNotPay threats for unauthorized practice of law without a license, a misdemeanor in several states. Id. Though DoNotPay ceased its efforts, this did not end the incursion of use of AI in the legal field.

In May 2023, plaintiff’s counsel in Mata v. Avianca Inc., Case No. 22-CV-1461 (S.D.N.Y., filed Feb. 22, 2022), were called out by opposing counsel and Judge Kevin Castel that the cases cited by plaintiff’s counsel appeared to be “bogus.”[5] Counsel had used ChatGPT to research cases, including case citations and quotes, all of which were fake and fabricated by AI. After a hearing on the manner in how such cases made their way into a Federal Court filing, the Court issued sanctions against the counsel, including fines of $5,000 each.[6] This should serve as warning against unchecked use of AI in court.

In Federal courts, counsel are required by Federal Rule of Civil Procedure 11 (popularly known as “Rule 11”) to sign off on all documents filed with the Court. An attorney’s signature under Rule 11 acts as certification that everything in the filing is true and correct within the best of that attorney’s knowledge and belief.[7] This includes anything put in front of the Court, not just evidence but also the legal authorities and arguments put forth. In addition to Rule 11, individual judges have also started issuing orders requiring all attorneys appearing before them to disclose any use of AI and to certify that human review was conducted on any AI generated content.[8]

While AI can be useful in drafting certain documents, caution is advised regarding the reliability of AI generated content. OpenAI has the following in its ChatGPT Terms of Use:

Accuracy. Artificial intelligence and machine learning are rapidly evolving fields of study. We are constantly working to improve our Services to make them more accurate, reliable, safe and beneficial. Given the probabilistic nature of machine learning, use of our Services may in some situations result in incorrect Output that does not accurately reflect real people, places, or facts. You should evaluate the accuracy of any Output as appropriate for your use case, including by using human review of the Output.[9]

Accordingly, OpenAI disclaims the accuracy of the information it provides. While most companies have a blanket indemnification in their terms of use, the specificity of OpenAI’s “Accuracy” section should give pause to any attorney seeking to use the platform as a resource.

Attorneys must also consider client-confidentiality when using AI. Inputting client-specific facts into an AI tool could cause data mishandling and a possible breach of client-confidentiality. Each AI tool uses its own servers, outside a firm’s infrastructure, to provide the digital analysis requested. The security of these servers is unknown. Relinquishing control over client data must be a factor in considering the use of AI in a legal practice.

Rewards of AI Use

Ultimately, the greatest benefit that AI provides attorneys is time savings. However, AI is not limited to quickly creating output for an attorney. There are a plethora of AI tools that can assist in the day to day administrative tasks that often burden attorneys and the judicial system. For example, Gideon is an AI chatbot that can help streamline client-intake questions.[10] Another AI tool, Diligen, assists attorneys by reviewing contracts for specific clauses or changes in a document and can provide a summary of lengthy documents.[11] Courts in New York and California have used the Correctional Offender Management Profiling for Alternative Sanctions (“COMPAS”) system, an AI tool that assesses the likelihood a defendant is to reoffend, in bail determinations.[12]

AI can also help in several other areas of daily legal work such as analysis of discovery documents for relevant or confidential information, or proofreading briefs.[13] What could be hours of legal billing to find the proverbial needle in the haystack can be reduced with a few clicks of the mouse, saving the client legal fees and providing the attorney additional hours of capacity for other tasks. AI also provides attorneys with the ability to assess voluminous documents at the onset of a matter, reducing the amount of time needed to accurately understand the full breadth of the matter from the beginning of an engagement and analyze potential risks and outcomes, rather than sift through pages to establish timelines or sequences of events.


Ultimately, AI further allows attorneys to pass on savings to their clients and improve their client relationships — first, by reducing the number of hours necessary to complete time-consuming tasks, and second, by allowing attorneys to time to tackle the matters and/or issues that cannot be handled by AI, such as signing under penalty of perjury. However, both attorneys and their clients should be wary of over-relying on AI. For now, AI can’t guarantee the accuracy of its byproduct (or interface with clients). But these technologies continue to evolve, and we may need to reconsider our position, and reliance, on AI, in the near future.

[1] How Artificial Intelligence is Used in Legal Practice | Bloomberg Law

[2] Generative AI in law firms: For many, such technologies are still a great unknown | Reuters

[3] AI in Law: The Transformation of Legal | Clio

[4] An AI robot lawyer was set to argue in court. Real lawyers shut it down. : NPR

[5] Lawyer apologizes for fake court citations from ChatGPT | CNN Business


[7] Federal Rules of Civil Procedure Rule 11. Note that most state courts have an identical requirement.


[9] Terms of use (





[View source.]

DISCLAIMER: Because of the generality of this update, the information provided herein may not be applicable in all situations and should not be acted upon without specific legal advice based on particular situations.

© DarrowEverett LLP | Attorney Advertising

Written by:

DarrowEverett LLP

DarrowEverett LLP on:

Reporters on Deadline

"My best business intelligence, in one easy email…"

Your first step to building a free, personalized, morning email brief covering pertinent authors and topics on JD Supra:
*By using the service, you signify your acceptance of JD Supra's Privacy Policy.
Custom Email Digest
- hide
- hide