If Bill Gates walked into a bar in the poorest part of town, the average patron there would be a billionaire. Statistically speaking, that is. And while this data-driven observation is meaningless, it serves as a helpful reminder that statistical analysis – when carelessly deployed – can yield absurd and unhelpful information.
Marketers would be making a grave error indeed if they surmised that, based on average income, our bar patrons were good prospects for a custom-built, hydrogen-powered superyacht or even, somewhat further down market, a pair of tickets to hear Kendrick Lamar live this summer. As statisticians know, average values are misleading when the dataset contains extreme values or data that is not distributed in a familiar “Bell curve” arrangement. This is the reason why real estate marketers describe home values in terms of median prices. Unlike average prices, a median price – the single price separating the lower half of property values from the higher half in the market under consideration – prevents a single, high-value property from overpricing other properties in the market.
Given the pace of technological innovation in artificial intelligence and the increasing amounts of legally relevant data available to digest, it seems inevitable that predictive legal analytics will become pervasive and unremarkable in law practice. Computers are faster, and likely soon will be smarter, than even the largest teams of individual lawyers.
Better Data Makes for Better Decisions
The Bill Gates scenario is a suitable lens through which to view the family of artificial intelligence technologies that claim an ability to predict courtroom outcomes. These tools are offered by all major legal research providers as well as a growing number of startups, commonly marketed under the terms “predictive analytics,” “litigation analytics,” or “legal analytics.” Broadly speaking, legal analytics technologies claim the ability to:
- analyze historical data, judge rulings, opposing counsel behavior, and case facts to predict likely outcomes
- predict potential jury verdicts and likely settlement ranges
- uncover patterns in how specific judges rule and how opposing counsel behaves
- predict how judges will interpret legal issues or rule on causes of action that could be raised
- provide lawyers with data that can be shared with clients regarding likely litigation outcomes, plausible timelines, and litigation costs
- suggest favorable or unfavorable jurisdictions for particular causes of action
- provide likely outcome predictions early in the litigation process, helping lawyers focus on the most promising cases or causes of action
- eliminate human bias in litigation strategy and settlement negotiations
Whether or not these tools can deliver on their promises depends largely on the available data and the suitability of the statistical tools providing the analysis.
Take, for example, the task of predicting how a judge will rule on a particular matter. Possessing a proficient crystal ball in this area seems like a game-changer for lawyers weighing whether to file, try, or settle a case. Some tools examine publicly available data from the federal courts’ PACER (Public Access to Court Electronic Records) database, while others plumb somewhat non-obvious data points. One such predictive analytics vendor claims an 81% accuracy rate by examining – among dozens of other factors – a judge’s net worth, where they attended law school, and whether they worked at a small or large law firm before becoming a judge. Political affiliation and gender also play roles in some predictive analytics tools.
To the extent that predictive analytics tools actually provide the benefits they claim, their value is unquestionably compelling. In litigation, the party with the best information will always have a clear advantage, whether in settlement negotiations or at trial.
Is it valuable to know that federal court judges grant 99% of motions for pro hac vice admission? Probably not. Or that 52% of contested summary judgment motions are granted? Again, most likely not given the nearly infinite number of possibly dispositive differences among cases. On the other hand, an analysis of jury-delivered damage awards in very similar cases across a number of possible jurisdictions might be highly valuable to a large corporate entity with legal exposure to those sorts of claims. The same might be said for knowing that a particular judge had a marked tendency to permit or exclude expert testimony.
Predictive legal analytics, if tied to a specific type of legal claim in a particular jurisdiction, might also be useful for parties weighing a decision whether to litigate or arbitrate their claims. Litigation analytics tools, which can uncover the experience that lawyers have in handling specific types of cases, are frequently consulted when hiring outside counsel or when formulating litigation strategy against known or likely opposing lawyers.
Litigation analytics tools have already been used by courts to provide an evidentiary basis for venue transfer rulings (e.g., which courts are busier than others) and for deciding reasonable hourly rates for attorneys’ fee demands (e.g., what is the experience level of a particular attorney for a particular matter).
Does Predictive Analytics Deliver Yet?
Given the pace of technological innovation in artificial intelligence and the increasing amounts of legally relevant data available to digest, it seems inevitable that predictive legal analytics will become pervasive and unremarkable in law practice. Computers are faster, and likely soon will be smarter, than even the largest teams of individual lawyers.
Today, however, computers are still having to prove their worth in the legal marketplace. According to a 2024 survey of legal professionals by legal analytics provider Lex Machina, 68% of lawyers at firms of all sizes are using legal analytics on case matters. Sixty-seven percent of survey respondents said that legal analytics tools are useful for pitching and demonstrating expertise to potential clients. In the area of litigation, 71% said that litigation analytics are useful for gaining competitive insights on opposing counsel, parties, and judges. Fifty-six percent of surveyed lawyers said they use these tools for formulating case strategy.
Lawyers assessing the value of predictive analytics technologies will also encounter the same hurdles presented by generative artificial intelligence tools: hard evidence of reliability is often lacking in vendor-supplied materials. In the end, however, litigators themselves will decide whether these tools do, in fact, provide value to their law practice and their clients.
This article is the third in a series on artificial intelligence in litigation practice. Previous articles examined generative artificial intelligence for legal research and artificial intelligence in electronic discovery.
