[Editor’s Note: EDRM is thrilled to welcome Doug Austin of eDiscovery Today to the platform.]
This seems like the perfect topic to cover for my very first blog post on the EDRM site, for which I’ll be starting to provide posts regularly! Last week, I received an email from a reader of eDiscovery Today in which she asked: Why don’t people use Technology Assisted Review (TAR)/predictive coding (and related technologies) more?
It’s a great question and the use of predictive coding (or lack thereof) is illustrated in the State of the Industry report that eDiscovery Today publishes every year, which is sponsored by EDRM. In this year’s report (which is summarized here), out of 281 respondents, only 25.9% of respondents said they use predictive coding in all or most of their cases, while 36.3% of respondents said they use it in very few or none of their cases.
“Legal professionals tend to assume that their opposing counterparts know what they’re doing when it comes to keyword search (even though many don’t), but they only tend to want proof that they know what they’re doing when they’re using TAR.
Doug Austin, Editor, eDiscovery Today
Even worse, those numbers reflect less usage than the previous year, where 31.1% of respondents said they use predictive coding in all or most of their cases (5.2% higher than this year), and 32.8% of respondents said they use it in very few or none of their cases (3.5% lower than this year). In other words, a smaller percentage of respondents uses it most of the time and a larger percentage of respondents hardly uses it.
Does this mean the use of TAR is regressing? Not necessarily. I think the fact that this year’s survey had over 50% more respondents than last year’s survey (which had 183 respondents) means that it illustrates a broader, more realistic view of the industry’s use of it. Keep in mind these are still people who chose to take an eDiscovery survey, so those results are probably still more optimistic than the legal community overall.
So, why don’t more people use TAR? Here are 3 reasons:
Lack of Understanding: Simply put, people avoid what they don’t understand. And – despite many efforts to educate the legal community on it – they still don’t understand how machine learning technologies work. To many, it’s a “black box” that they don’t understand and feel ill equipped to defend in court if they are required to do so.
Conversely, most legal professionals think they understand keyword search (because they learned it in law school with Westlaw or Lexis, or they know how to perform a Google search), but many don’t understand keyword search best practices when it comes to discovery, including testing search results and the null sets for those results. This recent example illustrates that many legal professionals don’t understand keyword search either. But most think they understand it, so they prefer that approach over something they know they don’t understand.
Unwillingness to Change: Even if they do understand the technology and recognize the efficiency and effectiveness of TAR, many legal professionals are still unwilling to change their workflows to incorporate it. When there are a lot of documents to review, the mindset is to put more reviewers on it. In some cases, this may be because there is a concern that offering a different approach may reduce billable hours (there, I said it!), while in other cases, it may simply be a resistance to change.
This case that we discussed in the April case law webinar illustrates that – the defendants were looking at reviewing 225,000 documents at a cost of between $140,000 and $235,000, with a team of 15 reviewers. Yet, there was no mention of the consideration of TAR anywhere. It’s natural for people to stick with what they know unless forced to change, even if they recognize there may be a more efficient way to do it.
More Transparency is Expected with TAR: Another reason that legal professionals avoid using TAR is that opposing parties tend to want more transparency on TAR processes than they do on the traditional keyword search approach. Why? That’s a good question and it probably relates to reasons 1 and 2 above.
Maura R. Grossman and Gordon V. Cormack noted in their paper last year The eDiscovery Medicine Show (which I covered here) the “misconception…that only the TAR tool should be subject to validation, while keyword culling and manual review should be exempt, as they have always been.” Legal professionals tend to assume that their opposing counterparts know what they’re doing when it comes to keyword search (even though many don’t), but they only tend to want proof that they know what they’re doing when they’re using TAR.
In last year’s State of the Industry report, Judge Andrew Peck (ret.), who authored the first ruling to approve the use of TAR over ten years ago in Da Silva Moore, stated as a reason that more people don’t use TAR: “Part of the problem remains requesting parties that seek such extensive involvement in the process and overly complex verification that responding parties are discouraged from using TAR.”
It shouldn’t be that way, but it is. So, how do we get people to start using TAR? I’ll discuss that in my next post! Yes, my first post for EDRM is a two-parter! Stay tuned!
