The RAND Corporation’s much-awaited report assessing bid protests of Department of Defense (DOD) procurements is out. The report fulfills a Congressional mandate in the National Defense Authorization Act (NDAA) for FY 2017 to analyze a variety of issues relating to protests of DOD acquisitions. The report appears to be thorough and well balanced and contains a number of useful observations and data-driven recommendations.
At a high level, the report (which was limited to DOD procurements) corroborates what contractors and their protest lawyers have been saying about bid protests in general for a long time:
There is no epidemic of bid protests. Less than 0.3 percent of all DOD procurements are protested – regardless of whether one counts by contract or by total contract dollars. Not quite as low as the probability of getting hit by lightning, but lower than your chance of getting audited by the IRS. And that is a large part of what the American bid protest system is: an audit program largely funded by protesters to provide greater accountability for public spending.
Nor is there an epidemic of “frivolous” bid protests. Year after year, more than 40 percent of all bid protests filed at the GAO consistently result in the procuring agencies taking corrective action, either voluntarily or as a result of GAO sustaining the protest. And this high success rate is despite the overwhelming legal deference given to procuring agencies under GAO’s reasonableness standard and the Court’s “arbitrary and capricious” standard.
Although incumbents are more likely to protest, there is no clear evidence that incumbents are filing meritless protests to eke out more performance in the face of the inevitable end of a contract. In fact, RAND’s data suggest that DOD task order protests filed by incumbents have approximately a whopping 70 percent chance of resulting in corrective action.
Disappointed offerors sometimes file protests in frustration after being stonewalled by agencies with evasive or meaningless debriefings. More on that below.
Although the full statutory protest timeline at the GAO extends to 100 days, approximately half of all GAO protests of DOD procurements are resolved within a month, and approximately 70 percent are resolved within 60 days. Protests taken to a full written decision on the merits, of course, trend closer to the 100-day deadline.
Given the starkly differing views on bid protests between industry and agency personnel (another interesting topic covered by the RAND report), a few of the above findings may come as a surprise to some of our friends in the Government. Some of the findings, however, were somewhat surprising, including the fact that over half of all DOD protests at both the GAO and the Court of Federal Claims were filed by small businesses. We also were surprised that nearly 8 percent of GAO protests of DOD procurements involved acquisitions valued at less than $100,000, including some protests of micro-purchases of less than $3,500.
We weren’t surprised to read that sufficient data simply do not exist to allow analysis of some of the topics Congress requested be studied. It would have been fascinating, for example, to gain insight into agency-level protests across DOD agencies. As we’ve discussed before, agency-level protests are something of a black box, but in certain circumstances may be better for resolving procurement problems that traditional GAO protests. A dearth of information prevented RAND from exploring this poorly-understood method of protest.
The report also includes a number of sensible recommendations. Our favorite is the recommendation to enhance the quality of debriefings – a subject we’ve addressed before here and here – which is also something the Office for Federal Procurement Policy has been urging. Poor debriefings have been a problem for a long time and directly lead to an increase in protests, both because disappointed offerors are desperate to get some sliver of information and because opaque debriefings often make offerors think agencies are hiding something. Some agencies proactively have taken the initiative to improve their debriefings. The RAND report mentions the Air Force’s exemplary extended debriefing pilot program. And, outside the scope of the RAND report, we’re fans of NASA’s already robust approach to debriefings. The recently enacted NDAA for FY 2018 requires DOD to enact regulations to improve debriefings, so this RAND recommendation is one we hope to see put into action soon. One hopes civilian agencies will follow DOD’s lead.
The RAND report also makes other recommendations: urging caution about shortening the GAO’s 100-day timeline for resolving protests; urging caution about further reducing jurisdiction over task order protests; recommending an expedited or more limited process for challenging acquisitions valued at less than $100,000; ensuring that any protest reforms take into account the fact that small businesses file most protests; and various suggestions for improving data collection at the GAO, the Court of Federal Claims, and procuring agencies.
We could come up with a few more recommendations, such as streamlining the GAO protest process by requiring agencies to produce the entire procurement record (rather than having the parties waste time and money fighting over which bits and pieces are supposedly relevant to the protest grounds as pled), as is standard practice at the Court of Federal Claims. But that’s a topic for another day. Our only recommendation now is that you take a look at the RAND report for yourself.