Report on Research Compliance 22, no. 1 (January, 2025)
In 2023, after two years of study, an NIH task force proposed a series of recommendations to improve stewardship of research it funds, including that the agency adopt “stopping rules” that would allow poorly designed or low enrollment trials to be paused or terminated. At the time, task force co-chair Debara Tucci decried “so-called crappy small trials [that] came up time and time again” during COVID-19, saying there was a desire to “change that trajectory.”[1]
NIH also needs to ensure that trial design is appropriate, the task force said, and institutions have a role to play. Tucci, director of the National Institute on Deafness and Other Communication Disorders, said NIH must grapple with “how the studies that are submitted are reviewed.” The challenge is to “make sure that the studies as they’re conceived of are not underpowered, that they’re conceived in a way that they’re likely to be able to address the question that they’re supposed to address,” she said. The task force also suggested that NIH limit the number of no-cost extensions granted to researchers.
Larry Tabak, NIH principal deputy director, seemed to embrace the recommendations. NIH officials need “the discipline to stop supporting underpowered boutique studies, which, quite frankly, many investigators around the country have built and sustained their careers on,” he said during the June 2023 meeting of the NIH Advisory Committee to the Director (ACD). Although ACD members also signaled their support, no vote was taken on the recommendations, and it’s not clear what became of them; the task force appears to be inactive.
But the issue of nonperforming, under-performing and low-quality trials hasn’t gone away. In fact, as of mid-December, journal publishers had pulled close to 500 COVID-19-related papers, according to Retraction Watch, with another 18 earning an “expression of concern.” In June, the Food and Drug Administration added its voice to the chorus seeking to beef up trials, issuing draft guidance calling for sponsors to report enrollment goals and progress in meeting them.[2]
Now, the HHS Secretary’s Advisory Committee on Human Research Protections (SACHRP) has waded into the fray, issuing its own set of recommendations on what it calls “foreseeably uninformative” research, crafted to raise awareness and provide assistance to institutional review boards (IRBs) so that they can “resist” approving such studies, in the words of former SACHRP chair Stephen Rosenfeld.
“Poor design, lack of methodologic/statistical expertise, the under-appreciated impact of resource limitations, and simple wishful thinking may lead to uninformative research projects that are foreseeably unlikely to meet their goals, waste time and resources, and devalue the contributions of research subjects,” according to the recommendations SACHRP approved at its October meeting.[3] The recommendations focus on “the role of the IRB system, and more generally the role of Human Research Participant Protection Programs, in assessing whether research is designed to be ‘informative.’” See the framework for the recommendations.[4]
Rosenfeld, who chaired SACHRP from December 2016 to January 2021, led the drafting of the recommendations as a member of a SACHRP subcommittee. In an interview with RRC, Rosenfeld said he believes “there should be a response to the number of retractions, to the findings [about uninformative studies]. It’s my feeling that we owe the public an accounting for that and a response to it, if we expect them to trust us.”
The retractions underscore that during the pandemic, “unprecedented numbers of people were exposed” to interventions in studies that did not reach their endpoints, he pointed out.
RRC asked Rosenfeld if uninformative research could also be termed “junk” studies or science. “We talked about this” as the recommendations were being drafted, he said. “There are a lot of reasons [studies] end up being uninformative. Sometimes it’s wishful thinking. It’s not wrong to call it junk science, but I think the connotations are a bit strong.”
No Automatic Approvals
Regarding the specific recommendations, “If the IRB can anticipate [that] what people are being asked to do really has no future benefit for others or themselves and is foreseeably uninformative, that’s a betrayal of trust” for members to approve such a study, Rosenfeld said, acknowledging that this is “a strong way of putting it.”
IRBs are “constantly under pressure to move quicker to approve, to not ask for too many changes,” he added. “I think even an explicit recognition from SACHRP that this goes on might give some strength to IRBs to choose to resist this kind of thing. They have to be able to resist.“
The recommendations also address the types of minimal-risk research that IRBs consider, such as a retrospective medical records review that may be proposed by residents whose medical fellowship programs require them to do research.
When it comes to approving such research, in particular, “there’s always tension because there are other agendas,” said Rosenfeld. “The IRB sits at the nexus of all sorts of conflicting interests, and the temptation is often when it doesn’t involve risks of physical harm to participants to say, ‘Well, that’s okay.’ I do think IRBs deal with that all the time. I feel strongly, and I hope it comes across in the recommendations, that that’s not appropriate, particularly at a time when we struggle with trust in science and in its institutions.”
The new recommendations reflect “the view that it is important that, even if something doesn’t present a risk of physical harm or present risks that are greater than people would face in everyday life, you’re asking them to participate in a research study. Whether that involves risk or not, it involves burden, it involves sharing of personal information,” Rosenfeld said. “And that’s done with the understanding that it’s a valuable use of their time.”
Typically, IRBs conduct reviews “study-by-study” and “rarely take into account the context of the research in terms of what other studies are going on,” which can lead to duplicative and ultimately uninformative research. “IRBs rely almost entirely on what the investigator has written in the protocol as far as background justification,” he said.
IRB Role to Ask: Is It Worth Doing?
For IRB members, “there’s no acceptable, ethical justification for putting people at risk or for asking them to contribute their time to something, to a question that has already been definitively answered,” Rosenfeld said. He added that “there is a justification for reproducibility studies” and it “is important to do further studies, but you just have to acknowledge that’s what you’re doing. And the IRB should know about it. The IRB’s role is explicitly to ask whether it’s worth doing enough to ask people to put themselves at risk or to give their time.”
He added that studies that are small should be designed to be combined with others.
One issue that was “of great interest” to SACHRP was what should be done when “a study is approved, and it’s well-designed and it looks great, and then you do continuing reviews, and you find that it’s not going to meet its enrollment numbers,” Rosenfeld said.
The recommendations state that “continuing review should confirm the ability of a site or research program to enroll enough subjects to meet the goals of its design,” and that IRB members should “ask for an updated plan that provides justification for continuing the study or should explore orderly termination of the research or the site in a way that protects subjects already enrolled, maximizes the value of their participation, and prevents others from being exposed to risks for no scientific purpose.”
If a decision is made to terminate a trial for lack of accrual or other reasons, IRBs and investigators can turn to recommendations SACHRP approved in March. These address when it is in participants’ best interest to allow some or all of the study procedures, care or treatments to be provided outside of a formal trial.[5]
Rosenfeld said he hoped the recommendations give IRB members “something to think about and something to appeal to within the institutional context” when faced with research that they believe could be uninformative.
1 Theresa Defino, “‘No More Boutique Studies’: NIH Mulls ‘Stopping Rules,’ Other Changes to Improve Clinical Trials,” Report on Research Compliance 20, no. 7 (July 2023), https://bit.ly/4greDas.
2 Theresa Defino, “FDA Diversity Plan Draft Guidance Calls For Periodic Reports, Addresses Waivers,” Report on Research Compliance 21, no. 9 (September 2024), https://bit.ly/4flJuUP.
3 U.S. Department of Health and Human Services, Office for Human Research Protections, “SACHRP Recommendations on Assessment Considerations on Uninformative Research,” October 22, 2024, content last reviewed October 28, 2024, https://bit.ly/4gCDI1R.
4 Theresa Defino, “SACHRP Recommendations on ‘Uninformative Research,’” Report on Research Compliance 22, no. 1 (January 2025).
5 Theresa Defino, “SACHRP: With Careful Review, Research Procedures May Continue After Trial Halt,” Report on Research Compliance 21, no. 5 (May 2024), https://bit.ly/3BqJyEL.
[View source.]
