Report on Research Compliance Volume 20, no. 10 (October, 2023)
Please don’t send uncorrected transcripts of interviews conducted via Zoom or other means to the National Science Foundation (NSF) Office of Inspector General (OIG). Engaging in a “document dump” to OIG also doesn’t serve institutions or federal research misconduct investigators.
As research integrity officers (RIOs) know, making a finding of misconduct—defined as fabrication, falsification or plagiarism—is no small feat. First, there’s the inquiry phase, then, if warranted, a full investigation. Committee members must be impaneled, evidence gathered and protected, witnesses interviewed. The process should be handled with the utmost professionalism and well, integrity to ensure fairness and lack of bias.
And then there’s the report—a written accounting confirming or knocking down the allegations that are sent to the federal oversight agency. For HHS-funded investigators, that is the Office of Research Integrity. When the support comes from NSF, it’s OIG, specifically the Division of Research Integrity and Administrative Investigations in the Office of Investigations.
The tips on transcripts and the warning against a ton of unorganized information come from Valerie Hillgren, an investigative scientist in the division. In July, Hillgren—who joined OIG in 2016 after decades as a research scientist—gave a presentation on writing the report to a select audience of RIOs and others.[1] The meeting was not open to the news media, nor was it recorded. However, OIG posted slides from the workshop, and Hillgren agreed to an interview with RRC.
The first meeting of its kind, OIG planned the workshop to be an in-person event held in the summer of 2020—and then COVID-19 intervened. Three years later, OIG had the workshop but decided to keep it online to make attendance easier, Hillgren told RRC.
The purpose of the meeting was to “familiarize research integrity officers with our office and our procedures. We initially sent the invitation to the RIOs of all the institutions that had received over $1 million in funding from NSF in fiscal year 2022, as well as any RIOs from universities where we may have had cases. And then, we encouraged the RIOs to forward [the meeting information] to other interested parties. The Association of Research Integrity Officers helped us out by including our invitation in their newsletter,” Hillgren explained.
To choose workshop topics, “We brainstormed together and talked about, ‘What are some general things that we would like to get out there, what sort of information would we like to share with the RIOs?’ For us, an important thing is the report that we get back after an investigation because that’s the basis that we use to start our own independent assessment of the facts—the report that they write and the evidence that they give us,” she added.
Although her talk was titled “The Institution’s Investigation Report: Best and Worst Practices,” Hillgren said she spent more time on best practices—and report requirements. “We certainly did not want to appear as if we were trying to call out institutions for subpar reports,” she said. “We view our relationship with the RIOs as a collaborative enterprise. What we were hoping here is that by sharing with them what we need in a report, it would make both of our jobs easier.”
In her remarks and discussion with RRC, Hillgren focused on the final report that an institution develops at the end of an investigation. However, she also addressed the notification related to an inquiry that, in some instances, must be sent to OIG.
Requirements Differ for OIG-Referred Cases
OIG can request that an institution conduct a misconduct inquiry or move straight to an investigation in cases when OIG has already done some digging.
If the institution initiates a research misconduct inquiry on its own, it only needs to notify OIG “if there’s going to be an investigation” that follows, Hillgren said. In this case, RIOs “use the link…for our online complaint form to provide us notice that they have completed an inquiry,” Hillgren said. The link is https://oig.nsf.gov/hotline.
“Alternatively, RIOs may notify the Director Aaron Manka (amanka@nsf.gov). I would like to emphasize, though, that this applies to inquiries that the university instituted on their own, and not to an inquiry that we referred. If we referred the inquiry, the RIO should direct the inquiry report to the case lead,” she said. “What we need to know at that point is the subject’s name, what are the allegations, what’s the connection to NSF, and a brief discussion of the evidence and the conclusions warranting an investigation.”
If the institution drops a case at the point of inquiry that it initiated, OIG doesn’t need to be notified, she said. However, “If we referred the inquiry, the RIO should direct the inquiry report to the case lead.”
When OIG initiates a case, it sends the institution a referral letter, asking the institution to determine if the subject committed fabrication, falsification or plagiarism.
If the answer is no, a report is still required “if we referred the investigation to them. They have to explain why they didn’t make a finding,” Hillgren said. “We do an independent reassessment. We have had cases where they don’t make a finding, but we do.”
If misconduct was found, the institution must state whether the behavior was a significant departure from accepted practices.
Describe Investigative Steps
The report also needs to make clear how/in what context the research misconduct occurred, as well as whether the actions were “recklessly, knowingly or intentionally” committed. The committee may find the actions were all three, just one, or some combination—Hillgren noted that the words are separated by an “or.”
She once received a report that simply stated that there was fabrication, falsification or plagiarism; it was a significant departure and committed intentionally. “That was the entire report,” Hillgren said.
OIG also wants to know if the individual received training in the responsible and ethical conduct of research, if the actions were part of a pattern and whether they “have a significant impact.”
The report must include specifics about the allegation or allegations, such as the number of the figure or image manipulated or a description of text plagiarized and where these appeared—including the journal citation if published.
Further, it should describe the investigative steps, including whether evidence was sequestered, the evidence that was reviewed, identities of interviewed individuals and the names of committee members. If applicable, the report should also discuss the “backstory of the allegations,” a timeline and “aggravating or mitigating factors,” according to Hillgren’s slides.
The report needs to specify how the committee reached its conclusions. The issue of the investigator’s “motive” may enter into this analysis, but the committee is not required to specifically address this, she said, nor how the misconduct was uncovered.
Using Zoom? Send Recordings
“Those two things are very interesting and I like to know [them], too, but they’re not actually part of a research misconduct finding. It’s not necessary for you to know why they did it. It can be helpful” to assess the level of intent, Hillgren said.
“Documents that support the committee’s conclusions also need to be part of the report. These include data files or logs, paper or proposal drafts, emails and interview transcripts.” Her slides also note that “to the extent possible, we prefer searchable PDFs” and also that “official transcripts are the best.”
Many organizations conduct interviews via Zoom and simply send in verbatim transcripts riddled with errors and inaccuracies.
“When I look at Zoom transcripts, sometimes [I think] ‘I don't know where it came up with that,’” Hillgren told RRC. She added that receiving uncorrected Zoom transcripts “is fairly common” but unacceptable. Take the time to correct them, she said—and send the recordings along to OIG. If an accurate transcript or a recording is not included with the report, “we need some good detailed notes of what happened in the interview.”
The two “biggest mistakes” related to reports that Hillgren sees are failing to provide “a detailed analysis that refers to supporting documents” and a lack of organization for supporting documents.
“Sometimes we get reports that just have the committee’s conclusions without providing any analysis of how the committee reached that conclusion,” Hillgren said. “And there’s also reports that do a great job of including that analysis, but they don't provide any references, or citations to the evidence that supports the statements that are being made.”
OIG May Request Additions to Reports
On the other hand, as noted earlier, institutions might give OIG too much information. “We have gotten the entire hard drive [that] includes personal material that we don’t have any interest in seeing. We certainly don’t want to have to try and sort through to try and find what we need,” Hillgren said. “Sometimes institutions provide us absolutely everything they sequester…sort of a document dump. We really only need the documents that are relevant to their findings.”
Documents also may lack what Hillgren called “logical names,” which “makes it difficult for us to go through the documents and assess them.”
Her slides provided an example of appropriate file names, including final investigation report and numbered attachments—complaint, complainant interview transcript, paper draft, published paper, complainant interview transcript, subject interview transcript and lab policy, among them.
Hillgren did not address the average length of a report or a range of pages. “The contents of the report are more of a concern to us than the length of the report,” she told RRC. “We don’t keep track of the length of reports, and we really don’t want to give RIOs the false impression that there is a length range they should be shooting for when preparing a report.”
OIG may kick back a report but may not ask for a complete do-over.
“We do get reports that are, from our perspective, lacking in various ways, but if we look at it and we see that maybe we can still work with what is there, we will. [But] it just makes our work more difficult and time-consuming,” she said.
Hillgren added that “sometimes they’ve done a good investigation,” but it’s not reflected in the report.
“What usually happens is, when [we] send a report back…we're typically asking them to elaborate on a certain aspect or maybe provide some more evidence that they didn’t supply with the report. It’s pretty uncommon for us to ask an institution to completely rewrite a report,” she said.
Preliminary Meeting With OIG Lays Foundation
OIG doesn’t want what it expects in a report to come as a surprise—in fact, officials take steps to make sure that’s not the case.
“When we refer the investigation to the institution, we usually offer to meet with the RIO and the committee to talk about the investigation process and what we need in the report. We tell them…we need documentation of everything because we’re working off of the report. We’re document-based. We’re looking at the evidence, their report. I would say that currently almost all committees take us up on the offer,'” Hillgren said, adding, “We generally conduct the meetings virtually.”
In addition to sharing information with the research compliance community, OIG gathered some of its own from workshop attendees with regard to report writing. Two-thirds of participants who responded to a poll question indicated they use a template for committee reports.
OIG does not offer its own template. “When we refer either an investigation or an inquiry, we expect the institution to carry out whichever process that we request consistent with whatever its policies are,” Hillgren said.
Answers to the other poll question indicated that RIOs’ level of involvement in report writing varies. Forty percent said they substantially review the report, providing feedback on its content; 32% “tell the committee what to write or write to report themselves;” 18% provide “general guidance on structure and content;” 7% have no involvement; and 3% conduct a “light review,” focusing on “things like typos and adherence to the template,” Hillgren said.
More OIG Resources, Workshop Slides Available
Another OIG resource could help with report writing. In February, Megan E. Wallace, assistant inspector general for investigations, penned a “Dear Colleague” letter that provides an overview of the agency’s misconduct investigative process.[2] Wallace noted that final reports should “include a description and explanation of any actions recommended and/or imposed by the institution.”
Wallace also pointed out that OIG submits the institution’s report along with its own recommendations to NSF for final adjudication. Enforcement actions NSF can take range from debarment, a return of funds and award suspension to a requirement for training and a letter of reprimand. OIG reports actions NSF takes on misconduct findings in its semiannual reports to Congress.
In addition to her talk on report writing, Hillgren and another investigative scientist discussed other types of OIG investigations during the workshop, including those involving violations of award terms and conditions and human subjects or animal welfare regulations. Slides were also posted from five other presentations; two were focused on the basics of conducting investigations into allegations referred by OIG and plagiarism data and ways to prevent the conduct.[3]
OIG considered the workshop—which had more than 200 participants from 161 institutions—”a great success, and we would like to repeat it in the future,” Hillgren told RRC. It is not expected to be an annual event, however, and OIG has not established a date for a second meeting.
1 Valerie J. Hillgren, The Institution’s Investigation Report: Best and Worst Practices, National Science Foundation Office of Inspector General, 2023 Research Integrity Workshop, https://bit.ly/3EEd3ky.
2 Megan E. Wallace, “Dear Colleague Letter,” National Science Foundation Office of Inspector General, February 6, 2023, https://bit.ly/3Rluuhn.
3 National Science Foundation Office of Inspector General, 2023 Research Integrity Workshop, https://oig.nsf.gov/resources-outreach/outreach.
[View source.]
