• Users Online: 182
  • Home
  • Print this page
  • Email this page
Home About us Editorial board Ahead of print Current issue Search Archives Submit article Instructions Contacts Login 

 Table of Contents  
Year : 2021  |  Volume : 34  |  Issue : 2  |  Page : 65-67

What Can COVID Teach Us about Research Integrity?

Trustee, World Association of Medical Editors (WAME)

Date of Web Publication28-Sep-2021

Correspondence Address:
Margaret Winker
Trustee, World Association of Medical Editors (WAME)

Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/0970-258X.326760

How to cite this article:
Winker M. What Can COVID Teach Us about Research Integrity?. Natl Med J India 2021;34:65-7

How to cite this URL:
Winker M. What Can COVID Teach Us about Research Integrity?. Natl Med J India [serial online] 2021 [cited 2021 Oct 22];34:65-7. Available from: http://www.nmji.in/text.asp?2021/34/2/65/326760

The pre-Covid world faced many issues including extreme politics, nationalism and racism; huge wealth disparities were widespread. The Covid pandemic, with more than 160 million cases and 3.3 million deaths reported worldwide to date,[1] amplified these problems through disinformation and conspiracy theories and affecting impoverished and essential workers most of all. Research and publishing likewise had their weaknesses highlighted by the Covid pandemic.

Peer review, the lynchpin of scholarly publishing, depends on experts willing to devote volunteer time to carefully evaluate what is and is not reported, often while juggling their own research, a busy clinical schedule and other responsibilities. Editors must identify red flags and missing elements while similarly juggling multiple responsibilities and the many deadlines publishing a journal involves. With the onset of the Covid pandemic, however, time was a luxury no one had—not researchers, who could not instantly plan and conduct well-designed multicentre studies, enrol sufficient numbers of patients and check and recheck data; not editors and reviewers, who had less time to carefully review research and ask for additional information as necessary; not clinicians, who sought answers in the myriad but inadequate observational studies (freely available, at least)[2] to decide which best applied to their patients; and least of all, patients, who were getting ill and dying while their families and society in general were distraught and clamouring for information.

The Covid crisis had several consequences for research. First, researchers studied what information was readily available to them, inundating journals with submissions. Second, the volume of research submitted to journals overtaxed editors and reviewers,[3] and the latter used different review criteria to avoid slowing time to publication.[4] At least one major journal accepted some research without external peer review.[5] Some journals inadvertently published duplicate submissions, as they quickly went through peer review to publication simultaneously.[6] Third, the volume of research published overwhelmed clinicians and others trying to find the best available evidence,[7] which was not up to usual standards.[8]

What were the consequences of a rushed and overloaded system? To date, 109 articles have been retracted, plus an additional 12 retracted due to journal error, 4 retracted and reinstated and 6 Expressions of Concern issued.[9] Two of the most notable retractions were published in the Lancet[10] and N Engl J Med,[11] journals generally not lacking in resources. In brief, vascular surgeon Sapan Desai and co-authors published two observational studies ostensibly of the Surgisphere database, one concluding that hydroxychloroquine for Covid was associated with an increased risk of death[10] and another that patients with cardiovascular disease were more likely to die from Covid.[11] When asked for more information, as stated in the retraction notices, all authors other than company founder and CEO Sapan Desai were ‘unable to complete an independent audit of the data’[12] and ‘all the authors were not granted access to the raw data and the raw data could not be made available to a third-party auditor’.[13]

Could these issues have been identified before acceptance? A spokesperson for N Engl J Med said, ‘We have limited experience with reviewing or publishing studies like this one, which used a large database based on electronic medical records. The reviewers and editors asked the authors questions about the data sources and data validity. The editors accepted the authors' responses, rather than asking for help from reviewers with expertise in this type of data. In the future, our review process of big data research will include reviewers with such specific expertise.’[14] However, other researchers found that had existing reporting guidelines for research using medical databases been adhered to, the study limitations would have been apparent.[15] Given these issues, should peer review and reporting be more transparent? Suggested guidelines describing the specific peer review conducted are rarely used.[16] Publishing the peer reviews themselves could make their strengths and limitations more apparent.[17]

Furthermore, retraction is not the end of the story: research frequently continues to be cited after it has been retracted, without indicating that the studies were retracted.[18] According to the N Engl J Med website, the Surgisphere study has been cited 382 times to date.[11]

Preprints are one way to enable sufficient time for peer review while not delaying access, and they have served as both a lifeline and a source of misinformation during Covid.[19] As a lifeline, some badly-needed descriptive information about the clinical characteristics of Covid was quickly made available.[20] At the same time, some research, notably the use of hydroxychloroquine and azithromycin for Covid, was widely disseminated before the study was peer-reviewed and found to be of poor quality (fortunately, Covid preprints were rapidly published in peer-reviewed journals––at least according to a preprint study).[21] Predictably, some media sources were unwilling to heed the prominent warnings on preprints. Medarxiv states: ‘This article is a preprint and has not been certified by peer review (what does this mean?). It reports new medical research that has yet to be evaluated and so should not be used to guide clinical practice.’ However, careful, discrete, deliberate reporting is unlikely when the public hangs on every Covid word while the 24x7 news media compete for clicks.

Covid research has had many successes. Vaccines were produced and published in record time; treatments were researched, preprinted and published and understanding of the new illness evolved quickly.[22] But what are the lessons for medical publishing? For starters, requiring the use of existing reporting guidelines[23] could help reviewers and editors conduct better and more efficient reviews. Authors and journals should have tools to identify retracted articles to prevent continued inaccurate citations. Preprints help provide rapid access for researchers and clinicians, yet the media persist in reporting unreviewed results to the public: better solutions are needed. When extraordinary events occur and rapid results are essential, relying on a voluntary system of peer review may not be enough. Healthcare systems, governments and/or funders could underwrite emergency peer review teams to enable experts to thoroughly review important submitted research and/or review preprints in real time. What could be more important than getting the best possible carefully evaluated research out to clinicians, public health professionals and the public as quickly as humanly possible?

Perhaps research needs its own Operation Warp Speed to coordinate design of comprehensive, thorough, efficient observational and interventional research across medical systems through an infrastructure designed to support it; efficient journals and well-qualified peer reviewers to enable expeditious publications and timely follow-up research on studies that generate wasted effort and expense (e.g. emphasizing deep cleaning of surfaces while ignoring masks). Study facilitation and rapid reviewers alone are not enough; losing experienced researchers in a pandemic because they cannot get childcare is surely a social crisis in need of a solution.[24]

Covid has exposed existing weaknesses in both societies and journals. Journals need to learn from this moment and use the tools that are available such as requiring reporting guidelines. Transparent peer review can help expose its strengths and weaknesses (from unaddressed comments or from poor quality reviews). Preprints have their place, but only for researchers and clinicians with the capacity to evaluate them. Moreover, preparation for the next pandemic may need to include comprehensive research planning, from design to publication. Covid is having devastating effects throughout the world, but if we learn from this moment, its legacy could have some benefits as well.

Conflicts of interest. None declared

  References Top

COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU), Johns Hopkins School of Medicine Covid Resource Center. Available at https://coronavirus.jhu.edu/ map.html (accessed on 13 May 2021).  Back to cited text no. 1
Finley K. Global officials call for free access to Covid-19 research. Wired; 13 Mar 2020. Available at www.wired.com/ story/global-officials-call-free-access-covid-19-research/ (accessed on 13 May 2021).  Back to cited text no. 2
Else H. How a torrent of COVID science changed research publishing—In seven charts. Nature 2020;588:553.  Back to cited text no. 3
Horbach SP. No time for that now! Qualitative changes in manuscript peer review during the Covid-19 pandemic. Res Eval 2021;rvaa037. doi: 10.1093/reseval/rvaa037.  Back to cited text no. 4
Bauchner H, Fontanarosa PB, Golub RM. Editorial evaluation and peer review during a pandemic: How journals maintain standards. JAMA 2020;324:453–4.  Back to cited text no. 5
Marcus A. Researchers publish the same COVID-19 paper three times. Retraction Watch; 29 Oct 2020. Available at https://retractionwatch.com/2020/10/29/researchers-publish-the-same-covid-19-paper-three-times (accessed on 13 May 2021).  Back to cited text no. 6
Brainard J. Scientists are drowning in COVID-19 papers. Can new tools keep them afloat? Science 2020. Available at www.sciencemag.org/news/2020/05/scientists-are-drowning-covid-19-papers-can-new-tools-keep-them-afloat (accessed on 13 May 2021). doi: 10.1126/science.abc7839.  Back to cited text no. 7
Whitmore KA, Laupland KB, Vincent CM, Edwards FA, Reade MC. Changes in medical scientific publication associated with the COVID-19 pandemic. Med J Aust 2020;213:496–9.e1.  Back to cited text no. 8
Retracted coronavirus (COVID-19) papers. Retraction Watch. Available at https://retractionwatch.com/ retracted-coronavirus-covid-19-papers/ (accessed on 13 May 2021).  Back to cited text no. 9
Mehra MR, Desai SS, Ruschitzka F, Patel AN. RETRACTED: Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: A multinational registry analysis. Lancet 2020. Available at www.sciencedirect.com/science/article/pii/S0140673620311806?via%3Dihub (accessed on 13 May 2021).  Back to cited text no. 10
Mehra MR, Desai SS, Kuy S, Henry TD, Patel AN. Cardiovascular disease, drug therapy, and mortality in Covid-19. N Engl J Med 2020;382:e102.  Back to cited text no. 11
Mehra MR, Ruschitzka F, Patel AN. Retraction—hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: A multinational registry analysis. Lancet 2020;395:1820.  Back to cited text no. 12
Mehra MR, Desai SS, Kuy S, Henry TD, Patel AN. Retraction: Cardiovascular disease, drug therapy, and mortality in Covid-19. N Engl J Med 2020;382:2582.  Back to cited text no. 13
Davey M. Covid-19 studies based on flawed Surgisphere data force medical journals to review processes. The Guardian; 12 Jun 2020. Available at www.theguardian.com/world/2020/jun/12/covid-19-studies-based-on-flawed-surgisphere-data-force-medical-journals-to-review-processes (accessed on 13 May 2021).  Back to cited text no. 14
Benchimol EI, Moher D, Ehrenstein V, Langan SM. Retraction of COVID-19 pharmacoepidemiology research could have been avoided by effective use of reporting guidelines. Clin Epidemiol 2020;12:1403–20.  Back to cited text no. 15
Black C, Kowalczuk M, Alam S, Moylan E, Jones L, Mehmani B, et al. Standard taxonomy for peer review (2.0). International Association of Scientific, Technical and Medical Publishers; 29 Sep 2020. Available at https://osf.io/aynr5/ (accessed on 13 May 2021).  Back to cited text no. 16
Meadows A. What does transparent peer review mean and why is it important? Scholarly Kitchen. 3 Aug 2017. Available at https://scholarlykitchen.sspnet.org/2017/08/03/transparent-peer-review-mean-important/ (accessed on 13 May 2021).  Back to cited text no. 17
Piller C. Many scientists citing two scandalous COVID-19 papers ignore their retractions. Science 2021. Available at www.sciencemag.org/news/2021/01/many-scientists-citing-two-scandalous-covid-19-papers-ignore-their-retractions (accessed on 13 May 2021).  Back to cited text no. 18
Guterman EL, Braunstein LZ. Preprints during the COVID-19 pandemic: Public health emergencies and medical literature. J Hosp Med 2020;10:634–6.  Back to cited text no. 19
Tufekci Z. 3 ways the pandemic has made the world better. The Atlantic; 18 Mar 2021. Available at www.theatlantic. com/health/archive/2021/03/three-ways-pandemic-has-bettered-world/618320/ (accessed on 13 May 2021).  Back to cited text no. 20
Kodvanj I, Homolak J, Virag D, Trkulja V. Publishing of COVID-19 preprints in peer-reviewed journals, preprinting trends, public discussion and quality issues. bioRxiv 2020. doi: 10.1101/2020.11.23.394577.  Back to cited text no. 21
Conroy G. How COVID-19 could make science kinder. Nature Index; 24 Feb 2021. Available at www.natureindex. com/news-blog/coronavirus-could-make-science-research-peer-review-more-constructive (accessed on 13 May 2021).  Back to cited text no. 22
EQUATOR Network. Available at www.equator-network.org (accessed on 13 May 2021).  Back to cited text no. 23
COVID is amplifying the inadequacy of research-evaluation processes. Nature 2021;591:7.  Back to cited text no. 24


Similar in PUBMED
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

  In this article

 Article Access Statistics
    PDF Downloaded52    
    Comments [Add]    

Recommend this journal