This story is part of The Cancer Letter’s ongoing coverage of COVID-19’s impact on oncology. A full list of our coverage, as well as the latest meeting cancellations, is available here.
Early in the War Against Cancer, when huge amounts of federal funds were suddenly funneled into cancer research, many scientists and clinicians working in other fields suddenly found it convenient—if not essential—to incorporate cancer into the title of their grant applications.
The hope was the application would be directed to a National Cancer Institute review panel, instead of a less-funded institute. The situation became so acute, some cynics commented more people were living from than dying from cancer. Fortunately, some of these re-directed research efforts were productive in unpredictable ways and helped us make substantial progress against cancer, even if total victory is not yet ours.
Sadly, some diseases can be cured by medical interventions, others not. When not, are there other approaches to control or cure? One possibility is to try to publish a disease to death, a therapy strategy first proposed by my late colleague Prof. David Golde from UCLA. Here I consider whether this strategy is working in the fight against severe acute respiratory syndrome-coronavirus-2 (SARS-CoV-2) pandemic and the associated coronavirus infectious disease-2019 (COVID-19).
To test this hypothesis I queried PubMed on May 16, 2020 for citations using the search terms SARS-CoV-2 and/or COVID-19. There were 12,959 hits since January 2020—or roughly 162 citations per day. I confirmed this by comparing this number with a similar PubMed search I did on May 14, 2020. The difference of 484 citations is consistent with a recent publication rate of 220 per day.
This is one publication for every four daily deaths from COVID-19 in the U.S. on June 8, 2020. And this is only for citations covered by PubMed. I also found 3915 publications on medRxiv and 925 on bioRxiv [1]. The figures from the World Health Organization, which tracks every manuscript on the virus and its disease submitted in their journals irrespective of publication, would be much greater.
How to explain this burst of publications? Can many high-quality studies be done so quickly? Unlikely. In fact, of 1,556 studies of COVID-19 listed in ClinicalTrials.gov, only 249 (16%) were phase III trials, and fewer than 100 included more than 100 subjects [2]. Given the baseline estimate, 85% of clinical research is not useful or wrong—we may be pushing this estimate to 90 or 95% this year [3,4].
One explanation of this publications deluge is the opportunity the pandemic offers authors and journals. Some journals (but not BJH) have lowered their criteria for acceptance. Publications of series of two or five subjects are appearing in high impact factor journals which would otherwise have appeared, if at all, in the Lesotho Journal of Plant Biology.
Although this change may be motivated by the goal to rapidly disseminate information about SARS-CoV-2 and COVID-19, it is also possible some journals and authors are jumping on the SARS-CoV-2 and COVID-19 bandwagon. (I am also guilty publishing several typescripts on this subject; εκπειραζοντες αυτον οι ιερεις ινα εχωσιν κατηγοριαν αυτου—the saints have been accused of this accusation) My mentor, Prof. Martin Cline, cautioned: No data is better than bad data [5].
Other forces may be operating. Submissions to scientific and medical journals increase dramatically over the Christmas and New Year holidays and on weekends [6]. Most scientists’ laboratories are closed and clinicians not directly involved in treating persons with COVID-19 have reduced clinical responsibilities and work from home via telemedicine. Their options: help with online schooling, cook dinner, vacuum (Dyson V7 highly recommended) or hide in your (newly designated) home office and complete a long-delayed typescript. The choice between publish or perish has never been starker.
Another issue I considered was that many, if not most, of this surge of publications are from Chinese authors. Figure 1 shows data on numbers of publication by geographic region and country [7]. Although China surpassed the U.S. in 2015, they trailed the EU in 2016. This is likely to change in 2020.
Another growth industry is the publication of management guidelines for SARS-CoV-2 and/or COVID-19. A PubMed search on May 16, 2020 found 599 SARS-CoV-2 and/or COVID-19 guidelines. For the BJH, I list only guidelines directed towards persons with hematological disorders, including hematopoietic cell transplant recipients and recipients of cell therapies such as chimeric antigen receptor (CAR) T-cells including one from the U.K. National Institute for Health and Care Excellence (NICE) [8]; one from an international expert panel [9]; one from the European Bone Marrow Transplant Group (EBMT) [10]; and one from the American Society for Transplantation and Cellular Therapy (ASTCT) [11].
There were 52 authors of the international expert panel guidelines—making me suspicious so many physicians could agree on anything, save authorship on a publication. I contacted six co-authors I know, none of whom had cared for someone with COVID-19. Furthermore, none of these four guidelines is listed in the National Guidelines Clearinghouse [12].
My next step was to evaluate the quality of these guidelines using criteria of the Infectious Diseases Society of America (Figure 2; [13]). Readers will not be surprised the four guidelines received a C for Strength of Recommendation (Poor evidence) and a III for Quality of Evidence (Evidence from opinions of respected authorities… without clinical trials data).
Nevertheless, recommendations in these guidelines, although not evidence-based, seem sensible and may be useful. The risk is that they will be awarded the imprimatur of delivering quality health care absent anything better. Wiser people than me have commented on the value of consensus in decision making. For example, Abba Eban, an ex-Israel Foreign Minister noted: Consensus means that lots of people say collectively what nobody believes individually.
Michael Crichton, physician and author commented: Historically, the claim of consensus has been the first refuge of scoundrels; it is a way to avoid debate by claiming that the matter is already settled. Whenever you hear the consensus of scientists agrees on something or other, reach for your wallet, because you’re being had.
Limitations of consensus guidelines and their detrimental effect on critical thinking are discussed by others, by Profs. Gianni Barosi and me, and by Shaun McCann (who also evaluates guidelines on wine making) [14-19].
What proof have I that my criticism of these guidelines is valid? Might I be biased? As a test I performed a series of controlled experiments in mice. Animals were divided into 2 cohorts, one was fed shredded versions of the 4 guidelines, and the other cohort, shredded blank paper (placebo). After a week I combined mice in each cohort and placed them in a large cage into which I put a block of cheese labeled Conquer COVID-19 with a marker pen (Sounds like The Patchwork Mouse? [20]).
My initial experiments failed for 2 reasons: (1) the first cheese I tried was Époisses de Bourgogne—because it has a soft rind, the Conquer COVID-19 quickly became invisible; and (2) even without the writing, the mice were quickly asphyxiated (think Stinking Bishop, Pont L’Evequ or Petit Muenster). To appreciate this danger, be advised it is illegal in France to open an Époisses de Bourgogne on a public transport.
My second attempt using Shropshire Blue was more successful, even though it was challenging to read Conquer COVID-19 between the veins. The bottom line, however, is there was no statistically significant difference in the time it took the mice in either cohort to consume the Shropshire Blue, whether they were fed shredded guidelines or placebo.
Other innovative strategies to conquer SARS-CoV-2 and COVID-19 have also been tried, such as having interminable meetings, perhaps a way of talking a disease to death. A bonus of these frequently remote audio-only meetings during the SARS-CoV-2 pandemic, is one can, in your sleeping costume, finish breakfast, check emails, and complete typescripts on SARS-CoV-2 and COVID-19 to submit (see above). Unfortunately, it seems the talking cure will not cure COVID-19 [21]. (Apologies to Josef Brueur and Anna O.)
Coming back to the strategy of publishing a disease to death, we have been there before with hairy cell leukemia. In the 1970s, many people with this disease were referred to Profs. Golde and me at UCLA, or Prof. Harvey Golumb at the University of Chicago. We were in a vigorous academic competition but the only intervention we had was splenectomy, effective in some people but not a cure in most.
What to do? Golde suggested: If we can’t cure hairy cell leukemia, perhaps we can publish it to death. Can it work? Who knows? Fortunately, for hairy cell leukaemia we now have cladribine, pentostatin, rituximab and interferon amongst others. For SARS-CoV-2 and COVID-19 we await a safe and effective vaccine, and safe and effective therapies.
Lastly, we may never know if the strategy of publishing COVID-19 to death worked, because the SARS-CoV-2 pandemic may subside during or soon after this deluge of publications. Was this merely an association or cause-and-effect? Many associations, regardless of how strong, are not cause-and-effect. Take, for example, the correlation between per capita cheese consumption (not only Époisses de Bourgogne) and likelihood of dying by becoming tangled in one’s bedsheets [22].
Causal inference is tricky. Perhaps, when the dust settles, we will have time for a rigorous evaluation of what is effective (and, perhaps more importantly, what is not) and we can be better prepared for the next coronavirus pandemic.
Acknowledgements
RPG acknowledges support from the National Institute of Health Research (NIHR) Biomedical Research Centre funding scheme.
This article was modified from a typescript published June 1 in the British Journal of Haematology.
Conflict of Interest: I have no fiscal interest in makers of Époisses de Bourgogne, Stinking Bishop, Pont L’Evequn nor Petit Muenster. However, during the lockdown, shipments of these would be greatly appreciated and can be sent to 11693 San Vincente Blvd, Los Angeles, CA USA 90049-1533.
References
https://clinicaltrials.gov/ct2/results?cond=covid&term=&cntry=&state=&city=&dist
Glasziou PP, Sanders S, Hoffman T. Waste in Covid-19 research. BMJ.2020;369:m1847.
Ioannis J. Why most clinical research is not useful. PLoS Med. 2016;13:e1002049.
Barnett A, Newburn I, Schroter S. Working 9 to 5, not the way to make an academic living: observational analysis of manuscript and peer review submission over time. BMJ 2019;367:l6460.
https://www.nsf.gov/statistics/2018/nsb20181/assets/1387/overview.pdf
Zeidan AM, Boddu PC, Patnaik MM. Special considerations in the management of adult patients with acute leukaemias and myeloid neoplasms in the COVID-19 Era: Recommendations by an International Expert Panel. Lancet Haematol.2020;in press.
Khan AR, Khan S, Zimmerman V, Baddour LM, Tleyjeh IM. Quality and strength of evidence of the Infectious Diseases Society of America clinical practice guidelines. Clin Infect Dis. 2010;51:1147-56.
Shekelle PG, Ortiz E, Rhodes S, Morton SC, Eccles MP, Grimshaw JM, Woolf SH. Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? JAMA. 200;286:1461-7.
Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS.Evidence based medicine: what it is and what it isn’t. 1996. Clin Orthop Relat Res. 2007;455:3-5.
Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ; GRADE Working Group. What is “quality of evidence” and why is it important to clinicians? BMJ. 2008;336:995-8.
Metlay JP, Armstrong KA. Annals Clinical Decision Making: Weighing Evidence to Inform Clinical Decisions. Ann Intern Med. 2020;172:599-603.
Barosi G, Gale RP. Is there expert consensus on expert consensus?Bone Marrow Transplant. 2018;53:1055-60.
McCann SR. COVID-19, HCT and wine. Bone Marrow Transplant. 2020; doi: 10.1038/s41409-020-0932-3. [Epub ahead of print].
Hixson J. The Patchwork Mouse. 1976; Anchor Press, New York, NY, USA.
http://commentary-gonzalo86.blogspot.com/2020/04/interminable-meetings-found-ineffective.html))
Zheng C, Dai R, Gale RP, Zhang MJ. Causal interference in randomized clinical trials. Bone Marrow Transplant. 2020;55:4-8.
