The first 10 000 COVID-19 papers in perspective: are we publishing what we should be publishing?

The members of the University Vita-Salute San Raffaele COVID-19 literature monitoring working group are available in Appendix 1.

The COVID-19 pandemic has led to an unprecedented focus of the world’s scientific community on one topic. To quantify, we have calculated that 4% of all scientific outputs during the last 5 months have been about COVID-19; this has increased from 0.3% in February, to 1.2% in March, 4.5% in April, 6.5% in May, 8.3% in June and 6.6% in July. We systematically retrieved and critically assessed the first 10 000 PubMed indexed papers on COVID-19. They were published between 20 January and 7 May 2020, with an average of nearly 100 new papers added every day, published in 1881 different scientific journals. Fewer than 8% of journals have published half of the total production, and 7 journals alone have indexed more than 100 papers each. By contrast, 43.3% of journals only published one paper on COVID-19. Unsurprisingly, the largest amount of papers, one-fourth of the 10,000, were published in the USA, the country with the largest COVID-19 burden and ranking first in the 2019 Nature Index for quality research,¹ followed by China (22.2%), Italy (9%), the UK (7.6%) and France (3.2%).

This surge of publications that has emerged during the current pandemic suggests that it is important to take a step back and ask two key questions. First, are we publishing what we should be publishing? Second, are we publishing the way we should be publishing?

Are we publishing what we should be publishing? While the nature of the questions of interest to the COVID-19 pandemic have been rapidly evolving, perhaps it is helpful to consider some early thoughts about the core questions to be addressed. On January 22, two days after the first-ever paper on COVID-19 (called 2019-nCoV at the time) was indexed in Pubmed, Nature listed the six key questions scientists should be asking.² Among these were: how does the virus spread? Can infected people spread the virus without showing symptoms? How deadly is the virus? Where did the virus come from? What can we learn from the virus’s genetic sequence? Can a drug be developed to treat the coronavirus? Although acknowledging that some research questions are quicker to answer than others, and that some research answers are moving targets, almost 5 months after none of them is fully answered. Later in February 2020, a more detailed list of 14 epidemiological research priorities, the severity of the disease, immunity, and impact of control and mitigation measures, among others, was identified as essential to inform effective public health responses to COVID-19;³ in March, the Science Translational Medicine Editorial framed key questions for pandemic prevention, identifying selected pathogen- and society-based variables to be measured.⁴ We are far from answering all these questions, from fully understanding COVID-19 epidemiology or from having quantified the impact of different control measures. While science takes time, and not understanding is not an indictment of purposeful science, it is worth asking whether the deluge of science has been asking the right questions. The largest share of COVID-19 papers thus far has focused on clinical management descriptions of hospitalized cases, and reflections on the implications of the COVID-19 emergency on different clinical specialties (29.7%). Over time, the percentage of papers reporting surveillance or epidemiological data have been decreasing (from 56% of all COVID-19 papers at the beginning of February to 10% at the beginning of May); little has so far been published on new therapies and treatment evaluation (4.4%), although trends in this regard are increasing. Other COVID-19-related papers in the literature include health services research (6.3%), mental health (3.5%), aspects related to communication in times of emergency (2.5%) and economic impacts (0.5%).

Are we publishing the way we should be publishing? Addressing the second question is trickier. As scientific output around COVID-19 evolves over time, we find, consistent with other efforts that are systematically monitoring the literature,⁵ both poor adherence to identified research priorities and a predominance of opinion over data. Centrally, more than 60% of published papers on COVID-19 are opinion pieces not reporting original data. While several outlets, including The Economist , have commented positively on COVID-19 publishing trends, underlining how the virus “has changed the way scientists do their work and talk to each other”,⁶ it remains unclear to us that overall the trends in publication are indeed positive.

The central questions seems to be: is the published literature meant to truly inform clinical and public health practice and decision making? Does a literature that is predominantly expository serve that purpose? On the positive side, the COVID-19 public health emergency context has pushed journals to laudable efforts to fast track peer reviews and publishers to waive publication fees and provide free access to articles’ content and encouraged a preprint model of publication, the latter carrying both pro and con arguments. It has highlighted the role of new tools based on machine learning and artificial intelligence that are available to support methodologists in conducting systematic reviews or assessing research quality.⁷ And it has shown us that scientific publications might become live documents, constantly updated. On the negative side, this moment has made foggier the distinction between data-driven and expository outputs, with important implications for how the work of science is communicated. It also has revealed a divide between the production of science with its ultimate aims, of taking us towards individual and population well-being.

Galileo Galilei in the Dialogue Concerning the Two Chief World Systems⁸ warned science had to deal with the ‘sensible world and not a world of paper’. How would he react to so few published papers on COVID-19 report original data? How would he react to the rapid dissemination of inaccurate and exaggerated information?⁹ The idea of a ‘sensible world’ was the revolt of scientists against philosophers writing their opinions devoid of empiric observation and physical fact. One wonders whether we are entering a new Galilean age where science and empiricism need to regain the upper hand, focusing on questions that address key scientific need and prioritizing data over opinion in an effort to solve a global problem.

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Conflicts of interest: Authors report no conflicts of interest.

Appendix 1

The members of the University Vita-Salute San Raffaele COVID-19 literature monitoring working group are as follows: Andrea Amerio, MD PhD; Lorenzo Bellini, MD; Daria Bucci, MD; Michele Capraro, MD; Giovanni Gaetti, MD; and Stefano Salvati, MD.