Be Careful With Big Data: Reanalysis of Patient Characteristics and Outcomes of 11 721 Patients With Coronavirus Disease 2019 Hospitalized Across the United States

To the Editor—We read with interest the article published by Fried et al [1] in Clinical Infectious Diseases. We wish to alert the editor about the robustness of this work. First, the groups in their study were not homogeneous, such as in other coronavirus disease 2019 big data studies [2]. More patients were intubated in the hydroxychloroquine group than in the non-hydroxychloroquine group (24.9% vs 12.2%). Seven percent of patients were intubated on the first day of hospitalization. While data for the mechanical ventilation subgroup of patients in the remdesivir group are provided in the supplementary data, this is not the case for the hydroxychloroquine group. In the hydroxychloroquine group, 93% of patients had pneumonia vs 79% in the nonhydroxychloroquine group (Supplementary Table 3 in Field et al [1]). The risk of being trapped in a Simpson’s paradox-like situation [3] is high.

This brings us to our main point of concern, which is the reliability of the data used in the article. Among the 11 authors, 7 are affiliated with a data collection company, namely, Target Pharmasolutions. In the article, they provide little detail on how the data were collected. The authors state that the data came “from a commercial insurance claims database that requires a data sharing agreement and data license for access.” They also specify that the data “were acquired from a commercially available source representing adults receiving inpatient care between February 15 and April 20, 2020 at 245 hospitals across 38 states in the US.” The hospital names are not provided, and whether those hospitals agreed to have their data used in such a study is not mentioned. The information available on the Target Pharmasolutions company website does not provide further details on the data collection mechanism.

There are some points that caught our attention. For instance, we do not understand how 99.4% of patients treated with hydroxychloroquine were treated in urban hospitals compared with 65% of untreated patients (Supplementary Table 3 in Fried et al [1]), while patients are distributed in a more balanced manner between teaching and nonteaching hospitals, as well as in the most urbanized (Northeast) and less urbanized (Midwest) regions of the United States. Likewise, the mortality rate of 70.5% among patients who received mechanical ventilation (Table 2 in Fried et al [1]) does not appear to be compatible with the data published in the literature. No information is given on any verification of individual data by any infectious diseases specialist, so it is possible that patient registration errors were included in the general study.

The scandal caused by the retraction of 2 articles initially published in The Lancet [4] and in the New England Journal of Medicine [5] by a data collection company, Surgisphere, that was unable to demonstrate the reliability of its data emphasizes the importance of data traceability. We therefore ask the editors of Clinical Infectious Diseases to ensure the scientific community, as The Lancet editors did for the article published by Merah et al, that the data presented in the article published by Fried et al meet the most essential reliability criteria.

Notes

Our Marseille group used widely available generic drugs distributed by many pharmaceutical companies.

Potential conflicts of interest. All authors: No reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed. Our Marseille group used widely available generic drugs distributed by many pharmaceutical companies.