Comparison of 3 Nucleic Acid Amplification Tests and a Rapid Antigen Test with Culture for the Detection of Group A Streptococci from Throat Swabs

Abstract

Background

Recently, the US Food and Drug Administration cleared 3 nucleic acid amplification test (NAAT) assays for detection of Streptococcus pyogenes [group A Streptococcus (GAS)] in pharyngeal specimens. However, there are limited studies evaluating the performance of these NAAT assays.

Methods

We compared the results of 3 NAATs (cobas Liat, Luminex Aries, and Cepheid Xpert Xpress) and a rapid antigen assay (Quidel QuickVue in-line strep A) with the accepted gold standard method, bacterial culture.

Results

Sixty-eight throat swab specimens collected between August and October 2017 were tested. Compared to bacterial culture, the sensitivities, specificities, positive predictive value, and negative predictive value for detecting GAS were as follows: cobas Liat: 100%, 97.4%, 96.7%, and 100%; Cepheid Xpert: 100%, 97.4%, 96.7%, and 100%; Luminex Aries: 95.2%, 100%, 100%, and 95.5%. The Quidel QuickVue in-line strep A assay showed poor sensitivity, detecting only 5.2% of culture-positive specimens.

Conclusion

The 3 NAATs have high sensitivity when compared with bacterial culture for detection of GAS. With rapid turnaround time and ease of use, these tests can be considered as reliable point-of-care tests for the diagnosis of GAS, replacing the need for back-up culture.

Impact Statement

The information presented in this article will benefit children and adults in diagnosing group A Streptococcus (GAS) pharyngitis. Our results indicate that the 3 nucleic acid amplification test (NAAT) assays have high sensitivity and specificity when compared with bacterial culture for detection of GAS. With rapid turnaround time and ease of use, NAAT assays can be considered as point-of-care tests for diagnosis of GAS, replacing the need for back-up culture.

Pharyngitis accounts for over 13 million office visits annually in the US. Acute pharyngitis is most often caused by viruses; however, approximately 12%–30% of cases in children and 5%–15% of cases in adults are caused by group A Streptococcus (GAS)² (1). Unfortunately, there is no one sign or symptom that has a sufficiently high likelihood ratio to successfully diagnose GAS. Even when accepted criteria are used to predict GAS, such as the Centor Criteria (tonsillar exudates, swollen tender anterior cervical lymph nodes, fever, and lack of cough), one study showed only 53% of patients with culture-positive sore throat met all 4 criteria (2). Current Infectious Disease Society of America guidelines suggest swabbing the throat for testing with a rapid antigen detection test (RADT) and/or culture to help differentiate between viral and GAS pharyngitis (3). RADTs have been shown to have high specificity; however, owing to poor sensitivity (70% to 90%), the Infectious Disease Society of America recommends throat culture in children and adolescents with negative RADT tests. Currently, at the University of Chicago Medicine (UCM) the QuickVue In-Line strep A (Quidel Corporation) test kit is being used as the point-of-care test for detection of GAS. Per the manufacturer's specifications, this test has 95% sensitivity and 98% specificity (4). However, independent analysis of this test has shown various results, with 71% sensitivity and 98.5% specificity in one study (5) and 73.9% sensitivity and 86.8% specificity in another (6). Thus, considering these independent findings and our institutional testing experiences, common practice still maintains the confirmation of negative RADT results with bacterial culture.

The major drawback for GAS bacterial culture is the turnaround time required for the detection of a positive result. A negative RADT entails a 24–48 h waiting period for culture confirmation before determining whether to prescribe antibiotics. Pharyngitis is the third most common diagnosis associated with antibiotic therapy (43 antibiotic prescriptions per 1000 people) (7). National guidelines suggest prescribing antibiotics only for streptococcal pharyngitis; however, studies have shown 56.2% of children and 72.4% of adults presenting with acute pharyngitis are prescribed antibiotics, although 37% of children and 18% of adults actually test positive for GAS (7). The overall low sensitivity of RADTs, long turnaround time for bacterial culture, and inappropriate antibiotic prescribing for acute pharyngitis has left the laboratory in need of more rapid and accurate methods for the diagnosis of GAS.

In 2017, the 3 nucleic acid amplification test (NAAT) assays received clearance from the US Food and Drug Administration. The assays amplify targeted regions of Streptococcus pyogenes DNA. Sensitivity, specificity, positive predictive value, and negative predictive value as reported by the respective manufacturers are as follows: cobas Liat strep A (Roche Molecular Systems Inc.): 98.3%, 94.2%, 88.1%, and 99.2% (8); Xpert Xpress strep A (Cepheid Inc.): 100%, 94.1%, 84.1%, and 100% (9); and Aries group A strep (Luminex Corporation): 97.5%, 97.8%, 94.0%, and 99.1% (10). However, there are limited studies evaluating the performance of these recently approved NAAT assays (11, 12). Here, we performed a comparison study of the 3 NAAT platforms to bacterial culture, which is performed routinely in our laboratory. Additionally, we compared the NAAT and bacterial culture results with RADT.

MATERIALS AND METHODS

Sixty-eight throat swab specimens were collected from nonconsecutive patients. Sixty-two of these were from patients who presented to the UCM emergency department or clinics between August 23, 2017, and October 31, 2017, and the remainder were received from an outside laboratory. Forty-four specimens were collected from children (<18 years old), and 18 were collected from adults (>18 years old); age was not available for the 6 additional samples received from the outside laboratory. All specimens were collected with a double BBL Culture Blue top swab (Becton Dickinson) and transported to the laboratory. The 2 swabs were used to inoculate 5% sheep blood and strep-selective agar plates (Remel), which were then incubated at 37 °C for 48 h. After inoculation, the swabs were returned to their transport tubes and were frozen at −80 °C. On the day of molecular testing, swabs were removed from the freezer and allowed to thaw for 2 h at ambient temperature. Swabs were then transferred to 1-mL Amies medium and vortex mixed. The 1-mL Amies suspension was then used to perform molecular testing. The 6 samples received from the outside laboratory were collected with double BBL Culture Blue top swabs and transferred to 1-mL Amies medium. The aliquots were then frozen at −30 °C, transported frozen, and kept frozen until molecular testing could be performed.

Testing was performed by laboratory personnel in the clinical microbiology laboratory on all 68 specimens with cobas Liat strep A and Xpert Xpress strep A assay, whereas only 42 samples were tested with Aries group A strep assay because this assay was included later in the study. Testing on all 3 platforms was performed according to manufacturer package inserts. Routine clinical point-of-care testing was performed by nursing staff in the emergency department or in clinic only on the UCM specimens for which the rapid assay was ordered by the clinician. Only 51 of the 62 pharyngeal specimens were therefore tested with the Quidel QuickVue in-line strep A rapid assay, according to manufacturer's specifications.

This study was granted exempt status from our institutional review board.

RESULTS

Of the 68 samples tested, 29 GAS culture-positive samples were positive on both the cobas Liat strep A and Xpert Xpress strep A assays, 38 culture-negative samples were negative on both the cobas Liat strep A and Xpert Xpress strep A assays, and 1 culture-negative sample tested positive on both the cobas Liat strep A and Xpert Xpress strep A assays (Table 1).

Table 1.

Performance of 3 NAATs and Quidel QuickVue in-line strep A rapid antigen assay in comparison to bacterial culture.

	Bacterial culture
	Positive, no.	Negative, no.	Total, no.
Cobas Liat strep A assay
Positive	29	1	30
Negative	0	38	38
			68
Xpert Xpress strep A assay
Positive	29	1	30
Negative	0	38	38
			68
Aries group A strep assay
Positive	20a	0	20
Negative	1	21a	22
			42
Quidel QuickVue strep A assay
Positive	1	1	2
Negative	18	31	49
			51

a

Five samples that initially resulted as invalid were rerun (repeat results: 3 negative, 2 positive).

Of the 42 samples tested with the Aries group A strep assay, 5 initially were “invalid.” Per the Aries group A strep assay package insert, these samples were retested with the residual swab transport medium and a new assay cassette. Repeat testing showed 20 of 21 GAS culture-positive samples tested positive and 21 culture-negative samples tested negative by Aries (Table 1).

In addition to PCR and bacterial culture, 51 of the 62 UCMC samples were initially tested with the Quidel QuickVue in-line strep A rapid assay. Of the 19 culture-positive specimens, 18 returned a negative result from the Quidel QuickVue in-line strep A kit (Table 1). Of the 32 culture-negative samples, 31 tested negative with this RADT (Table 1). Compared to bacterial culture, all of the NAAT assays displayed excellent performance, with sensitivities and specificities exceeding 95% (Table 2). Performance of the Liat and Xpert assays was identical, achieving 100% sensitivity and 97.4% specificity. The Quidel QuickVue in-line strep A assay, however, showed very poor sensitivity, detecting only 5.2% of culture-positive specimens (Table 2).

Table 2.

Sensitives, specificities, and positive and negative predictive values for 3 NAATs and Quidel QuickVue In-Line strep A assay compared to the results of culture for the detection of group A Streptococci from throat swabs.

Type of assay	Sensitivity (95% CI), %	Specificity (95% CI), %	PPV (95% CI), %	NPV (95% CI), %
Roche cobas Liat	100 (88.06–100)	97.4 (86.5–99.9)	96.7 (82.8–99.9)	100 (90.75–100)
Cepheid Xpert	100 (88.06–100)	97.4 (86.5–99.9)	96.7 (82.8–99.9)	100 (90.75–100)
Luminex Aries	95.2 (76.2–99.9)	100 (83.9–100)	100 (83.2–100)	95.5 (77.2–99.9)
Quidel QuickVue	5.3 (0.1–26.0)	96.9 (83.8–99.9)	50 (1.3–98.7)	63.3 (48.3–76.6)

DISCUSSION

Our results indicate that the 3 NAATs have high sensitivity and specificity for detecting GAS when compared with bacterial culture. We observed 100% sensitivity with Xpert Xpress strep A and cobas Liat strep A assays. For Aries group A strep assay, the sensitivity was 95.2%. In a recent study by Wang et al. 2017 (12), the cobas Liat strep A assay demonstrated improved sensitivity (97% Liat vs 84.5% RADT) and essentially equivalent specificity (93.3% Liat vs 95.3% RADT) when compared with reference culture method. In a second study, performance of the cobas Liat strep A assay was equivalent to that of a laboratory-developed S. pyogenes LightCycler PCR assay. The sensitivity and specificity of the cobas Liat group A strep test were 100% and 98.3%, respectively, when LightCycler PCR assay was considered as the gold standard (11).

In comparison to most RADT kits, which have a 5- to 10-min turnaround time (13), the cobas Liat strep A and Xpert Xpress strep A assays have turnaround times <20 min (11, 12). The cobas Liat strep A and the Xpert Xpress strep A assays received waived testing status under CLIA, whereas the Aries group A assay has not received the CLIA-waived status to date. Given the overall performance of the NAAT assays, their rapid result times, and ease of use, waiting for confirmatory culture results in possible GAS pharyngitis may no longer be necessary. Notably, the Aries group A strep and Cepheid Xpert Xpress strep A product inserts still recommend confirmation by culture for negative results (9, 10). However, the product insert for the cobas Liat strep A does not contain such a statement; in fact, the manufacturer clearly states in their promotional materials that confirmatory culture is not needed (8). Additionally, the use of NAATs can confirm the clinical diagnosis quickly and also encourage appropriate and timely use of antibiotics. The rapid negative NAAT results could eliminate the need for unnecessary antibiotic courses that may be prescribed while awaiting culture results.

RADTs have been shown to have variable sensitivity for detection of GAS (1, 2); however, our study highlights the extremely poor performance (5.2% sensitivity) of the Quidel QuickVue in-line strep A rapid assay. Meta-analyses have shown a high variability in sensitivity, ranging from 68% to 92% (1). This variability has been attributed to severity of clinical symptoms, inoculum size, technical training, and personnel conducting the tests (1). It is also possible that highly sensitive molecular testing is detecting low bacterial burden or carrier states. In such cases, we would also expect to see variability in colony counts on solid media, based on inoculum. The effect of inoculum was not assessed in our study, although it is known that differences in inoculum may lead to significantly wide variation in sensitivity of rapid antigen streptococcal assays (14). Moreover, rapid antigen testing was not performed by laboratory personnel but by a variety of nursing staff in the emergency department and clinic settings. It is known that sample acquisition and technique greatly influence sensitivity of these RADTs (15). Therefore, although we cannot point to a single factor accounting for the extremely low sensitivity of the rapid antigen assay observed in our study, it is likely that multiple factors contributed. Larger studies would be needed to replicate the overall poor performance of the RADT observed in our study. Nonetheless, our findings support the current Infectious Disease Society of America guidelines, which suggest that a negative RADT should be confirmed by culture to rule out false-negative cases among pediatric patients (3). Recent evidence indicates that even among adults, a similar approach may be beneficial (16).

Limitations of our study include a relatively small sample size and that the effect of inoculum was not assessed, which may have contributed to the extremely low sensitivity of the RADT. Further, chart review was not performed to assess clinical symptoms, treatment decisions, or patient follow-up. Larger comparative studies testing asymptomatic patients with both RADTs and molecular methods may help reflect the effect of the carrier state, which may have influenced our results. Additionally, our study was performed during a time of year with expected higher rates of culture-positive GAS; different results might have been observed if testing occurred at other times of the year when acute GAS pharyngitis is less prevalent.

In conclusion, all 3 NAATs evaluated in the present study have high sensitivity when compared with bacterial culture for the detection of GAS. With rapid turnaround time and ease of use, these tests can be considered as point-of-care or near-patient tests for the diagnosis of GAS. However, cost of these tests and reimbursement issues may limit their widespread use. Moreover, their high sensitivity may result in the detection of GAS carriers and consequently in unnecessary treatment. Therefore, testing should only be performed for clinically significant cases of pharyngitis and results must be interpreted with care. Future studies evaluating the carrier state and effect of these tests on treatment will be beneficial in the effort to use these rapid molecular tests prudently.

2 Nonstandard abbreviations

 
• GAS

  group A Streptococcus

 
• RADT

  rapid antigen detection test

 
• UCM

  University of Chicago Medicine.

Author Contributions:All authors confirmed they have contributed to the intellectual content of this paper and have met the following 4 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved.

Authors' Disclosures or Potential Conflicts of Interest:Upon manuscript submission, all authors completed the author disclosure form.

Employment or Leadership: None declared.

Consultant or Advisory Role: None declared.

Stock Ownership: None declared.

Honoraria: A. Charnot-Katsikas has received speaking honoraria from Cepheid.

Research Funding: Roche, Cepheid, and Luminex provided test kits for study.

Expert Testimony: None declared.

Patents: None declared.

Role of Sponsor: The funding organizations played no role in the design of study, choice of enrolled patients, review and interpretation of data, preparation of manuscript, or final approval of manuscript.

Acknowledgments

The authors would like to thank Roche, Cepheid, and Luminex for supplying test kits. Additionally, the authors thank Roche and Luminex for loaning instrumentation for the use in this evaluation. The authors would also like to thank the following technologists: Angelina Valencia, Antonia Turner, and Melinda Wills, who performed the NAATs in the microbiology laboratory.