This study evaluated the clinical performance of the cobas 4800 HPV Test (Roche Molecular Systems, Pleasanton, CA) for high-risk human papillomavirus (HR-HPV) testing with individual HPV-16/HPV-18 genotyping in women 21 years or older with atypical squamous cells of undetermined significance (ASC-US). Women (N = 47,208) were recruited in the United States during routine screening, and liquid-based cytology and HPV testing were performed. The ASC-US prevalence was 4.1% (1,923/47,208), and 1,578 women underwent colposcopy with valid results. The cobas 4800 HPV Test demonstrated performance comparable to the Hybrid Capture 2 test (QIAGEN, Gaithersburg, MD) for the detection of cervical intraepithelial neoplasia (CIN) grade 2 or worse and grade 3 or worse. HPV-16/HPV-18+ women had a greater absolute risk of CIN 2 or worse compared with pooled HR-HPV+ and HR-HPV– women (24.4%, 14.0%, and 0.8%, respectively).
The cobas 4800 HPV Test is clinically validated for ASC-US triage. HPV-16/HPV-18 genotyping can identify women at highest risk for high-grade cervical disease, and this additional risk stratification may be used in formulating patient management decisions.
In North America and Europe, the current preferred approach to managing women with atypical squamous cells of undetermined significance (ASC-US) is testing for high-risk types of human papillomavirus (HR-HPV) whenever liquid-based cytology is used.1–3 Because HR-HPV testing can be carried out using the residual liquid-based cytology specimen, this approach eliminates the need for an additional office visit for repeated testing and allows HR-HPV– women to be assured that they do not have a significant lesion without undue delay. It also eliminates the need for colposcopy for 40% to 60% of women and has been shown to have a favorable cost-effectiveness ratio.4–8
There is awareness in the cervical cancer screening community that analytic and clinical validation of the performance of an HR-HPV test in the intended-use populations is necessary before a test can be considered acceptable for widespread clinical implementation.9–11 It has been recommended that an HR-HPV test have a clinical sensitivity of 92% (± 3%) for detection of cervical intraepithelial neoplasia (CIN) grade 3 (CIN 3). Moreover, the test should have sufficient specificity to ensure that women without disease are not overly studied because of a positive test result.9 Failure to meet benchmark criteria for clinical sensitivity and clinical specificity has potential safety implications for women undergoing cervical cancer screening.11
Even with carefully validated tests, the clinical usefulness of HR-HPV testing in women with ASC-US is limited by the fact that, on average, 43% will be HR-HPV+, while the prevalence of CIN 2 or worse (to include CIN 2, CIN 3, adenocarcinoma in situ, and invasive cancer) in published studies is only 10.3%.7,12 This suggests that there may be clinical benefit in further stratifying HR-HPV+ women with ASC-US to reduce unnecessary colposcopy or follow-up. Because HPV-16 and HPV-18 are associated with approximately 70% of all invasive cervical carcinomas,13 genotyping for HPV-16/HPV-18 might identify the women at highest risk for CIN 2 or worse and provide a clinically useful stratification of cervical disease risk.
In this analysis, we evaluated the clinical performance of the recently developed cobas 4800 HPV DNA Test (Roche Molecular Systems, Pleasanton, CA) for pooled HR-HPV and individual HPV-16 and HPV-18 genotypes among women with ASC-US cervical cytology results enrolled in the ATHENA (Addressing THE Need for Advanced HPV Diagnostics) HPV Study.
Materials and Methods
We enrolled 47,208 women 21 years or older undergoing routine cervical cancer screening into the ATHENA HPV study between May 2008 and August 2009 at 61 clinical centers across the United States. Study inclusion/exclusion criteria were as follows: 21 years or older; not pregnant; intact uterus; willing to undergo colposcopy and biopsy within 12 weeks, if required; no treatment for CIN in the preceding 12 months; and no current or planned participation in a clinical trial for HPV treatment. The current analysis is focused on 1,578 (82.3%) of the 1,918 women who had ASC-US cytology; all 1,578 women underwent colposcopy and had valid HPV tests and cervical biopsy results. The protocol was approved by the institutional review boards of all study sites; all women provided written informed consent before undergoing any study procedures.
Cytology and HPV Testing
At the enrollment visit, women provided written informed consent and demographic information and medical history. Two liquid-based cervical cytology samples (ThinPrep, Hologic, Bedford, MA) were collected from each participant using a plastic spatula and an endocervical brush according to the manufacturer’s instructions. The first cervical sample was used for cytology and for testing with 3 separate HR-HPV assays: the second-generation cobas 4800 HPV Test, the first-generation AMPLICOR HPV Test, and the first generation LINEAR ARRAY High Risk HPV Genotyping Test (Roche Molecular Systems). Cytology evaluation was conducted at 4 accredited clinical laboratories in the United States and reported using the Bethesda 2001 nomenclature14: LabCorp, Burlington, NC; DCL Medical Laboratories, Indianapolis, IN; Scott & White Memorial Hospital, Temple, TX; and TriCore Reference Laboratories, Albuquerque, NM.
The cobas 4800 HPV Test simultaneously detects a total of 14 HR-HPV types: HPV-16 individually, HPV-18 individually, and 12 pooled HR-HPV genotypes (31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68), in addition to a separate β-globin control. The cobas 4800 HPV Test was performed at the same 4 study laboratories that conducted the cytology evaluation and also at Roche Molecular Systems. The AMPLICOR HPV Test detects 13 HR-HPV types (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, and 68; optical density cutoff ≥0.2), and the LINEAR ARRAY High Risk HPV Genotyping Test detects 16 HR-HPV types (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, 68, 73, and 82). These HPV tests were performed at the 4 study laboratories stated previously. The second cervical cytology sample was tested according to the manufacturer’s instructions with the Hybrid Capture 2 (hc2) test (QIAGEN, Gaithersburg, MD) that detects 13 HR-HPV types (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, and 68) 15; hc2 testing was conducted at the 4 main study laboratories. Both cobas 4800 testing and hc2 testing were performed independently and without knowledge of other HPV or cytology test results.
Women with ASC-US cytology results were referred for colposcopy. Participants and colposcopists were masked to the patients’ cytology and HPV test results. Colposcopy with biopsy and/or endocervical curettage (ECC) were performed within 12 weeks of enrollment according to a standardized protocol that included biopsy of all visible cervical lesions or, in women with a satisfactory colposcopy but without visible cervical lesions, a random biopsy at the squamocolumnar junction. All women with an unsatisfactory colposcopy had ECC. Biopsy and ECC specimens were reviewed in a blinded manner by a panel of 3 pathologists (Central Pathology Review) and diagnosed using standard criteria and the CIN terminology.16
Performance characteristics of HPV testing (sensitivity, specificity, positive predictive value, and negative predictive value) for identification of high-grade cervical disease (≥CIN 2 and ≥CIN 3) were determined using standard statistical tests. The absolute risks (ARs) and relative risks (RRs) of high-grade cervical disease with the respective 95% confidence intervals (CIs) were determined for different categories of HPV DNA results (by the cobas 4800 HPV Test). HPV results of genotypes 16 and 18 were analyzed as individual results and as combined HPV-16/HPV-18 results (genotype 16 and/or 18 positive). Genotype 16+ results included positive for genotype 16 alone, with or without genotype 18, and with or without 12 other HR-HPV types present. Genotype 18+ results included positive for genotype 18 alone, with or without a positive result for 12 other HR-HPV, and negative result for genotype 16. Twelve other HR-HPV+ samples were positive only for 1 or more of these high-risk types.
Demographics and Medical History
Of the 47,208 women 21 years or older enrolled into the ATHENA study, 1,923 (4.1%) had an ASC-US cytology result. Of these 1,923 women, 5 had invalid/missing HPV test results; 1,620 (84.5%) with valid HPV test results underwent colposcopy, and of these, 1,578 had valid biopsy results, 32 had invalid biopsy results, and 10 had no biopsy done. Analysis was therefore performed for 1,578 evaluable women. The percentages of women with ASC-US who underwent colposcopy were similar for HR-HPV+ and HR-HPV– women, 86.4% and 83.5%, respectively. Characteristics of the 1,578 evaluable women are shown in Table 1. The mean age was 37.1 years (± SD, 11.3 years) and 23.1% were postmenopausal. Only 4.3% had been vaccinated against HPV, and 0.3% were immunocompromised for any reason.
HR-HPV Prevalence and Biopsy Results
The overall prevalence rates of HR-HPV (14 genotypes) and HPV-16 and HPV-18 detected with the cobas 4800 HPV Test were 32.6%, 8.2%, and 2.9%, respectively Table 2. The prevalence of HR-HPV (14 genotypes) and HPV-16 and HPV-18 declined with increasing age (Table 2). The overall prevalence of HR-HPV detected with the hc2 test was 31.5% and also declined with increasing age (Table 2). Among women with ASC-US, biopsy-confirmed CIN 1, CIN 2, and CIN 3 were identified in 10.0% (158/1,578), 2.2% (34/1,578), and 2.9% (46/1,578) of women, respectively. No cases of invasive cervical cancers or adenocarcinoma in situ were detected. The prevalence of CIN 2 or worse was 5.1% (80 cases), and the prevalence of CIN 3 or worse was 2.9% (46 cases). Genotypes 16 and/or 18 were detected in 8% of women without CIN, 18% of women with biopsy-confirmed CIN 1, 44% of women with CIN 2, and 61% of women with CIN 3 or worse Figure 1.
cobas 4800 HPV Test Performance Characteristics
The sensitivity, specificity, positive predictive value, and negative predictive value of the cobas 4800 HPV Test and hc2 test for the detection of CIN 2 or worse and CIN 3 or worse are shown in Table 3. The performance of the cobas 4800 HPV Test is very similar to that of the hc2 test for all standard parameters of test performance for the CIN 2 or worse and CIN 3 or worse end points. Both tests were highly concordant (90.6% [95% CI, 89.1%–92.0%] for <CIN 2 and 96.2% [95% CI, 89.3%–98.7%] for ≥CIN 2) with minor nonsignificant disagreement Table 4.
Risk Estimates by HPV Test Result
The AR of CIN 2 or worse was 18.6-fold higher among women who were HR-HPV+ (14 types) by the cobas 4800 HPV Test compared with HR-HPV– women: AR = 14.0% (95% CI, 11.3%–17.3%) vs 0.75% (95% CI, 0.4%–1.5%) with an RR of 18.6 (95% CI, 9.0–38.4) Table 5 and Table 6. The AR for CIN 2 or worse was greatest among women who were HPV-16+ (AR, 31.5%).
The RR for CIN 2 or worse among women who were HPV-16+ was 42.0 (95% CI, 20.1–87.5) compared with women who were HR-HPV– (14 types) and 3.7 (95% CI, 2.4–5.7) compared with women who were positive for the 12 other HR-HPV (excluding HPV-16 and HPV-18: Table 6). A similar effect was observed for CIN 3 or worse (Table 6). Overall, the relative risk of CIN 3 or worse was 70.9 (95% CI, 21.8–231.1) among women who were HPV-16+ compared with women who were HR-HPV– (14 types) and 4.5 (95% CI, 2.5–8.2) compared with women who were positive for the 12 other HR-HPV (excluding HPV-16 and HPV-18; Table 6).
The ATHENA study enrolled 47,208 subjects undergoing routine cervical cancer screening in the United States. A key objective of this trial was to clinically validate HR-HPV testing for the triage of women 21 years or older with an ASC-US cytology result using the newly developed cobas 4800 HPV Test. This polymerase chain reaction–based test detects 14 HR-HPV genotypes and provides separate yet integrated results for HPV-16 and HPV-18. The enrolled population was largely unvaccinated and nonsmoking, with a racial distribution reflective of the United States. The cervical cancer screening history of the study participants was also reflective of the United States, with 90% reporting a history of a screening with cervical cytology in the 5 years before enrollment.
The prevalence of ASC-US among women enrolled into ATHENA was 4.1%. This is similar to the mean reporting rate of ASC-US among US laboratories participating in the 2003 College of American Pathologists surveys (4.65%).17 Although the ASC-US rate was similar to that in the United States as a whole, the prevalence of HR-HPV (14 types) detected by the cobas 4800 HPV Test was somewhat lower than reported in other studies from the United States and from other countries. Among women 21 years or older with ASC-US cytology in the ATHENA study, the overall HR-HPV rate was 32.6%. For comparison, the prevalence of HR-HPV in women with ASC-US in the ASC-US/Low-Grade Squamous Intraepithelial Lesion Triage Study (ALTS) was 48.0%,18 and in the recently published phase III trial of the Cervista HPV HR test (Hologic), it was 57.1%.19,20 Similarly, a 2005 meta-analysis of published studies reported a pooled HR-HPV positivity rate of 44.7% in women with ASC-US.12 The lower HR-HPV positivity rate observed among women with ASC-US in the ATHENA study most likely reflects differences in the age distribution of women enrolled into the studies. The median age of women with ASC-US in the ATHENA study was 37.1 years compared with 29 years for ALTS21 and 31.0 years for the Cervista study.20 Differences in HR-HPV prevalence in the ASC-US population may also reflect variations in the criteria used by different laboratories to diagnose ASC-US, which is well recognized to be a poorly reproducible cytologic interpretation.22 This possibility is further supported by the finding that the overall prevalence of HR-HPV detected by the hc2 test in women with ASC-US in the ATHENA study was 31.5%, which is similar to the prevalence detected using the cobas 4800 HPV Test, and makes intrinsic assay variability due to excess polymerase chain reaction sensitivity highly unlikely.
Clinical validation of the cobas 4800 HPV Test (14 high-risk genotypes) in women with ASC-US was achieved by determining its performance characteristics for the detection of CIN 2 or worse and CIN 3 or worse and by comparing its performance with that of the hc2 test. Sensitivity rates for CIN 2 or worse and CIN 3 or worse were 90.0% (95% CI, 81.5%–94.8%) and 93.5% (95% CI, 82.5%–97.8%), respectively. These findings were comparable to the performance in the same population of the hc2 test, which is considered to be an established standard in the United States. Moreover, it is similar to that found in the 2005 meta-analysis of published studies of HR-HPV testing for ASC-US that reported a pooled HR-HPV sensitivity of 94% for CIN 2 or worse12 and is comparable to that found for the Cervista HPV HR test in women with ASC-US (92.8% [95% CI, 84.1%–96.8%] for ≥CIN 2)19,20 and, therefore, meets the benchmarks of sensitivity established by expert consensus for clinically valid HPV tests in the United States and Europe.9,10 The specificity for high-grade disease detected by the cobas 4800 HPV Test was also comparable to the hc2 test in the current study (70.5% [95% CI, 68.1%–72.7%] vs 71.1% [95% CI, 68.8%–73.4%] for ≥CIN 2, respectively). Both tests had higher specificity than has been reported for the Cervista HPV HR test (44.2% [95% CI, 41.5%–46.9%] for ≥CIN 2 and 43% [95% CI, 40.3%–45.7%] for ≥CIN 3).19,20
One potential advantage of the cobas 4800 HPV Test is that it provides information on HPV-16 and HPV-18 separately. HPV-16 and HPV-18 are generally considered particularly high-risk genotypes because they account for approximately 70% of invasive cervical cancers globally.13 In the ATHENA study, specific HPV genotyping was found to have a dramatic impact on the AR and RR of CIN 2 or worse and CIN 3 or worse in women with ASC-US. The AR of CIN 2 or worse among women who were HPV-16+ was 31.5%, and for CIN 3 or worse, it was 20.0%. The RR of CIN 2 or worse for women who were HPV-16+ vs women positive for a non–HPV-16/HPV-18 HR-HPV (12 other genotypes) was 3.7, and the RR of CIN 3 or worse for women positive for HPV-16 vs women positive for 12 other HR-HPV genotype(s) was 4.5. It is interesting that despite its being associated with approximately 10% of squamous cell carcinomas of the cervix, 34% of adenocarcinomas, and a very high fraction of rare but aggressive neuroendocrine carcinomas,13,23,24 HPV-18 positivity at baseline alone did not confer a significantly increased relative risk for CIN 2 or worse or CIN 3 or worse compared with the combination of all 12 other HR-HPV genotypes. This may be reflective of the finding that, in prospective trials, high-grade CIN associated with HPV-18 takes longer to develop or remains clinically occult, perhaps in the endocervical canal, compared with precancer associated with HPV-16.25–27 The effect of HPV-18 on the risk of high-grade disease will be further investigated and better defined by the 3-year follow-up phase of the ATHENA study.
In the ALTS, HPV-16 was also found to confer the greatest risk of high-grade CIN among women with ASC-US or low-grade squamous intraepithelial lesion. The 2-year cumulative risks (contrasted with the immediate cross-sectional risks above) of CIN 2 or worse or CIN 3 or worse in women infected with only HPV-16 were 50.6% (95% CI, 44.1%–57.2%) and 39.1% (95% CI, 32.9%–45.7%), respectively. Risk of CIN 2 or worse for women infected with other HR-HPV genotypes ranged from a high of 29.5% (95% CI, 20.3%–40.2%) for HPV-31 to a low of 4.7% (95% CI, 0.6%–15.8%) for HPV-59.28 In addition, as in the ATHENA study, the risk of CIN 2 or worse among women with HPV-18 in the ALTS (18.3% [95% CI, 10.6%–28.4%]) was not dramatically higher than for many other HR-HPV types.
The clinical implications of the enhanced risk of CIN 2 or worse associated with being HPV-16+ are worthy of further consideration. The risk of CIN 2 or worse in women with ASC-US infected with other HR-HPV genotypes is high enough to warrant colposcopy. However, given the poor sensitivity of colposcopy,29 consideration should be given to more aggressive disease ascertainment and more intensive follow-up for HPV-16+ women with ASC-US not found to have CIN 2 or worse at initial colposcopy. Therefore, these new data on genotype-related risk, which are an intrinsic product of the cobas 4800 HPV Test process, should be highly useful if guidelines for the management of ASC-US are considered in need of revision.
This study has a number of strengths. These include the large number of subjects who had ASC-US cytology results, underwent colposcopy, and had valid HPV tests and cervical biopsy results (n = 1,578); the fact that all women with ASC-US were referred to colposcopy, with 84.5% (1,620/1,918) undergoing the procedure; and that the colposcopy was performed masked to the cytology result and the woman’s HPV status. In addition, a standard protocol was used for colposcopy, which included a “random” cervical biopsy in women without colposcopic lesions, and biopsy specimens underwent a masked consensus central pathology review. The results presented herein clinically validate the cobas 4800 HPV Test for use in women with ASC-US cytology. In addition, they demonstrate that the use of an HR-HPV genotyping assay that individually detects HPV-16 allows identification of women at highest risk of biopsy-proven high-grade CIN in a population in which HR-HPV testing is recommended.
We thank Catherine Behrens, MD, PhD, for assistance in analysis of the data and in writing the manuscript. Editorial assistance was provided by Health Interactions, London, England.
ATHENA Central Pathology Review Panel: Thomas C. Wright, Jr, MD, Department of Pathology, Columbia University School of Medicine, New York, NY; Mark H. Stoler, MD, University of Virginia Health System, Charlottesville; and Alexander Ferenczy, MD, Department of Pathology, McGill University, Montreal, Canada.
* ATHENA study testing sites and participants were as follows:
Laboratory testing sites: LabCorp, Burlington, NC, B.A. Body; Roche Molecular Systems, Pleasanton, CA, A. Butcher; DCL Medical Laboratories, Indianapolis, IN, C. Eisenhut; Scott & White Memorial Hospital, Temple, TX, A. Rao; and TriCore Reference Laboratories, Albuquerque, NM, S. Young.
The institutions and principal investigators who participated in the study were as follows: Comprehensive Clinical Trials, West Palm Beach, FL, R. Ackerman; Green Clinic, Ruston, LA, R. Anders; Philadelphia Clinical Research, Philadelphia, PA, E. Andruczyk; Visions Clinical Research, Boynton Beach, FL, K. Aqua; Women’s Health Specialist, Costa Mesa, CA, R. Black; Mount Vernon Clinical Research, Atlanta, GA, S. Blank; Tennessee Women’s Care, Nashville, P. Bressman; Chattanooga Medical Research, Chattanooga, TN, K. Brody; OB/GYN Specialists of the Palm Beaches, West Palm Beach, FL, J. Burigo; Segal Institute for Clinical Research, North Miami, FL, S. Chavoustie; SC Clinical Research Center, Columbia, SC, M. Davis; Bluegrass Clinical Research, Louisville, KY, A. Donovan; Delaware Valley OB-GYN and Infertility Group, Plainsboro, NJ, S. Eder; Advanced Research Associates, Corpus Christi, TX, C. Eubank; Advanced Clinical Concepts, West Reading, PA, S. Fehnel; Miami Research Associates, Miami, FL, R. Feldman; Center for Women’s Health of Lansdale, Lansdale, PA, R. Filosa; Blue Skies Center for Women, Colorado Springs, CO, S. Fowler; Visions Clinical Research, Tucson, AZ, C. Goldberg; Impact Clinical Trials, Las Vegas, NV, R. Groom; Physicians’ Research Options, Lakewood, CO, J. Grube; Four Rivers Clinical Research, Paducah, KY, P. Grumley; Medical Network for Education and Research, Decatur, GA, P. Hadley; Women’s Health Research, Phoenix, AZ, M. Harris; Impact Clinical Trials, Los Angeles, CA, L. Hazan; HWC Women’s Research Center, Englewood, OH, J. Huey; Texas Medical Center, Houston, M. Jacobs; Mobile OB/GYN, Mobile, AL, S. Kleinpeter; Altus Research, Lake Worth, FL, S. Lederman; Tacoma Women’s Specialist, Tacoma, WA, J. Lenihan, Jr; Phoenix OB-GYN Association, Moorestown, NJ, B. Levine; The Woman’s Clinic, Boise, ID, K. Lowder; Impact Clinical Trials, Los Angeles, CA, N. Lurvey; eCast, North Charleston, SC, J. Martin, Jr; State of Franklin Healthcare Associates Research, Johnson City, TN, R. McDavid; Quality of Life Medical & Research Center, Tucson, AZ, J. McGettigan; Eastern Carolina Women’s Center, New Bern, NC, J. Michelson; Tidewater Clinical Research, Virginia Beach, VA, F. Morgan; St John’s Center for Clinical Research, Jacksonville, FL, R. Myers; M & O Clinical Research, Ft Lauderdale, FL, K. Osman; Lyndhurst Gynecologic Associates, PA, Winston-Salem, NC, R. Parker, Jr; Enterprise Women’s Center, Enterprise, AL, J. Pollard; Salt Lake Research, Salt Lake City, UT, A. Rappleye; Women’s Health Care at Frost Street, San Diego, CA, R. Reagan; Atlanta North Gynecology Center for Research, Roswell, GA, H. Reisman; Women’s Clinical Research, Newburgh, IN, L. Rogers; Jacksonville Center for Clinical Research, Jacksonville, FL, N. Sager; Women’s OB-GYN, Saginaw, MI, G. Sieggreen; Clinical Research Consultants, Hoover, AL, W. Somerall, Jr; Edinger Medical Group Research Center, Fountain Valley, CA, M. Sperling; Health Awareness, Jupiter, FL, R. Surowitz; Physician Care Clinical Research, Sarasota, FL, M. Swor; Woman’s Health Practice, Champaign, IL, S. Trupin; Clinical Trials Management, Covington, LA, A. Tydings; Advanced Research Associates, Dallas, TX, K. Waldrep; Fellows Research Alliance, Savannah, GA, D. Walland; Fellows Research Alliance, Hilton Head, SC, D. Walland; Women’s Care Florida, Tampa, W. Wilkerson; Advanced Research Associates, McAllen, TX, W. Wilson; Precision Trials, Phoenix, AZ, S. Wininger; and Yassear Clinical Research, Carmichael, CA, S. Yassear.