Clinical Test Performance of a Rapid Point-of-Care Syphilis Treponemal Antibody Test: A Systematic Review and Meta-analysis

Abstract We reviewed relevant syphilis diagnostic literature and conducted a meta-analysis to address the question, “What is the sensitivity and specificity of the Syphilis Health Check, a rapid qualitative test for the detection of human antibodies to Treponema pallidum.” The Syphilis Health Check is the only rapid syphilis test currently cleared by the Food and Drug Administration (FDA). We conducted a systematic review and a meta-analysis using Bayesian bivariate random-effects and fixed-effect models to create pooled estimates of sensitivity and specificity of the Syphilis Health Check. We identified 5 test evaluations published in the literature and 10 studies submitted to the FDA and for a Clinical Laboratory Improvement Amendments waiver application. The pooled sensitivity (95% CI) from the laboratory evaluations (n = 5) was 98.5% (92.1–100%), while pooled specificity was 95.9% (81.5–100.0%). The pooled sensitivity for prospective studies (n = 10) was 87.7% ( 71.8–97.2%), while pooled specificity was 96.7% (91.9–99.2%). Using nontreponemal supplemental testing, the sensitivity improved to a pooled sensitivity of 97.0% (94.8–98.6%). The Syphilis Health Check may provide accurate detection of treponemal antibody.

Syphilis, caused by infection with the bacterium Treponema pallidum, is a re-emerging public health problem that can lead to significant complications if left untreated. In addition, syphilis has been demonstrated to facilitate human immunodeficiency virus acquisition and transmission [1][2][3]. In the United States, syphilis rates have increased nearly every year since 2000. In 2017, the rate was 9.5 per 100 000, the highest rate since 1993 [4]. While men who have sex with men account for the majority of cases reported to the Centers for Disease Control and Prevention, the increases in syphilis incidence among women (up 155.6% between 2013 and 2017 [4]) are of significant concern due to the impact that syphilis can have on pregnant women and their fetuses.
Routine screening and early treatment are essential to reduce the burden and impact of this disease. Fortunately, syphilis is curable; generally, penicillin G is utilized to treat persons in all stages of syphilis [5]. Congenital syphilis can be prevented by screening early in pregnancy, treating syphilis-infected pregnant women, and preventing reinfection [6,7].
Globally, over the past 15 years, rapid tests that can be used at the point-of-care have been developed that detect treponemal antibody using fingerstick whole-blood specimens. In global settings, rapid syphilis tests play an important role in prompt diagnosis [8]. These tests can be performed without laboratory processing, at the point-of-care and while the patient waits, thus reducing the potential for loss to follow-up [9][10][11][12][13]. With a shortened time to diagnosis, patients may be treated much more quickly, resulting in reduced complications from untreated infection as well as reducing the spread of syphilis to others. Most rapid tests that are available globally detect treponemal antibody and therefore cannot be used to determine if the infection is current and requires treatment. Testing with a nontreponemal test (anticardiolipin antibody test) is therefore still recommended in some settings to inform patient management [14,15]. However, a nonreactive treponemal test may be useful to rapidly rule out syphilis.
The Syphilis Health Check (Diagnostics Direct, LLC, Stone Harbor, NJ) is a rapid qualitative test for detection of human antibodies to T. pallidum in serum, plasma, or whole blood. The test utilizes antihuman immunoglobulins gold conjugate and highly purified T. pallidum recombinant proteins for specific detection of anti-T. pallidum antibodies [16]. That test detects both immunoglobulin (Ig) G and IgM using TP-15, TP-17, and TP-44 recombinant syphilis antigens. The test works by lateral flow of the sample through the absorbent device where, in the presence of T. pallidum antibodies, the antihuman immunoglobulins/protein dye conjugate binds to the human immunoglobulins forming an antigen-antibody complex. That complex produces a pink-colored band to indicate a reactive result. In the absence of anti-T. pallidum antibodies, no line is present in the reaction zone to indicate a nonreactive result.
A reagent control mechanism is included in the test in which unbound conjugate binds to the reagents in the control zone producing a pink-colored band. This band must be present for the test to be valid.
To review the performance of the Syphilis Health Check we conducted a meta-analysis that combines data from all available evaluations of the Syphilis Health Check. We created pooled estimates of sensitivity and specificity for the rapid test.

METHODS
We searched Medline, Embase, CINAHL, Scopus, and the Cochrane Library databases using search terms that included the following: ( . Abstracts were reviewed for relevance. We excluded articles that were not in English; were duplicates; had no authors; had no abstracts; were opinion, case reports, or reviews; and were not related to syphilis or point-of-care tests. Studies and data were included that evaluated the Syphilis Health Check rapid test. In addition to those studies identified in the literature, we included the studies from the manufacturer's Food and Drug Administration (FDA) application [17] for clearance of the Syphilis Health Check test as well as the Clinical Laboratory Improvement Amendments (CLIA) waiver application study data that were described in the package insert (REV P 02/17) [16]. The FDA studies included retrospective studies using stored or purchased serum specimens. The prospective FDA and CLIA studies used prospectively collected clinical specimens.
For the meta-analysis of diagnostic accuracy, the data extracted included study title, authors, year of publication, reference test type, specimen type, and study location. We calculated sensitivity and specificity for each study individually and used the exact binomial method to calculate 95% confidence intervals (CIs).
We used Bayesian bivariate random-effects and fixed-effect models meta-analysis to calculate pooled sensitivity and specificity of the Syphilis Health Check test using SAS PROC MCMC (SAS Institute, Cary, NC) [18]. We considered the fixed-effect model to be a better fit if the deviance information criterion (DIC) was less than 10 larger than the DIC of the random-effects model [ie, (DIC FIXED -DIC RANDOM ) <10]; otherwise, the random-effects model was used to calculate pooled estimates of sensitivity and specificity [19]. SAS version 9.4 was used for all other data analyses.

RESULTS
We identified 5 test evaluations published in the literature as well as studies submitted to the FDA and for the CLIA waiver application that we included in our meta-analysis. All studies (n = 15) [17,[20][21][22][23][24] used treponemal tests as the reference tests, while some included nontreponemal test result information. We separated performance results by study type, laboratorybased retrospective versus prospective. There were 10 prospective studies identified-4 from the literature describing clinic-based prospective evaluations, 1 from the CLIA application, and 5 from the FDA application. Among the prospective studies, some studies used whole-blood fingerstick specimens and some used sera ( Table 1).
The sensitivity and specificity estimates for each study are included in Tables 2-4. The sensitivity and specificity estimates from the prospective studies (n = 10) ranged from 50.0% to 100% and 50% to 100%, respectively. For laboratory-based evaluations on stored serum specimens (n = 5), the sensitivity and specificity estimates ranged from 88.7% to 100% and 83.3% to 100%, respectively. The pooled sensitivity for prospective studies was 87.7% (95% CI, 71.8-97.2%) and the pooled specificity was 96.7% (95% CI, 91.9-99.2%) ( Table 2). For the 4 prospective studies identified in the literature, the sensitivity was lower than that in all of the 5 FDA studies and with wide CIs. We pooled the results from those 4 prospective studies identified from the literature in a random-effects model and found a sensitivity of 68.6% (95% CI, 35.0-90.9%) and a specificity of 95.2% (95% CI, 84.4-99.2%). In addition, we pooled the results from just the prospective CLIA and FDA studies using a random-effects model and found the sensitivity to be 95.2% (95% CI, 83.6-99.7%) and the specificity to be 96.8% (95% CI, 87.1-99.8%).
Most (4/5) of the laboratory-based studies were from the FDA clearance application and all used sera. The pooled sensitivity from the laboratory evaluations was 98.5% (95% CI, 92.1-100%) and the pooled specificity was 95.9% (95% CI, 81.5-100.0%) ( Table 3). The laboratory study identified in the literature had the lowest sensitivity compared with the FDA laboratory evaluations.
Of the prospective studies, 4 had reference testing algorithms that included or were stratified by nontreponemal test results (Table 4). Each study used rapid plasma reagin (RPR), a nontreponemal test, in their diagnostic algorithm: Matthias et al [20] and Nakku-Joloba et al [21] had data available on RPR reactivity such that we were able to stratify the results to define positivity as those both RPR and treponemal reference test reactive and nonreactive as those both RPR and treponemal reference test nonreactive in Table 4. Fakile et al [23] also defined positivity as both RPR and treponemal reference test reactive and defined negative as those RPR and treponemal reference test nonreactive, RPR reactive and treponemal reference as nonreactive, or RPR nonreactive and treponemal reference test reactive. The CLIA waiver study used an antitreponemal chemiluminescent immunoassay (CIA) as a screening test and confirmed it with RPR. If the RPR was nonreactive, the T. pallidum particle agglutination assay was used as the tiebreaker [16]. With those reference algorithms, sensitivity improved to a pooled sensitivity of 97.0% (95% CI, 94.8-98.6%).

DISCUSSION
We reviewed prior publications and regulatory data to summarize the performance of the Syphilis Health Check test. We found that the Syphilis Health Check test had over 87% sensitivity and 96% specificity in prospective studies. In addition, when using nontreponemal results to inform infection status, the Syphilis Health Check had even higher sensitivity (97%), which is clinically important given that those who require treatment may be those who have both reactive treponemal and nontreponemal results. By combining data from several studies, the precision around sensitivity and specificity estimates increased. Additionally, we showed how the test performed in multiple settings across different specimen types. The sensitivity of the test tended to be much higher in the FDA trial studies compared with the studies identified in the literature, where the pooled sensitivity was 68.6%. The FDA trials included rigorous training and oversight while the studies described in the literature did not include methods of quality monitoring. Programs supporting ongoing quality control and quality assurance should be implemented where these rapid tests are used. In addition, the FDA prospective studies used sera for testing on the Syphilis Health Check while the other prospective studies used whole-blood specimens. This difference may have also contributed to the higher sensitivity observed in most of the FDA trials.
The Syphilis Health Check rapid test is currently the only rapid point-of-care whole-blood test for syphilis that has FDA clearance. However, many other manufacturers have commercially available, rapid point-of-care tests for syphilis, some of which have undergone regulatory approval processes in settings outside the United States. For example, the World Health Organization (WHO) has a prequalification of an in vitro diagnostics process that consists of review of products' safety, performance, design, and manufacture. The SD Bioline HIV/Syphilis Duo (Alere, Waltham, MA) is an example of a rapid test that includes treponemal antibody detection that is WHO prequalified. Other multiplex rapid tests have been CE marked, an indication that the The results of the pooled analysis suggest that the Syphilis Health Check had good performance for the detection of treponemal antibody, although further studies are warranted to understand why the test performed better in the studies for FDA submission than in other clinical settings. The 4 prospective studies included in our meta-analysis that were not for FDA submission had a low sensitivity (68.6%). The summary findings confirm that treponemal antibody may be detected outside of traditional laboratory settings. Future studies should stratify by stage of syphilis to better understand how the test performs at each stage. In addition, further research is needed to understand if the test performs well for immunocompromised patients and those with coinfections. One of the studies we included in our analysis included samples from pregnant women; however, those used stored sera, thus future research may be warranted that includes prospective clinical testing among pregnant women. Use of rapid point-of-care treponemal tests may facilitate the timely detection, treatment, and ultimately the enhanced control of syphilis. Differences in findings between FDA submission studies and studies conducted by other investigators point to the need for ongoing quality-assurance programs including training, competency assessments, quality-control measures, and proficiency testing with implementation of these tests. Those aspects are key to a successful testing program and should be implemented whenever utilizing a point-of-care assay.