-
PDF
- Split View
-
Views
-
Cite
Cite
Elizabeth L Lewandrowski, Sarah E Turbett, Lise E Nigrovic, Erik H Klontz, John A Branda, Comparative Evaluation of Commercial Test Kits Cleared for Use in Modified Two-Tiered Testing Algorithms for Serodiagnosis of Lyme Disease, The Journal of Infectious Diseases, Volume 230, Issue Supplement_1, 15 August 2024, Pages S76–S81, https://doi.org/10.1093/infdis/jiae229
- Share Icon Share
Abstract
Modified 2-tiered testing (MTTT) for Lyme disease utilizes automatable, high throughput immunoassays (AHTIs) in both tiers without involving western immunoblots, offering performance and practical advantages over standard 2-tiered testing (STTT; first-tier AHTI followed by immunoglobulin M (IgM) and immunoglobulin G (IgG) western immunoblots). For MTTT, Centers for Disease Control and Prevention recommends using AHTI test kits that have been cleared by Food and Drug Administration (FDA) specifically for this intended use. We evaluated performance of FDA-cleared MTTT commercial test kits from 3 manufacturers by comparing with STTT results.
We performed MTTT (total antibody AHTI with reflex to separate IgM and IgG AHTIs) using test kits from Diasorin, Gold Standard Diagnostics (GSD), and Zeus Scientific on 382 excess serum samples submitted to the clinical laboratory for routine Lyme disease serologic testing in July 2018, measuring agreement between MTTT and STTT using the κ statistic.
Overall agreement with STTT was 0.87 (95% confidence interval [CI], .77–.97) using Diasorin assays (almost perfect agreement), 0.80 (95% CI, .68–.93) using GSD assays (substantial agreement) and 0.79 (95% CI, .68–.90) using Zeus assays (substantial agreement). For detection of IgM reactivity, agreement between MTTT and STTT was 0.70 (.51–.90; substantial), 0.63 (95% CI, .44–.82; substantial) and 0.56 (95% CI, .38–.73; moderate), respectively. For detection of IgG reactivity, MTTT/STTT agreement was 0.73 (95% CI,.58–.88), 0.78 (95% CI, .62–.94), and 0.75 (95% CI, .60–.90), respectively (substantial agreement in all cases).
MTTT results obtained using commercial test kits from 3 different manufacturers had substantial to almost perfect agreement with STTT results overall and moderate to substantial agreement for IgM and IgG detection independently. Commercial MTTT tests can be used broadly for the diagnosis of Lyme disease.
Lyme disease is a growing concern with approximately a half million cases diagnosed annually in the United States alone [1]. Laboratory support for the diagnosis usually involves 2-tiered serology performed at 1 or more time points during the course of suspected infection [2]. Standard 2-tiered serologic testing for Lyme disease begins with an automatable, high-throughput immunoassay (AHTI) as the first-tier test; immunoglobulin M (IgM) and immunoglobulin G (IgG) western immunoblots are reflexively added after a reactive (positive or equivocal) first-tier test [3]. Recently, the US Centers for Disease Control (CDC) updated its guidance for the laboratory diagnosis of Lyme disease, stating that a modified 2-tiered testing (MTTT) algorithm—in which a reactive first-tier AHTI is followed by 1 or more second-tier AHTIs, without the use of western immunoblots—is an acceptable alternative to standard 2-tiered testing (STTT), with the stipulation that laboratories should use test kits approved by the US Food and Drug Administration (FDA) specifically for use in MTTT algorithms [4].
In general, MTTT improves clinical sensitivity compared with STTT among patients with early Lyme disease, especially those with a single erythema migrans lesion [5]. Additionally, patients with acute Lyme neuroborreliosis or carditis have higher rates of positive results using MTTT compared with STTT, because MTTT dispenses with the “1-month rule” that is part of STTT interpretive criteria [5]. According to this rule, which is intended to bolster clinical specificity, IgM reactivity alone should not be considered supportive evidence of active Lyme disease if symptoms have been present for greater than 1 month [3]. However, a subset of patients with acute Lyme neuroborreliosis or carditis will meet IgM but not IgG immunoblot criteria up to 6–8 weeks after the onset of symptoms, and STTT may misclassify these patients as seronegative [6].
MTTT has several important practical advantages compared with STTT. First, the second-tier test has lower complexity. Second, results are obtained objectively using instrumented readers rather than making visual comparisons, as in western immunoblotting, which is more subjective and prone to false-positive results caused by over-interpretation of faint bands [7]. Last, costs are lower [8] and turnaround time may be shorter, especially if both first- and second-tier testing can be performed locally without sample transfer to a reference laboratory. However, a barrier to wide-spread adoption has been the lack of FDA-cleared MTTT commercial test kits on the US market.
Three manufacturers offer Lyme disease AHTI serology kits for use in MTTT algorithms: Diasorin, Inc (Stillwater, MN), Gold Standard Diagnostics (Davis, CA), and Zeus Scientific (Scarborough, ME). Although MTTT has been extensively evaluated, performance varies depending on the particular AHTIs used in the algorithm [9]. To this end, we performed a head-to-head clinical evaluation of commercially available tests kits cleared for use in MTTT algorithms using discard samples collected from patients undergoing evaluation for Lyme disease.
METHODS
The study was approved by the Mass General Brigham Institutional Review Board.
Patient Samples
During a previous a study for FDA premarket evaluation of Zeus Scientific MTTT test kits [10], we analyzed 900 consecutive excess serum samples submitted for routine Lyme disease serologic testing to the clinical microbiology laboratory at Massachusetts General Hospital in the summer of 2018. No clinical information was available on the subjects whose samples were included. Although the previous study involved 900 samples, for the current study we performed additional analysis on the first 400 samples, all of which had been collected in the month of July 2018. Among those 400 samples, we excluded 18 due to insufficient quantity to perform the additional research assays. Thus, 382 samples were further analyzed in the current study to obtain additional results.
Serologic Testing
During the previous study, samples were analyzed using Zeus Scientific assays as listed in Table 1. The Zeus Scientific results were previously reported in a separate publication as part of a larger data set [10]. For the current study, we further analyzed a subset (n = 382) of the same samples using Diasorin and subsequently Gold Standard Diagnostics (GSD) assays (Table 1). Those results have not been previously reported. Sample analysis was performed following an MTTT algorithm in which second-tier IgM and IgG AHTIs were performed only when the associated first-tier total antibody (IgM/IgG) AHTI test result was reactive (positive or equivocal). For comparison, an STTT algorithm was applied using the same samples, in which second-tier IgM and IgG western immunoblots were performed only when the first-tier total antibody AHTI was reactive. Immunoblots were interpreted according to CDC criteria, except the 1-month rule was not applied because information about the duration of symptoms was unavailable to the investigators. STTT western immunoblot results reported here were generated in the previous study [10] using Borrelia burgdorferi IgM and IgG Marblot tests (MarDx Diagnostics, Inc, Carlsbad, CA). We do not report in this article the results of STTT or MTTT obtained separately for clinical diagnostic purposes. All commercial assays used in this study were performed according to the manufacturers’ instructions.
Manufacturer . | Assay . | Antibody Class Detected . | Antigens . | Application . |
---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | IgM/IgG | VlsE from B burgdorferi and B. garinii; OspC from B. afzelii | 1st tier test in MTTT or STTT algorithm |
Liaison Lyme IgM CLIA | IgM only | OspC from B. afzelii strain pKo; VlsE from B burgdorferi strain B31 | 2nd tier test in MTTT algorithm | |
Liaison Lyme IgG CLIA | IgG only | VlsE from B burgdorferi strain B31; C6 peptide from B burgdorferi VlsE | 2nd tier test in MTTT algorithm | |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | IgM/IgG | VlsE and OspC from B burgdorferi | 1st tier test in MTTT or STTT algorithm |
Borrelia burgdorferi IgM ELISA | IgM only | Undefined mixture of purified or recombinant protein antigens from B burgdorferi | 2nd tier test in MTTT algorithm | |
Borrelia burgdorferi IgG ELISA | IgG only | Undefined mixture of purified or recombinant protein antigens from B burgdorferi | 2nd tier test in MTTT algorithm | |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | IgM/IgG | VlsE1 and pepC10 from B burgdorferi | 1st tier test in MTTT or STTT algorithm |
Borrelia burgdorferi IgM ELISA | IgM only | Whole-cell antigen from B burgdorferi | 2nd tier test in MTTT algorithm | |
Borrelia burgdorferi IgG ELISA | IgG only | Whole-cell antigen from B burgdorferi | 2nd tier test in MTTT algorithm |
Manufacturer . | Assay . | Antibody Class Detected . | Antigens . | Application . |
---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | IgM/IgG | VlsE from B burgdorferi and B. garinii; OspC from B. afzelii | 1st tier test in MTTT or STTT algorithm |
Liaison Lyme IgM CLIA | IgM only | OspC from B. afzelii strain pKo; VlsE from B burgdorferi strain B31 | 2nd tier test in MTTT algorithm | |
Liaison Lyme IgG CLIA | IgG only | VlsE from B burgdorferi strain B31; C6 peptide from B burgdorferi VlsE | 2nd tier test in MTTT algorithm | |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | IgM/IgG | VlsE and OspC from B burgdorferi | 1st tier test in MTTT or STTT algorithm |
Borrelia burgdorferi IgM ELISA | IgM only | Undefined mixture of purified or recombinant protein antigens from B burgdorferi | 2nd tier test in MTTT algorithm | |
Borrelia burgdorferi IgG ELISA | IgG only | Undefined mixture of purified or recombinant protein antigens from B burgdorferi | 2nd tier test in MTTT algorithm | |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | IgM/IgG | VlsE1 and pepC10 from B burgdorferi | 1st tier test in MTTT or STTT algorithm |
Borrelia burgdorferi IgM ELISA | IgM only | Whole-cell antigen from B burgdorferi | 2nd tier test in MTTT algorithm | |
Borrelia burgdorferi IgG ELISA | IgG only | Whole-cell antigen from B burgdorferi | 2nd tier test in MTTT algorithm |
Abbreviations: CLIA, chemiluminescent immunoassay; IgG, immunoglobulin class G; IgM, immunoglobulin class M; ELISA, enzyme-linked immunosorbent assay; MTTT, modified 2-tiered testing; OspC, outer surface protein C; STTT, standard 2-tiered testing; VlsE, vmp-like sequence, expressed.
Manufacturer . | Assay . | Antibody Class Detected . | Antigens . | Application . |
---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | IgM/IgG | VlsE from B burgdorferi and B. garinii; OspC from B. afzelii | 1st tier test in MTTT or STTT algorithm |
Liaison Lyme IgM CLIA | IgM only | OspC from B. afzelii strain pKo; VlsE from B burgdorferi strain B31 | 2nd tier test in MTTT algorithm | |
Liaison Lyme IgG CLIA | IgG only | VlsE from B burgdorferi strain B31; C6 peptide from B burgdorferi VlsE | 2nd tier test in MTTT algorithm | |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | IgM/IgG | VlsE and OspC from B burgdorferi | 1st tier test in MTTT or STTT algorithm |
Borrelia burgdorferi IgM ELISA | IgM only | Undefined mixture of purified or recombinant protein antigens from B burgdorferi | 2nd tier test in MTTT algorithm | |
Borrelia burgdorferi IgG ELISA | IgG only | Undefined mixture of purified or recombinant protein antigens from B burgdorferi | 2nd tier test in MTTT algorithm | |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | IgM/IgG | VlsE1 and pepC10 from B burgdorferi | 1st tier test in MTTT or STTT algorithm |
Borrelia burgdorferi IgM ELISA | IgM only | Whole-cell antigen from B burgdorferi | 2nd tier test in MTTT algorithm | |
Borrelia burgdorferi IgG ELISA | IgG only | Whole-cell antigen from B burgdorferi | 2nd tier test in MTTT algorithm |
Manufacturer . | Assay . | Antibody Class Detected . | Antigens . | Application . |
---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | IgM/IgG | VlsE from B burgdorferi and B. garinii; OspC from B. afzelii | 1st tier test in MTTT or STTT algorithm |
Liaison Lyme IgM CLIA | IgM only | OspC from B. afzelii strain pKo; VlsE from B burgdorferi strain B31 | 2nd tier test in MTTT algorithm | |
Liaison Lyme IgG CLIA | IgG only | VlsE from B burgdorferi strain B31; C6 peptide from B burgdorferi VlsE | 2nd tier test in MTTT algorithm | |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | IgM/IgG | VlsE and OspC from B burgdorferi | 1st tier test in MTTT or STTT algorithm |
Borrelia burgdorferi IgM ELISA | IgM only | Undefined mixture of purified or recombinant protein antigens from B burgdorferi | 2nd tier test in MTTT algorithm | |
Borrelia burgdorferi IgG ELISA | IgG only | Undefined mixture of purified or recombinant protein antigens from B burgdorferi | 2nd tier test in MTTT algorithm | |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | IgM/IgG | VlsE1 and pepC10 from B burgdorferi | 1st tier test in MTTT or STTT algorithm |
Borrelia burgdorferi IgM ELISA | IgM only | Whole-cell antigen from B burgdorferi | 2nd tier test in MTTT algorithm | |
Borrelia burgdorferi IgG ELISA | IgG only | Whole-cell antigen from B burgdorferi | 2nd tier test in MTTT algorithm |
Abbreviations: CLIA, chemiluminescent immunoassay; IgG, immunoglobulin class G; IgM, immunoglobulin class M; ELISA, enzyme-linked immunosorbent assay; MTTT, modified 2-tiered testing; OspC, outer surface protein C; STTT, standard 2-tiered testing; VlsE, vmp-like sequence, expressed.
Statistical Analysis
Differences in percent agreement were considered statistically significant if the 2-tailed P value was <.05 as determined using Fisher exact test; 95% confidence intervals (CIs) were calculated using the modified Wald method. Agreement between 2-tiered algorithms was measured using Cohen κ statistic. Based on the κ value, the level of agreement was determined according to published standards: 0.81 to 1.00 (almost perfect), 0.61 to 0.80 (substantial), 0.41 to 0.60 (moderate), 0.21 to 0.40 (fair), and 0 to 0.20 (slight) [11]. All analyses were performed using publicly available software (GraphPad QuickCalcs).
RESULTS
Overall Agreement Between MTTT and STTT Algorithms
We compared results obtained using STTT with those obtained using 3 different commercial MTTT methods (Table 2). STTT was performed using the same first-tier AHTI test kit as the MTTT method being compared, with reflex to third-party IgM and IgG western immunoblots. Overall agreement with STTT ranged from “substantial” (Zeus Scientific and GSD assays; κ 0.79 and 0.80 respectively) to “almost perfect” (Diasorin, κ 0.87). Positive percent agreement (PPA) was 100% using Zeus Scientific assays (25/25), 96% using Diasorin assays (22/23), and 87% using GSD assays (20/23) (P = .10 to .48). Small numerical (but nonsignificant) differences were observed in negative percent agreement (NPA): Diasorin assays, 99% NPA (354/359); GSD assays, 98% NPA (353/359); and Zeus Scientific assays, 97% NPA (345/357); P = .09 to 1.0.
Agreement Between MTTT and STTT Algorithms Performed With Commercial Test Kits From Different Manufacturers
MTTT Test Kit Manufacturer . | 1st Tier Test Used in MTTT and Comparator STTT Algorithm . | 2nd Tier Tests Used in MTTT Algorithm . | Positive Percent Agreement Between MTTT and STTT (95% CI) . | Negative Percent Agreement Between MTTT and STTT (95% CI) . | Overall Percent Agreement Between MTTT and STTT (95% CI) . | κ (95% CI) . |
---|---|---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | Liaison Lyme IgM and IgG CLIAs | 96 (77–99) P = .61 vs GSD P = .48 vs Zeus | 99 (97–99.5) P = 1.0 vs GSD P = .09 vs Zeus | 98 (97–99) P = .60 vs GSD P = .23 vs Zeus | 0.87 (.77–.97) |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | Borrelia burgdorferi IgM and IgG ELISAs | 87 (67–96) P = .61 vs Diasorin P = .10 vs Zeus | 98 (96–99) P = 1.0 vs Diasorin P = .16 vs Zeus | 98 (96–99) P = .60 vs Diasorin P = .66 vs Zeus | 0.80 (.68–.93) |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | Borrelia burgdorferi IgM and IgG ELISAs | 100 (84–100) P = .48 vs Diasorin P = .10 vs GSD | 97 (94–98) P = .09 vs Diasorin P = .16 vs GSD | 97 (94–98) P = .23 vs Diasorin P = .66 vs GSD | 0.79 (.68–.90) |
MTTT Test Kit Manufacturer . | 1st Tier Test Used in MTTT and Comparator STTT Algorithm . | 2nd Tier Tests Used in MTTT Algorithm . | Positive Percent Agreement Between MTTT and STTT (95% CI) . | Negative Percent Agreement Between MTTT and STTT (95% CI) . | Overall Percent Agreement Between MTTT and STTT (95% CI) . | κ (95% CI) . |
---|---|---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | Liaison Lyme IgM and IgG CLIAs | 96 (77–99) P = .61 vs GSD P = .48 vs Zeus | 99 (97–99.5) P = 1.0 vs GSD P = .09 vs Zeus | 98 (97–99) P = .60 vs GSD P = .23 vs Zeus | 0.87 (.77–.97) |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | Borrelia burgdorferi IgM and IgG ELISAs | 87 (67–96) P = .61 vs Diasorin P = .10 vs Zeus | 98 (96–99) P = 1.0 vs Diasorin P = .16 vs Zeus | 98 (96–99) P = .60 vs Diasorin P = .66 vs Zeus | 0.80 (.68–.93) |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | Borrelia burgdorferi IgM and IgG ELISAs | 100 (84–100) P = .48 vs Diasorin P = .10 vs GSD | 97 (94–98) P = .09 vs Diasorin P = .16 vs GSD | 97 (94–98) P = .23 vs Diasorin P = .66 vs GSD | 0.79 (.68–.90) |
Abbreviations: CI, confidence interval; CLIA, chemiluminescent immunoassay; ELISA, enzyme-linked immunosorbent assay; GSD, Gold Standard Diagnostics; IgG, immunoglobulin class G; IgM, immunoglobulin class M; MTTT, modified 2-tiered testing; STTT, standard 2-tiered testing.
Agreement Between MTTT and STTT Algorithms Performed With Commercial Test Kits From Different Manufacturers
MTTT Test Kit Manufacturer . | 1st Tier Test Used in MTTT and Comparator STTT Algorithm . | 2nd Tier Tests Used in MTTT Algorithm . | Positive Percent Agreement Between MTTT and STTT (95% CI) . | Negative Percent Agreement Between MTTT and STTT (95% CI) . | Overall Percent Agreement Between MTTT and STTT (95% CI) . | κ (95% CI) . |
---|---|---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | Liaison Lyme IgM and IgG CLIAs | 96 (77–99) P = .61 vs GSD P = .48 vs Zeus | 99 (97–99.5) P = 1.0 vs GSD P = .09 vs Zeus | 98 (97–99) P = .60 vs GSD P = .23 vs Zeus | 0.87 (.77–.97) |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | Borrelia burgdorferi IgM and IgG ELISAs | 87 (67–96) P = .61 vs Diasorin P = .10 vs Zeus | 98 (96–99) P = 1.0 vs Diasorin P = .16 vs Zeus | 98 (96–99) P = .60 vs Diasorin P = .66 vs Zeus | 0.80 (.68–.93) |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | Borrelia burgdorferi IgM and IgG ELISAs | 100 (84–100) P = .48 vs Diasorin P = .10 vs GSD | 97 (94–98) P = .09 vs Diasorin P = .16 vs GSD | 97 (94–98) P = .23 vs Diasorin P = .66 vs GSD | 0.79 (.68–.90) |
MTTT Test Kit Manufacturer . | 1st Tier Test Used in MTTT and Comparator STTT Algorithm . | 2nd Tier Tests Used in MTTT Algorithm . | Positive Percent Agreement Between MTTT and STTT (95% CI) . | Negative Percent Agreement Between MTTT and STTT (95% CI) . | Overall Percent Agreement Between MTTT and STTT (95% CI) . | κ (95% CI) . |
---|---|---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | Liaison Lyme IgM and IgG CLIAs | 96 (77–99) P = .61 vs GSD P = .48 vs Zeus | 99 (97–99.5) P = 1.0 vs GSD P = .09 vs Zeus | 98 (97–99) P = .60 vs GSD P = .23 vs Zeus | 0.87 (.77–.97) |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | Borrelia burgdorferi IgM and IgG ELISAs | 87 (67–96) P = .61 vs Diasorin P = .10 vs Zeus | 98 (96–99) P = 1.0 vs Diasorin P = .16 vs Zeus | 98 (96–99) P = .60 vs Diasorin P = .66 vs Zeus | 0.80 (.68–.93) |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | Borrelia burgdorferi IgM and IgG ELISAs | 100 (84–100) P = .48 vs Diasorin P = .10 vs GSD | 97 (94–98) P = .09 vs Diasorin P = .16 vs GSD | 97 (94–98) P = .23 vs Diasorin P = .66 vs GSD | 0.79 (.68–.90) |
Abbreviations: CI, confidence interval; CLIA, chemiluminescent immunoassay; ELISA, enzyme-linked immunosorbent assay; GSD, Gold Standard Diagnostics; IgG, immunoglobulin class G; IgM, immunoglobulin class M; MTTT, modified 2-tiered testing; STTT, standard 2-tiered testing.
Agreement Between MTTT and STTT Algorithms in Detection of IgM Reactivity
All MTTT and STTT algorithms evaluated in this study used separate IgM and IgG antibody class-specific tests in the second tier, as opposed to a single total antibody test in the second tier, which is an acceptable alternative in MTTT [5]. Although the IgM and IgG second-tier commercial AHTIs are intended to be applied in parallel when used for real-world clinical diagnostic testing, for evaluative purposes we also generated results using antibody class-specific MTTT and STTT algorithms and determined agreement between them (Table 3). For example, an IgM class-specific MTTT algorithm (MTTT-IgM) starts with a total antibody AHTI and reflexes to an IgM-specific AHTI only; an IgM class-specific STTT algorithm (STTT-IgM) starts with a total antibody AHTI and reflexes to IgM-specific western immunoblot only. By analyzing results this way, agreement in detection of IgM or IgG reactivity can be assessed independently.
Agreement Between MTTT-IgM and STTT-IgM Algorithms Performed With Commercial Test Kits From Different Manufacturers
MTTT Test Kit Manufacturer . | 1st Tier Test Used in MTTT-IgM and Comparator STTT-IgM Algorithm . | 2nd Tier Test Used in MTTT-IgM Algorithm . | Positive Percent Agreement Between MTTT-IgM and STTT-IgM (95% CI) . | Negative Percent Agreement Between MTTT-IgM and STTT-IgM (95% CI) . | Overall Percent Agreement Between MTTT-IgM and STTT-IgM (95% CI) . | κ (95% CI) . |
---|---|---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | Liaison Lyme IgM CLIA | 91 (60 to > 99) P = 1.0 vs GSD P = .46 vs Zeus | 98 (96–99) P = .35 vs GSD P = .02 vs Zeus | 98 (96–99) P = .50 vs GSD P = .05 vs Zeus | 0.70 (.51–90) |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | Borrelia burgdorferi IgM ELISA | 100 (70–100) P = 1.0 vs Diasorin P = 1.0 vs Zeus | 97 (94–98) P = .35 vs Diasorin P = .20 vs Zeus | 98 (96–99) P = .50 vs Diasorin P = .27 vs Zeus | 0.63 (.44–.82) |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | Borrelia burgdorferi IgM ELISA | 100 (73–100) P = .46 vs Diasorin P = 1.0 vs GSD | 95 (92–97) P = .02 vs Diasorin P = .20 vs GSD | 95 (92–97) P = .05 vs Diasorin P = .27 vs GSD | 0.56 (.38–.73) |
MTTT Test Kit Manufacturer . | 1st Tier Test Used in MTTT-IgM and Comparator STTT-IgM Algorithm . | 2nd Tier Test Used in MTTT-IgM Algorithm . | Positive Percent Agreement Between MTTT-IgM and STTT-IgM (95% CI) . | Negative Percent Agreement Between MTTT-IgM and STTT-IgM (95% CI) . | Overall Percent Agreement Between MTTT-IgM and STTT-IgM (95% CI) . | κ (95% CI) . |
---|---|---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | Liaison Lyme IgM CLIA | 91 (60 to > 99) P = 1.0 vs GSD P = .46 vs Zeus | 98 (96–99) P = .35 vs GSD P = .02 vs Zeus | 98 (96–99) P = .50 vs GSD P = .05 vs Zeus | 0.70 (.51–90) |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | Borrelia burgdorferi IgM ELISA | 100 (70–100) P = 1.0 vs Diasorin P = 1.0 vs Zeus | 97 (94–98) P = .35 vs Diasorin P = .20 vs Zeus | 98 (96–99) P = .50 vs Diasorin P = .27 vs Zeus | 0.63 (.44–.82) |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | Borrelia burgdorferi IgM ELISA | 100 (73–100) P = .46 vs Diasorin P = 1.0 vs GSD | 95 (92–97) P = .02 vs Diasorin P = .20 vs GSD | 95 (92–97) P = .05 vs Diasorin P = .27 vs GSD | 0.56 (.38–.73) |
Abbreviations: CI, confidence interval; CLIA, chemiluminescent immunoassay; ELISA, enzyme-linked immunosorbent assay; IgM, immunoglobulin class M; MTTT, modified 2-tiered testing; STTT, standard 2-tiered testing.
Agreement Between MTTT-IgM and STTT-IgM Algorithms Performed With Commercial Test Kits From Different Manufacturers
MTTT Test Kit Manufacturer . | 1st Tier Test Used in MTTT-IgM and Comparator STTT-IgM Algorithm . | 2nd Tier Test Used in MTTT-IgM Algorithm . | Positive Percent Agreement Between MTTT-IgM and STTT-IgM (95% CI) . | Negative Percent Agreement Between MTTT-IgM and STTT-IgM (95% CI) . | Overall Percent Agreement Between MTTT-IgM and STTT-IgM (95% CI) . | κ (95% CI) . |
---|---|---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | Liaison Lyme IgM CLIA | 91 (60 to > 99) P = 1.0 vs GSD P = .46 vs Zeus | 98 (96–99) P = .35 vs GSD P = .02 vs Zeus | 98 (96–99) P = .50 vs GSD P = .05 vs Zeus | 0.70 (.51–90) |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | Borrelia burgdorferi IgM ELISA | 100 (70–100) P = 1.0 vs Diasorin P = 1.0 vs Zeus | 97 (94–98) P = .35 vs Diasorin P = .20 vs Zeus | 98 (96–99) P = .50 vs Diasorin P = .27 vs Zeus | 0.63 (.44–.82) |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | Borrelia burgdorferi IgM ELISA | 100 (73–100) P = .46 vs Diasorin P = 1.0 vs GSD | 95 (92–97) P = .02 vs Diasorin P = .20 vs GSD | 95 (92–97) P = .05 vs Diasorin P = .27 vs GSD | 0.56 (.38–.73) |
MTTT Test Kit Manufacturer . | 1st Tier Test Used in MTTT-IgM and Comparator STTT-IgM Algorithm . | 2nd Tier Test Used in MTTT-IgM Algorithm . | Positive Percent Agreement Between MTTT-IgM and STTT-IgM (95% CI) . | Negative Percent Agreement Between MTTT-IgM and STTT-IgM (95% CI) . | Overall Percent Agreement Between MTTT-IgM and STTT-IgM (95% CI) . | κ (95% CI) . |
---|---|---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | Liaison Lyme IgM CLIA | 91 (60 to > 99) P = 1.0 vs GSD P = .46 vs Zeus | 98 (96–99) P = .35 vs GSD P = .02 vs Zeus | 98 (96–99) P = .50 vs GSD P = .05 vs Zeus | 0.70 (.51–90) |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | Borrelia burgdorferi IgM ELISA | 100 (70–100) P = 1.0 vs Diasorin P = 1.0 vs Zeus | 97 (94–98) P = .35 vs Diasorin P = .20 vs Zeus | 98 (96–99) P = .50 vs Diasorin P = .27 vs Zeus | 0.63 (.44–.82) |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | Borrelia burgdorferi IgM ELISA | 100 (73–100) P = .46 vs Diasorin P = 1.0 vs GSD | 95 (92–97) P = .02 vs Diasorin P = .20 vs GSD | 95 (92–97) P = .05 vs Diasorin P = .27 vs GSD | 0.56 (.38–.73) |
Abbreviations: CI, confidence interval; CLIA, chemiluminescent immunoassay; ELISA, enzyme-linked immunosorbent assay; IgM, immunoglobulin class M; MTTT, modified 2-tiered testing; STTT, standard 2-tiered testing.
Results obtained using the Zeus MTTT-IgM algorithm were in “moderate” agreement with STTT-IgM using the same first-tier total antibody AHTI followed by an IgM-specific western immunoblot (κ 0.56). The Diasorin and GSD MTTT-IgM algorithms were in “substantial” agreement with STTT-IgM (κ 0.70 and 0.63, respectively). PPA was similar across the 3 MTTT-IgM algorithms when each was compared with STTT-IgM (91% to 100%; P = .46 to 1.0). NPA with STTT-IgM differed significantly between the Diasorin MTTT-IgM algorithm (98%) and the Zeus MTTT-IgM algorithm (95%; P = .02). NPA with STTT-IgM was 97% using the GSD MTTT-IgM algorithm, which was not significantly different compared with Diasorin MTTT-IgM (98%, P = .35) or Zeus MTTT-IgM (95%, P = .20).
Agreement Between MTTT and STTT Algorithms in Detection of IgG Reactivity
In this analysis, we compared IgG class-specific 2-tiered algorithms: MTTT-IgG, which starts with a total antibody AHTI and reflexes to an IgG-specific AHTI only, and STTT-IgG, which starts with the same total antibody AHTI and reflexes to an IgG-specific western immunoblot only. Using any of the 3 manufacturers’ MTTT-IgG method, results were in “substantial” overall agreement with STTT-IgG (κ 0.73–0.78; Table 4). PPA between MTTT-IgG and STTT-IgG methods ranged from 68% to 84%, but the differences were nonsignificant (P = .45 to 1.0). The GSD MTTT-IgG method produced the highest NPA compared with STTT-IgG (99.7%), which was significantly different from results obtained using the Diasorin MTTT-IgG method (98%, P = .04) but not the Zeus MTTT-IgG method (P = .07).
Agreement Between MTTT-IgG and STTT-IgG Algorithms Performed With Commercial Test Kits From Different Manufacturers
MTTT Test Kit Manufacturer . | 1st Tier Test Used in MTTT-IgG and Comparator STTT-IgG Algorithm . | 2nd Tier Test Used in MTTT-IgG Algorithm . | Positive Percent Agreement Between MTTT-IgG and STTT-IgG (95% CI) . | Negative Percent Agreement Between MTTT-IgG and STTT-IgG (95% CI) . | Overall Percent Agreement Between MTTT-IgG and STTT-IgG (95% CI) . | κ (95% CI) . |
---|---|---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | Liaison Lyme IgG CLIA | 84 (62–95) P = .45 vs GSD P = 1.0 vs Zeus | 98 (96–99) P = .04 vs GSD P = .79 vs Zeus | 97 (95–98) P = .48 vs GSD P = 1.0 vs Zeus | 0.73 (.58–.88) |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | Borrelia burgdorferi IgG ELISA | 68 (46–85) P = .45 vs Diasorin P = .48 vs Zeus | 99.7 (98 – > 99) P = .04 vs Diasorin P = .07 vs Zeus | 98 (96–99) P = .48 vs Diasorin P = .63 vs Zeus | 0.78 (.62–.94) |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | Borrelia burgdorferi IgG ELISA | 80 (58–93) P = 1.0 vs Diasorin P = .48 vs GSD | 98 (96–99) P = .79 vs Diasorin P = .07 vs GSD | 97 (95–99) P = 1.0 vs Diasorin P = .63 vs GSD | 0.75 (.60–.90) |
MTTT Test Kit Manufacturer . | 1st Tier Test Used in MTTT-IgG and Comparator STTT-IgG Algorithm . | 2nd Tier Test Used in MTTT-IgG Algorithm . | Positive Percent Agreement Between MTTT-IgG and STTT-IgG (95% CI) . | Negative Percent Agreement Between MTTT-IgG and STTT-IgG (95% CI) . | Overall Percent Agreement Between MTTT-IgG and STTT-IgG (95% CI) . | κ (95% CI) . |
---|---|---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | Liaison Lyme IgG CLIA | 84 (62–95) P = .45 vs GSD P = 1.0 vs Zeus | 98 (96–99) P = .04 vs GSD P = .79 vs Zeus | 97 (95–98) P = .48 vs GSD P = 1.0 vs Zeus | 0.73 (.58–.88) |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | Borrelia burgdorferi IgG ELISA | 68 (46–85) P = .45 vs Diasorin P = .48 vs Zeus | 99.7 (98 – > 99) P = .04 vs Diasorin P = .07 vs Zeus | 98 (96–99) P = .48 vs Diasorin P = .63 vs Zeus | 0.78 (.62–.94) |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | Borrelia burgdorferi IgG ELISA | 80 (58–93) P = 1.0 vs Diasorin P = .48 vs GSD | 98 (96–99) P = .79 vs Diasorin P = .07 vs GSD | 97 (95–99) P = 1.0 vs Diasorin P = .63 vs GSD | 0.75 (.60–.90) |
Abbreviations: CI, confidence interval; CLIA, chemiluminescent immunoassay; ELISA, enzyme-linked immunosorbent assay; IgG, immunoglobulin class G; MTTT, modified 2-tiered testing; STTT, standard 2-tiered testing.
Agreement Between MTTT-IgG and STTT-IgG Algorithms Performed With Commercial Test Kits From Different Manufacturers
MTTT Test Kit Manufacturer . | 1st Tier Test Used in MTTT-IgG and Comparator STTT-IgG Algorithm . | 2nd Tier Test Used in MTTT-IgG Algorithm . | Positive Percent Agreement Between MTTT-IgG and STTT-IgG (95% CI) . | Negative Percent Agreement Between MTTT-IgG and STTT-IgG (95% CI) . | Overall Percent Agreement Between MTTT-IgG and STTT-IgG (95% CI) . | κ (95% CI) . |
---|---|---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | Liaison Lyme IgG CLIA | 84 (62–95) P = .45 vs GSD P = 1.0 vs Zeus | 98 (96–99) P = .04 vs GSD P = .79 vs Zeus | 97 (95–98) P = .48 vs GSD P = 1.0 vs Zeus | 0.73 (.58–.88) |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | Borrelia burgdorferi IgG ELISA | 68 (46–85) P = .45 vs Diasorin P = .48 vs Zeus | 99.7 (98 – > 99) P = .04 vs Diasorin P = .07 vs Zeus | 98 (96–99) P = .48 vs Diasorin P = .63 vs Zeus | 0.78 (.62–.94) |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | Borrelia burgdorferi IgG ELISA | 80 (58–93) P = 1.0 vs Diasorin P = .48 vs GSD | 98 (96–99) P = .79 vs Diasorin P = .07 vs GSD | 97 (95–99) P = 1.0 vs Diasorin P = .63 vs GSD | 0.75 (.60–.90) |
MTTT Test Kit Manufacturer . | 1st Tier Test Used in MTTT-IgG and Comparator STTT-IgG Algorithm . | 2nd Tier Test Used in MTTT-IgG Algorithm . | Positive Percent Agreement Between MTTT-IgG and STTT-IgG (95% CI) . | Negative Percent Agreement Between MTTT-IgG and STTT-IgG (95% CI) . | Overall Percent Agreement Between MTTT-IgG and STTT-IgG (95% CI) . | κ (95% CI) . |
---|---|---|---|---|---|---|
Diasorin | Liaison Lyme Total Antibody Plus CLIA | Liaison Lyme IgG CLIA | 84 (62–95) P = .45 vs GSD P = 1.0 vs Zeus | 98 (96–99) P = .04 vs GSD P = .79 vs Zeus | 97 (95–98) P = .48 vs GSD P = 1.0 vs Zeus | 0.73 (.58–.88) |
Gold Standard Diagnostics | Borrelia burgdorferi VlsE-OspC IgG/IgM ELISA | Borrelia burgdorferi IgG ELISA | 68 (46–85) P = .45 vs Diasorin P = .48 vs Zeus | 99.7 (98 – > 99) P = .04 vs Diasorin P = .07 vs Zeus | 98 (96–99) P = .48 vs Diasorin P = .63 vs Zeus | 0.78 (.62–.94) |
Zeus Scientific | Borrelia VlsE1/pepC10 lgG/IgM ELISA | Borrelia burgdorferi IgG ELISA | 80 (58–93) P = 1.0 vs Diasorin P = .48 vs GSD | 98 (96–99) P = .79 vs Diasorin P = .07 vs GSD | 97 (95–99) P = 1.0 vs Diasorin P = .63 vs GSD | 0.75 (.60–.90) |
Abbreviations: CI, confidence interval; CLIA, chemiluminescent immunoassay; ELISA, enzyme-linked immunosorbent assay; IgG, immunoglobulin class G; MTTT, modified 2-tiered testing; STTT, standard 2-tiered testing.
DISCUSSION
We evaluated the performance of commercial Lyme disease serologic test methods that have been cleared by FDA for use in modified 2-tiered testing (MTTT) algorithms by comparing with STTT results obtained using the same samples. Although one version of MTTT involves the use of a total antibody (polyvalent) AHTI in both the first and second tier of testing [5], we focused our study on an alternative version, which begins with a total antibody AHTI but reflexes to separate IgM- and IgG-specific AHTIs when the first-tier test is reactive (positive or equivocal) [5]. This immunoglobulin class-differentiating MTTT approach mirrors the reference method, known as standard 2-tiered testing (STTT), in which a screening AHTI is followed by separate IgM- and IgG-specific western immunoblots. We also use an immunoglobulin class-differentiating MTTT algorithm in our clinical laboratory because we believe that diagnostic accuracy can be improved by correlating clinical features and estimated duration of illness with the maturity of the antibody response, as roughly indicated by the presence or absence of IgM to IgG class switching.
We used all currently available commercial test kits intended for immunoglobulin class-differentiating MTTT (manufactured by Diasorin, Gold Standard Diagnostics, and Zeus Scientific) to perform MTTT using 382 remnant serum samples that were collected during peak Lyme disease season and submitted for routine clinical Lyme disease testing. As would be expected from FDA-cleared assays with the same intended use, the commercial MTTT methods performed comparably to each other and to the reference method (STTT). Overall agreement between MTTT and STTT was “substantial” (Zeus Scientific and GSD) or “almost perfect” (Diasorin) based on κ coefficient values, which had overlapping 95% CIs. There were also no significant differences between MTTT methods in their PPA or NPA with STTT.
In our subanalysis, we measured agreement between MTTT and STTT methods in detection of IgM- or IgG-specific reactivity. Regarding IgM reactivity, κ coefficient values for overall agreement were similar across commercial MTTT platforms, with overlapping 95% CIs, although the Zeus Scientific value was formally classified as demonstrating “moderate” agreement while the others fell into the “substantial” agreement category. NPA in IgM reactivity between MTTT and STTT methods was significantly different between the platform with lowest NPA (Zeus Scientific, 95%) and highest NPA (Diasorin, 98%; P = .02), whereas other differences in IgM-specific NPA were nonsignificant and there were no significant differences in IgM-specific PPA. Regarding IgG-specific reactivity, all the commercial MTTT methods were in “substantial” overall agreement with STTT based on the κ statistic. We found no significant differences in PPA across MTTT platforms when IgG-specific MTTT results were compared with IgG-specific STTT results. However, the difference between the highest measured IgG-specific NPA (99.7%, obtained using Gold Standard Diagnostics test kits) and the lowest (98%, obtained using Diasorin test kits) was significant (P = .04).
Thus, significant differences in performance between the 3 commercial MTTT methods were limited to certain IgM- or IgG-specific NPA results, and these differences offset one another such that overall (not immunoglobulin class-specific) NPA was comparable, as was overall (positive and negative) agreement. Notably, imperfect negative agreement between the MTTT methods and the reference method (STTT) was expected, as MTTT is more sensitive compared with STTT in early infection [5, 6, 9, 10, 12–19] and we focused the study on patients with suspected Lyme disease who presented during peak season for early-stage infection. Because overall performance was comparable between the 3 manufacturer's methods, the choice between them may come down to practical considerations such as instrument preference, procedural complexity, or cost.
In our view, STTT methods should remain available alongside MTTT when needed. MTTT is more sensitive compared with STTT in erythema migrans cases and can usually be performed with shorter turnaround time and less expense. However, STTT provides some information in specialized cases that MTTT does not. By individually and separately assessing serum reactivity to a number of individual B burgdorferi antigens, via western immunoblotting, STTT can provide more detailed information about the extent and maturity of the antibody response than MTTT would. We believe that diagnostic accuracy can be enhanced by correlating this information with disease manifestations of the individual patient, especially in complex or atypical suspected cases of Lyme disease (other than atypical erythema migrans lesions).
This study has several additional limitations. The archived, frozen serum samples were analyzed iteratively using 3 commercial MTTT systems, with a freeze-thaw cycle occurring each time. Although suboptimal, in our experience this limited amount (3 iterations) of freeze-thaw cycles does not appreciably affect reproducibility. Another limitation is that the study's sample size was not large enough to detect small differences in performance between the MTTT methods with statistical validity. A third limitation is that we used discarded clinical samples and did not have access to clinical information about the patients. This limits the data analysis to agreement between MTTT methods and the reference method (STTT) but prevents calculation of clinical sensitivity and specificity. Furthermore, it means that we could not apply the 1-month rule when interpreting immunoblot results for STTT, because information about the duration of symptoms was unavailable to the investigators. Thus, it is possible that some STTT-positive cases with a positive IgM immunoblot alone may be falsely positive.
In conclusion, we performed a real-world clinical laboratory assessment of modified 2-tiered serologic testing for Lyme disease using FDA-cleared commercial test methods. We demonstrated that results obtained using these methods correlate well with those obtained using the reference method, standard 2-tiered testing with western immunoblots, including immunoglobulin class (IgM or IgG) differentiation. Commercial MTTT methods are clinically valid, and ideally both modified and standard 2-tiered testing algorithms should be available for clinicians to choose depending on the individual clinical circumstances.
Notes
Acknowledgment. We are grateful to Vedrana Eleta, Dina Ryan, Kathleen Cichonski, Danielle Accardi, Julius Torres, Sarah Ma, and Christine Keselica for technical assistance.
Author contributions. J. A. B., E. L. L., and S. E. T. conceived the study. J. A. B. supervised the study and prepared the manuscript. All authors assisted with data analysis and interpretation, and edited, reviewed, and approved the manuscript.
Financial support. This study includes data generated during a previous study that was funded by Zeus Scientific, Inc. Supplies and reagents for sample analysis were provided by the test kit manufacturers at no charge.
Supplement sponsorship. This article appears as part of the supplement “Lyme Disease,” sponsored by the Lyme Disease Program at Massachusetts General Hospital and a generous gift from the Morse family.
Potential conflicts of interest. J. A. B. has received research funding from Analog Devices, Inc, Zeus Scientific, Inc., Immunetics, Pfizer, DiaSorin, and bioMerieux; and has been a paid consultant to T2 Biosystems, Diasorin, Roche Diagnostics, Flightpath Biosciences, and Tarsus Pharmaceuticals. S. E. T. has received royalties from UpToDate and grant support from the Centers for Disease Control and Prevention. All other authors report no potential conflicts.
All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
References
Author notes
Presented in part: Symposium in honor of Allen C. Steere, MD, Massachusetts General Hospital, Boston, MA, 26 October 2023.