Context:

Laboratories are increasingly shifting to new automated 25-hydroxyvitamin D (25-OHD) assays, with subsequent variability in results.

Objective/Setting:

We describe the experience at our center with such a shift and illustrate its clinical implications.

Methods:

25-OHD levels were measured in 494 patients using Immunodiagnostic Systems RIA (IDS-RIA) and DiaSorin Liaison assays. Sources of variability between the assays were investigated in a subset of 83 samples, retested in the reference laboratory in the United States, and by reviewing the performance reports issued by the International Vitamin D External Quality Assessment Scheme, DEQAS. 25-OHD cut-points for target levels were used to compare the two assays.

Results:

25-OHD concentrations were significantly lower when measured with Liaison as compared to IDS-RIA: mean bias was −5 ng/ml, range was −38.1 to 18.7 ng/ml, P < 0.001; the absolute bias was independent of 25-OHD value. Interassay variability was also detected in values obtained in the reference laboratory and in DEQAS reports. Using 20 ng/ml as the target 25-OHD level, 52% of patients required treatment when tested by Liaison, as opposed to 36% by IDS-RIA (P < 0.001). Using 30 ng/ml as the desirable level, the proportions were 79 and 64%, respectively (P < 0.001). The two assays agreed in only 41–68% of subjects, proportions that depended on criteria used to define agreement.

Conclusion:

A change in 25-OHD assays has a significant impact on results, patient classification, and treatment recommendations. Such variability cannot be ignored when deriving and applying vitamin D guidelines. It also renders universal assay standardization a pressing call.

Approximately 1 billion people worldwide are suspected to have vitamin D deficiency or insufficiency (1). Subjects in the Middle East have some of the lowest values of 25-hydroxyvitamin D (25-OHD) in the world (2) and do so across the life cycle (26). The impact of vitamin D deficiency on musculoskeletal health is well known (1, 2), and hypovitaminosis D has been associated with cancer, cardiovascular disease, diabetes, and autoimmune disorders, but the evidence as to causality is still inconclusive (1, 2).

Serum 25-OHD is a precursor of the active form, calcitriol, and is the more fat-soluble moiety of the two. Its serum level is therefore the most useful marker of vitamin D nutritional status, reflecting endogenous synthesis and dietary intake (7), and is widely used clinically to guide replacement needs. Controversy exists as to the appropriate level of circulating 25-OHD (8). Recent recommendations include a target above 20 ng/ml (50 nmol/liter) by the Institute of Medicine (7) and of 30 ng/ml, by the International Osteoporosis Foundation (9) and The Endocrine Society 2011 Clinical Practice Guidelines (10).

Several assays are available for the measurement of 25-OHD levels, but significant variability between different assays and laboratories has been reported (11, 12). The choice between assays differs between countries and between different laboratories, which may reflect impact not only of geographic variations but also of inherent assay variations, thus limiting comparisons across populations. This is further compounded by the need for time and volume efficiency in running 25-OHD, leading many institutions worldwide to shift to the more rapid automated techniques available (11).

The impact of this variability between assays is that single universal thresholds might not be adequate to define vitamin D status. Furthermore, results from a certain laboratory might not reflect the actual 25-OHD level of a patient, which would result in under- or overreplacement (12).

In this paper, we systematically quantify the sources of variability in 25-OHD measurements that occurred during the shift in assays from a traditional RIA to a more rapid automated assay at the American University of Beirut-Medical Center (AUB-MC), a tertiary care center in Lebanon. We also illustrate the clinical and research implications of such variability.

Subjects and Methods

Study population and 25-OHD samples

An empirical number of 494 samples were randomly chosen among samples received for 25-OHD determination at AUB-MC in the period of spring to summer 2010. The population tested consisted mainly of outpatients (97%), mostly adults (97%), of female gender (75%). Samples were concurrently measured using Immunodiagnostic Systems RIA (IDS-RIA) (IDS, Boldon, UK) and DiaSorin Liaison Total rapid automated assay (DiaSorin, Stillwater, MN).

After noting the difference in measurements between the two assays run at our center and to check for interassay (RIA vs. Liaison), interlaboratory (AUB-MC vs. U.S. laboratories), and intermanufacturer (IDS vs. DiaSorin) variability, a subset of 83 samples (the most recent batch of samples) out of the initial 494 was sent to DiaSorin in Stillwater, Minnesota, to be retested.

Assays used, laboratory certification, and quality control

The Endocrinology Core Laboratory of the Department of Internal Medicine assayed samples for 25-OHD using IDS-RIA, whereas the Clinical Chemistry Laboratory of the Department of Pathology and Laboratory Medicine assayed samples using DiaSorin Liaison.

Our institution has been a participant of the Vitamin D External Quality Assessment Scheme (DEQAS) for several years (www.deqas.org). DEQAS evaluates the performance of participating laboratories quarterly (13) (Supplemental Data, published on The Endocrine Society's Journals Online web site at http://jcem.endojournals.org). The DEQAS performance of both assay methods during the transition period (July 2010 to January 2011) was also examined, providing additional information on the systematic change noted in 25-OHD values.

The Clinical Chemistry Laboratory also receives quality assurance, evaluation, and accreditation from the College of American Pathologists (CAP) (www.cap.org).

25-OHD values are reported in nanograms per milliliter. To convert to nanomoles per liter (SI units), multiply by 2.496.

Assay methodology

IDS-RIA

This is a liquid phase RIA where 25-OHD concentration is inversely proportional to the bound radioactivity. The manufacturer reports an analytical range of 1.6–160.0 ng/ml, an intraassay precision [coefficient of variation (CV)] of 5.0–6.1% for values between 10.6 and 60.4 ng/ml, and interassay precision (CV) of 7.3–8.2% for values between 7.84 and 54.4 ng/ml. Cross-reactivity is reported to be 75% to 25-OHD2 and 100% to 25-OHD3 (www.idsplc.com). In our laboratory, using a sample of 10 patients, the intraassay precision (CV) (assayed in duplicates) is 4.62 ± 2.9%, and the interassay precision is 5.9 ± 5.9%.

DiaSorin Liaison

DiaSorin Liaison uses chemiluminescent immunoassay. 25-OHD concentration is inversely proportional to the obtained chemiluminescent signal. The manufacturer reports an analytical range of 4–150 ng/ml, an intraassay variability of 7.7–12.7% for values between 5.8 and 35 ng/ml, and an interassay variability of 11.6–25% for values between 5.8 and 35 ng/ml. Cross-reactivity is reported by the manufacturer to be 104% to 25-OHD2 and 100% to 25-OHD3 (www.diasorin.com). However, in our laboratory, it is estimated to be 90% for 25-OHD3 and 67% for 25-OHD2, based on the CAP Accuracy-Based Vitamin D survey 2011. In our laboratory, the intraassay precision (CV) for duplicate analysis is 4.61 ± 3.2%, and the interassay precision (CV) is 9.68 ± 5.35%, based on a sample of 30 patients. Interassay precision using manufacturer's quality control samples shows precision of 10.2% and 7.8% for mean 25-OHD concentrations of 15.3 and 53.7 ng/ml, respectively (n = 55).

Statistical analysis

Continuous variables were checked for normality using Shapiro-Wilk test and were summarized using means and sd, or medians with range as applicable. Variability between RIA and Liaison was evaluated through the CV = 100*sd/mean. Difference in the mean 25-OHD levels was tested using paired t test/Wilcoxon signed rank test as applicable, or Mann-Whitney rank sum test. Measurements of these assays were regressed against each other, and a linear regression model fit was generated accordingly. Subjects were further categorized according to their 25-OHD levels, and differences in their distribution using the two assays were tested using χ2 test. Weighted Cohen's κ coefficient, measuring agreement between the two assays, was also calculated with its 95% confidence interval. Bland-Altman plots of the difference in 25-OHD levels between both assays (or their percentage difference) vs. their average [IDS-RIA + Liaison)/2] in all 494 subjects were generated. The trend of moving average of difference was also calculated based on ordered sub-cohorts (n = 50 each) with 50% overlapping. Analysis and model fitting were done using SigmaPlot 11.0 software (Systat Software Inc., San Jose, CA) and Stata 9 software (StataCorp, College Station, TX). P < 0.05 was considered significant.

Results

Comparison between IDS-RIA and DiaSorin Liaison assays

Full cohort (n = 494)

Serum 25-OHD levels obtained on 494 patients using DiaSorin-Liaison (mean, 21.3 ± 11.8 ng/ml) were systematically and significantly different from the corresponding levels obtained with IDS-RIA (mean, 26.3 ± 12.7 ng/ml; P < 0.001), with an interassay CV of 20.6 ± 14.2%. Relevant descriptive statistics of values obtained with each assay and differences between the two are summarized in Table 1. The systematic negative bias between the two assays was reflected in a leftward shift of the DiaSorin Liaison curve in relation to the IDS-RIA curve (Fig. 1, A and B). The absolute bias averaged 5 ng/ml and was fairly independent of the mean 25-OHD value (Figs. 1B and 2A), but increased when evaluated as percentage difference vs. mean 25-OHD level (Fig. 2B).

Table 1.

Summary of 25-OHD levels obtained on the full cohort at AUB-MC Lebanon and on the sub-cohorts at AUB-MC and Stillwater-USA, using both assays

TestnMeansdMedianMinimumMaximumStatistically different from*
AUB-MC Lebanon (FC)       
    IDS-RIA (ng/ml)a 494 26.3 12.7 25.0 5.4 86.2 b 
    DiaSorin Liaison (ng/ml)b 494 21.3 11.8 19.6 4.0 91.5 a 
    Difference (ng/ml)b − a 494 −5.0 5.9 −4.5 −38.1 18.7  
    Difference (%) 494 −25.1 25.1 −24.6 −98.7 92.8  
AUB-MC Lebanon (SC)       
    IDS-RIA (ng/ml)c 83 26.4 12.4 24.0 7.4 74.8 d, e, f 
    DiaSorin Liaison (ng/ml)d 83 20.1 11.3 18.8 4.0 57.0 c, e, f 
    Difference (ng/ml)d − c 83 −6.3 7.0 −5.6 −27.2 32.6  
    Difference (%) 83 −32.7 29.4 −31.1 −103.0 82.1  
Stillwater-USA (SC)       
    DiaSorin RIA (ng/ml)e 83 21.7 9.1 20.5 5.0 45.6 c, d 
    DiaSorin Liaison (ng/ml)f 83 21.4 11.8 20.1 1.6 58.9 c, d 
    Difference (ng/ml)f − e 83 −0.3 5.4 −1.1 −17.4 31.7  
    Difference (%) 83 −9.1 26.5 −4.9 −101.9 88.1  
TestnMeansdMedianMinimumMaximumStatistically different from*
AUB-MC Lebanon (FC)       
    IDS-RIA (ng/ml)a 494 26.3 12.7 25.0 5.4 86.2 b 
    DiaSorin Liaison (ng/ml)b 494 21.3 11.8 19.6 4.0 91.5 a 
    Difference (ng/ml)b − a 494 −5.0 5.9 −4.5 −38.1 18.7  
    Difference (%) 494 −25.1 25.1 −24.6 −98.7 92.8  
AUB-MC Lebanon (SC)       
    IDS-RIA (ng/ml)c 83 26.4 12.4 24.0 7.4 74.8 d, e, f 
    DiaSorin Liaison (ng/ml)d 83 20.1 11.3 18.8 4.0 57.0 c, e, f 
    Difference (ng/ml)d − c 83 −6.3 7.0 −5.6 −27.2 32.6  
    Difference (%) 83 −32.7 29.4 −31.1 −103.0 82.1  
Stillwater-USA (SC)       
    DiaSorin RIA (ng/ml)e 83 21.7 9.1 20.5 5.0 45.6 c, d 
    DiaSorin Liaison (ng/ml)f 83 21.4 11.8 20.1 1.6 58.9 c, d 
    Difference (ng/ml)f − e 83 −0.3 5.4 −1.1 −17.4 31.7  
    Difference (%) 83 −9.1 26.5 −4.9 −101.9 88.1  
*

Letters indicate significant statistical difference (P < 0.05) compared to specific assay/cohort:

a

, IDS-RIA (FC);

b

, DiaSorin Liaison (FC);

c

, IDS-RIA (SC);

d

, DiaSorin Liaison (SC);

e

, DiaSorin RIA;

f

, DiaSorin Liaison. FC, Full cohort; SC, sub-cohort.

Table 1.

Summary of 25-OHD levels obtained on the full cohort at AUB-MC Lebanon and on the sub-cohorts at AUB-MC and Stillwater-USA, using both assays

TestnMeansdMedianMinimumMaximumStatistically different from*
AUB-MC Lebanon (FC)       
    IDS-RIA (ng/ml)a 494 26.3 12.7 25.0 5.4 86.2 b 
    DiaSorin Liaison (ng/ml)b 494 21.3 11.8 19.6 4.0 91.5 a 
    Difference (ng/ml)b − a 494 −5.0 5.9 −4.5 −38.1 18.7  
    Difference (%) 494 −25.1 25.1 −24.6 −98.7 92.8  
AUB-MC Lebanon (SC)       
    IDS-RIA (ng/ml)c 83 26.4 12.4 24.0 7.4 74.8 d, e, f 
    DiaSorin Liaison (ng/ml)d 83 20.1 11.3 18.8 4.0 57.0 c, e, f 
    Difference (ng/ml)d − c 83 −6.3 7.0 −5.6 −27.2 32.6  
    Difference (%) 83 −32.7 29.4 −31.1 −103.0 82.1  
Stillwater-USA (SC)       
    DiaSorin RIA (ng/ml)e 83 21.7 9.1 20.5 5.0 45.6 c, d 
    DiaSorin Liaison (ng/ml)f 83 21.4 11.8 20.1 1.6 58.9 c, d 
    Difference (ng/ml)f − e 83 −0.3 5.4 −1.1 −17.4 31.7  
    Difference (%) 83 −9.1 26.5 −4.9 −101.9 88.1  
TestnMeansdMedianMinimumMaximumStatistically different from*
AUB-MC Lebanon (FC)       
    IDS-RIA (ng/ml)a 494 26.3 12.7 25.0 5.4 86.2 b 
    DiaSorin Liaison (ng/ml)b 494 21.3 11.8 19.6 4.0 91.5 a 
    Difference (ng/ml)b − a 494 −5.0 5.9 −4.5 −38.1 18.7  
    Difference (%) 494 −25.1 25.1 −24.6 −98.7 92.8  
AUB-MC Lebanon (SC)       
    IDS-RIA (ng/ml)c 83 26.4 12.4 24.0 7.4 74.8 d, e, f 
    DiaSorin Liaison (ng/ml)d 83 20.1 11.3 18.8 4.0 57.0 c, e, f 
    Difference (ng/ml)d − c 83 −6.3 7.0 −5.6 −27.2 32.6  
    Difference (%) 83 −32.7 29.4 −31.1 −103.0 82.1  
Stillwater-USA (SC)       
    DiaSorin RIA (ng/ml)e 83 21.7 9.1 20.5 5.0 45.6 c, d 
    DiaSorin Liaison (ng/ml)f 83 21.4 11.8 20.1 1.6 58.9 c, d 
    Difference (ng/ml)f − e 83 −0.3 5.4 −1.1 −17.4 31.7  
    Difference (%) 83 −9.1 26.5 −4.9 −101.9 88.1  
*

Letters indicate significant statistical difference (P < 0.05) compared to specific assay/cohort:

a

, IDS-RIA (FC);

b

, DiaSorin Liaison (FC);

c

, IDS-RIA (SC);

d

, DiaSorin Liaison (SC);

e

, DiaSorin RIA;

f

, DiaSorin Liaison. FC, Full cohort; SC, sub-cohort.

Fig. 1.

Number (A) or proportion (B) of subjects by 25-OHD level in the total cohort (n = 494) using the DiaSorin Liaison (gray) and the IDS-RIA (black) assays.

Fig. 1.

Number (A) or proportion (B) of subjects by 25-OHD level in the total cohort (n = 494) using the DiaSorin Liaison (gray) and the IDS-RIA (black) assays.

Fig. 2.

A, Bland-Altman plot of the difference between DiaSorin Liaison and IDS-RIA 25-OHD levels vs. their average [(IDS-RIA + Liaison)/2] in all 494 subjects (symbols). Solid horizontal line, Mean difference between the two assays (or bias); dashed horizontal lines, 95% confidence interval; thick line, moving average of difference (trend) calculated based on ordered sub-cohorts (n = 50 each) with 50% overlapping. B, Corresponding Bland-Altman plot of the percentage difference [100*(Liaison-IDS/RIA)/average). C, Frequency distribution of the percentage difference in 25-OHD levels among all 494 subjects.

Fig. 2.

A, Bland-Altman plot of the difference between DiaSorin Liaison and IDS-RIA 25-OHD levels vs. their average [(IDS-RIA + Liaison)/2] in all 494 subjects (symbols). Solid horizontal line, Mean difference between the two assays (or bias); dashed horizontal lines, 95% confidence interval; thick line, moving average of difference (trend) calculated based on ordered sub-cohorts (n = 50 each) with 50% overlapping. B, Corresponding Bland-Altman plot of the percentage difference [100*(Liaison-IDS/RIA)/average). C, Frequency distribution of the percentage difference in 25-OHD levels among all 494 subjects.

Subcohort (n = 83)

The means, medians, and distribution of 25-OHD levels obtained at AUB-MC (Lebanon) for the sub-cohort were similar to those obtained in the total cohort (P = 0.96 for IDS-RIA, and P = 0.33 for DiaSorin Liaison), and hence representative of the total sample (Table 1).

The reanalysis results at Stillwater-USA varied from those obtained at AUB-MC. The mean and median 25-OHD levels obtained with DiaSorin Liaison were closely matched to the ones obtained with DiaSorin RIA, as opposed to the ones obtained with IDS-RIA (Table 1).

Comparison between both assays using the DEQAS reports as a reference

Review of the quarterly DEQAS reports during the transition (July 2010 to January 2011) revealed the same bias for 25-OHD values obtained at AUB-MC with both assays. The mean value obtained in our laboratory using IDS-RIA had a positive bias compared with DEQAS generated mean all laboratory trimmed mean (ALTM), ranging between 2.5 and 31.5% (with only one result giving a negative bias of −3.9%). Conversely, the mean value obtained with DiaSorin Liaison had a consistent negative bias when compared with the ALTM, ranging between −4.1 and −18.4%. This negative bias persisted, but with a smaller magnitude when a new DiaSorin Liaison instrument was installed, ranging between −9.3 and +5.5%, thus coming closer to the ALTM.

Potential sources of variability between IDS RIA and DiaSorin Liaison

Single linear regression analysis results correlating 25-OHD data pairs across assays (within laboratory—full and sub-cohort) and across laboratories (sub-cohort) are shown in Table 2. Regression equations A (total cohort) and D (sub-cohort) were comparable, again underscoring that the sub-cohort was representative of the total cohort. The best combination (slope, intercept, R2) was observed when one compared 25-OHD values obtained using the same manufacturer and same method, even when measured in different laboratories (equation B). The worse combination was observed using differing manufacturers with either the same method (equation C) or differing methods (equation D). Results from assays provided by the same manufacturer but using different methods (equation G) correlated better than those provided by different manufacturers using different methods (equation D). However, although the mean 25-OHD values obtained using the same manufacturer but different assay methods in the sub-cohort were almost identical (Table 1), the match at the individual level illustrated in the regression equation was somewhat suboptimal (equation G). Regression curves detailing all equations are shown in the Supplement Data.

Table 2.

Linear regression model fit comparing various 25-OHD assays

Test comparison (y vs. x)nLinear regression (y = ax + b)R2
    A. DiaSorin Liaison (L) vs. IDS-RIA (L) 494* y = 0.82x − 0.31 0.79 
Lab comparison (sub-cohort)    
    B. DiaSorin Liaison (S) vs. DiaSorin Liaison (L) 83# y = 1.02x + 0.88 0.95 
Manufacturer comparison (sub-cohort)    
    C. DiaSorin RIA (S) vs. IDS-RIA (L) 83# y = 0.66x + 4.2 0.81 
Method comparison (sub-cohort)    
    D. DiaSorin Liaison (L) vs. IDS-RIA (L) 83# y = 0.76x + 0.04 0.69 
    E. DiaSorin Liaison (S) vs. IDS-RIA (L) 83# y = 0.80x + 0.12 0.71 
    F. DiaSorin Liaison (L) vs. DiaSorin RIA (S) 83# y = 1.08x − 3.3 0.75 
    G. DiaSorin Liaison (S) vs. DiaSorin RIA (S) 83# y = 1.17x − 4.0 0.81 
Test comparison (y vs. x)nLinear regression (y = ax + b)R2
    A. DiaSorin Liaison (L) vs. IDS-RIA (L) 494* y = 0.82x − 0.31 0.79 
Lab comparison (sub-cohort)    
    B. DiaSorin Liaison (S) vs. DiaSorin Liaison (L) 83# y = 1.02x + 0.88 0.95 
Manufacturer comparison (sub-cohort)    
    C. DiaSorin RIA (S) vs. IDS-RIA (L) 83# y = 0.66x + 4.2 0.81 
Method comparison (sub-cohort)    
    D. DiaSorin Liaison (L) vs. IDS-RIA (L) 83# y = 0.76x + 0.04 0.69 
    E. DiaSorin Liaison (S) vs. IDS-RIA (L) 83# y = 0.80x + 0.12 0.71 
    F. DiaSorin Liaison (L) vs. DiaSorin RIA (S) 83# y = 1.08x − 3.3 0.75 
    G. DiaSorin Liaison (S) vs. DiaSorin RIA (S) 83# y = 1.17x − 4.0 0.81 

Assays that provide identical values should have a slope of 1, an intercept of zero, and an R2 of 100%. Corresponding regressions are shown in Fig. 3 (*) and Supplemental Fig. 1 (#). L, AUB-MC Lebanon; S, Stillwater-USA.

Table 2.

Linear regression model fit comparing various 25-OHD assays

Test comparison (y vs. x)nLinear regression (y = ax + b)R2
    A. DiaSorin Liaison (L) vs. IDS-RIA (L) 494* y = 0.82x − 0.31 0.79 
Lab comparison (sub-cohort)    
    B. DiaSorin Liaison (S) vs. DiaSorin Liaison (L) 83# y = 1.02x + 0.88 0.95 
Manufacturer comparison (sub-cohort)    
    C. DiaSorin RIA (S) vs. IDS-RIA (L) 83# y = 0.66x + 4.2 0.81 
Method comparison (sub-cohort)    
    D. DiaSorin Liaison (L) vs. IDS-RIA (L) 83# y = 0.76x + 0.04 0.69 
    E. DiaSorin Liaison (S) vs. IDS-RIA (L) 83# y = 0.80x + 0.12 0.71 
    F. DiaSorin Liaison (L) vs. DiaSorin RIA (S) 83# y = 1.08x − 3.3 0.75 
    G. DiaSorin Liaison (S) vs. DiaSorin RIA (S) 83# y = 1.17x − 4.0 0.81 
Test comparison (y vs. x)nLinear regression (y = ax + b)R2
    A. DiaSorin Liaison (L) vs. IDS-RIA (L) 494* y = 0.82x − 0.31 0.79 
Lab comparison (sub-cohort)    
    B. DiaSorin Liaison (S) vs. DiaSorin Liaison (L) 83# y = 1.02x + 0.88 0.95 
Manufacturer comparison (sub-cohort)    
    C. DiaSorin RIA (S) vs. IDS-RIA (L) 83# y = 0.66x + 4.2 0.81 
Method comparison (sub-cohort)    
    D. DiaSorin Liaison (L) vs. IDS-RIA (L) 83# y = 0.76x + 0.04 0.69 
    E. DiaSorin Liaison (S) vs. IDS-RIA (L) 83# y = 0.80x + 0.12 0.71 
    F. DiaSorin Liaison (L) vs. DiaSorin RIA (S) 83# y = 1.08x − 3.3 0.75 
    G. DiaSorin Liaison (S) vs. DiaSorin RIA (S) 83# y = 1.17x − 4.0 0.81 

Assays that provide identical values should have a slope of 1, an intercept of zero, and an R2 of 100%. Corresponding regressions are shown in Fig. 3 (*) and Supplemental Fig. 1 (#). L, AUB-MC Lebanon; S, Stillwater-USA.

Fig. 3.

Comparison of Liaison vs. IDS-RIA 25-OHD levels in all study patients (n = 494). Symbols represent individual data points using both assays. Gray areas represent concordance (or agreement) segments at increasing 25-OHD levels [A (≤10 ng/ml), E (10–20 ng/ml), and I (≥20 ng/ml)]. Numbers in parentheses next to each letter represent the number of individuals in each area segment. Dashed line, Line of identity; solid line, linear regression model fit (y = 0.82x − 0.31; R2 = 0.79).

Fig. 3.

Comparison of Liaison vs. IDS-RIA 25-OHD levels in all study patients (n = 494). Symbols represent individual data points using both assays. Gray areas represent concordance (or agreement) segments at increasing 25-OHD levels [A (≤10 ng/ml), E (10–20 ng/ml), and I (≥20 ng/ml)]. Numbers in parentheses next to each letter represent the number of individuals in each area segment. Dashed line, Line of identity; solid line, linear regression model fit (y = 0.82x − 0.31; R2 = 0.79).

Impact of assay methodology (IDS-RIA vs. DiaSorin Liaison) on patient classification, magnitude of difference between 25-OHD levels obtained, and corresponding concordance rates

Patient classification

In the total cohort, using IDS-RIA results, 6% of patients were classified as having a 25-OHD level less than or equal to 10 ng/ml, 29% between 10 and 20 ng/ml, and 65% at least 20 ng/ml. The classification was different when using DiaSorin Liaison: 17% were labeled as having a 25-OHD level less than or equal to 10 ng/ml, 34% between 10 and 20 ng/ml, and 49% at least 20 ng/ml (P < 0.001; Fig. 1B).

Using a threshold of 20 ng/ml to define vitamin D deficiency, as per The Endocrine Society guidelines (10), 52% (100 − 48%) of patients would then be classified as deficient using DiaSorin Liaison, compared with 36% (100 − 64%) using IDS-RIA (P < 0.001). For a target 25-OHD level of 30 ng/ml, proposed by the same guidelines (10), 79% (100 − 21%) of patients would then require treatment using DiaSorin Liaison (25-OHD level <30 ng/ml), compared with 64% (100 − 36%) according to results obtained by IDS-RIA (P < 0.001).

Magnitude of difference between the two assays

To further evaluate differences between assays, the difference in 25-OHD levels obtained with both assays was calculated for each subject: 87% of subjects differed by less than 10 ng/ml, 11% by 10–20 ng/ml, and 2% by at least 20 ng/ml. Similarly, the proportion of subjects that varied by less than 20% was 41%, between 20 and 40% was 29%, and by more than 40% was 30% (Fig. 2C). Using the DEQAS-defined proportion for agreement between any assay and the ALTM of 25%, the proportion of values that varied by less than 25% across the two assays was 246 of 494 (50.0%) pairs, or just under half of the subjects.

Concordance rates

Concordance proportions depended on the criteria used to define agreement. Indeed, in the section above, concordance varied between 41 and 50%, but using our selected three-category approach, the overall concordance rate for classification between the two assays was 68% (Fig. 3). The weighted κ was 0.55 (P < 0.001; 95% confidence interval, 0.484–0.616), suggesting moderate agreement.

Among patients classified as having 25-OHD no greater than 10 ng/ml by Liaison (83 of 494), one third (27 of 83) only received the same classification by RIA, whereas two thirds (55 of 83) were stratified into the 10–20 ng/ml category, and 1% (1 of 83) had a 25-OHD above 20 ng/ml (Fig. 3). Among 172 subjects considered to have a 25-OHD level of 10–20 ng/ml by Liaison, half (88 of 172) had a 25-OHD above 20 ng/ml by IDS-RIA (Fig. 3). Among patients classified as having 25-OHD below 30 ng/ml by Liaison (390 of 494), 80% (311 of 390) received the same classification by RIA, whereas the remaining 20% (79 of 390) were considered to have a 25-OHD of at least 30 ng/ml.

Discussion

It is becoming increasingly common practice for laboratories to shift from 25-OHD classical RIA to more rapid, high-output, automated assays. The implications of this change have not been previously systematically evaluated. In this study, we demonstrate substantial differences in the 25-OHD values obtained with the two assays and their profound implications on intervention using recent vitamin D guidelines.

Serum 25-OHD levels assayed with the newly adopted automated assay DiaSorin Liaison were, on average, 5 ng/ml lower than those measured by IDS-RIA (range, −38.1 to +18.7 ng/ml). The difference abated with the use of differing methodologies (that is Liaison and RIA) but the same manufacturer (that is DiaSorin, Stillwater-USA), pointing to an element of intermanufacturer variability. But that by itself did not fully account for the variability observed. Indeed, regression analyses between different assay methods within the same manufacturer revealed that variability still persisted. The fact that the mean values obtained with DiaSorin Liaison and DiaSorin RIA in the sub-cohort were identical reflects a quality assurance measure introduced by the manufacturer to ensure that values obtained when switching methodologies do not differ, at least at the mean level. In short, the variability observed in Lebanon was thus reproducible, unexplained solely by human and laboratory errors, and reflects a genuine difference between methods and manufacturers.

In 2007, Kimball and Vieth (11) compared new automated 25-OHD assays, including the older version of Liaison, to the routinely used RIA. The Liaison assay correlated the most poorly with DiaSorin RIA (R2 = 0.502) (11). Results obtained with DiaSorin Liaison were again negatively biased when compared with liquid chromatography-tandem mass spectrometry (LC-MS/MS), considered by many as the possible “gold standard” for 25-OHD measurement (12, 14), and when compared with high-pressure liquid chromatography-atmospheric pressure chemical ionization-mass spectrometry (15).

Variability between 25-OHD assays was similarly illustrated by DEQAS. The mean imprecision (CV) of participating laboratories decreased from 32% in 1994 to 15.3% in 2009, but results submitted were still biased. The majority of methods in the 2008–2009 distribution were positively biased with respect to the ALTM, with the IDS-RIA having the highest bias of 10.2%. The exception was the DiaSorin RIA and Liaison assays, which had a mean bias of −2.1 and −7.4%, respectively. The DiaSorin Liaison and IDS-RIA gave mean values that differed by as much as 17.5% (16), compared with 20.6% using our data. The percentage bias observed with IDS-RIA and DiaSorin Liaison was similarly observed in the 2009–2010 and the 2010–2011 distributions (Fig. 4). This reported bias may not have illustrated the full magnitude of the actual differences based on individual data points or laboratories because the figure displayed mean values. Moreover, the assay bias relative to ALTM varied substantially from 1 yr to another, as reported for IDS-RIA, iSYS, and Roche, which further contributed to variability in results (Fig. 4).

Fig. 4.

Mean (%) bias ± sem for various 25-OHD methods compared with the ALTM in 2009–2010 and 2010–2011 (current date). ALTM is the overall mean of 25-OHD levels obtained from all involved laboratories regardless of methodology used, at each cycle. DEQAS data were published by permission of the quality assurance program organizer, Mr. Graham Carter.

Fig. 4.

Mean (%) bias ± sem for various 25-OHD methods compared with the ALTM in 2009–2010 and 2010–2011 (current date). ALTM is the overall mean of 25-OHD levels obtained from all involved laboratories regardless of methodology used, at each cycle. DEQAS data were published by permission of the quality assurance program organizer, Mr. Graham Carter.

The reasons for the negative bias in Liaison compared with RIA and for the variability between different assays are multiple. RIA and Liaison assays are affected by the so-called matrix effect. Being lipophilic, 25-OHD is prone to interference from other lipids, a problem that is not encountered when using chromatography and competitive protein-binding assays (17). Another contributing factor would be a difference in the detection of 25-OHD metabolites, 25-OHD2 and 25-OHD3. Chromatographic methods can effectively measure and report both metabolites, whereas under-detection is reported with both IDS-RIA and DiaSorin Liaison (18, 19), in contrast to the manufacturers' claims. Clinically, this would lead to under- or overestimation of the actual 25-OHD level depending on the assay used and on whether the patient is being supplemented with ergocalciferol (vitamin D2) or cholecalciferol (vitamin D3). Another potential source of variability between assays is the reported cross-reactivity of the 3-epi-25-OHD3 (3-epimer) with some 25-OHD assays. The 3-epimer has the same mass and thus a similar fragmentation pattern to 25-OHD3 in LC-MS/MS. It can then be mistakenly reported as 25-OHD, which could explain the positive bias of this assay. According to the manufacturer, the 3-epimer is undetectable by Liaison (www.diasorin.com); however, no such data are available for IDS-RIA. Interference from the 3-epimer is mainly significant in infants, where it forms 15–41% of the total 25-OHD. In adults, it forms 2.5–17% and should not be neglected (20). In our study, none of the 494 patient samples studied were derived from infants, and less than 3% belonged to the pediatric age group (ages 1–18 yr); the remaining samples were from adults.

The impact of the observed variability in 25-OHD results with a shift to rapid and efficient automated assays is major. DiaSorin RIA is the first 25-OHD assay approved for clinical diagnosis by the U.S. Food and Drug Administration (FDA) and the reference method used to determine the currently available cutoffs for vitamin D treatment (21). Although it used to be the most widely used and the adopted method by the Centers for Disease Control and Prevention (CDC) for epidemiological surveys until 2005–2006, it is labor intensive and became limited in its ability to be automated (11, 22). Among 899 laboratories participating in DEQAS in January 2011, the DiaSorin Liaison Total assay was the most commonly used commercial platform, accounting for 38.7% of participating laboratories, compared with 3.7% using DiaSorin RIA assay and 1.8% using IDS-RIA assays. As a result of this change in assays, universal thresholds defining an adequate 25-OHD level might not be applicable, and assay-specific adjustments in cutoff points may be needed.

Using a 25-OHD level of 20 ng/ml as a threshold for treatment (7), 52% of the 494 patients studied in this paper would have been treated based on 25-OHD levels obtained with Liaison, whereas only 36% would have required replacement based on IDS-RIA results. Similarly, using a cutoff of 20 ng/ml, Snellman et al. (15) reported vitamin D insufficiency among 43% of 204 patients using Liaison compared with 22% using IDS-RIA. Using an arbitrary threshold of insufficiency of 32 ng/ml, Binkley et al. (23) showed that the proportion of subjects below this cutoff varied between 17 and 90% with a change between two different laboratories using RIA.

Findings from this and other studies highlight the need to improve quality assessment of 25-OHD assays. In 2004, the DEQAS performance target required participants to get 80% or more of their results within ± 30% of ALTM. This target was only achieved by 59% of participants (24). Moreover, in DEQAS, values obtained by specific centers, being compared against the ALTM, are biased by the most commonly used method, as opposed to a defined universal reference standard. Additional means are thus needed to standardize 25-OHD assays. Although not recommended, one way might be to derive a statistical correction factor between assays (25). Another way would be to derive a standard reference against which methods might be validated. The National Institute of Standards and Technology (NIST) and the National Institutes of Health (NIH) Office of Dietary Supplements (ODS) worked together to develop standard reference materials (SRM) with different levels of 25-OHD2, 25-OHD3, and the 3-epimer, known as SRM 972 and SRM 2972. Values for SRM were assigned using LC-MS/MS, chosen by NIST as the reference measurement procedure (20, 26). When compared with the reference measurement procedure, the bias already observed for various 25-OHD assays participating in DEQAS was reproducible, and the ALTM was close to the proposed true value with a mean bias of +3.3% (17). Unfortunately, available SRM are still not perfect, and NIST is currently working on new reference sera. Standardization of assays against SRM 972 has already been recommended in the United Kingdom (22). Moreover, a roundtable of experts convened to discuss 25-OHD in the National Health and Nutrition Examination Survey (NHANES) suggested the use of LC-MS/MS as proposed by the NIST, and the need to work with manufacturers to improve the comparability of measurement procedures across laboratories (27, 28). Relevant initiatives for standardization of 25-OHD assays are also available from the CAP (www.cap.org) and the collaboration between the ODS of the NIH, the CDC, and NIST (29).

We have described in detail substantial variation between two 25-OHD assays and have provided an overview of this variability for other 25-OHD assays. With the increased clinical demand for measuring 25-OHD, many institutions are shifting from older assays, based on which treatment thresholds were derived, to the more rapid and automated techniques available. DiaSorin Liaison is currently the most commonly used platform by laboratories participating in DEQAS, and it shows a consistent negative bias compared with several assays and with the DEQAS ALTM or the NIST standards. The implications of variability within and between assays on the relevance and applicability of guidelines/recommendations, be it at the public health or at the individual level, are substantial. Clinicians are to be aware of that fact, being faced with variable and noncomparable 25-OHD results, where it is still not clear which method is giving the most accurate results. This may lead to misinterpretation of the actual vitamin D status of patients and the possibility of under- or overtreatment. Furthermore, patients having 25-OHD levels ordered serially should be done by the same method. Assay variability is a problem that has been previously encountered with the measurement of lipids, glycosylated hemoglobin and other variables, and it has been solved through global standardization efforts (30) (www.ngsp.org). As observed by Binkley et al. (23), 25-OHD assay variability exists, is confounding the diagnosis of hypovitaminosis D, and is thus a pressing issue. It needs to be addressed through the implementation of quality assurance programs in laboratories performing 25-OHD assays, pushing forward the agenda for global standardization of assays, the consideration of assay-specific cutoff points, and the provision of guidance for the implementation of current vitamin D recommendations (Institute Of Medicine, The Endocrine Society, etc.), ideally providing desirable ranges for 25-OHD as opposed to discrete cutoffs, thus adjusting for assay differences.

Acknowledgments

The authors thank Mr. Graham Carter, administrator at DEQAS, for sharing his experience on variation in 25-OHD assays and providing us with DEQAS data summarizing mean percentage bias for various assay methods for 2008–2009 and 2009 to the current date; Mr. Joshua Soldo at DiaSorin Stillwater for rerunning 25-OHD levels using DiaSorin RIA and DiaSorin Liaison at their laboratory; and Mr. Ali Hammoudi for assistance with some of the graphics.

Disclosure Summary: The authors declare no conflict of interest.

*

M.B. and R.T.D. are co-primary authors.

Abbreviations

     
  • ALTM

    All laboratory trimmed mean

  •  
  • CV

    coefficient of variation

  •  
  • LC-MS/MS

    liquid chromatography-tandem mass spectrometry

  •  
  • 25-OHD

    25-hydroxyvitamin D

  •  
  • SRM

    standard reference material.

References

1.

Holick
MF
2007
Vitamin D deficiency.
N Engl J Med
357
:
266
281

2.

Arabi
A
,
El Rassi
R
,
El-Hajj Fuleihan
G
2010
Hypovitaminosis D in developing countries—prevalence, risk factors and outcomes.
Nat Rev Endocrinol
6
:
550
561

3.

El-Hajj Fuleihan
G
,
Nabulsi
M
,
Choucair
M
,
Salamoun
M
,
Hajj Shahine
C
,
Kizirian
A
,
Tannous
R
2001
Hypovitaminosis D in healthy schoolchildren.
Pediatrics
107
:
E53

4.

Fuleihan
GE
,
Deeb
M
1999
Hypovitaminosis D in a sunny country.
N Engl J Med
340
:
1840
1841

5.

Gannagé-Yared
MH
,
Chemali
R
,
Yaacoub
N
,
Halaby
G
2000
Hypovitaminosis D in a sunny country: relation to lifestyle and bone markers.
J Bone Miner Res
15
:
1856
1862

6.

Arabi
A
,
Baddoura
R
,
Awada
H
,
Salamoun
M
,
Ayoub
G
,
El-Hajj Fuleihan
G
2006
Hypovitaminosis D osteopathy: is it mediated through PTH, lean mass, or is it a direct effect?
Bone
39
:
268
275

7.

Ross
AC
,
Manson
JE
,
Abrams
SA
,
Aloia
JF
,
Brannon
PM
,
Clinton
SK
,
Durazo-Arvizu
RA
,
Gallagher
JC
,
Gallo
RL
,
Jones
G
,
Kovacs
CS
,
Mayne
ST
,
Rosen
CJ
,
Shapses
SA
2011
The 2011 report on dietary reference intakes for calcium and vitamin D from the Institute of Medicine: what clinicians need to know.
J Clin Endocrinol Metab
96
:
53
58

8.

Lips
P
2004
Which circulating level of 25-hydroxyvitamin D is appropriate?
J Steroid Biochem Mol Biol
89–90
:
611
614

9.

Dawson-Hughes
B
,
Mithal
A
,
Bonjour
JP
,
Boonen
S
,
Burckhardt
P
,
Fuleihan
GE
,
Josse
RG
,
Lips
P
,
Morales-Torres
J
,
Yoshimura
N
2010
IOF position statement: vitamin D recommendation for older adults.
Osteoporos Int
21
:
1151
1154

10.

Holick
MF
,
Binkley
NC
,
Bischoff-Ferrari
HA
,
Gordon
CM
,
Hanley
DA
,
Heaney
RP
,
Murad
MH
,
Weaver
CM
2011
Evaluation, treatment, and prevention of vitamin D deficiency: an Endocrine Society Clinical Practice Guideline.
J Clin Endocrinol Metab
96
:
1911
1930

11.

Kimball
SM
,
Vieth
R
2007
A comparison of automated methods for the quantitation of serum 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D.
Clin Biochem
40
:
1305
1310

12.

Lai
JK
,
Lucas
RM
,
Banks
E
,
Ponsonby
AL
,
Ausimmune Investigator Group
2011
Variability in vitamin D assays impairs clinical assessment of vitamin D status
.
Intern Med J

13.

Carter
GD
,
Carter
R
,
Jones
J
,
Berry
J
2004
How accurate are assays for 25-hydroxyvitamin D? Data from the International Vitamin D External Quality Assessment Scheme.
Clin Chem
50
:
2195
2197

14.

Zerwekh
JE
2008
Blood biomarkers of vitamin D status.
Am J Clin Nutr
87
:
1087S
1091S

15.

Snellman
G
,
Melhus
H
,
Gedeborg
R
,
Byberg
L
,
Berglund
L
,
Wernroth
L
,
Michaëlsson
K
2010
Determining vitamin D status: a comparison between commercially available assays.
PloS ONE
5
:
e11555

16.

Carter
GD
,
Berry
JL
,
Gunter
E
,
Jones
G
,
Jones
JC
,
Makin
HL
,
Sufi
S
,
Wheeler
MJ
2010
Proficiency testing of 25-hydroxyvitamin D (25-OHD) assays.
J Steroid Biochem Mol Biol
121
:
176
179

17.

Carter
GD
2011
Accuracy of 25-hydroxyvitamin D assays: confronting the issues.
Curr Drug Targets
12
:
19
28

18.

Glendenning
P
,
Taranto
M
,
Noble
JM
,
Musk
AA
,
Hammond
C
,
Goldswain
PR
,
Fraser
WD
,
Vasikaran
SD
2006
Current assays overestimate 25-hydroxyvitamin D3 and underestimate 25-hydroxyvitamin D2 compared with HPLC: need for assay-specific decision limits and metabolite-specific assays.
Ann Clin Biochem
43
:
23
30

19.

Roth
HJ
,
Schmidt-Gayk
H
,
Weber
H
,
Niederau
C
2008
Accuracy and clinical implications of seven 25-hydroxyvitamin D methods compared with liquid chromatography-tandem mass spectrometry as a reference.
Ann Clin Biochem
45
:
153
159

20.

Stepman
HC
,
Vanderroost
A
,
Van Uytfanghe
K
,
Thienpont
LM
2011
Candidate reference measurement procedures for serum 25-hydroxyvitamin D3 and 25-hydroxyvitamin D2 by using isotope-dilution liquid chromatography-tandem mass spectrometry.
Clin Chem
57
:
441
448

21.

Wagner
D
,
Hanwell
HE
,
Vieth
R
2009
An evaluation of automated methods for measurement of serum 25-hydroxyvitamin D.
Clin Biochem
42
:
1549
1556

22.

de la Hunty
A
,
Wallace
AM
,
Gibson
S
,
Viljakainen
H
,
Lamberg-Allardt
C
,
Ashwell
M
2010
UK Food Standards Agency Workshop Consensus Report: the choice of method for measuring 25-hydroxyvitamin D to estimate vitamin D status for the UK National Diet and Nutrition Survey.
Br J Nutr
104
:
612
619

23.

Binkley
N
,
Krueger
D
,
Cowgill
CS
,
Plum
L
,
Lake
E
,
Hansen
KE
,
DeLuca
HF
,
Drezner
MK
2004
Assay variation confounds the diagnosis of hypovitaminosis D: a call for standardization.
J Clin Endocrinol Metab
89
:
3152
3157

24.

Carter
GD
,
Carter
C
,
Gunter
E
,
Jones
J
,
Jones
G
,
Makin
H
,
Sufi
S
2004
Measurement of vitamin D metabolites: an international perspective on methodology and clinical interpretation.
J Steroid Biochem Mol Biol
89–90
:
467
471

25.

Hyppönen
E
,
Turner
S
,
Cumberland
P
,
Power
C
,
Gibb
I
2007
Serum 25-hydroxyvitamin D measurement in a large population survey with statistical harmonization of assay variation to an international standard.
J Clin Endocrinol Metab
92
:
4615
4622

26.

Phinney
KW
2008
Development of a standard reference material for vitamin D in serum.
Am J Clin Nutr
88
:
511S
512S

27.

Yetley
EA
,
Pfeiffer
CM
,
Schleicher
RL
,
Phinney
KW
,
Lacher
DA
,
Christakos
S
,
Eckfeldt
JH
,
Fleet
JC
,
Howard
G
,
Hoofnagle
AN
,
Hui
SL
,
Lensmeyer
GL
,
Massaro
J
,
Peacock
M
,
Rosner
B
,
Wiebe
D
,
Bailey
RL
,
Coates
PM
,
Looker
AC
,
Sempos
C
,
Johnson
CL
,
Picciano
MF
2010
NHANES monitoring of serum 25-hydroxyvitamin D: a roundtable summary.
J Nutr
140
:
2030S
2045S

28.

Schleicher
RL
,
Encisco
SE
,
Chaudhary-Webb
M
,
Paliakov
E
,
McCoy
LF
,
Pfeiffer
CM
2011
Isotope-dilution ultra performance liquid chromatography-tandem mass spectrometry method for simultaneous measurement of 25-hydroxyvitamin D2, 25-hydroxyvitamin D3 and 3-epi-25-hydroxyvitamin D3 in human serum.
Clin Chim Acta
412
:
1594
1599

29.

Coates
P
2011
Notice of vitamin D standardization program
.
Federal Register
.

30.

Centers for Disease Control and Prevention
Laboratory quality assurance and standardization programs
. Accessed 23 August 2011.

Supplementary data