Comparison of Proteomic Measurements Across Platforms in the Atherosclerosis Risk in Communities (ARIC) Study

Abstract

Background

The plasma proteome can be quantified using different types of highly multiplexed technologies, including aptamer-based and proximity-extension immunoassay methods. There has been limited characterization of how these protein measurements correlate across platforms and with absolute measures from targeted immunoassays.

Methods

We assessed the comparability of (a) highly multiplexed aptamer-based (SomaScan v4; Somalogic) and proximity-extension immunoassay (OLINK Proseek^® v5003; Olink) methods in 427 Atherosclerosis Risk in Communities (ARIC) Study participants (Visit 5, 2011–2013), and (b) 18 of the SomaScan protein measurements against targeted immunoassays in 110 participants (55 cardiovascular disease cases, 55 controls). We calculated Spearman correlations (r) between the different measurements and compared associations with case-control status.

Results

There were 417 protein comparisons (366 unique proteins) between the SomaScan and Olink platforms. The average correlation was r = 0.46 (range: −0.21 to 0.97; 79 [19%] with r ≥ 0.8). For the comparison of SomaScan and targeted immunoassays, 6 of 18 assays (growth differentiation factor 15 [GDF15], interleukin-1 receptor-like 1 [ST2], interstitial collagenase [MMP1], adiponectin, leptin, and resistin) had good correlations (r ≥ 0.8), 2 had modest correlations (0.5 ≤ r < 0.8; osteopontin and interleukin-6 [IL6]), and 10 were poorly correlated (r < 0.5; metalloproteinase inhibitor 1 [TIMP1], stromelysin-1 [MMP3], matrilysin [MMP7], C-C motif chemokine 2 [MCP1], interleukin-10 [IL10], vascular cell adhesion protein 1 [VCAM1], intercellular adhesion molecule 1 [ICAM1], interleukin-18 [IL18], tumor necrosis factor [TNFα], and visfatin) overall. Correlations for SomaScan and targeted immunoassays were similar according to case status.

Conclusions

There is variation in the quantitative measurements for many proteins across aptamer-based and proximity-extension immunoassays (approximately 1/2 showing good or modest correlation and approximately 1/2 poor correlation) and also for correlations of these highly multiplexed technologies with targeted immunoassays. Design and interpretation of protein quantification studies should be informed by the variation across measurement techniques for each protein.

Introduction

The advent of highly multiplexed technology to simultaneously measure thousands of proteins has fueled a recent boom in proteomics research. Methods for measuring the human plasma proteome include an aptamer-based (1) approach and a proximity-extension immunoassay (2) approach. These highly multiplexed approaches provide the relative concentrations of proteins, using different methodologies. The aptamer approach, for example, can be impacted by features such as three-dimensional epitope shape modification or masking in multi-molecule complexes. The extent to which protein measurements correlate across platforms is relatively uncharacterized (3, 4). Characterizing the comparability across highly multiplexed proteomics platforms is crucial for synthesizing findings that utilize different platforms to measure the human proteome. Few studies have examined correlations of relative protein measurements across platforms and with absolute measures from targeted immunoassays used in research and clinical settings (3, 5–8).

Using data from the Atherosclerosis Risk in Communities (ARIC) Study, we firstly assessed the comparability of highly-multiplexed aptamer-based (SomaScan by SomaLogic) and proximity-extension immunoassay (Olink) methods in 427 participants. We also correlated relative abundance on both platforms, including cystatin-C, to estimated glomerular filtration rate (eGFR) and to absolute measures of plasma cystatin-C (clinical immunoassay). Secondly, we tested the comparability of 18 aptamer-based protein measurements against targeted immunoassays in 110 participants (55 cardiovascular disease [CVD] cases and 55 controls). The study design allowed us to conduct an exploratory study of how differences might translate to associations with CVD. Additionally, among 29 participants, we explored correlations of 16 proteins that were measured on all platforms (SomaScan, Olink, and targeted immunoassays).

Materials and Methods

Study Design

The ARIC Study is a community-based cohort study which began in 1987 to 1989 when the 15 792 participants were middle-aged. The participants were recruited from 4 US communities: Forsyth County, North Carolina; Jackson, Mississippi; suburban Minneapolis, Minnesota; and Washington County, Maryland. Visit 5 occurred in 2011 to 2013 and was attended by 6538 participants. At Visit 5, participants underwent phlebotomy. At each ARIC site, the plasma underwent standardized processing and storage in −80°C freezers until the specimens were shipped on dry ice to the ARIC central laboratory (7, 9). Not previously (never) thawed ARIC Visit 5 samples were thawed using a standardized quick thaw/refreeze protocol, aliquoted, and shipped on dry ice to Somalogic and Olink for analysis (see online Supplemental Methods). Samples had one freeze–thaw cycle when arriving at the site of lab analysis (Somalogic and Olink) and were analyzed in single analysis in a randomized and blinded fashion at each laboratory site.

Aptamer-based proteomic platform.

The SomaScan version 4 platform uses multiplexed modified DNA-based aptamer technology (Somalogic) (1). The relative concentrations of 5284 aptamers were quantified on plates at Somalogic using processes previously described with relative fluorescence intensity calibrated using standards on each plate and normalized for plate variation (1, 10–12) and overall concentration of all proteins in a sample using adaptive normalization by maximum likelihood (ANML). Briefly, short single-stranded DNA with modified nucleotides, known as slow off-rate modified aptamers (SOMAmers), were used as protein-binding reagents. These SOMAmers have sequences that can be quantified using DNA detection technology.

Using blind duplicates of the specimens from 187 (4%) ARIC Study Visit 5 participants with available SOMAmer data, we derived coefficients of variation using a method by Bland and Altman (13) (CV_BA = $e^{\surd mean(variance(\ln (X)))}-1$⁠). We excluded 329 SOMAmers with a CV_BA >50%, a variance <0.01 on the log scale, or binding to mouse Fc-fusion, contaminants, or non-proteins, which resulted in 4955 SOMAmers (4712 unique proteins) for potential comparison to Olink proteins. The median inter-assay CV_BA for the SOMAmers was 6.6%. The median split sample reliability coefficient (intraclass correlation coefficient) was 0.96. We transformed Somalogic normalized values using the following equation [log₂(SomaScan protein abundance) + log₂(1/dilution)] to account for dilutions (1:5, 1:200, and 1:20000).

Immunoassay proteomic platform.

The Olink Proseek^® v.5003 platform uses a multiplex immunoassay approach based on proximity-extension assay technology (2). Briefly, pairs of oligonucleotide-labeled antibodies were added to the plasma samples and could bind to the target protein present. DNA hybridization occurred when the pair of oligonucleotide-labeled antibodies were in close proximity. This sequence was quantified using a standard qPCR technique. Five Olink panels (CVD II, CVD III, inflammation, organ damage, and cardiometabolic) were assayed.

The relative concentrations of the 460 proteins (92 proteins per panel; 11 proteins were measured on 2 panels) were measured within each plasma sample. Olink data was represented as normalized protein expression values, which were expressed on the log₂ scale. Using 29 blind duplicates in the ARIC Study, the median inter-assay CV_BA for the Olink proteins was 3.4% and the median split sample reliability coefficient was 0.86. Relative protein concentrations below the limit of detection (LOD) were imputed to the LOD. Specimens underwent 1:2025 dilution for the cardiometabolic panel and the CVD III panel underwent 1:100 dilution. To account for dilutions, we added log₂(1/dilution) to the Olink normalized protein expression values that were already expressed on the log₂ scale.

Targeted immunoassays.

Immunoassays were performed in a Clinical Laboratory Improvement Amendments (CLIA) and College of American Pathologists (CAP) accredited laboratory (with prior experience in laboratory developed tests and high complexity testing) according to manufacturers’ protocols. The Human Magnetic Luminex Assay (R&D System) was used to measure 14 analytes: growth differentiation factor 15 (GDF15), interleukin-1 receptor-like 1 (ST2), osteopontin, interleukin-6 (IL6), interstitial collagenase (MMP1), metalloproteinase inhibitor 1 (TIMP1), stromelysin-1 (MMP3), matrilysin (MMP7), C-C motif chemokine 2 (MCP1), interleukin-10 (IL10), vascular cell adhesion protein 1 (VCAM1), intercellular adhesion molecule 1 (ICAM1), interleukin-18 (IL18), tumor necrosis factor (TNFα). Four adipokines were measured with ELISAs (adiponectin, leptin, resistin by R&D Systems; visfatin by Invitrogen).

Multiple configurations of the Luminex Assay were used for ST2, IL6, TNFα, IL10 and TIMP1. Luminex assays were performed according to manufacturer’s protocol and intra- and inter-assay CVs are listed in online Supplemental Table 1. Assay CVs for the adipokines are shown in online Supplemental Table 2. GDF15 was also measured using the clinical Elecsys immunoassay (Roche Diagnostics) on an automated Cobas e411 immunoanalyzer. The assay has an LOD of 400 pg/mL and an inter-assay CV of 4.8% in our cohort. Details on the targeted immunoassays and other relevant measurements are provided in the Supplemental Methods.

We log₂-transformed these immunoassay measurements for consistency with the protein measures from the SomaScan and Olink platforms.

Study design for comparison of highly multiplexed platforms.

The study design for the comparison of highly multiplexed platforms is depicted in Fig. 1. The SomaScan and Olink platforms were used to analyze previously stored plasma samples obtained at ARIC Visit 5. Visit 5 specimens were measured on the SomaScan platform in 2018 while Olink was measured in 2019. The relative abundance of 5284 proteins were measured at Somalogic using previously stored plasma from ARIC Visit 5 participants (n > 5000). Stored plasma samples for 500 ARIC participants (250 cases, 250 controls) were tested on the Olink platform. Eligible cases included those without prevalent heart failure (HF) at Visit 5 but who developed HF subsequently. Eligible controls were those without prevalent HF at Visit 5 who did not develop HF over the same time period and matched based on age and sex. Incident HF was defined using the Framingham Heart Study HF definition (14) and/or as participants who were hospitalized at least once with a HF diagnosis without a prior HF diagnosis.

Fig. 1.

Study design for the proteomic comparisons of highly multiplexed aptamer-based (SomaScan) and proximity-extension immunoassay methods (Olink Proteomics) and the study design for the targeted comparisons of SomaScan and immunoassays (research and clinical use). For the comparison of highly multiplexed platforms, there were 417 SomaScan vs Olink protein assay comparisons in total (366 unique proteins). In exploratory analyses, there were 29 participants with 16 protein measurements (TNFα, MMP1, leptin, GDF15, MMP7, VCAM1, resistin, ST2, IL6, osteopontin, MCP1, IL18, TIMP1, MMP3, IL10, and ICAM1) that overlapped across the SomaScan, Olink, and targeted immunoassay platforms (Olink TNFα was excluded in this subset of n = 29 due 100% of Olink values being below the LOD).

Study design for targeted comparison.

We used data from a pilot study conducted using stored plasma samples of ARIC Visit 5 (2011 to 2013) participants. Participants with prevalent CVD (defined by coronary heart disease, stroke, or HF) at Visit 5 were not included in this pilot study. Participants with incident CVD by December 31, 2017, were eligible to be cases. Controls included those without incident CVD, who did not die within 5 years after Visit 5. The pilot study included 58 cases and 58 controls. Cases were balanced by age (≥73 or <73 years, median), sex, race, and eGFR (≥60 or <60 mL/min/1.73 m²). Controls were frequency-matched to the age (±10 years), sex, race and eGFR groupings of cases. There were 3 participants (2 cases, one control) whose results did not meet SomaScan QC standards. Therefore, to maintain the matched nature of the study, our final analytic sample size for our targeted comparison included 110 participants (55 cases, 55 controls).

Statistical Analysis

Statistical analysis was performed using R version 4.1.2.

Analysis for comparison of highly multiplexed platforms.

Olink and SomaScan protein values were presented on the log₂ scale. For proteins with multiple measurements on the same platform, we report pairwise comparisons for all measurements, e.g., delta-like non-canonical Notch ligand 1 has 3 comparisons for one Olink immunoassay and 3 SomaScan aptamers (see Supplemental Methods). We graphed Spearman correlation coefficients (r) and reported corresponding P-values for the 417 protein pairs found in both Olink and SomaScan with vs without excluding outliers (>3 standardized residuals from linear regression). After excluding outliers, we summarized the correlations using histograms and summary statistics (mean, median, and percentiles). We reported the intercept, slope from unadjusted linear regression (x = SomaScan, y = Olink) and provided summary statistics (mean, SD, SD/mean) for each Olink and SomaScan protein value after accounting for dilutions. We correlated the proteins on each platform with clinical measures of kidney function—eGFR and cystatin-C are strongly related to the plasma proteome (15)—to try to determine which assay is more biologically relevant to kidney function when the assays are poorly correlated. In an exploratory analysis, we examined how cross-platform correlations for the proteins varied across the dilution bins.

Analysis for comparison between SomaScan and targeted immunoassays.

We assessed the comparability of 18 plasma analyte pairs measured using aptamer-based (SomaScan) vs targeted immunoassay methods. We generated Spearman correlation coefficients. Where multiple immunoassay configurations (e.g., Multiplex-1, Roche) were used, we report correlations for each configuration compared to SomaScan and inter-immunoassay correlations. We used logistic regression to obtain age, sex, and race-adjusted odds ratios (OR) and 95% CIs for incident CVD (i.e., case definition) per log₂(analyte).

Results

Comparisons of the Highly Multiplexed Platforms

On the SomaScan platform, we had measurements for 4955 SOMAmers (4712 unique proteins) in 5327 ARIC Study participants. On the Olink platform, we measured abundances of 460 proteins (448 unique) in 500 individuals. There were 427 participants with both SomaScan and Olink measurements and there were 417 protein assay comparisons (366 unique proteins) with overlap between the SomaScan and Olink platforms, after excluding proteins with CV_BA >50% or negative protein values. In the 427 ARIC participants, the mean age was 78 years (SD = 5), 44% were women, and 24% self-identified their race as Black (Table 1).

Table 1.

Participant characteristics of the highly multiplexed platform and targeted protein assay comparisons: the ARIC Study (2011 to 2013).^a

	Highly multiplexed platform comparison (SomaScan vs Olink)	Targeted assay comparison (SomaScan vs targeted immunoassays)b
		Overall	Cases	Controls
n	427	110	55	55
Age, years	77.9 (5.4)	74.2 (4.5)	74.8 (5.3)	73.5 (3.6)
Female	187 (43.8%)	54 (49.1%)	27 (49.1%)	27 (49.1%)
Black	101 (23.7%)	48 (43.6%)	24 (43.6%)	24 (43.6%)
Body mass index, kg/m2	29.1 (5.6)	29.9 (5.5)	30.9 (6.4)	28.9 (4.3)
eGFR	67.9 (18.3)	64.5 (21.3)	61.7 (23.6)	67.4 (18.5)
Diabetes	150 (36.0%)	39 (36.1%)	23 (42.6%)	16 (29.6%)
Fasting glucose, mg/dL	115.5 (35.3)	120.4 (45.2)	125.9 (53.0)	115.0 (35.6)
Hb A1c, %	6.1 (1.1)	6.2 (1.2)	6.5 (1.5)	5.9 (0.6)
Hypertension	336 (79.8%)	87 (79.1%)	48 (87.3%)	39 (70.9%)
Systolic BP, mmHg	133 (20.1)	132.4 (15.9)	134.4 (17.7)	130.4 (13.7)
Ejection fraction, %	63.4 (8.8)	64.5 (6.9)	63.1 (8.3)	66.0 (4.8)
Incident coronary heart disease	62 (14.5%)	11 (10.0%)	11 (20.0%)	0 (0%)
Incident HF	224 (52.5%)	29 (26.4%)	29 (52.7%)	0 (0%)
Incident stroke	22 (5.2%)	15 (13.6%)	15 (27.3%)	0 (0%)

Abbreviations: HbA1c, hemoglobin A1c; BP, blood pressure; HF, heart failure; eGFR, estimated glomerular filtration rate.

^aData are presented as mean (standard deviation) or n (%).

^bIn the targeted assay comparison, baseline participant characteristics were similar according to cardiovascular disease case status (P-values > 0.05), except for ejection fraction (P-value = 0.02).

The mean correlation of the proteins overlapping on the SomaScan and Olink platforms was 0.46 (median: 0.53, range: −0.21 to 0.97, interquartile range: 0.13 to 0.76). The distribution of the protein correlations was bimodal, with good to excellent correlations for about half of the protein pairs (19%: r ≥ 0.8; 34%: 0.5 ≤ r < 0.8) and poor correlations for the other half (47%: r < 0.5, including 51 r ≤ 0) (Fig. 2). One of the modes was at a correlation near 0. There were 295 correlations with P-values < 1.2 × 10⁻⁴ (<0.05/417). Protein correlations were similar without removing outliers (see online Supplemental Fig. 1). Individual Spearman correlations (P-values) with and without removal of outliers are provided in online Supplemental Table 3. For each analyte, the intercept and slope from linear regression and the number of outliers removed (outliers per protein comparison: mean 4.4, median 4, range: 0 to 13) are also reported in Supplemental Table 3. For proteins with >1 measurement on platforms, the average intra-platform protein correlation was r = 0.32 (range: −0.25 to 0.98) on SomaScan (online Supplemental Table 4) and was r = 0.91 (range: 0.85 to 0.98) on Olink (online Supplemental Table 5).

Fig. 2.

Histogram of Spearman correlations between the 417 overlapping protein assay comparisons for the Olink and SomaScan platforms with outliers removed. Outliers for each protein defined by a protein abundance >3 standardized residuals based on simple linear regression with SomaScan protein (x-variable) and Olink protein (y-variable).

Average correlations for the overlapping proteins measured on Olink and SomaScan according to dilution bins on the 2 platforms are shown in online Supplemental Fig. 2. Cross-platform correlations were lowest for proteins measured in the most concentrated samples (i.e., Olink no dilution, SomaScan 1:5 dilution), which included the majority of proteins on each platform (64% SomaScan, 56% Olink; mean correlation of 0.37 in Olink panels without dilution vs 0.56 in Olink panels with dilutions). No other clear patterns emerged according to dilution bins.

Both platforms had proteins that had good to excellent negative correlations with eGFR (Fig. 3; SomaScan had 19 proteins with r < −0.5, Olink had 44 proteins with r < −0.5 while most other proteins were poorly correlated (|r| < 0.5) with eGFR. The proteins with good to excellent negative correlations with eGFR on both platforms tended to be more negatively correlated with Olink than with the SomaScan quantification. Of note, SomaScan and Olink measurements of cystatin-C were strongly correlated (r = 0.71) and were highly inversely correlated with eGFR on both platforms (SomaScan-eGFR: r = −0.87, Olink-eGFR: r = −0.79) and with plasma cystatin-C (measured by traditional immunoassays, SomaScan-cystatin-C: r = 0.89, Olink-cystatin-C: r = 0.81). Proteins that were not correlated with eGFR on SomaScan often had a similarly poor correlation for Olink quantification but a number of the proteins showed a markedly more negative correlation with Olink than SomaScan (Fig. 3).

Fig. 3.

Scatterplot of the correlations of the 417 overlapping proteins measured on the Olink and SomaScan platforms against eGFR. Spearman correlations of the proteins measured using SomaScan vs eGFR (x-axis) and Spearman correlations of the proteins measured using Olink vs eGFR (y-axis). The diagonal line is the identity line. Overlapping protein measurements are labeled by Entrez Gene symbol: ADM, adrenomedullin; AMBP, alpha-1-microglobulin; BAMBI, BMP and activin membrane-bound inhibitor homolog; CAPG, macrophage-capping protein; CD4, T-cell surface glycoprotein CD4; CD5, T-cell surface glycoprotein CD5; CD46, membrane cofactor protein; CD93, complement component C1q receptor; COL18A1, endostatin; CSF1, macrophage colony-stimulating factor 1; CST3, cystatin-C; DCN, decorin; EFEMP1, EGF-containing fibulin-like extracellular matrix protein 1; EGFR, epidermal growth factor receptor; EPHB4, ephrin type-B receptor 4; FABP4, fatty acid-binding protein, adipocyte; FAS, tumor necrosis factor receptor superfamily member 6; Gal9, galectin-9; IGFBP6, insulin-like growth factor-binding protein 6; IL10RB, interleukin-10 receptor subunit beta; IL15RA, interleukin-15 receptor subunit alpha; IL16, interleukin-16; IL18BP, interleukin-18-binding protein; KLK6, kallikrein-6; LTBR, tumor necrosis factor receptor superfamily member 3; MFAP5, microfibrillar-associated protein 5; PAPPA, pappalysin-1; PDL1, basement membrane-specific heparan sulfate proteoglycan core protein; PGF, placenta growth factor; PLC, basement membrane-specific heparan sulfate proteoglycan core protein; RARRES2, retinoic acid receptor responder protein 2; SLAMF1, signaling lymphocytic activation molecule; SPON2, spondin-2; TIMP1, metalloproteinase inhibitor 1; TF, serotransferrin; TFF3, trefoil factor 3; TNC, tenascin; TNFR1, tumor necrosis factor receptor superfamily member 1A; TNFR2, tumor necrosis factor receptor superfamily member 1B; TNFRSF9, tumor necrosis factor receptor superfamily member 9; TNFRSF10A, tumor necrosis factor receptor superfamily member 10A; TNFRSF14, tumor necrosis factor receptor superfamily member 14; TRAILR2, tumor necrosis factor receptor superfamily member 10B; UMOD, uromodulin; UPAR, urokinase plasminogen activator surface receptor; VEGFA, vascular endothelial growth factor A; VSIG2, V-set and immunoglobulin domain-containing protein 2.

Comparisons of Highly Multiplexed Platforms vs Targeted Immunoassay Methods

There were 110 participants (mean age 74 years, 49% were women, 44% self-identified their race as Black) in the targeted comparison of the 18 analytes. Characteristics of the 110 participants included in the targeted comparison study are shown in Table 1 and are reported overall and by case status. Given the 1:1 matching, participants were similar in terms of age, sex, race, and eGFR.

Assays with excellent overall correlation (r ≥ 0.8) between SomaScan and targeted immunoassays included GDF15, ST2, MMP1, adiponectin, leptin, and resistin (Table 2). Assays with modest overall correlation (0.5 ≤ r < 0.8) included osteopontin and IL6. Assays with poor overall correlation (r < 0.5) included TIMP1, MMP3, MMP7, MCP1, IL10, VCAM1, ICAM1, IL18, TNFα, and visfatin. Inter-immunoassay correlations were high except for the modest inter-immunoassay correlation for TNFα (r = 0.59) and poor inter-immunoassay correlations for TIMP1 (r = 0.18) and IL10 (r = 0.17). Correlations tended to be similar when stratified by case status, except for TIMP1 (Table 2).

Table 2.

Spearman correlations for 18 targeted analytes measured using the SomaScan platform (aptamer-based) and targeted immunoassay methods and odds ratios (95% CIs) for incident cardiovascular disease: the ARIC Study (2011 to 2017).

		Overall		Control		Case		OR for incident CVD (95% CI)a
Protein	Platform	Correlation with SomaScan	Inter-immunoassay correlation	Correlation with SomaScan	Inter-immunoassay correlation	Correlation with SomaScan	Inter-immunoassay correlation
GDF15	SomaScan	—b	—	—	—	—	—	2.81 (1.38–6.16)
	Multiplex-1	0.83	0.88	0.84	0.81	0.81	0.90	2.75 (1.54–5.35)
	Roche	0.92		0.92		0.90		2.51 (1.44–4.72)
ST2	SomaScan	—	—	—	—	—	—	1.09 (0.51–2.35)
	ELISA	0.91	0.84	0.89	0.73	0.92	0.91	1.31 (0.64–2.79)
	Multiplex-1	0.88		0.82		0.92		1.67 (0.77–3.88)
Osteopontin	SomaScan	—	—	—	—	—	—	3.24 (1.23–10.02)
	Multiplex-1	0.56	—	0.53	—	0.56	—	1.69 (1.05–2.97)
IL6	SomaScan	—	—	—	—	—	—	2.72 (0.98–8.40)
	hsELISA	0.69	0.91 (vs hsMP)	0.66	0.91 (vs hsMP)	0.73	0.92 (vs hsMP)	1.41 (0.96–2.11)
	hsMultiplex	0.63	0.81 (vs MP1)	0.62	0.82 (vs MP1)	0.63	0.80 (vs MP1)	1.38 (0.93–2.10)
	Multiplex-1	0.50	0.76 (vs hsELISA)	0.55	0.78 (vs hsELISA)	0.46	0.75 (vs hsELISA)	1.52 (0.76–3.13)
MMP1	SomaScan	—	—	—	—	—	—	0.97 (0.61–1.52)
	Multiplex-2	0.87	—	0.89	—	0.86	—	1.14 (0.76–1.72)
TIMP1	SomaScan	—	—	—	—	—	—	2.60 (0.74–12.56)
	ELISA	0.30	0.18	0.01	0.06	0.57	0.26	3.20 (1.21–9.52)
	Multiplex-1	0.11		0.22		−0.04		2.38 (0.21–28.17)
MMP3	SomaScan	—	—	—	—	—	—	1.90 (0.46–8.25)
	Multiplex-2	−0.16	—	−0.23	—	−0.10	—	1.61 (0.95–2.81)
MMP7	SomaScan	—	—	—	—	—	—	1.76 (0.84–3.88)
	Multiplex-2	0.49	—	0.24	—	0.61	—	1.34 (0.97–1.88)
MCP1	SomaScan	—	—	—	—	—	—	1.05 (0.47–2.33)
	Multiplex-1	0.46	—	0.26	—	0.53	—	2.35 (1.13–6.08)
IL10	SomaScan	—	—	—	—	—	—	3.72 (1.11–16.69)
	hsMultiplex	0.03	0.17	−0.02	0.17	0.04	0.17	1.05 (0.84–1.32)
	Multiplex-1	−0.17		−0.21		−0.19		1.07 (0.72–1.59)
VCAM1	SomaScan	—	—	—	—	—	—	3.46 (0.94–13.95)
	Multiplex-1	0.36	—	0.31	—	0.37	—	1.36 (0.68–2.73)
ICAM	SomaScan	—	—	—	—	—	—	1.26 (0.60–2.68)
	Multiplex-1	−0.02	—	−0.02	—	−0.04	—	1.46 (0.90–2.44)
IL18	SomaScan	—	—	—	—	—	—	2.11 (0.98–4.79)
	Multiplex-1	0.14	—	−0.01	—	0.19	—	3.01 (1.24–7.86)
TNFα	SomaScan	—	—	—	—	—	—	2.06 (0.59–8.82)
	hsMultiplex	0.16	0.59	0.32	0.72	0.08	0.50	1.54 (0.78–3.15)
	Multiplex-1	0.14		0.21		0.07		1.75 (0.62–5.16)
Adiponectin	SomaScan	—	—	—	—	—	—	0.70 (0.39–1.22)
	ELISA	0.91	—	0.91	—	0.91	—	0.88 (0.56–1.37)
Leptin	SomaScan	—	—	—	—	—	—	1.02 (0.69–1.51)
	ELISA	0.95	—	0.97	—	0.95	—	1.06 (0.84–1.37)
Resistin	SomaScan	—	—	—	—	—	—	3.41 (1.47–8.91)
	ELISA	0.86	—	0.82	—	0.88	—	3.14 (1.66–6.52)
Visfatin	SomaScan	—	—	—	—	—	—	0.34 (0.08–1.13)
	ELISA	−0.05	—	−0.02	—	−0.08	—	1.12 (0.86–1.49)

^aBolded values indicate a statistically significant OR.

^bNot applicable.

GDF15 (per log₂) measured using SomaScan (OR 2.81 [95% CI, 1.38–6.16]) and both the immunoassays (Multiplex-1 OR 2.75 [95% CI, 1.54–5.35]; Roche OR 2.51 [95% CI, 1.44–4.72]) were strongly associated with incident CVD (Table 2). Higher levels of osteopontin measured using the SomaScan (OR _{per log2(osteopontin)} 3.24 [95% CI, 1.23–10.02]) and the Multiplex-1 immunoassay configuration (1.69 [95% CI, 1.05–2.97]) were associated with elevated odds of future CVD. Higher levels of TIMP1, MCP1, and IL18 from one of the targeted assays were associated with incident CVD, while measurements based on the SomaScan were not statistically significantly associated with incident CVD. Higher levels of resistin measured using the SomaScan (OR _{per log2(resistin)}: 3.41 [95% CI, 1.47–8.91]) and the targeted immunoassay (3.14 [95% CI, 1.66–6.52]) were associated with incident CVD.

There were 29 participants with measurements for 16 analytes (TNFα, MMP1, leptin, GDF15, MMP7, VCAM1, resistin, ST2, IL6, osteopontin, MCP1, IL18, TIMP1, MMP3, IL10, and ICAM1) that overlapped across all platforms. We did not provide correlations for TNFα because TNFα concentrations for all 29 participants were below the Olink LOD. In this small subset, there was substantial variation in pairwise correlations of the protein measurements across platforms (online Supplemental Table 6). Pairwise correlations across the platforms/configurations were generally good or modest for MMP1, leptin, GDF15, MMP7, VCAM1, resistin, ST2, IL6, and osteopontin. For the remaining proteins (MCP1, IL18, TIMP1, MMP3, IL10, and ICAM1), Olink tended to have higher correlations with the targeted immunoassay, however correlations for these proteins were largely poor or modest, except for Olink IL10 vs hsMultiplex (r = 0.76) and Olink MMP3 vs Multiplex-2 (r = 0.87).

Discussion

In this study, we assessed the comparability of 2 proteomic platforms using aptamer-based (SomaScan version 4) and proximity-extension immunoassay (Olink Proseek v.5003) methods. We also evaluated the comparability of 18 targeted protein measurements using aptamer-based methods against targeted immunoassays, and in 29 participants examined correlations across all platforms. While many proteins were highly correlated across platforms, a substantial number were not correlated at all. Heterogenous correlations between proteins assays can result from the assays measuring different features (epitopes, folding, modifications) of the same protein, the same features differently, or features outside the target protein (mis-identification, interference). Understanding assay agreement and heterogeneity is important for scientists interpreting protein measures and biological associations in studies that utilize proteomic platforms.

Previous studies comparing highly multiplexed proteomics platforms (aptamer-technology vs immunoassay) have similarly reported a substantial range in the correlations for the overlapping proteins. One study reported on 871 overlapping plasma proteins measured on the aptamer-based (SomaScan version 4) and immunoassay (Olink) from 485 participants (4). The distribution of Spearman correlation coefficients (median r = 0.38, range: −0.61 to 0.96)) was similar to our study (Fig. 2). They reported that 64% of identified protein quantitative trait loci (pQTLs) were concordant across the platforms, which supports that the majority of overlapping, measurable proteins are measuring the intended protein targets. However, pQTLs may not directly translate to changes in circulating protein abundance. Another study reported that 106 of 163 pQTLs identified using the Somalogic platform were also identified on the Olink platform (16). This research, together with our findings, suggest many of the overlapping proteins measured on popular multiplex assays may be capturing the intended biologic targets; however, quantitative agreement can vary substantially for many other proteins.

Most other comparison studies of proteomic platforms have been based on smaller sample sizes and/or older panels which measured fewer proteins (online Supplemental Table 7) (3, 6, 16–18). One of the prior comparison studies reported a similar distribution in SomaScan (version 1.1k) vs Olink protein correlations (median 0.36, range: −0.58 to 0.93) to our study (Fig. 2) for 425 overlapping protein measurements among 10 myocardial infarction patients (48 plasma samples due to repeated measurements) (3). There were 249 correlations of SomaScan aptamers vs Olink reported in the prior comparison that overlapped with our study, and the overlapping correlations were quite comparable to our findings (online Supplemental Fig. 3, r = 0.59, P < 0.001). Potential reasons for the differences in correlations between studies may relate to the different SomaScan versions (version 4 vs 1.1k), sample size (n = 425 vs n = 10), and populations (community-based study vs myocardial infarction patients).

Our findings inform thoughtful application and interpretation of assays in the rapidly evolving and expanding field of proteomics. Researchers should be cognizant of potentially divergent findings across platforms, specifically those using multiplex immunoassays vs aptamer-based methods (approximately 1/2 of the proteins had good or modest correlations and approximately 1/2 poor correlations). Assay comparisons provide information about agreement but validity requires knowing the biologically relevant aspect of each protein. Immunoassays often disagree with each other so disagreement with a proteomic platform does not definitively imply which is superior. It is also important to consider assay characteristics as factors that could influence comparisons across platforms. For example, the SomaScan platform reported values below the LOD, while protein values below the LOD on Olink were imputed to the LOD (average % below the LOD in ARIC was 1% on SomaScan and 10% on Olink). Olink values for 3 proteins (interferon-γ [IFNγ], interleukin-2 [IL2], leukotriene A4 hydrolase [LTA4H]) in ARIC were all below the LOD and could not be correlated with SomaScan measurements. Correlations were very poor when the majority of values on one platform were below the LOD (e.g., interleukin-13, IL2). Additionally, the SomaScan version 4 has approximately 5000 aptamers available while fewer proteins were measured on the Olink panels. Our findings suggest that approximately 1/2 of the overlapping proteins from the different platforms are highly (19%) or moderately (34%) correlated but the other 1/2 approximately had poor correlations (r < 0.5) and may be capturing different information.

Limitations include, first, the long-term storage of the plasma specimens and possible protein degradation. However, we have previously demonstrated in ARIC the high reliability of protein markers over time using the SomaScan platform (7). Second, for both Olink and SomaScan, relative protein abundance (not absolute concentrations) was examined and may not be on the same scale. Third, power was limited, given the case-control design, to examine associations with incident events. However, we still found that all measures of GDF15, a known cardiac biomarker (19), were predictive of incident CVD. Finally, extrapolation of the percentage of protein assay agreement to the full and growing coverage (approximately 7000 assays in SomaScan and approximately 3000 assays in Olink latest platforms) should be done with caution but we should expect substantial variation between assays for different proteins.

Strengths of this analysis include the large sample size and rich assortment of phenotypic measurements. We were able to compare analytes across multiple assays (including targeted immunoassays), and to compare these analytes to eGFR—a biologically relevant consideration for many proteins—in efforts to identify which assay was perhaps of greater biologic relevance. Many proteins shared inverse correlations with eGFR, however, a fraction of the Somalogic proteins had much weaker correlations with eGFR than the correlations found for those proteins measured on Olink (Fig. 3, and Supplemental Table 3). Redundant protein quantification across (and possibly within) platforms could increase our knowledge of disease associations.

In this study of 427 participants with 417 overlapping protein comparisons (366 unique proteins) measured using 2 different highly multiplexed proteomic platforms, we found that correlations between platforms varied substantially. Future research is needed to characterize the complex underlying traits or pathways contributing to discordant findings of specific proteins across platforms. Overall, researchers should consider the specific performance characteristics of each assay for each protein in planning and interpreting studies that involve proteomic measurements. Continued efforts to standardize untargeted (and targeted) assays with detailed comparisons of when and how they align is important. Future studies should also examine how well untargeted platforms allow for assessment of change in proteins over time.

Data Availability

The data generated in this study may be made available upon reasonable request. Consistent with a prespecified policy for access of ARIC data, requests may be submitted to the ARIC steering committees for review. The analytic methods and study materials may be made available to other researchers for purposes of reproducing the results or replicating the procedure upon reasonable request.

Supplementary Material

Supplementary material is available at Clinical Chemistry online.

Nonstandard Abbreviations

ARIC, Atherosclerosis Risk in Communities; HF, heart failure; pQTL, protein quantitative trait loci; GDF15, growth differentiation factor 15; ST2, interleukin-1 receptor-like 1; MMP1, interstitial collagenase; IL6, interleukin-6; TIMP1, metalloproteinase inhibitor 1; MMP3, stromelysin-1; MMP7, matrilysin; MCP1, C-C motif chemokine 2; IL10, interleukin-10; VCAM1, vascular cell adhesion protein 1; ICAM1, intercellular adhesion molecule 1; IL18, interleukin-18; TNFα, tumor necrosis factor; eGFR, estimated glomerular filtration rate; CVD, cardiovascular disease; SOMAmers, Slow Off-rate Modified Aptamers; LOD, limit of detection.

Author Contributions

The corresponding author takes full responsibility that all authors on this publication have met the following required criteria of eligibility for authorship: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved. Nobody who qualifies for authorship has been omitted from the list.

Authors’ Disclosures or Potential Conflicts of Interest

Upon manuscript submission, all authors completed the author disclosure form. Disclosures and/or potential conflicts of interest:

Employment or Leadership

None declared.

Consultant or Advisory Role

R.C. Hoogeveen has received consulting fees from Denka Seiken. C.M. Ballantyne has been a consultant to Denka Seiken, Abbott Diagnostics, and Roche. J. Coresh is a scientific advisor to Somalogic. F. Zannad, Amgen, Applied Therapeutics, AstraZeneca, Bayer, Boehringer, Cardior, Cereno, CEVA, Cellprothera, CVRx, Novartis, NovoNordisk, and ACCELERON.

Stock Ownership

F. Zannad, G3 Pharma, Cereno, and Cardiorenal.

Honoraria

F. Zannad, Boehringer, Merck, Bayer, and Vifor Fresenius.

Research Funding

The Atherosclerosis Risk in Communities Study has been funded in whole or in part with Federal funds from the National Heart, Lung, and Blood Institute (NHLBI), National Institutes of Health (NIH), Department of Health and Human Services, under contract nos. (75N92022D00001, 75N92022D00002, 75N92022D00003, 75N92022D00004, 75N92022D00005). Research reported in this publication was supported by the NIH/NHLBI grants T32HL007024 (M.R. Rooney), K24HL152440 (E. Selvin), R01-HL134320 (C.M. Ballantyne and E. Selvin), R01-HL146907 (C.E. Ndumele), NIH/NIDDK grant R01DK089174 (E. Selvin), and the American Heart Association grant 20SFRN35120152. The research leading to these results has received funding from the European Union Commission’s Seventh Framework Programme under grant agreement no. 305507 (HOMAGE [Heart Omics in Ageing consortium]). Johns Hopkins University has signed a collaboration agreement with Somalogic to conduct SomaScan of stored ARIC samples at no charge in exchange for the rights to analyze linked ARIC phenotype data. R.C. Hoogeveen has received research grants (to institution) from Denka Seiken. C.M. Ballantyne has received research grants (to institution) from Abbott and Roche. D.J. Couper, funding from NHLBI to institution. O. Tang, NIH/NLHBI T32HL007024 and NIH/NIDDK F30DK120160. J. Coresh, NIH and National Kidney Foundation. M.E. Grams, NIH to institution. W. Tang, NIH to institution.

Expert Testimony

F. Zannad, Hogan and Lovells.

Patents

None declared.

Role of Sponsor

The funding organizations played no role in the design of study, choice of enrolled patients, review and interpretation of data, preparation of manuscript, or final approval of manuscript.

Acknowledgments

The authors thank the staff and participants of the ARIC study for their important contributions.