Technology and Tool Development for BACPAC: Qualitative and Quantitative Analysis of Accelerated Lumbar Spine MRI with Deep-Learning Based Image Reconstruction at 3T

Abstract Objectives To evaluate whether combining fast acquisitions with deep-learning reconstruction can provide diagnostically useful images and quantitative assessment comparable to standard-of-care acquisitions for lumbar spine magnetic resonance imaging (MRI). Methods Eighteen patients were imaged with both standard protocol and fast protocol using reduced signal averages, each protocol including sagittal fat-suppressed T2-weighted, sagittal T1-weighted, and axial T2-weighted 2D fast spin-echo sequences. Fast-acquisition data was additionally reconstructed using vendor-supplied deep-learning reconstruction with three different noise reduction factors. For qualitative analysis, standard images as well as fast images with and without deep-learning reconstruction were graded by three radiologists on five different categories. For quantitative analysis, convolutional neural networks were applied to sagittal T1-weighted images to segment intervertebral discs and vertebral bodies, and disc heights and vertebral body volumes were derived. Results Based on noninferiority testing on qualitative scores, fast images without deep-learning reconstruction were inferior to standard images for most categories. However, deep-learning reconstruction improved the average scores, and noninferiority was observed over 24 out of 45 comparisons (all with sagittal T2-weighted images while 4/5 comparisons with sagittal T1-weighted and axial T2-weighted images). Interobserver variability increased with 50 and 75% noise reduction factors. Deep-learning reconstructed fast images with 50% and 75% noise reduction factors had comparable disc heights and vertebral body volumes to standard images (r2≥ 0.86 for disc heights and r2≥ 0.98 for vertebral body volumes). Conclusions This study demonstrated that deep-learning-reconstructed fast-acquisition images have the potential to provide noninferior image quality and comparable quantitative assessment to standard clinical images.


Introduction
Lower back pain (LBP) is a major cause of disability responsible for limiting work related activities and ultimately a cause of deteriorating quality of life. 1 The impact of LBP results in loss of work productivity and the healthcare expenditure alone ranges from $50 to $91.8 billion yearly in the United States. 2,3 Magnetic resonance imaging (MRI) is extensively used in the clinical evaluation of LBP, diagnosing abnormalities in the vertebral bodies, intervertebral discs, nerve roots, and spinal cord related to degenerative disease, osteoporotic compression fracture, trauma, infectious and inflammatory conditions, and tumors. Even though imaging findings of abnormalities are not always associated with LBP, 4-6 a growing number of studies have demonstrated that MRI findings can function as important biomarkers associated with LBP. [7][8][9] Currently, a standard clinical lumbar spine MRI protocol consists of high-resolution T 1 -weighted and T 2 -weighted 2D fast-spin echo (FSE) sequences in multiple orientations and requires approximately 30-minute scan time. When intravenous contrast injection is required, additional post-contrast sequences are acquired, further increasing the total scan time. Remaining still in the MRI scanner for a long duration can be a challenge for patients particularly when in pain, and can generate motion artifacts. Reducing acquisition time would improve patient experience and image quality in the setting of LBP. Furthermore, a shorter clinical protocol time would allow for adding special quantitative imaging 10,11 as part of routine imaging to look beyond structural abnormalities.
Numerous strategies have been developed for fast acquisition; partial Fourier, 12,13 parallel imaging, 14,15 and compressed sensing, 16 which exploit k-space data redundancy or spatial correlation, are the most common schemes to shorten acquisition times. However, these acceleration approaches suffer from reduced signal-to-noise ratio (SNR) or blurring, [17][18][19] and application of those would not provide sufficient imaging quality for clinical 2D FSE lumbar spine imaging that normally uses multiple signal averaging (number of excitations [NEX] > 1) to achieve sufficient SNR.
Recently, deep-learning (DL) based methods have been rapidly developed for various MRI research areas including undersampled data reconstruction, 20,21 segmentation, 22 super-resolution, 23 and denoising. 24 In particular, a DL reconstruction method employing a deep convolutional neural network, which directly applies to raw k-space data, has been developed to improve SNR and to reduce ringing artifacts (commercially available as AIR Recon DL, GE Healthcare). 25 The feasibility of this particular DL reconstruction to improve SNR while providing similar or improved diagnostic capability compared to conventional reconstruction has been validated for high-resolution post-operative pituitary imaging, 26 late gadolinium enhancement cardiac imaging, 27 and clinical assessment of the prostate cancer, 28 the peripheral nerve, 29 and the hip and shoulder, 30,31 but has not been validated comprehensively for clinical lumbar spine MRI at 3 T yet.
In this study we evaluated the application of DL reconstruction to fast, reduced-NEX acquisitions for 2D clinical lumbar spine MRI in comparison to standard acquisitions through both qualitative and quantitative analysis. Qualitative analysis was based on radiologists' evaluation. Quantitative analysis was performed by applying previously developed DL-based segmentation algorithms to segment intervertebral discs and vertebral bodies and to compare extracted disc heights and vertebral body volumes, 32 which can be important biomarkers associated with spine degeneration [33][34][35] and biomechanical modeling. 36

Image acquisition and reconstruction
This study was conducted in accordance with and approval by the local institution review board with waived informed consent. Images and raw k-space data from 18 patients with LBP, scanned between March to June 2020 at our institution, were retrospectively collected. A GE 3 T SIGNA Premier MRI scanner and a table-embedded 32-channel spine posterior coil array (GE Healthcare, Waukesha, WI, USA) were used for imaging.
Two different clinical protocols were used at our institution for lumbar spine MRI, a musculoskeletal (MSK) radiology lumbar spine protocol and a neuroradiology lumbar spine protocol. The MSK radiology protocol was tailored for better depiction of orthopedic and rheumatologic disorders while the neuroradiology protocol for better depiction of spinal cord and peripheral nerve disorders (exploits a longer TE for T 2 -weighted sequences). These standard lumbar spine protocols consisted of sagittal fat-saturated T 2 -weighted 2D fast spin-echo (FSE) (SAG T2 FS) sequence, sagittal T 1 -weighted 2D FSE (SAG T1) sequence, axial T 2 -weighted 2D FSE (AX T2) sequence, axial T 1 -weighted 2D FSE sequence, and coronal T 1 -weighted 2D FSE sequence. These clinical sequences used a NEX of 1-2 to achieve SNR for a high diagnostic confidence, and all imaging parameters were within ranges recommended by the Back Pain Consortium (BACPAC) Spine Imaging Working Group. 37 For each patient, imaging using one of these clinical protocols based on physicians' order was performed, and then imaging with a fast protocol consisting of SAG T2 FS, SAG T1, and AX T2 sequences, with only a NEX reduced to 0.5-1, was also performed. With 0.5 NEX prescription, partial Fourier was applied along the k y dimension. 12,13 Table 1 shows image parameters for SAG T2 FS, SAG T1, and AX T2 sequences for both MSK radiology and neuroradiology protocols, with denoting NEX for both standard and fast acquisition protocols and approximate scan times.
The vendor-supplied prototype version of AIR Recon DL was applied to raw k-space data from the reduced-NEX fast acquisition protocol offline. AIR Recon DL pipeline included a deep convolutional neural network with 4.4 million parameters in approximately 10 000 kernels, and designed to improve input data's SNR while enhancing sharpness and reducing ringing artifacts. 25 The deep convolution neural network was trained using pairs of images representing nearperfect images (high-resolution images with minimal ringing and very low noise levels) and synthesized images of lower resolution versions with more truncation artifacts and higher noise levels. 25 Data with 0.5 NEX acquisition was also included for training data sets. Image augmentation included rotations and flips, intensity gradients, phase manipulations, and additional Gaussian noise, to add robustness, and over 4 million unique image/augmentation combinations over various anatomy were used for training. 25 The raw data was reconstructed using the algorithm with three different noise reduction factors, 25%, 50%, and 75%. Vendor's standard reconstruction computer with multiple CPU cores was used for reconstruction, and each DL reconstruction took approximately one minute.

Qualitative assessment and statistical analysis
For all of three sequences (SAG T2 FS, SAG T1, and AX T2), original images from the standard clinical acquisitions and fast acquisitions, and DL-reconstructed images from the fast acquisitions with 25%, 50%, and 75% noise reduction factors (will name those as Standard images, Fast images, and Fast-DL images (or Fast DL25, Fast DL50, and Fast DL75 images), respectively) were evaluated. Patient information was anonymized, and image details such as the NEX, the application of DL reconstruction algorithm, and DL reconstruction noise reduction factor, were all removed from the DICOM header. Qualitative image assessment was performed by three board-certified radiologists specializing in neuroradiology or MSK radiology, CC (neuroradiologist), TML (MSK radiologist) and JVM (neuroradiologist), with 25, 26, and 5 years of experience, respectively. All image series from the two lumbar spine protocols were randomly ordered for each sequence (90 series for each sequence, five series for each of 18 patients) and presented to the radiologists. The radiologists graded each series sequentially for five different categories, "Apparent SNR," "Ability to Discern Anatomical Structures," "Diagnostic Confidence," "Overall Image Quality," and "Presence of Artifacts." The first four categories were rated on a Likert scale of 1 to 5 (1-5: poor to excellent), while the last metric was rated as binary, 0 (no artifacts) and 1 (artifacts), as described in Table 2.
Radiologists were blinded to each other's rating.
Noninferiority testing 38,39 was performed to compare the average scores of Fast and Fast-DL images from the three radiologists to those from Standard images. The null and alternative hypotheses for the first four categories were: H 0 : l D À l S Àd l and H 1 : l D À l S > Àd l where l D represents the average scores from Fast or Fast-DL groups while l S represents those from Standard groups. The noninferiority margin Àd l was set as À0.5. For the "Presence of Artifacts," the null and alternative hypotheses were: H 0 : l D À l S ! d b and H 1 : l D À l S < d b where the noninferiority margin d b was set as 0.2 (as a lower difference was improvement with this category). A one-sided Wilcoxon signed-rank test was performed to assess noninferiority, and P values were calculated using normal approximation and continuity correction. 40,41 The P values were adjusted using the false discovery rate (FDR) to correct for multiple testing. 42 Interobserver agreement of scores by the three radiologists for each category and each sequence was conducted using a Conger's kappa (j) coefficient with quadratic weighting. 43 The coefficient was interpreted as follows: j < 0, no agreement; 0 < j 0.2, slight agreement; 0.2 < j 0.4, fair agreement; 0.4 < j 0.6, moderate agreement; 0.6 < j 0.8, substantial agreement; and 0.8 < j 1, almost perfect agreement.

Automatic segmentation and quantitative analysis
Two convolutional neural networks incorporating 2D V-Net architecture, 44 previously trained with 27 and 24 SAG T1 patient data sets (for intervertebral discs and vertebral bodies respectively) 32 were used to segment intervertebral discs and vertebral bodies separately. After acquiring segmentation masks on each image slice, 2D slice masks were reconstructed into 3D volume masks and postprocessed to smooth, fill holes, and identify largest connected components. Disc segmentation masks were used to calculate intervertebral disc height by computing a 3D centroid for each disc to find the most central slice and extracting the shortest side length from a minimum bounding rectangle of the disc segmentation. The vertebral body segmentation masks were used to calculate each vertebral body volume by summing the number of foreground pixels for each body and converting to patient-based space. Segmented discs (mostly located between T12 and S1 vertebral bodies) or vertebral bodies (among T12-S1 vertebral bodies) were manually matched between Standard, Fast, and Fast-DL images. These derived quantities from Fast and Fast-DL images were compared to those from Standard images using correlation analysis.

Patient population
The study included 18 patients (10 males and 8 females), with patient age ranging from 31 to 82 years (median age of 61 years). Thirteen out of 18 patients were imaged with the MSK radiology protocol while the other five patients were imaged with the neuroradiology protocol. Various common degenerative spine abnormalities were demonstrated on their MR images such as Modic changes in vertebral endplates, disc osteophytes, annular fissures, disc protrusion, disc extrusion, spinal and foraminal stenoses, and facet arthropathy and cyst, and some patients had previous surgery or metal implants. Figure 1 compares Standard, Fast, and Fast-DL images from a 63-year-old male patient with LBP for more than six weeks, acquired with the MSK lumbar spine protocol. Sagittal images ( Figure 1A-B) show Modic type I endplate changes in the posterior L3 and L4 vertebral bodies (fibrovascular endplate change) and an L3-L4 disc extrusion resulting in mild central canal stenosis. On axial images at L3-L4 disc ( Figure 1C), mild facet arthropathy and moderate right foraminal narrowing were identified. Fast images provided a lower SNR than Standard images as expected, but the reduction of noise was perceived with Fast-DL images while degeneration in vertebral bodies, discs, spinal cord, and facet joints was well visualized. Images from a 73-year-old male patient acquired with the neuroradiology protocol are shown in Figure 2. SAG T2 FS images show decreased contrast over discs and intervertebral bodies while increased contrast between spinal cord and cerebrospinal fluid compared to Figure 1 due to a higher TE. This patient had dessiccated (dehydrated) intervertebral discs and small Schmorl's nodes (endplate disc herniations) in the L1, L3, L4, and L5 vertebral bodies ( Figure 2B). Disc protrusions and mild-moderate central canal stenoses at L3-L4 and L4-L5 discs ( Figure 2A) and mild facet arthropathy with mild left foraminal narrowing ( Figure 2C) were also identified. These abnormalities were again well-visualized on Fast-DL images. Figure 3 compares Standard, Fast, and Fast DL50 images from other two patients (78 year-old and 39 year-old males) visualizing different anatomical structures/pathologies. The patient in Figure 3A had transitional lumbosacral anatomy and had moderate neuroforaminal stenosis at the L5-S1 disc level and severe L4 and L5 facet arthropathy. Figure 3B shows central annular fissure (bright signal depicted by an arrow). These abnormal structures were well delineated with Fast DL50 images again.

Qualitative assessment
The mean Likert scores over the three radiologists for all 18 patients are summarized in Figure 4 as box plots for the categories of "Apparent SNR," "Ability to Discern Anatomical Structures," "Diagnostic Confidence," and "Overall Image Quality." The means and standard deviations of the average scores from the three radiologists are presented in Table 3 for the five categories including the "Presence of Artifacts." The mean scores from the Fast groups were lower than those from the Standard groups for the first four categories, but with DL reconstruction, the mean scores improved. Table 4 shows the results of the noninferiority tests. The (Pseudo)median 40 of the difference scores between Fast/Fast DL groups and Standard groups as well as the P values from the one-sided tests for the specified noninferior margins are indicated. The mean scores from Fast groups were statistically inferior to those from standard groups for four to five categories of all sequences. However, with Sag T2 FS, all Fast DL groups with the three different noise reduction factors showed statistical noninferiority compared to the Standard groups across all categories. For SAG T1, Fast DL25 and Fast DL50 images received inferior scores for most of categories, while Fast DL75 images showed noninferiority for "Apparent SNR," "Ability to Discern Anatomical Structures," and "Diagnostic Confidence." For AX T2, Fast DL50 images attained statistically noninferior scores for "Apparent SNR," "Ability to Discern Anatomical Structures," and "Diagnostic Confidence." Conger's j coefficients to check interobserver agreement are shown in Table 5. There was no to slight agreement on "Presence of Artifacts" (j < 0.2), but for other criteria, fair to moderate agreement was demonstrated for SAG T2 (0.255 < j < 0.550), and slight to moderate agreement for SAG T1 (0.051 < j < 0.550). The AX T2 sequence exhibited less Technology and Tool Development for BACPAC S153 agreement (À0.003 < j < 0.516) than the other two sequences. Overall, Fast DL75 groups and Fast DL50 groups showed the least agreement among the radiologists, suggesting varying levels of perception of DL-reconstructed images.
Automatic segmentation and quantitative analysis Figure 5 shows standard SAG T1 images (A, D) and compares segmentation masks of intervertebral discs (B, E) and vertebral bodies (C, F) attained from Standard, Fast, and Fast-DL images of two patients. Patient A was the same with the patient in Figure 1. Segmentation masks shown were right after applying convolutional neural networks and before applying additional post-processing. With Standard images, all discs were segmented well; however, with Fast images and Fast-DL images using a lower noise reduction factor, some pixels within the disc regions were not included for the L1-L2 disc mask (Patient A) and L4-L5 disc mask (Patient B), respectively. By incorporating DL reconstruction with a 50% or 75% noise reduction factor, both discs were segmented well, similarly to Standard images. For the vertebral bodies, segmentation worked well with Fast images and the Fast-DL images except the T12 vertebral body that included some imaging artifacts at the interface of the vertebral body and fat, probably caused by 0.5 NEX (partial Fourier) acquisition. Correlation plots of intervertebral disc heights and vertebral body volumes extracted from segmentation masks of Fast and Fast-DL images vs those from Standard images are shown in Figure 6. Linear equations and the correlation of   determination (r 2 ) were also denoted. Significant correlations were observed for all cases for both the intervertebral disc height and vertebral body volume. By applying DL reconstruction, r 2 further increased, and the highest r 2 was achieved when the noise reduction factor was 50% for disc heights (r 2 ¼ 0.87) and when the noise reduction factor was 75% for vertebral body volumes (r 2 ¼ 0.99). Table 6 compares the mean and standard deviations of extracted disc heights and intervertebral body volumes over the segmented discs and vertebral bodies. The mean and standard deviation (SD) of disc height from Standard images was 10.69 6 2.30 mm, and the difference was À1.31%, À0.84%, and À0.84% for Fast, Fast DL25, Fast DL50, and Fast DL75 images, respectively. The mean intervertebral body volume of Standard images was 26.98 6 9.33 cm 3 , and the difference was À2.19%, À1.45%, À0.79%, and À0.52% for the other image groups, respectively.

Discussion
In this work, we demonstrate that fast image acquisition with DL reconstruction has a potential to provide clinically diagnostic image quality for lumbar spine MRI while significantly shortening the exam duration. Additionally, we showed that convolutional neural networks trained with standard clinical images can reliably segment both intervertebral discs and vertebral bodies on Fast-DL images. Our patients had various LBP-associated imaging findings including degeneration in discs, vertebral bodies, facet joints with spinal canal and foraminal nerve compromise, while some patients had prior  The coefficients were interpreted as follows: j < 0, no agreement; 0 < j 0.2, slight agreement; 0.2 < j 0.4, fair agreement; 0.4 < j 0.6, moderate agreement; 0.6 < j 0.8, substantial agreement; and 0.8 < j 1, almost perfect agreement. a Bold text was used with agreement > 0. 4. surgery, thus increasing the generalizability of our assessment and analysis to the broad-spectrum of clinical pathologies. Although the radiologists evaluated "Apparent SNR" using a 1-5 scale without any quantitative measurement, we found that their mean "Apparent SNR" scores were, to some extent, in agreement with the expected SNR variations across different imaging groups. Fast images had a theoretical SNR reduction of 29/50/ 29% compared to Standard images, based on NEX differences, for SAG T2 FS, SAG T1, and AX T2, respectively. The mean scores for Fast images were lower than those for Standard images for all sequences. With SAG T2 FS and AX T2, DL reconstruction with noise reduction factors of 25% or higher provided mean scores that were noninferior to those of Standard images, while with SAG T1, only DL reconstruction with only a noise reduction factor of 75% provided a noninferior score (the P values for Fast DL50 images of SAG T1 for non-inferiority was .079).
For comparing qualitative scores between different imaging groups, we used a noninferiority test 38,39 instead of a standard significance test. A significance test is to determine whether there is a significance difference between groups (ie, one group is superior to the other) by checking if the P values is smaller than .05. However, with this test, justifying similarity with P ! 0.05 is not appropriate as a high P values can be resulted from a lack of evidence in difference or a lack of statistical power due to a small sample size. 45 An equivalent/noninferiority test has been specifically proposed to demonstrate similarity by checking whether the confidence interval of the difference is within a predetermined equivalence or noninferiority margin. In this study, we used the noninferiority test as we wanted to prove whether Fast-DL images are not worse than Standard images (can be equivalent or superior) rather than these are equivalent to Standard images. There is no standard about what should be noninferiority/equivalent margins for radiology research yet, 46 but here we used the noninferiority margins of À0.5 for a 5point scale and 0.2 for a binary scale, which were 12.5 and 20% of total ranges of scores respectively. Based on qualitative assessment, the Fast-DL images with all of 25, 50, and 75% noise reduction factors from SAG T2 FS were noninferior (comparable or superior) to the standard clinical images in all categories, "Apparent SNR," "Ability to Discern Anatomical Structures," "Diagnostic Confidence," "Overall Image Quality," and "Presence of Artifacts." With either SAG T1 or AX T2, Fast DL50 and Fast DL75 images provided statistically noninferior scores for four out of 10 comparisons. However, statistically inferior scores were recorded for "Overall Image Quality" and "Presence of Artifacts" on Fast DL50 and Fast DL75 images in both sequences. We believe that this was not due to problems with DL reconstruction, but was rather due to enhanced motion/flow artifacts with partial Fourier acquisition (0.5 NEX) of the fast protocol, and DL reconstruction might make the artifacts more apparent due to increased SNR. Enhanced wrapping/ghost artifacts with DL reconstruction have been previously reported. 27,29 For the fast protocol, parallel imaging can be also used by exploiting multi-coil elements of posterior array coils. We expect motion/flow artifacts would not be that enhanced with the use of parallel imaging. For reconstruction, parallel imaging reconstruction can be applied to fill missing k-space data first, followed by DL reconstruction.
There was interobserver variability when assessing image quality by three radiologists. We observed that radiologist 1 (neuroradiologist) perceived standard images to have overall better image quality, while radiologists 2 (MSK radiologist) and 3 (neuroradiologist) were favorable to DL-reconstructed images, yielding the lowest Conger's coefficients for Fast DL75 images. These variabilities made demonstrating statistical noninferiority more challenging. Radiologists have inherent biases, with some preferring realistic image textures and others sharper and smooth images, 47 similarly to what DL reconstructions provide. In addition, differences in clinical disease experience between neuroradiologists and MSK radiologists may further explain the variability in interpretation of spine imaging with differing anatomical focuses. 48 To exploit DL-reconstructed images in clinics, there is a need to validate DL-reconstructed images with many more radiologists with different background.
Despite the presence of interobserver variability, DL-based segmentation algorithms that were trained using clinical standard SAG T1 images worked well for Fast-DL images. Segmentation results visually looked better on Fast-DL images than on Fast images, and extracted disc height and vertebral body volumes were better correlated between Fast DL50/Fast DL75 images and Standard images. Even though further  The highest correlation was achieved when the noise reduction factor was 50% for disc heights (r 2 ¼ 0.87) and when the noise reduction factor was 75% for vertebral body volumes (r 2 ¼ 0.99). investigations are needed, the DL-reconstructed images seemed to be robust to machine algorithms. This study has several limitations. First, the sample size of this study might not be sufficient to fully evaluate the ability of DL reconstruction in a wide range of lumbar spine pathologies. Secondly, the methods for qualitative assessment by radiologists can be improved. For example, the "Ability to Discern Anatomical Structures" was rated as one score, even though there are multiple complicated structures to evaluate. Individual scorings for delineation of vertebral bodies, discs, spinal cords, nerve roots, and facet joints, as used in studies such as Bratke G et al. 49 or Sun et al. 50 can allow for more comprehensive and objective evaluation. Furthermore, the presence of artifacts was rated as a binary score, but it may not be appropriate to evaluate the overall level of artifacts from various sources. Additionally, the ratio of NEX used between the fast and standard protocols was not equivalent between difference sequences. If the ratio had been kept equivalent, the effects of DL reconstruction might have been fairly evaluated between different sequences. Another issue might have arisen from MR data acquisition order during patient scans; fast protocol acquisitions were always performed after standard protocol acquisitions. This might have resulted in more motion artifacts in Fast images though we believe the effects would be minimal as imaging time for the fast protocols were under 10 minutes. Randomization of the order might have eliminated this potential problem. Lastly, these fast scans were not directly compared with other known fast imaging methods such as partial Fourier, parallel imaging, and compressed sensing, and data from only one vendor's MRI scanner were involved in this study.

Conclusions
This study demonstrates that DL reconstruction combined with fast acquisitions has a potential to provide diagnostic image quality noninferior to standard-care-of images. The difference in qualitative scores between Standard images and Fast-DL images varied with image parameters and sequences, but overall, this lumbar spine protocol, which was 52% faster, was able to provide scores noninferior to the standard protocol for apparent SNR, visualization of anatomical structures, and diagnostic confidence. However, larger studies involving multiple vendors are needed to assess the accuracy of these fast scans in diagnosing various pathologies and to compare the performance with other fast imaging methods such as parallel imaging and compressed sensing. We also validated that previously trained convolutional neural networks based on standard clinical SAG T1 images can reliably segment discs and vertebral bodies of DL-reconstructed images. The combination of fast acquisition, DL reconstruction, and DL-based image analysis will potentially enable a path for an efficient clinical workflow for patients with LBP.