SIGNAL-TO-NOISE RATIO RATE MEASUREMENT IN FLUOROSCOPY FOR QUALITY CONTROL AND TEACHING GOOD RADIOLOGICAL IMAGING TECHNIQUE

Abstract Visibility of low-contrast details in fluoroscopy and interventional radiology is important. Assessing detail visibility with human observers typically suffers from large observer variances. Objective, quantitative measurement of low-contrast detail visibility using a model observer, such as the square of the signal-to-noise ratio rate (SNR2rate), was implemented in MATLAB™ and evaluated. The expected linear response of SNR2rate based on predictions by the so-called Rose model and frame statistics was verified. The uncertainty in the measurement of SNR2rate for a fixed imaging geometry was 6% based on 16 repeated measurements. The results show that, as expected, reduced object thickness and x-ray field size substantially improved SNR2rate/PKA,rate with PKA,rate being the air kerma area product rate. The measurement precision in SNR2rate/PKA,rate (8–9%) is sufficient to detect small but important improvements, may guide the selection of better imaging settings and provides a tool for teaching good radiological imaging techniques to clinical staff.


INTRODUCTION
The assessment of the performance of an imaging system is ultimately a measure of the amount of diagnostic information that an operator can derive for a specific task (1) . Evaluations of x-ray systems performance must also consider absorbed doses to patients' organs. Clinical image quality of the imaging system can be evaluated using receiver operating characteristics (2) or visual grading of specific image criteria (3) . However, physical image quality indices such as contrast, noise, artifacts and spatial and temporal resolution are more commonly considered in quality control measurement. Favorable characteristics of these indexes should include clinical relevance, reproducibility, accuracy, precision, sensitivity and ease of measurement.
Low-contrast detail detectability is an important image quality index in fluoroscopy and is primarily dependent on contrast, sharpness and background noise. Various methods are used to assess the imaging system's performance in this respect. For quality control purposes, evaluation of the visibility of various low-contrast details by human observers is common; for example, threshold contrast detection and possibly multiple-alternative forced-choice detection experiments using low-contrast cylinder discs test objects. These experiments are typically limited by problems of subjectivity and lack of precision (4)(5)(6) . This is because human visual detection is observerdependent, and it is difficult to define, communicate and maintain a criterion on what is actually visible in a reliable way. SKE/BKE (signal/background known exactly) is the simplest and most ideal task where the target to be detected is fully known and variation in the image data is due to stochastic effects (7) . Under these circumstances, a class of objective mathematical ideal model observers, derived from statistical decision theory, can estimate signal-to-noise ratio (SNR) based on the theoretically most efficient use of information. No general correlation between the physical image quality indices and clinical image quality exists (1,(8)(9)(10) . Model observers (11,12) can still fulfill a role in routine quality control of the imaging system performance if most of the favorable characteristics mentioned previously are met. In addition, analysis of the effect of an imaging equipment parameter on SNR and patient dose indices, such as dose rate, mode of operation, imaging geometry, x-ray field size and photon energy, forms a basis for its clinical operation and if properly analyzed, it can be a useful teaching tool for the operator. The ratio between this image quality index and the patient dose index is a figure of merit (FOM) and is here computed as FOM K = SNR 2 rate /K rate and FOM KA = SNR 2 rate /P KA,rate and sometimes called dose efficiency. Here K rate is the incident air kerma . SDD is source to contrast detail distance and SID is source to image detector distance rate at the phantom surface and P KA,rate the air kerma area product rate. The objectives of this work were to (1) encode and validate the SNR 2 rate model observer software used in FluoroQuality in MATLAB™ to (2) explore the model observer usefulness for quality control on a fluoroscopy unit and to (3) perform measurements of FOM K and FOM KA as a tool for teaching imaging physics to clinical staff and optimizing radiological protection.

Model observer and signal-to-noise rate measurements
In the current study, ideal and quasi-ideal model observers have been used for measurements of the accumulating rate of the square of the SNR, SNR 2 rate of contrast details (13,14) on two fluoroscopy units. The SNR 2 rate detection index is the natural choice as FOM considering the integration of information over time in real-time x-ray viewing. The methods are based on experiments of binary response, which require two hypotheses: H 1 : signal present and H 0 : signal absent. The decision criterion in statistical decision theory is based on the rating of confidence for a decision between the two hypotheses: H 1 and H 0 . The degree of confidence that a certain image belongs to either H 1 or H 0 is quantified by a conditional decision variable (CDV) (1) . An assumption according to this theory is that CDVs from the two sets of images under the same imaging conditions will be grouped into one of two normal distributions belonging to each class. Detection performance is therefore expressed in terms of the separation between these two distributions (1,7) .
A quasi-ideal DC and high frequency suppressing model observer SNR 2 rate (7) was implemented in a MATLAB™ (version 2019a, The MathWorks, Inc, Natick, Massachusetts, USA) code (15) . This model observer is constructed from the difference between the mean signal of the image frames (here 900 or 1024) containing a low-contrast detail and the same number of frames in the same part of the image detector without the low-contrast detail. The model observer template is then cross-correlated with each image frame separately with and without the contrast detail to form the observer's CDVs. Specific image frames analyzed were sequentially removed from the image stack. The remaining images were used to form the observer template, in order to minimize bias.
The SNR of single frame (SNR sf ) was computed from the average difference and variances of the two conditional distributions: one for signal present and signal absent cases. However, neighboring frames in a sequence are not independent, and hence a lagfactor, F [unit s −1 ], is calculated from the spatialtemporal noise power spectrum to account for the number of independent frames per second such that SNR 2 rate = SNR 2 sf . F, for details see Tapiovaara (7) .

Imaging system, instrumentation and measurements of FOM
Images and dosimetric quantities were collected from two Siemens fluoroscopy systems at Linköping University Hospital (Axiom Artis Zee MP and Cios Alpha, Siemens Healthineers, Erlangen, Germany). Images were sent to the picture archiving and communication system (PACS) or saved to a USB-flash drive for further image analysis using FluoroQuality (7) and a validated in-house MATLAB™ code (15) . Figures of merit with respect to K rate and P KA,rate i.e. FOM K and FOM KA were studied as a function of phantom thickness, source to-object distance and x-ray field size.

Imaging geometry and imaging parameters
In the FOM measurements, the patient was represented by a stack of homogeneous polymethylmethacrylate (PMMA) blocks with a surface area of 30 × 30 cm 2 , positioned on the patient couch with the mattress removed. The thickness of the PMMA block, the distances between the x-ray focal spot and phantom and the x-ray beam area were systematically varied. Cylinder-shaped, test objects were positioned on top of the PMMA block ca 10 cm away from the image detector. Figure 1a shows a schematic view of the imaging geometry used with the fixed fluoroscopy system (Setup 1). using Setup 2 In the quality control measurements, the PMMA slab was replaced by a 2 mm thick copper filter (99.9% Cu, Cambridge Ltd, Huntingdon, UK) placed outside of the collimator housing (Figure 1b). The test object was positioned in the center of the beam on the image detector in order to obtain an easily reproducible imaging condition with a mobile C-arm (Setup 2). Hence, the measurement in Setup 2 is done with minimal intervention and magnification and therefore with limited influence of the focal spot size. This setup is more easily reproduced and do not involve a heavy PMMA block and patient couch. Acquisition modes, imaging parameters, contrast details and automatic dose rate control (ADRC) parameters for the two measurement setups are given in Table 1.

Uncertainty estimation
The relative uncertainty in SNR 2 rate for different experiments was estimated to 7.1-9.4% by: where σ Stat is the statistical uncertainty in the image analysis due to a limited number of image samples, estimated to 6-8% (see Figure 2b). A quadratic uncertainty term was then added for an experiment when a parameter X was altered between setups. σ rel,X is the relative uncertainty in X and B x the slope of its linear relation with SNR 2 rate . A 1 cm display uncertainty in couch height results in a change in magnification (M) of the contrast detail (which affect SNR 2 rate ) and σ rel,M 2 was estimated to 1.6-2.0%. The uncertainty in the measurements of x-ray field size σ rel,FS is 2.4-4.6%.
The variation in dose index between subsequent measurements was estimated to 2% σ rel,D from the spread of P KA -rate readings acquired in Setup 2. The accuracy in the calibration of the instruments PDC (P KA -meter, Radcal, Monrovia USA) and T20 (air kerma meter, RTI Group, Mölndal Sweden) were 2.4% (k = 2) and 1.7% (k = 2), respectively. The uncertainty in the figures of merit FOM KA and FOM K was estimated to 7.9-9.4% in Setup 1 experiments, where dose indices and SNR 2 rate were treated as independent variables. Figure 2a shows the influence on SNR 2 rate of the area of a 3 mm thick Al cylindrical disc contrast detail using Setup 1. Figure 2b shows a linear increase of the relative statistical uncertainty in SNR 2 rate when plotted against the inverse of the square root of the number of image frames used in the analysis. Using 1000 frames, the uncertainty (1 standard deviation, Figure 3: Repeated P KA,rate and SNR 2 rate measurements using Setup 2 with a Siemens Cios Alpha mobile C-arm over a period of 4 months. The contrasting detail was a 4 mm thick and 6 mm diameter Al-cylinder. The solid lines indicate the mean value and the dashed line indicate ±2 SDs (or 14%) in SNR 2 rate SD) in SNR 2 rate is ca 7%. Figure 2c shows the linear increase in SNR 2 rate with increasing tube current as indicated by the K rate measured at the image detector (Setup 2). Figure 3 shows results of 16 repeated measurements of SNR 2 rate and P KA,rate over 4 months. The SD in SNR 2 rate from repeated measurements was 6%. The results indicate that the imaging system was stable. Table 2 show SNR 2 rate , K rate , P KA,rate , SNR 2 rate /K rate and SNR 2 rate /P KA,rate as a function of PMMA phantom thickness (Figure 4), source to contrast detail distance ( Figure 5) and x-ray field size ( Table 2). The results were expected and consistent with our experiences. The changes were due to the specific way the ADRC system was designed to approximately maintain air kerma rate at the image detector surface behind the anti-scatter grid.

DISCUSSION
The main finding in this study was that using a model observer to assess an image quality index, such as SNR 2 rate , allows you to estimate small changes in the performance of the imaging system with high precision (6%). This is an advantage for quality control or for selecting a more dose efficient imaging setting.
Good agreement of SNR 2 rate (within 1%) between results generated from the original FluoroQuality software (7) and the in-house, MATLAB™-based version (15) was found using identical image sets (Figure 2a). SNR 2 rate increases linearly with both area of the contrast detail (for fixed K rate ) and with K rate (for fixed area contrast detail, A) in agreement with the so-called Rose-model, SNR 2 rate ∝ M 2 C 2 A K rate , with C being the contrast and M the magnification.
We argue that the general trends of the variation of image quality index and dosimetric indices in   Table 2 are useful for teaching x-ray fluoroscopy physics and technology for clinical medical staff. They can be taught, discussed and reflected on during radiological protection training sessions with clinical staff. In fact, these and similar results are being used in training of resident radiologists in Linköping, Sweden. Tesselaar and Sandborg (11) evaluated the figures of merit of changing the dose rate, pulse rate and field of view on a Siemens Axiom Artis Zee MP. In the present study, we assessed the figures of merit of the same equipment while instead changing the phantom thickness, x-ray field size and patient couch height. The results in terms of 411 Figure 4: Image system characteristics for 10 different PMMA thicknesses from 14 to 30 cm with imaging conditions in Setup 1, but source image detector distance SID of 120 cm. With increasing PMMA thickness the tube current (20, 41, 83, 95, 97, 97, 97, 98, 101, 123 mA) and pulse length (3.5, 3.5, 3.4, 5.5, 7.8, 9.5, 11.6, 13.7, 16.3, 16.3 ms) increased while tube voltage and filtration were maintained (81 kV, 0.3 mm Cu filtration). The geometric magnification decreased as PMMA thickness increased. The fitted curves are not based on any model, but connect the data points to make the results more discernible variation of SNR 2 rate , K rate , P KA,rate , SNR 2 rate /K rate and SNR 2 rate /P KA,rate with the imaging parameters above were expected, but specific to this imaging system and its ADRC-settings.
The large increase in K rate and P KA,rate with increasing PMMA thickness is evident in Figure 4 for fixed x-ray beam size and couch height. Both dosimetric indices approximately doubled for every additional 4 cm PMMA. The tube current initially increased with increasing PMMA thickness from 14 to 20 cm, whereas pulse length was approximately maintained. As the PMMA slab thickness was further increased, the pulse length increased while tube current was approximately maintained. SNR 2 rate decreased rapidly with increasing PMMA thickness due to beam hardening and additional scatter to the image detector. The reduction in SNR 2 rate was furthermore caused by a reduced magnification of the contrast detail (5 mm thick soft tissue), as it was positioned even closer to the image detector since the couch height was fixed while PMMA slab thickness increased. Consequently SNR 2 rate /K rate and SNR 2 rate /P KA,rate decreased at an equally rapid rate with increasing PMMA thickness.
As the source to detail distance (SDD) increased (by increasing the couch height; see Figure 1), the tube current decreased since more scattered radiation contributed to the ADRC ( Figure 5). Source to image detector distance (SID), x-ray beam size, PMMA thickness and tube voltage were constant. Consequently K rate and P KA,rate also decreased, but K rate decreased more rapidly with increasing SDD due to the inverse square law. SNR 2 rate decreased with increasing SDD due to a decrease in magnification of the 15 mm thick low-density low contrast detail, lower photon fluence (decreasing tube current) and more scattered photons reaching the ADRC. FOM SNR 2 rate /P KA,rate decreased slowly with increasing SDD since SNR 2 rate decreased more rapidly than P KA,rate . SNR 2 rate /K rate , on the other hand, increased slowly with increasing SDD since K rate decreased more rapidly than SNR 2 rate . In Table 2, the ADRC-system responded to an increased amount of scattered radiation from an extended x-ray field size by decreasing the tube current. K rate therefore decreased slowly. Magnification of the 5 mm thick soft tissue contrast detail was maintained and so was the PMMA phantom thickness. Since the increase in x-ray field size was much larger than the decrease in tube current, the P KA,rate increased rapidly with x-ray field size. The reduced SNR 2 rate was caused by an added proportion of scattered radiation and reduced K rate . SNR 2 rate /K rate varied slowly with x-ray field size. However, SNR 2 rate /P KA,rate decreased rapidly since SNR 2 rate and P KA,rate changed in opposite directions.
Previous studies have used model observers for the assessment of image quality in fluoroscopy systems. Bertolini (16) used the Channelised Hotelling Observer model to assess possible significant differences between different imaging parameters on a General Electric (Discovery IGS740) fluoroscopy system. Their experiment is similar to the current study as it identifies imaging conditions with superior lowcontrast detectability on a homogeneous phantom.
Villa (6) developed a model observer approach to assess low-contrast detectability in dynamic imaging. In addition, they performed human observer performance assessments in the form of two -alternative forced-choice experiments and compared them with tuned model observers to identify best correlation. In contrast to our study, they did not explicitly compute a FOM nor attempt to use their image quality metric to quantify the quality of the specific angiography unit over time.
Samei (17) pointed out the importance of anatomical background for the detection of lung nodules by human observers. He quantified its importance, as the much larger peak contrast-diameter product needed to detect nodules in an anatomical varying background compared to in a homogeneous background (with only quantum noise), for achieving identical area under the receiver operating characteristic curve (ROC-curve). This aspect is overlooked in our work. Therefore, general trends of figures of merit in our work need to be validated in a more realistic scenario with anatomical background and using model observers tuned to the human visual system.
Assessing low-contrast resolution with a human observer is quick, but probably biased and imprecise as humans find it difficult to define and reliably maintain what is actually resolved. In order to detect small changes in low-contrast resolution, we argue that a model observer will produce results that are more reliable. The sensitivity of SNR 2 rate to detect changes in image noise is several times better than visual methods if one is limited to a reasonable number of human observers (5) . We find it useful not only to evaluate the image quality index SNR 2 rate but also to measure simultaneously a dose rate index (e.g. P KA,rate or K rate ) to ensure that the ADRC-system is operating as expected.
The disadvantage of the SNR 2 rate method is that it does not consider moving test objects and hence the effect of pulse length nor does it fully include the effect of the focal spot unsharpness if the test object is directly on top of the image detector housing. Moreover, a single type of test object may not be representative of all clinical tasks for which the system is used. The practical disadvantage of this model observer implementation is that it can be timeconsuming (typically 10-15 minutes) to extract manually and analyses the images. However, if the images can be sent to a server and analyzed automatically when imaging is completed, the extra time is not a concern.
Dehairs (18) implemented a spatio-temporal FOM [SdNR(u)] with a new ADRC strategy in dynamic imaging aiming to maintain the signal-to-noise level for a range of patient thicknesses. Contrary to what is found, for example in our Figure 4, using a conventional ADRC-system (where SNR 2 rate decreases with increasing phantom thickness), their ADRC strategy keeps signal-to-noise constant from ∼10 cm to 25 cm tissue-bone equivalent thickness and still results in an increase in their FOM, SdNR(u) 2 /AKR ref compared with conventional ADRC; AKR ref being the air kerma rate at the reference point. In effect, this new ADRC strategy adds an additional sixth parameter, the target detectability SdNR(u), to the traditionally used five parameters (tube voltage, tube current, pulse length, filtration and focal spot size).

CONCLUSION
We have successfully implemented the FluoroQuality computer program in MATLAB™. The precision in the estimation of SNR 2 rate in quality control is 6%. Our estimation of SNR 2 rate or of FOM (e.g. SNR 2 rate /K rate and SNR 2 rate /P KA,rate ) allows staff to identify small but important improvements. The objective nature of the data provides reliable and transportable information for quality control and for teaching radiological protection to clinical staff.