Predicting risk of metastases and recurrence in soft-tissue sarcomas via Radiomics and Formal Methods

Abstract Objective Soft-tissue sarcomas (STSs) of the extremities are a group of malignancies arising from the mesenchymal cells that may develop distant metastases or local recurrence. In this article, we propose a novel methodology aimed to predict metastases and recurrence risk in patients with these malignancies by evaluating magnetic resonance radiomic features that will be formally verified through formal logic models. Materials and Methods This is a retrospective study based on a public dataset evaluating MRI scans T2-weighted fat-saturated or short tau inversion recovery and patients having “metastases/local recurrence” (group B) or “no metastases/no local recurrence” (group A) as clinical outcomes. Once radiomic features are extracted, they are included in formal models, on which is automatically verified the logic property written by a radiologist and his computer scientists coworkers. Results Evaluating the Formal Methods efficacy in predicting distant metastases/local recurrence in STSs (group A vs group B), our methodology showed a sensitivity and specificity of 0.81 and 0.67, respectively; this suggests that radiomics and formal verification may be useful in predicting future metastases or local recurrence development in soft tissue sarcoma. Discussion Authors discussed about the literature to consider Formal Methods as a valid alternative to other Artificial Intelligence techniques. Conclusions An innovative and noninvasive rigourous methodology can be significant in predicting local recurrence and metastases development in STSs. Future works can be the assessment on multicentric studies to extract objective disease information, enriching the connection between the radiomic quantitative analysis and the radiological clinical evidences.


Lay Summary
Soft-tissue sarcomas of the extremities are a group of rare malignancies that may develop distant metastases or local recurrence, mainly in the lungs. A 3-year survival rate for patients with metastases is often lower for those who are not candidates for surgery; for these reasons, the earlier identification of patients with high risk of developing distant metastasis could potentially allow implementing more effective therapies. The Radiomic analysis will be performed used a public database of short tau inversion recovery and T2-weighted fat-saturated images. In addition, instead of using Artificial Intelligence, this paper introduces the possibility of: • exploiting mathematical methods together with Radiomics to generate a second-virtual opinion useful to radiologists and their co-workers when facing rare diseases; • localize the most important slices as a visual feedback for medical specialists; • meet the limits of the Artificial Intelligence techniques in the medical field.
The mathematical methods are called Formal Methods: they allow to build a rigorous model of a patient through the values of its radiomic features. The results in this article are greater than 0.70%: this suggests that radiomics and formal verification may be useful in predicting future metastases or local recurrence development in soft tissue sarcoma.

BACKGROUND AND SIGNIFICANCE
Soft-tissue sarcomas (STSs) are a group of malignancies which include a wide number of subtypes, all arising from the mesenchymal cells. More than 50 different categories are reported, as stated by the World Health Organization.
STSs are rare tumors and represent about 1% of all cancers. 1 Despite their low incidence, these malignancies are worrisome; in fact, about 25% of STSs develop distant metastases, representing the main factor leading to death, with a metastatic percentage that can reach about 50% for high-grade STSs. [2][3][4] The main site of metastatization is the lungs, which account for about 80% of lesions. 5 Prognosis of patients who develop metastases is generally: 3-year survival rate is lower than 50% for those undergoing surgical metastasectomy and lower than 20% in those who are not candidate to surgery. Thus, an imaging method, which potentially enables the prediction of metastases occurrence in this set of patients might be of high benefit. The median survival time after distant metastasis diagnosis is approximately 11.6 months. 2 Identifying patients who are at a high risk of developing distant metastasis at an early stage could potentially allow implementing more effective therapies. 6,7 Analysis of tumor heterogeneity on pathological samples obtained from biopsies may be challenging and the information may depend on which part of the tumor is sampled. 8 Attempts to solve this issue and to obtain better information from STSs have been done with the socalled "Radiomics", a field of imaging research which implies the analysis and extraction of large amount of data from medical images, using advanced quantitative characteristics and specific image-processing algorithms. 9,10 As a matter of fact, with radiomics will be possible to avoid the biopsy and to identify the patients with greater risk of metastases or local recidive. In addition, the help of radiomics during the follow-up can support clinicians to predict the behavior of tumors.
Promising results have been recently reported in the use of radiomics in musculoskeletal oncology, being mainly aimed for investigating conventional statistical methods/machine learning algorithm for musculoskeletal sarcomas, 11 for discriminating benign from malignant spine tumors, 12 for predicting patient's prognosis and treatment response. 7,13-15 Nevertheless, a common problem of radiomics studies is related to the considerable number of imaging features that are evaluated by the software 9 or to the poor understandability of the radiomic features in the clinical context. The first problem can have negative consequences on the accuracy of prediction models because some of these features may be redundant, useless, or highly correlated among each other.
Nowadays, various fields, including scientific research, data analysis, and medical diagnosis, utilize artificial intelligence (AI). Machine learning (ML) 16 represents the most commonly exploited AI technique in the medical field as it enables the analysis of data, detection of patterns, and derivation of conclusions also without explicit input. However, this technique often encounters problems related to the complexity of AI, which are frequently used as "black boxes" due to the inability to comprehend the processing from input to output. 17 For these reasons, the current study applies Formal Methods in the radiomics pipeline: these are based on mathematical logic and reasoning 18,19 that allow to model the radiomic data of a patient and to verify if this satisfies the properties belonging to a disease state. Formal Methods are a group of logical and mathematical procedures used to confirm and demonstrate the accuracy and the correctness of a computer system or software application.
Compared to AI techniques, Formal Methods permit to: • have a reduced dataset of patients and/or images for computing the model; • produce an explainable and understandable model; • easily reply the process, because a small number of parameters is required for the entire pipeline.

OBJECTIVE
The purpose of this study was to provide a Formal Method to predict distant metastases and local recurrence in STSs of the extremities. The proposed technique was noninvasive and without the need for biopsy, indeed radiomic features were computed from nonenhanced magnetic resonance images (MRI). To the best of our knowledge, Formal Methods for STSs have never been analyzed in the literature. MRI protocols were not uniform among the patients. From the whole MRI dataset, we selected only MRI scans T2-weighted fatsaturated (T2FS) or short tau inversion recovery (STIR); patients were labelled "metastases/local recurrence" (group B) or "no metastases/no local recurrence" (group A) as clinical outcomes; 4 patients with upper limb soft tissue sarcoma were excluded. In terms of texture, T2FS and STIR images are considered to be similar, and therefore, they were grouped together under one category. 7,21 Following these criteria, we included in our analysis a total of 47 patients from an overall number of 51 patients. Two different MRI exams (from 2 different patients) were used for modelling the property which will be automatically verified by a mathematical technique.

Segmentation and feature analysis
Segmentations for exams were obtained from the above mentioned public database; for this study, segmentations included visible edema. Every single segmentation was visually valued by a radiologist with 7 years' experience and modified if necessary. The 3D slicer software (4.13 version) was used for this step. 22 All the radiomic features were computed using Pyradiomics 3.0.1 (https://pyradiomics.readthedocs.io), a library for radiomic features extraction from medical imaging, 23  The hyperparameter for feature extraction 25 were as follows in Table 1; the remaining parameters were set to default. For each exam, features were extracted from each image separately.
Features used for Formal Method classifier were manually chosen, in order to describe the distribution of voxel intensities and shape characteristics, also with the support of Correlation Attribute Evaluation (Weka software version 3.8.5). 26,27 Figure 1 shows an example of the Radiomic pipeline: 102 features were extracted from the segmentation of a left tight pleomorphic sarcoma, and finally were selected 2 first-order features and 3 Shape 2D features. The next step was the discretization of extracted features, to simplify the translation in formal models. Authors divided the features in 3 different intervals: low, basal, and up with the equal-width partitioning. The discretized features were transformed into a formal model according to the Calculus of Communicating Systems (CCS), a process calculus introduced by Robin Milner. 28,29 More detailed information about this process are reported in the following section.

Formal Methods
Formal Methods 18,19,30,31 are techniques derived from the Computer Science field to verify the correctness of critical informatic systems. Hence, thanks to the mathematical theory, this methodology allows to build formal and rigorous representation of a system. As a matter of fact, this methodology is widely used in Cybersecurity, [32][33][34] Bioinformatics, 35 and Computer Science to verify the safety of complex system behaviors where there is the possibility of economic losses or deaths (automated air traffic management or banking transitions).
In the current study, instead of having a critical informatic system, authors considered the state of health of the patient: the methodology aims to verify the presence or the absence of a disease. After extracting the radiomic features, their numerical values were discretized in 3 levels; according to the keywords used in this methodology: a low level is represented by b1of3, a medium level is represented by b2of3, and the highest level is represented by b3of3. In order to create the formal model, radiomic features (Sphericity, Kurtosis, Skewness, Elongation, and Mesh Surface, as illustrated in the example below) were combined using the combination operator represented by the "." symbol, as shown in Figure 2.
The entire sequence of clinical or radiomics features is represented by a "model" (one model is generated for each subject or patient). The pattern of a disease is represented through a "property" or "formula"; the property is computed by analyzing the common pattern of the disease with the aid of mathematical logic algorithms.
For instance, a pattern can be composed of 2 consecutive values a and b, followed, even if nonimmediately, by other 2 consecutive values c and d. Such a statement can conveniently be expressed in a temporal logic as: The above representation is called "property" and, in case of a MRI exam, each row represents the discriminant radiomics feature values which correspond to the presence of a certain disease mark. In Figure 3, there is a practical example of property with a Shape feature called "Sphericity" and a First order feature called "Kurtosis". Their numerical values are described through the discretization in 3 levels and different operators, which are: • < and > are used for specify which action is performed; • _ is used to perform an 'OR' combination of the values; • min X is a function which define when the rule is satisfied; 36 • tt or ff means the termination condition of the property.
With the increase of radiomics studies on a certain disease, researchers can state (eg) that a low level of Kurtosis is indicative of a high risk of metastases or local recidive. Normally, the property is verified on a MRI exam independently of the number of slices contained. The agent responsible for verifying the absence or the presence of the property is called "Model checker": 18 it is a powerful automatic reasoning technique that allows a rigorous verification of the property on the model of the patient. Practically, this agent takes in input the model of the patient and the property of the disease status, verifying if the model satisfies the property; finally, the agent concludes with a simple result: the output is true if the model satisfies the property, otherwise it is false. If the output is true, this means the patient is affected by the disease status that the property describes (eg, the risk of metastases). If the result is false, researchers know the patient have a negligible risk of metastases and the methodology also return a counterexample with the explication why the model is considered false, which increments the understandability of the technique. In addition to the previous advantages, the use of Formal Methods can also help to localize the site of the sarcoma. Being a mathematical method, it allows to verify in which parts of the model the property is satisfied. In radiological terms, this functionality turns in a localization of the disease in the radiological exam. An example of the whole workflow, including radiomics and Formal Methods, is described in the Figure 4.

Clinical data
Our study population included 47 patients ( Table 2 and Table  "DBInformation" provided in the Supplementary Materials section.   MRI protocols were not uniform and only T2FS or STIR sequences were selected. Two exams were acquired in the sagittal plane, 4 exams were acquired in the coronal plane, and the remaining 41 exams were acquired in the axial plane. More details regarding to MRI acquisition protocols are reported in Table "MRIAcquisition" provided as Supplementary Materials.
In the public dataset, each individual patient did not have both STIR and T2FS sequences available; therefore, we selected the only available fluid-sensitive sequence. 7,21 Segmentation and feature analysis After visually assessment, 43 segmentations were retained, and 4 segmentations were manually changed, in order to better delineate the profile of the lesion.
From segmentations, in total 102 radiomics features were extracted from T2FS or STIR images. For formal modelling, we con-

Formal verification and statistical analysis
After generating the CCS models from all 47 patients, the disease property was generated by a radiologist and 2 computer science researchers, looking at 2 different exams, and it is shown in Table 3. Please note: in Formal Methods there is no division of training and testing, only the creation of the models, properties and their verification on the models. Formal Methods do not learn from any patterns or behaviors.
The following metrics were considered for evaluating the performance of the property to predict the development or not of metastases/ local recurrence (group B vs group A): specificity, sensitivity (also called recall), accuracy, positive predictive value and negative predictive value. Intercorrelation among selected features was calculated with the Spearman correlation coefficient. Furthermore, the clinical utility indexes (CUI) were calculated to take into account both occurrence and discrimination. 37 The value for CUI ranges from 0 to 1:

Explainability
Combining Radiomics and Formal Methods allows to obtain a "second-virtual opinion" for radiologists and clinicians. In addition, this method can also localize the slices where the property is satisfied, giving a visual feedback, as shown in Figure 6. This can be very useful when facing difficult radiological exams where the main tumoral component is not clearly visible.
For example, in Figure 6 the localization method is used on patient STS_038 (from group B) and, in Table 4 are depicted which features values are aligned with the formula.

DISCUSSION
This study provides a formal method to predict the development of distant metastases and local recurrence in STSs using MRI. The identified property obtained an accuracy of 0.74, a positive CUI of 0.606 and a negative CUI of 0.491. In the following section, we illustrate the choices in the materials and methods, the limitations,  Table 3. Temporal logic properties used for the diagnosis of patients affected by sarcoma with local recurrence and recent articles regarding the same topic. For this work, 3 planes were considered: sagittal, coronal, and axial; axial plane was the most frequent acquisition plane. We decided to include all exams, even if with different planes, because current routine MRI protocols for STSs include all 3 orthogonal planes. Regarding segmentations, it was preferred to include edema inside the segmentations, because STS cells are present histologically also in the tissues beyond the tumor. 38 In addition, the ability to analyze tumor cells beyond the gross tumor volume has relevant implications such as in treatment.
As regards Kurtosis and Skewness, according to references 40-43, positive skewness and higher kurtosis can represent increased heterogeneity and poorer prognosis in several tumours; indeed, researchers in reference 44 showed that the presence of at least 2 out of 3 characteristics (heterogeneity, necrosis, and peritumoral enhancement) was a predictor of overall survival and metastasis-free survival in STSs. Regarding to Elongation and Sphericity, researchers in reference 45 included both features for their Radiomics-T2FS model, aimed to differentiate low-grade from high-grade STSs.
The principal limitations of the proposed study are as follows.
The first limitation is the relatively small sample size. However, differently from other AI techniques, the proposed approach does not require many exams because there is not a training step.
Regarding the second limitation, Elongation and Sphericity have a Spearman correlation coefficient of 0.86. It means redundant data can be present in the proposed property; anyway, we decided to retain them both on the basis of the above-mentioned article. 45 The third limitation is the inclusion of patients with development of metastases (23 patients) and local recurrence (3 patients) in the same group B, considering these patients as a unique group.
The fourth limitation is due to differences in histology. According to references 46 and 47, the risk of distant metastasis in STSs ranges from 20% to almost 100% based on grading and histological type. Histology effect has not been investigated in this study and it could be explored in further studies.
The fifth limitation is the heterogeneity of MRI protocols and future works will verify the stability of the proposed approach through various image acquisition protocols.
Regarding the state-of-the-art, researchers in reference 48 trained a radiomics score for metastatic relapse-free survival in 35 patients with myxoid/round cell liposarcomas. The model, combining the radiomics score and relevant radiological features, achieved an AUC of 0.925.
Researchers in reference 49 found 5 contrast enhancement MRI radiomics models for predicting metastatic relapse-free survival, using 50 patients having high-grade sarcomas; the model with the highest integrative AUC obtained a value of 0.87.
Authors in reference 50 built a radiomics-based models to predict metastatic relapse at 2-years with a training data-set of 50 patients. On the testing cohort (20 patients), the best supervised model obtained an accuracy of 0.75.
In reference 51, the authors constructed and validated a radiomics method for prediction of distant metastasis in STSs. They used a training dataset with 54 sarcomas and a testing dataset with 23 sarcomas. The highest AUC and accuracy obtained were 0.902 and 0.913, respectively.
The above-mentioned articles 48,49,51 considered only one histotype or the same MRI scanner. Differently, the proposed study included different histologies and different MRI scanners (further details regarding histological types and MRI scanner protocols are provided in "DBInformation" and "MRIAcquisition"-Supplementary Materials section).
Researchers in reference 7 used the same public data-set of the current study. They combined the texture features of FDG-PET and MRI imaging to assess lung metastasis risk in STSs on 51 patients; the model achieved an AUC, sensitivity and specificity of 0.984, 0.955, and 0.926, respectively.
Authors in reference 52 built a model for the prediction of lung metastases in STSs, by optimizing MR and PET image acquisition protocols. From the same public data-set of our study, the researchers selected 30 patients for their research and the model obtained an AUC of 0.89.
However, both the previous articles 7,52 did not validate their results on independent datasets.  Table 4. Comparison between the values described in the property and those found in the radiological examination

Sphericity Kurtosis Skewness Elongation Meshsurface
Property High Low -Medium/High High The above literature confirms the novelties of the proposed Formal Method approach, which can be considered as a valid alternative to other AI techniques. Moreover, even if the present results are slightly lower than the state of the art, the proposed method is highly-available, indeed it is based only on routine MRI protocols.
The aim of AI is to develop computer systems capable of reasoning and contributing to various fields, such as interpreting natural language, 53 perceiving sensory information, and learning new information. These systems should emulate human intellect, performing tasks and improving their ability to perform them over time.
The most commonly used AI approaches are ML and deep learning (DL), which are often used interchangeably. ML 16 is a broad category of approaches and algorithms that analyze data and draw conclusions. DL, on the other hand, is a subset of ML that uses artificial neural networks to analyze extremely complex data. Creating and utilizing a ML system can often pose challenges and difficulties. The first issue may be the lack of data, as ML requires a large amount of training data to be effective; but finding relevant and high-quality data to train the computer model can be difficult. Furthermore, overfitting and underfitting can arise due to noise or unrepresentative data in the training dataset, leading to ML models that are unable to accurately generalize predictions or classifications to new data. Even with a perfect dataset, interpreting the results of an ML system can sometimes be challenging because the model may be difficult to explain. 17 Formal Methods can be utilized to analyze and evaluate the correctness and accuracy of computer systems by means of mathematical modeling of system behavior and the verification of specified properties. Formal Verification, Theorem Proving, Program Synthesis, and Model Checking are examples of Formal Methods techniques used to demonstrate a computer system's correctness with respect to specific requirements. 18 By enabling errors in the software to be detected and resolved during development, formal approaches can enhance the dependability and security of computer systems, minimizing the probability of catastrophic errors or system failures.
As a conclusion, despite the limitations, the current study suggests that Formal Methods can provide beneficial assistance for personalized medicine. As a matter of fact: • the availability of small datasets does not affect the robustness of the model and therefore the reliability of the results; • the construction through mathematical and rigorous methods allows to understand the production and the meaning of the property (avoiding the risk of a "black box" approach); • the entire process is supervised by radiologists and AI experts.

CONCLUSIONS
The proposed approach, based on Formal Methods, can be an alternative tool to predict the risk of local recurrence and metastases in STSs. If the data are confirmed from further validation, this technique may assist physicians in choosing the appropriate treatment for STSs and potentially improve patient survival. Future works can be the development of these mathematical methods to extrapolate the objective characteristics of the disease independently of MRI scanners.

SUPPLEMENTARY MATERIAL
Supplementary material is available at JAMIA Open online.