A novel generation adversarial network framework with characteristics aggregation and diffusion for brain disease classification and feature selection

Abstract

Imaging genetics provides unique insights into the pathological studies of complex brain diseases by integrating the characteristics of multi-level medical data. However, most current imaging genetics research performs incomplete data fusion. Also, there is a lack of effective deep learning methods to analyze neuroimaging and genetic data jointly. Therefore, this paper first constructs the brain region-gene networks to intuitively represent the association pattern of pathogenetic factors. Second, a novel feature information aggregation model is constructed to accurately describe the information aggregation process among brain region nodes and gene nodes. Finally, a deep learning method called feature information aggregation and diffusion generative adversarial network (FIAD-GAN) is proposed to efficiently classify samples and select features. We focus on improving the generator with the proposed convolution and deconvolution operations, with which the interpretability of the deep learning framework has been dramatically improved. The experimental results indicate that FIAD-GAN can not only achieve superior results in various disease classification tasks but also extract brain regions and genes closely related to AD. This work provides a novel method for intelligent clinical decisions. The relevant biomedical discoveries provide a reliable reference and technical basis for the clinical diagnosis, treatment and pathological analysis of disease.

Alzheimer’s disease, brain imaging genetics, brain disease classification, feature selection, feature information aggregation and diffusion generation adversarial network, deep learning

Introduction

The brain is the control center of human physiological activities, whereas brain diseases with complex causes have now become the major health concern of modern people [1, 2]. For a long time, scholars have found that there exist complex functional correlations between brain activities and gene expressions, which can be reflected through neuroimaging and genetic data [3]. On the one hand, neuroimaging data such as magnetic resonance imaging (MRI) can intuitively show the operative condition of the brain with radiographic techniques. Image characteristics are also the most common diagnostic indices [4, 5]. On the other hand, the physiological activities of the brain are strictly controlled by genetic materials like DNA and RNA, whose abnormalities will have a profound impact on the development of brain diseases [6–9] Specifically, single nucleotide polymorphism (SNP) can more accurately reflect the disease-related mutation [10, 11]. In such a context, imaging genetics has emerged as a new field to bridge these two aspects, where scholars can thus fully understand the internal genetic causes of brain lesions [12, 13].

Data from different omics usually contain complementary biological information, which can provide a wider horizon to pathology studies. In existing imaging genetics research, though fair performance improvements have been accomplished, the practice of data fusion is either to simply splice the feature matrices of the multi-level data [14] or to design regularization terms according to the characteristics of multimodal data to improve the objective functions of model training [15, 16]. In short, the fusion of imaging genetic data is not intuitive enough, which not only reduces the interpretability of intelligent learning methods but makes it difficult to ensure that the extracted fusion features have clinical importance. Comparatively, graph-based data representation can flexibly express the non-Euclidean relationships among nodes, and thus attracts increasing attention [17–19]. For example, Yang et al. [20] proposed a graph embedding method to study the complex association between circular RNAs and diseases, designed a deep network to extract discriminant line features and achieved good classification results in real datasets. Song et al. [21] constructed functional graphs and structural graphs with functional MRI (fMRI) and diffusion tensor imaging data and fused the multimodal information into edges through an effective calibration mechanism. Therefore, the first contribution of this paper is that we first introduce the network-based data representation into the field of imaging genetics. Specifically, the brain region-gene network (BG-network) is constructed based on the fMRI and SNP data, which can bring two benefits. First, the nonlinear association patterns between brain regions and genes are explicitly expressed as the edges in the network. Second, based on the topological structure of the network, we define the structural information of each node, and creatively propose a structural information aggregation model to realize the iterative evolution of the BG-networks, and then obtain the key structural sub-networks that can accurately characterize the distinctive characteristics of Alzheimer’s disease (AD) patients.

With the development of computational technology, computer-aided diagnosis and treatment based on artificial intelligence have become a vital means for disease risk prediction and diagnosis, which has improved the development of intelligent medicine [22–24]. The machine learning method has become a popular method to extract distinctive features that are more conducive to the classification of brain diseases [25–29]. For example, Illan et al. [30] used Bayesian networks to simulate the dependency between affected areas of AD to assist the diagnosis based on MRI data. Abdul et al. [31] developed a convolutional neural network (CNN) classification framework for AD, and the proposed framework achieved satisfactory results on a variety of classification tasks. Though deep learning methods often outperform machine learning due to their strong representation and modeling abilities, the inherent black-box characteristics of the deep learning make it difficult for scholars to understand the concrete process of model training, which reduces the application scope of deep learning approaches. In this paper, we have made improvements suitable for multi-level data analysis to the basic framework of the generative adversarial network (GAN). In the generator and discriminator, we design novel bidirectional cross convolution (BDC-C) and diffusion deconvolution (D-DC) operations according to the established information aggregation model, which helps our method to accurately identify the pathogenic features of AD.

This paper adopts a modeling-before-learning research pipeline and studies AD based on imaging genetic data. First, based on the association analysis of multimodal number features, BG-networks are constructed to represent the association pattern between the brain regions and genes. Second, we construct a novel feature information aggregation model, which clearly describes the information dissemination process between nodes. Then, we build a feature information aggregation and diffusion generative adversarial network (FIAD-GAN) based on the model, where the specific convolution operation is defined, and the GAN structure and adversarial training can accurately extract distinctive features. Finally, we test the performance of FIAD-GAN in the task of disease classification and feature extraction based on the imaging genetic data of AD patients. The results not only show the advancement of FAID-GAN compared with other competitive methods but also reveal multimodal biomarkers that may attribute good prospects in clinical diagnosis and further pathological research.

Methods

Framework

In this study, we propose the feature information aggregation model and a FIAD-GAN method to classify and extract the pathogenesis of AD. Figure 1 shows the overall framework of FIAD-GAN. First, the BG-networks are constructed based on brain imaging genetic data. Second, the feature information aggregation model is built, and the generator consisting of convolutional and deconvolutional layers is established accordingly, where the BDC-C and D-DC operation are designed for feature enhancement and network reconstruction. Finally, the discriminator is built to classify the input network and calculate the loss function. After the training, FIAD-GAN can accurately identify patients and extract diseased brain regions, causative genes and the association pattern among those pathogenies.

Figure 1

Framework of the proposed FIAD-GAN.

Datasets and preprocessing

The data of 870 samples are collected from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). The use of data is approved and authorized by ANDI, and the demographic characteristics are listed in Table 1. The data are collected and processed according to strict standards, which ensures homogeny. We have tested the difference in gender and age through the Chi-square test and t-test, and the respective P-values are 0.274 and 0.007. The former is greater than 0.05 and the latter is <0.05, indicating no significant difference. Each sample contains SNP data and fMRI data, and the preprocessing of data is based on Plink and DPARSF software [32, 33]. The main steps of SNP preprocessing include quality control on the deletion rate, heterozygosity and gender, SNP screening, and encoding (assuming that the C of a locus has minimum allele frequencies, the phenotypes of CC, GG and CG are, respectively, encoded as −1, 1 and 0). The main steps of fMRI data preprocessing include time layer alignment and head movement correction, spatial normalization and smoothing, delinearization and filtering, and covariate regression. The length of the series and sequences is determined as 70, which is a compromised result after evaluating the information richness and the balance of multimodal data. If the length is too short, there will be too less information contained in the series and sequences to afford effective analysis. If the length is too long, too few genes will be retained, making the modality balance hard to be kept. As a result, the sequences of 45 genes are obtained for each sample. Trimmed to the same length of 70, the time series of 116 brain regions were acquired based on the anatomical automatic labeling template, which is a commonly used template in imaging analysis [34].

Table 1

Basic information of the data

Variables	NC	EMCI	LMCI	AD
Sample number	237	197	203	233
Age (mean ± SD)	73.03 ± 6.06	73.43 ± 5.45	74.83 ± 7.92	71.55 ± 5.90
Gender (female/male)	142/95	108/89	108/95	151/82

All data are from the ADNI databases.

Construction of BG-network

It has been confirmed that the expression of genes at the microscopic level can lead to macroscopic changes in brain regions, and these associations may be closely related to AD [35–37]. To better characterize the inter-modal associations, the BG-network is constructed based on the sequence of genes and brain regions. The Pearson correlation coefficient (PCC) between nodes is calculated as the connection strength of each edge, which is presented as:

$$\begin{equation} {Pear}_{v_a,{v}_b}=\frac{len\sum{v}_a{v}_b-\sum{v}_a\sum{v}_b}{\sqrt{len\sum{v_a}^2-{\left(\sum{v}_a\right)}^2}\sqrt{len\sum{v_b}^2-{\left(\sum{v}_b\right)}^2}}, \end{equation}$$

(1)

where |${v}_a$| or |${v}_b$|is the time series or genetic sequence, |$len$| is the node sequence length. Accordingly, |$W\in{R}^{N\times N}$| is defined as the weight matrix of the BG-network, |$N$| represents the node number.

Feature information aggregation model of BG-network

In this section, the feature information aggregation model is designed for the key feature information extraction from BG-networks. The model can aggregate disease-specific information into key structural sub-networks by evolving the initial BG-network. Therefore, diseased brain regions and causative genes and their association patterns can be found according to the key structural sub-networks. The elements of the feature information aggregation model are as follows:

• BG-network set: |$G=\{{G}_i|i=1,\dots, m\}$|⁠, |${G}_i$|denotes the BG-network of the |$i$|-th sample (NC or AD patient).

• Edge weights: |${W}^{(i,n)}=\{{W}_{pq}^{(i,n)}|i=1,\dots, m,1\le p,q\le N\}$|⁠, |${W}_{pq}^{(i,0)}$| denotes the weights of edge |${E}_{pq}$| at the initial BG-network of |$i$|-th sample, which is obtained by calculating the PCC of nodes |$p$| and |$q$|⁠. |${W}_{pq}^{(i,n)}$| denotes the weights of edge |${E}_{pq}$| after the |$n$|-th feature information aggregation of the |$i$|-th sample, with larger weights indicating a stronger degree of connection between nodes |$p$| and |$q$|⁠.

• Key structural sub-network: |${KG}^i=\{({E}_{pq},{W}_{pq}^{(i,n)})|1\le p,q\le N\}$|⁠, |${KG}^i$| denotes the key structural sub-network obtained from the |$i$|-th sample BG-network after |$n$| times of feature information aggregation.

According to the above elements, the key structural sub-network is finalized through feature information aggregation. The following equations represent the feature information aggregation model.

$$ \left\{\begin{array}{@{}l}{W}^{\left(i,n\right)}=\left\{{W}_{pq}^{\left(i,n\right)}=\beta{\sum}_{i=1}^n\left({W}_{pk}^{\left(i,n-1\right)}+{W}_{kq}^{\left(i,n-1\right)}\right),1\le p,q\le N\right\}\kern1.4em (2)\\{}K{G}^i= Graph\left({W}^{\left(i,n\right)}\right)\kern15em (3)\\{}{P}^i= Classify\left(K{G}^i\right)\kern20.7em (4)\\{}\left( CG, DBR, AP\right)= Feature\_ extract\ \left({P}^i,K{G}^i\right),i=1,\dots, m\kern4.8em (5)\end{array}\right. $$

Equation (2) denotes the update of the edge weight |${W}^{(i,n)}$| during the |$n$|-th feature information aggregation process in the BG-network of |$i$|-th sample, where |$\beta \sum_{k=1}^n({W}_{pk}^{(i,n-1)}+{W}_{kq}^{(i,n-1)})$| denotes the feature information that edge |${E}_{pq}$| aggregates from its neighboring edges.

Equation (3) is the calculation of the key structure sub-network based on |${W}^{(i,n)}$|⁠.

Equation (4) represents the |$i$|-th sample classification, where |$K{G}^i$| represents the key structure sub-network of the |$i$|-th sample.

Equation (5) shows the process of feature extraction, where |$CG$| represents causative genes, |$DBR$| represents diseased brain regions and |$AP$| is the association patterns of brain regions and genes.

FIAD-GAN

GAN is becoming a popular research topic in deep learning because it can weigh the synergies and differences between data more accurately [38]. Combining GAN with the feature information aggregation model, FIAD-GAN framework is designed. The details are specified below.

Generator

As shown in Figure 2, the generator consists of a convolution part and a deconvolution part. The concrete description is as follows.

Figure 2

Flowchart of generator.

On the one hand, the weight matrix of the BG-network is updated with each iteration of the BDC-C operation, which leads to the topology change of the BG-network. In the existing deep learning methods based on network structure, cross convolution has been used to realize the feature information aggregation of network nodes in their neighborhood. However, the original approach only considers the one-way information propagation process from the source node to the target node, ignoring the bidirectional propagation of information on the same edge that may have different effects on the edge weight. Based on the established information aggregation model, BDC-C layers are designed, which decompose the information aggregation process between each pair of nodes into two directional edges in opposite directions, and therefore the key structural information can be captured more accurately. The following describes the specific |$n$|-order BDC-C operation. Based on the weight matrix |${W}^{(n-1)}$| of the |$n-1$| order BG-network, each edge aggregates the feature information from the |$n$|-order neighboring edges in the |$n$|-order BDC-C operation. As a result, the updated weight matrix |${W}^{(n)}$|is obtained by:

$$\begin{equation} {\displaystyle \begin{array}{l} {W}_{rc}^{(n)}= ReLU\left({\sum}_{i=1}^N\ \beta \left({W}_{ri}^{\left(n-1\right)}{A}_{ri}^{(n)}+{W}_{ic}^{\left(n-1\right)}{A}_{ic}^{(n)}+{W}_{ir}^{\left(n-1\right)}{A}_{ir}^{(n)} \right.\right.\\{}\qquad\quad \left.+\kern0.33em {W}_{ci}^{\left(n-1\right)}{A}_{ci}^{(n)}+{b}^{(n)}\right),\kern0.99em r,c=1,\dots, N;\kern0.33em r\ne c \end{array}} \end{equation}$$

(6)

where |${b}^{(n)}$| is the bias of the |$n$|-order BDC-C operation, |${A}^{(n)}\in{R}^{N\times N}$| denotes the |$n$|-th order BDC-C kernel, |$r$| and |$c$| are the row and column of the matrix, respectively, |$ReLU(\bullet )$| is the activation functions and |${W}_{rc}^{(n)}$| is the updated value of |${W}_{rc}^{(n-1)}$| through the |$n$|-order BDC-C operation.

On the other hand, the D-DC operation is designed to reconstruct the generative BG-network from the key structural sub-network. In the D-DC process proposed in this paper, each edge only diffuses the feature information into the first-order domain, so the shape of the deconvolution kernel is also a cross shape rather than a traditional square. Such a design is also suitable for our network-based data representation. The following is the process of the |$n$|-order D-DC operation. The input of the |$n$|-order D-DC operation is the weight matrix |${W}^{\prime (n-1)}$| obtained from the |${W}^{(n)}$| after |$n$|-|$1$| times D-DC operation. Then, |${W}^{\prime (n-1)}$| is deconvoluted by |$n$|-order diffusion, with each edge absorbing feature information from its |$n$|-order neighborhood. The updated weight matrix |${W}^{\prime (n)}$|is calculated by:

$$\begin{equation} {\displaystyle \begin{array}{c}{W}_{rc}^{\hbox{'}(n)}= ReLU\left(\sum \limits_{i=1}^N\ \left({W}_{ri}^{\hbox{'}\left(n-1\right)}{B}_{ri}^{(n)}+{W}_{ic}^{\hbox{'}\left(n-1\right)}{B}_{ic}^{(n)}\right)+{b}^{\hbox{'}(n)}\right),\\{}\kern0.50em r,c=1,\dots, N;r\ne c\end{array}} \end{equation}$$

(7)

where |${b}^{\prime (n)}$| is the bias of the |$n$|-order D-DC operation, |${B}^{(n)}\in{R}^{N\times N}$| is the |$n$|-order D-DC kernel and |${W}_{rc}^{\prime (n)}$|is the value of |${W}_{rc}^{\prime (n-1)}$|updated by the |$n$|-order D-DC operation. Generally, the layer number |$n$| can be any integer number greater than 1. In this paper, however, only two BDC-C layers and two D-DC layers are included. In general, stacking excessive convolution layers may cause over-smoothing and reduced accuracy. As experiments indicate, the two BDC-C layers can capture > 99% of the feature information in the BG-networks, so it is unnecessary to add more BDC-C layers that may bring extra calculation load.

Compared with the ordinary convolution and deconvolution in the traditional GAN-based approaches, the design of BDC-C and D-DC operations in this paper is based on more intuitive and reasonable mathematical modeling, and thus the interpretability and performance of the deep learning framework are improved. The former is used to aggregate the structural information in BG-network to obtain the key structural sub-network representing the association pattern between the multimodal data, from which we can better determine the discriminative features of AD patients. The latter is used for reconstructing the BG-network based on the key structural sub-network as the output of the generator.

Discriminator

The discriminator consists of BDC-C layers and fully connected (FC) layers and aims to discriminate whether the input BG-network is real or generated and classify the sample.

The purpose of the BDC-C layers is to enable edges to aggregate key feature information in their neighborhood, whose input is either the real BG-network or the generative BG-network reconstructed by the generator. The process of convolution is demonstrated using the n-order BDC-C as the example, which is defined as follows:

$$\begin{equation} {\displaystyle \begin{array}{c}{W}_{rc}^{(n)}= ReLU\left({\sum}_{i=1}^N\ \beta \left({W}_{ri}^{\left(n-1\right)}{C}_{ri}^{(n)}+{W}_{ic}^{\left(n-1\right)}{C}_{ic}^{(n)}+{W}_{ir}^{\left(n-1\right)}{C}_{ir}^{(n)}\right.\right.\\{}+\kern0.33em \left. {W}_{ci}^{\left(n-1\right)}{C}_{ci}^{(n)}+{c}^{(n)}\right),\kern1.32em r,c=1,\dots, N;\kern0.33em r\ne c\end{array}} \end{equation}$$

(8)

where |${c}^{(n)}$| denotes the bias of the |$n$|-order BDC-C operations, |${C}^{(n)}\in{R}^{N\times N}$| denotes the |$n$|-order BDC-C total kernel, |$r$| and |$c$| are the row and column number of the matrix, and |${W}_{rc}^{(n)}$| is the updated weight of |${W}_{rc}^{(n-1)}$| by the BDC-C operations.

The purpose of the FC layer is to extract the deeper feature information of the BG-network, and to determine the truth and class of the input BG-network. The equation of the |$L$|-layer FC layer is as follows:

$$\begin{equation} {X}^{l+1}=\left\{\begin{array}{@{}l} ReLU\left({W}^l{X}^l+{b}^l\right),\kern3.5em \ \ \ if\ l\ne L-1\\{}\left\{\begin{array}{@{}c} Sigmoid\left({W}^l{X}^l+{b}^l\right),\\{} Softmax\left({W}^l{X}^l+{b}^l\right),\end{array}\right.\kern0.50em \ \ \ if\ l=L-1\end{array}\right. \end{equation}$$

(9)

where |${X}^0= Ran({W}^{(n)})$|⁠, |${W}^{(n)}$|is the output of the BDC-C layer, |$Ran(\cdotp )$| denotes the straightening operation by row, |${W}^l$| denotes the weight matrix of the |$l$|-th FC layer; |$Sigmoid(\bullet )$| and |$Softmax(\bullet )$| are the activation functions. The output of |$Sigmoid$| indicates whether the input network is real or fake. The output of |$Softmax$| indicates the sample belongs to which category.

Loss function

The loss function of FIAD-GAN has two parts: the difference loss value |${L}_1$| between generative and real data, and the adversarial loss value |${L}_2$| between the generator and the discriminator. |${L}_1$| is defined as:

$$\begin{equation} {L}_1=\sum_{x=1}^{P_{data}(x)}\sum_{i=1}^N\sum_{j=1}^N{\left|{W}_{ij}^{x,(0)}-{W}_{ij}^{x,\prime (n)}\right|}^2, \end{equation}$$

(10)

where |${P}_{data}(x)$| is the real sample distribution, |${W}_{ij}^{x,(0)}$| is the element of the |$x$|-th BG-network weight matrix and |${W}_{ij}^{x,\prime (n)}$| denotes the weight value between nodes |$i$| and |$j$| in the weight matrix generated by the generator.

The adversarial loss |${L}_2$|⁠, which constitutes the adversarial training between generator and discriminator, is defined by the following equation:

$$\begin{equation} {\displaystyle \begin{array}{l}{L}_2= mi{n}_G ma{x}_DV\left(D,G\right)\\{}\quad ={E}_{x\sim{P}_d(x)}\left[\log D(x)\right]+{E}_{z\sim{P}_z(z)}\left[\log \Big(1-D\left(G(z)\Big)\right)\right]\ \end{array}}, \end{equation}$$

(11)

where |$G(\bullet )$| and |$D(\bullet )$| are the representative functions of the generator and discriminator, |${P}_d(x)$| represents the initial sample distribution and |${P}_z(z)$| is the sample distribution of the generated BG-network reconstructed by the generator |$G$|⁠.

In summary, the total function loss of FIAD-GAN is defined as follows:

$$\begin{equation} Loss={L}_1+{L}_2. \end{equation}$$

(12)

Feature extraction

After FIAD-GAN converges, the generator captures the key feature information of the BG-networks and the discriminator can accurately classify the samples. Accordingly, the diseased brain regions, causative genes and the association patterns of AD-related pathogenies can be further extracted.

First, based on the results of the FC layer, the predicted probability score of the sample is defined by:

$$\begin{equation} {Prob}^i={P}_i\cdotp{R}_i, \end{equation}$$

(13)

where |${P}_i$| and |${R}_i$| are the predicted result and real label. |${P}_i=({P}_{i_{11}},{P}_{i_{12}})$| is a 2D vector, where |${P}_{i_{11}}$| and |${P}_{i_{12}}$| denote the probability that the sample is predicted to be a NC or an AD patient, respectively, |${T}_i$| is a vector of |${R}^{1\times 2}$|⁠, |${T}_i={(1,0)}^T$| represents the sample is normal, and |${T}_i={(0,1)}^T$| represents the sample is AD.

Second, based on the key structure sub-network of the sample and the predicted probability score, the key feature matrix |$FW$| is obtained by

$$\begin{equation} FW=\frac{\sum_{i=1}^m{Prob}^i\times{KW}^i}{m}, \end{equation}$$

(14)

where |$K{W}^i$| is the weight matrix of the key structural sub-network, i.e. the output of the BDC-C layers.

Similarly, the upper triangle values in |$FW$| are the importance scores of the corresponding edges. For a specific edge |${E}_{v_r{v}_c}$|⁠, the importance score |${Score}_{E_{v_r{v}_c}}$| can be defined as:

$$\begin{equation} Scor{e}_{E_{v_r{v}_c}}=F{W}_{r,c},\kern0.91em 1\le r<c\le N \end{equation}$$

(15)

where |${Score}_{E_{v_r{v}_c}}$| denotes the importance score of edge |${E}_{v_r{v}_c}$|⁠.

The |${Score}_{v_r}(r=1,\dots, N)$| and |${Score}_{E_{v_r{v}_c}}(r,c=1,\dots, N)$| are sorted to obtain the set of node features |${Sub}_V$| and the set of edge features |${Sub}_E$|⁠. The optimal subsets of node and edge features were further selected by incremental search.

Evaluation indicators

Four traditional assessment indicators are applied for performance evaluation in this study, i.e. accuracy (ACC), sensitivity (SEN), specificity (SPE) and balance accuracy (BAC). The indicators are defined by:

$$\begin{equation} ACC=\frac{TP+ TN}{TP+ TN+ FP+ FN} \end{equation}$$

(16)

$$\begin{equation} SEN=\frac{TP}{TP+ FN} \end{equation}$$

(17)

$$\begin{equation} SPE=\frac{TN}{TN+ FP} \end{equation}$$

(18)

$$\begin{equation} BAC=\frac{SEN+ SPE}{2}, \end{equation}$$

(19)

where |$TP$|⁠, |$TN$|⁠,|$FP$|and|$FN$| denote true positive, true negative, false positive and false negative, respectively.

Results and discussion

Performance evaluation

Disease classification is the first and most important application of the method. With the ratio of 8:2, the training set and test set are derived from the original data and kept in the proportion of each class of sample. Taking the division of AD patients and normal controls (NCs) as an example, 189 of 237 NCs and 186 of 233 AD patients are divided into the training set, whereas the remaining data will form the test set. The classification performance of the competing methods was evaluated by the aforementioned four metrics. The baseline methods include RF, SVM, CNN and graph convolutional network (GCN), and two GAN methods whose discriminator is implemented respectively by CNN (CNN-GAN) and GCN (GCN-GAN). The structure of CNN-GAN and GCN-GAN was the same as FIAD-GAN. Therefore, they could be regarded as two variants of our method.

Table 2 shows the classification performance results on three tasks, and Figure 3 depicts the corresponding radar plots. It could be concluded that methods based on brain imaging genetic data generally outperform those based on a single modality of data. Also, the GAN-based methods outperformed the others. In addition, FIAD-GAN had the largest radar plot area among all tasks and all modalities. As the results indicate, the proposed algorithms of BDC-C and D-DC could fully use the complementary information in imaging genetic data, and accurately extract brain regions and genes highly related to diseases, providing an efficient new method for the diagnosis and pathogeny detection of AD.

Table 2

Classification performance results of different methods on three tasks under different modal data

Method	Modal	NC versus AD				EMCI versus LMCI				LMCI versus AD
		ACC (%)	SEN (%)	SPE (%)	BAC (%)	ACC (%)	SEN (%)	SPE (%)	BAC (%)	ACC (%)	SEN (%)	SPE (%)	BAC (%)
RF	fMRI	60.64	55.56	70.97	63.26	60.00	56.45	72.22	64.34	59.38	57.81	65.63	61.72
	SNP	59.57	52.63	70.27	61.45	57.50	57.14	58.82	57.98	58.33	57.38	60.00	58.69
	Dual	64.89	70.59	58.14	64.36	63.75	59.32	76.19	67.76	63.54	62.07	65.79	63.93
SVM	fMRI	65.96	50.00	80.00	65.00	62.50	62.07	63.64	62.85	64.58	59.02	74.29	66.65
	SNP	63.83	60.00	70.59	65.29	60.00	60.34	59.09	59.72	61.46	57.14	69.70	63.42
	Dual	68.08	65.52	69.23	67.37	71.25	66.67	80.77	73.72	67.71	66.67	69.24	67.95
CNN	fMRI	74.47	72.92	76.09	74.50	67.50	75.51	58.06	66.79	70.83	71.15	70.45	70.80
	SNP	69.15	63.16	77.78	70.47	66.25	62.50	75.00	68.75	68.75	79.17	58.33	68.75
	Dual	75.53	72.92	78.26	75.59	70.00	66.04	77.79	71.91	72.94	72.00	73.91	72.96
GCN	fMRI	74.47	70.00	79.55	74.77	68.75	64.29	79.17	71.73	70.83	64.29	80.00	72.14
	SNP	72.34	67.31	78.57	72.94	67.50	77.27	55.56	66.41	66.67	61.02	75.68	68.35
	Dual	76.60	70.00	84.09	77.05	72.50	70.59	75.86	73.23	73.96	73.47	74.47	73.97
CNN-GAN	fMRI	77.66	74.47	82.98	78.72	73.75	72.92	75.00	73.96	75.00	84.91	62.79	73.85
	SNP	74.47	71.42	77.78	74.60	72.50	67.92	81.48	74.70	72.92	67.92	79.07	73.50
	Dual	80.85	83.33	80.23	83.87	77.50	72.92	84.38	78.65	78.13	78.57	77.78	78.17
GCN-GAN	fMRI	78.72	76.09	81.25	78.67	75.00	60.00	81.82	70.91	76.04	72.92	79.17	76.04
	SNP	75.53	67.65	80.00	73.82	73.75	71.43	77.42	74.42	73.96	75.47	72.09	73.78
	Dual	81.91	89.47	80.00	84.74	78.75	75.00	84.38	79.69	79.17	78.26	80.00	79.13
FIAD-GAN	fMRI	87.23	77.27	90.28	83.38	85.00	85.71	84.21	84.96	84.38	85.37	83.64	84.50
	SNP	84.48	87.50	81.48	84.49	83.75	88.37	78.38	83.38	83.33	81.40	84.91	83.15
	Dual	93.62	95.35	92.16	93.75	91.25	91.67	90.63	91.15	91.95	92.11	91.84	91.97

The bold values indicate the highest results of each indicator in each group of experiments, and the proposed approach always performs the best.

Figure 3

The radar plot of FIAD-GAN method and other related methods, including RF, SVM, CNN, GCN, CNN-GAN and GCN-GAN. (A) AD versus NC task based on fMRI data; (B) EMCI versus LMCI task based on fMRI data; (C) LMCI versus AD task based on fMRI data; (D) AD versus NC based on SNP data tasks; (E) EMCI versus LMCI task based on SNP data; (F) LMCI versus AD task based on SNP data; (G) AD versus NC task based on fMRI and genetic data; (H) EMCI versus LMCI task based on fMRI and genetic data; (I) LMCI versus AD task based on fMRI and genetic data.

Evaluation of network construction method

To verify the superiority of the BG-network, the brain networks were constructed through single fMRI data and the gene networks were constructed through single SNP data. Meanwhile, Kendall correlation coefficient (KCC), Spearman correlation coefficient (SCC) and PCC were selected to verify the validity of data fusion.

Table 3 presents the classification performance of FIAD-GAN under different network construction methods. In the classification task distinguishing AD and NC, the PCC network construction had the best classification performance using both unimodal and multi-modal data. The method had the best performance when using multimodal data. Similarly, it could be observed that PCC with multimodal data construction performed the best in the other two tasks, which indicated that the PCC was able to better detect the association information in multimodal data, which can improve the classification of diseases.

Table 3

Performance comparison of different network construction methods based on different modal data

Data	Method	NC versus AD				EMCI versus LMCI				LMCI versus AD
		ACC (%)	SEN (%)	SPE (%)	BAC (%)	ACC (%)	SEN (%)	SPE (%)	BAC (%)	ACC (%)	SEN (%)	SPE (%)	BAC (%)
fMRI	SCC	78.72	76.09	81.25	78.67	75.00	72.92	78.13	75.52	77.08	73.47	80.85	77.16
	KCC	79.79	77.78	81.63	79.71	76.25	75.00	78.12	76.56	78.13	74.47	81.63	78.05
	PCC	87.23	77.27	90.28	83.38	85.00	85.71	84.21	84.96	84.38	85.37	83.64	84.50
SNP	SCC	75.53	76.6	74.47	75.53	72.50	71.15	75.00	73.08	73.96	70.00	78.26	74.13
	KCC	77.66	74.47	80.85	77.66	73.75	71.42	77.42	74.42	76.04	73.47	78.72	76.10
	PCC	84.48	87.50	81.48	84.49	83.75	88.37	78.38	83.38	83.33	81.40	84.91	83.15
Dual	SCC	82.98	83.33	82.69	83.01	78.75	76.09	82.35	79.22	79.17	78.26	80.00	79.13
	KCC	84.04	85.37	83.02	84.19	80.00	78.26	82.35	80.21	80.21	79.55	80.77	80.16
	PCC	93.62	95.35	92.16	93.75	91.25	91.67	90.63	91.15	91.95	92.11	91.84	91.97

The bold values indicate the highest results of each indicator under each classification task and each data modality, and the proposed approach always performs the best.

Effect of parameters on classification performance

On the one hand, when constructing the BG-network, the correlation between nodes in the network was calculated by PCC. Then, all edges whose correlation coefficients are less than a threshold |$\alpha$| were deleted. The correlation coefficients of the remaining edges were defined as the weights of the edges, by which the weight matrix of the network was constructed. The value of |$\alpha$| directly affected the density of edges in BG-network. On the other hand, in the feature information aggregation process, the proportion of feature information in the aggregating neighborhood of a node was |$\beta$|⁠. The value of |$\beta$| directly affected the change degree of feature information.

In summary, the best combination of |$\alpha$| and |$\beta$| could make FIAD-GAN reach the best performance. Letting |$\alpha, \beta =[\textrm{0.1,0.2,0.3}\dots, 1]$|⁠, the classification performance of the method was examined by ACC, as shown in Figure 4. According to the experimental results, when |$\alpha$||$=0.3$| and |$=0.6$|⁠, FIAD-GAN achieved the best ACC in the three classification tasks.

Figure 4

Classification accuracy under different parameter combinations in different classification tasks. (A) AD versus NC. (B) EMCI versus LMCI. (C) LMCI versus AD.

Validity of BDC-C

To verify that the designed BDC-C operation was vital to obtaining better classification performance, a visualization experiment was conducted in this section. The left part of Figure 5 shows the specific experimental steps. First, for a specific sample, we define the |$i$|-th row in the weight matrix as the feature information sequence of the |$i$|-th feature (a brain region or a gene). Second, for a specific feature |$i$|⁠, we average the feature information sequences of the |$i$|-th feature, respectively, in the AD and NC groups to obtain the standard feature information sequences of the feature in the two groups. Finally, the PCC is calculated between each feature information sequence and its corresponding standard sequence. The above process was carried out in the weight matrix before and after the BDC-C operation, respectively. The right part of Figure 5 shows the experimental results of some features. For better visualization, only the top four features extracted from each classification task were shown.

Figure 5

The structural information similarity of the most discriminative features.

In Figure 5, the fluctuation in the red lines was significantly weaker than that of the blue lines. Even for the samples having the same disease status, there were fluctuations in the similarity of the feature information between the same features. With the help of the proposed BDC-C operation, the information noise in features was suppressed. The similarity between samples in the same category tended to stabilize, which could improve the classification performance. The results uncovered the rationale for the improvement of classification ability brought by the proposed BDC-C operation. Compared with the ordinary convolution, the BDC-C operation takes the non-Euclidean structure of the graph into account, facilitating the efficient capture of the potential correlation in multimodal data. At the same time, the impact on performance caused by excessive model parameters is suppressed, improving the classification performance of the model.

Comparison with advanced methods

This section compared FIAD-GAN with some advanced methods [34, 39–42], covering several of the most common deep learning methods like CNN or GAN. The results are presented in Table 4, which also reported the sample numbers and data modality. The results showed that, among the compared methods, FIAD-GAN maintained satisfactory performance on all three classification tasks.

Table 4

Comparison with advanced methods

Method	Modality	Samples	ACC (%)
3D-CNN	MRI + SNP + EHR	267 (NC)/128 (AD)	86.00
C-GNN	fMRI	960 (NC)/592 (AD)	85.80
SVM	MRI	74 (EMCI)/38 (LMCI)	64.30
FSN + PFC	fMRI	29 (EMCI)/18 (LMCI)	80.85
C-GNN	fMRI	638 (LMCI)/592 (AD)	65.20
TLA	MRI	70 (LMCI)/75 (AD)	76.73
FIAD-GAN	fMRI + SNP	237 (NC)/233 (AD)	93.62
		197 (EMCI)/203 (LMCI)	91.25
		203 (LMCI)/233 (AD)	91.95

Biomedical discoveries and discussion

In terms of biological discovery, this work provides a reliable reference and technical basis for the clinical diagnosis, treatment and pathological analysis of diseases. This section displayed and analyzed these discoveries from different aspects.

First, based on the key structural sub-network, the importance scores of each node (brain region or gene) and edge (brain region-gene pair) were calculated. Then, they were sorted in descending order to find the optimal feature subset (optimal node feature subset and optimal edge feature subset) using the incremental search method. Figure 6 shows a visualization of 15 of the most distinctive brain region-gene pairs.

Figure 6

Visualization of the most discriminative brain region-gene pairs.

These brain region-gene pairs had unique guiding significance for further biomedical research. With the development of gene sequencing technology, it is rather difficult to screen out the gene-to-brain connections related to diseases from the entire human genome. The findings of this paper enabled researchers to focus on the regulation mode of certain specific genes to specific diseased brain regions, which may significantly improve the accuracy and speed of pathological research on complex brain diseases.

Second, Figure 7A shows the importance scores of the top 10 most discriminative brain regions. Figure 7B shows the locations of these brain regions, where the node sizes reflect the importance score. The extracted brain regions were consistent with other medical findings.

Figure 7

Location and importance scores of the most discriminative brain regions. (A) Importance of diseased brain regions. (B) Locations of diseased brain regions.

The inferior frontal gyrus had long been believed to be related to the development and degradation of lingual ability, yet recently, with the gradual deepening of the research on the function of brain regions, more and more scholars have found that the inferior frontal gyrus has a great relationship with the decline of memory ability and cognitive ability. Hay et al. [43] found a significant negative correlation between individual subjective cognitive ability and cerebral blood flow in the inferior frontal gyrus, proving that the inferior frontal gyrus may be an important biomarker of AD-like disease. As an important component, the Triangular part of the inferior frontal gyrus (IFGtriang) was shown to play a determinant role in memory generation [44]. Also, a significant increase in functional connectivity was detected in the orbital part of the right inferior frontal gyrus (ORBinf) in patients with cognitive impairment [45]. Kandilarova et al. [4] found that IFGtriang and ORBinf have significant regulatory effects on emotional abnormalities through 3 T-MRI scan data. Coincidentally, IFGtriang and ORBinf also have high importance scores in this study.

Gyrus rectus (REC), once regarded as the non-functional area of the brain, has been found evidently related to multiple cognitive functions like personality and thus may be linked with neurodegenerative diseases like AD. For example, Knutson et al. [46] found that the disinhibited behavior of patients with REC injury will be more significant, which is also one of the important criteria for the behavior of AD patients. Cajanus et al. [5] further observed a statistically significant association between the degree of atrophy in the REC region and the index of disinhibition in patients with neurodegenerative diseases. Shin et al. [47] studied MRI cortical thickness changes in the progression from MCI to dementia, finding that the right REC region was associated with the AD course.

Hippocampus (HIP) is the main control area of human memory and emotion regulation, whose atrophy has become a common and recognized biological characterization in AD research [48]. Recent studies have provided concrete evidence of its association with diseases such as AD from various perspectives. For example, the study by Tobin et al. [49] showed that the degree of neurogenesis in the hippocampal region is related to cognitive status. Specifically, the decrease in neurogenesis may indicate the onset of AD. Altuna et al. [37] found multiple DNA methylation sites related to AD in HIP, supporting the view that genes regulate AD development by affecting brain lesions.

Figure 8 shows all the genes extracted in this study and their importance scores, most of which were also supported by previous results. For example, the gene DAB1 regulates the synaptic neurotransmission in the adult brain and is involved in memory formation and maintenance [50]. The study by Blume et al. [13] showed that abnormal expression of DAB1 affects synapse generation in HIP, thereby affecting the generation of neuronal circuits with memory and learning functions, which may be a new way of the gene DAB1 affecting AD development. Kunkle et al. [51] conducted a genome-wide association study analysis of African-American AD and found some AD-related genes including LRP1B. CNTNAP2 has long been linked to neurodevelopmental disorders such as autism spectrum disorder. Based on gene knockout experiments, Varea et al. [52] studied the role of the gene CNTNAP2 in neuronal development and maturation, indicating the potential mechanism of this gene in neurodevelopmental diseases. As the results show, FIAD-GAN can precisely identify AD-related factors. For other atypical pathogenic factors not yet discussed, we plan to expand our dataset in the follow-up work and cooperate with clinicians to find more proof of the pathogenetic mechanism behind these factors. The identified brain regions and genes were distinctive between patients and normal people, which provides potential biomarkers for the clinical diagnosis and treatment of AD and can directly guide the development of transcranial magnetic stimulation therapy and targeted drugs.

Figure 8

Importance scores of risk genes. Genes with greater importance were considered as causative genes.

For the node features (brain regions and genes) extracted by all comparative methods, the discriminative ability was assessed through the standard t-test. Figure 9 displays the P-values of the top 30 features extracted by the six methods. As shown, the P-values for most of the distinguishing node features extracted by FIAD-GAN were <0.05. It could be concluded that the features extracted by FIAD-GAN had a good discriminating ability, which not only illustrated that the proposed method has better classification capability, but also proved that FIAD-GAN had a promising prospect in the application of clinical decisions.

Figure 9

P-values of the top 30 features extracted by different methods.

Finally, the functional connectivity developmental pattern of NC to AD was demonstrated by visualizing the brain functional connectivity network of the sample, as shown in Figure 10. The mean connectivity weights of brain functional connectivity in the NC group and AD group were calculated by Pearson correlation analysis. For better demonstration, the weights below a certain threshold were reset to 0. According to the exhibited results, as the disease develops, the number of functional connections significantly decreased and the strength of the connections diminished, indicating that some of the functional brain connections were disrupted during the disease progression of AD. In addition, most brain regions corresponding to the discriminative brain functional connectivity were consistent with those extracted by the method in this study, which from another aspect demonstrated the rationality and validity of FIAD-GAN. Also, the brain regions corresponding to the functional connections corrupted in AD patients were those who needed to be particularly concerned in the clinical treatment.

Figure 10

Brain functional connection networks. (A) NC. (B) AD.

Rich experimental results indicate that the effectiveness of FIAD-GAN in feature extraction is not derived from the experimental coincidence but from the design of methodology. Based on the premise that the brain region level abnormalities reflected by brain imaging are regulated by genes, the effective extraction of multi-modal features by the proposed method benefits from two key points. The first is network-based fusion feature representation. By abstracting pathogenetic factors as nodes and the inter-modal relationship as the edges, BG-networks implement the information fusion of two modalities and provide a premise for inter-modal feature learning. The second is the proposed FIAD-GAN method, where the convolution and deconvolution operations in the generator are designed according to the feature information aggregation model we established. FIAD-GAN can explore the aggregation and diffusion pattern of the feature information between brain regions and genes in the disease development process, and realize the fusion of the multi-modal information.

In addition, FIAD-GAN is a general method for multimodal feature extraction. In addition to AD, our method has been applied for classifying Parkinson’s disease, autism spectrum disorder and other diseases, where different modalities of data in public datasets such as PPMI or ABIDE are applied. We also conducted experiments based on the clinical data collected by the Xiangya Hospital, which also confirmed the effectiveness and universality of our method. Considering the diversity of diseases, it is hard to enumerate all existing datasets from the experimental perspective. However, from the perspective of a data analysis algorithm, it is promising that our work can be extended to a wider range of data.

Conclusions

The development of complex brain diseases like AD has profound genetic causes. To fully utilize the complementary information between fMRI data and genetic data, this paper proposes a fusion data representation using graph structure and proposes an effective GAN-based method to realize automatic disease classification and feature extraction. Therefore, we established the feature information aggregation model and designed the specific convolution operation. We design experiments based on fMRI and SNP data in ADNI, and the superiority of FIAD-GAN in terms of disease classification and feature selection is verified by rich experiments. Furthermore, we extracted brain regions and genes that are highly related to the development of AD, which may provide a reference for future disease research and clinical diagnosis.

Key Points

• This paper integrated multi-level medical data and represent it with a BG-network to explore AD from multiple perspectives, and it is one of the handful studies where network-based data representation is introduced.

• We established a more intuitive and reasonable feature information aggregation model to describe the pathological interaction among brain regions and genes.

• A novel deep learning framework named FIAD-GAN is proposed based on mathematical modeling, where the BDC-C and D-DC operations in the generator are specially designed for better key structural information capture and BG-network reconstruction. As the results indicate, the interpretability of the GAN framework has been improved, and performance was achieved in the classification and feature extraction.

• The proposed FIAD-GAN extracted salient features, which provide a reliable reference and technical basis for the clinical diagnosis, treatment and pathological analysis of AD-like diseases.

Data availability

The source code is available at: https://github.com/fmri123456/FIAD-GAN.git. Multi-modal data used in this study is obtained from the Alzheimer’s Disease Neuroimaging Initiative (https://adni.loni.usc.edu/).

Funding

National Natural Science Foundation of China (62072173), Natural Science Foundation of Hunan Province, China (2020JJ4432), Key Scientific Research Projects of Department of Education of Hunan Province (20A296), Key open project of Key Laboratory of Data Science and Intelligence Education (Hainan Normal University), Ministry of Education (DSIE202101), National Key R&D Program of China (2020YFB2104400), Medical humanities and Social Sciences project of Hunan Normal University, and Innovation & Entrepreneurship Training Program of Hunan Xiangjiang Artificial Intelligence Academy.

Author Biographies

Xia-an Bi is currently a professor in the Hunan Provincial Key Laboratory of Intelligent Computing and Language Information Processing, and College of Information Science and Engineering in Hunan Normal University. His current research interests include machine learning, brain science and artificial intelligence.

Yuhua Mao, Sheng Luo and Hao Wu are currently pursuing the MS or ME degree with the Department of Computing, School of Information Science and Engineering, Hunan Normal University, Changsha, China. Their research interests and expertise are in machine learning, big data analysis and artificial intelligence.

Lixia Zhang is currently a professor at the School of Information Science and Engineering, Hunan Normal University. Her current research interests include machine learning, large-scale graph algorithm research and artificial intelligence.

Xun Luo is currently an associate professor in the College of Information Science and Engineering in Hunan Normal University. His current research interests include big data analysis, algorithm design and deep learning.

Luyun Xu is currently an associate professor in the College of Business in Hunan Normal University. Her current research interests include knowledge network, brain science and artificial intelligence.