Multi-class Classification of Alzheimer’s Disease Using Deep Learning and Transfer Learning on 3D MRI Images

3.1 Dataset description

Alzheimer's Disease Neuroimaging Initiative (ADNI) is a longitudinal multicenter study aiming to identify diagnostic biomarkers (clinical, imaging, genetic, and biochemical) for early detection of Alzheimer's disease. screening for Alzheimer's disease and subsequent treatment [19]. Since ADNI contains more MRI data than any other publicly available source, we have used it to test the efficacy of our method. We used 375 samples with Alzheimer’s disease, 378 samples with mild cognitive impairment, 447 cognitively normal samples and total number of samples are 1200 images from the dataset. Grad Warp (which fixes distorted picture geometry caused by the gradient model), B1 Correction (which employs B1 calibration scans to even out image brightness), and N3 (which employs a histogram peak sharpening method to even out brightness) were all used as preliminary processing on all samples. The original data dimensions were changed to 113×137×113×3 so that all samples would be uniform in size and shape. As hippocampus volume is a strong indicator of Alzheimer's disease classification, the data reduction was performed by scaling rather than cropping to preserve hippocampi information. Table 1 represents the dataset description.

Table 1. Analysis of dataset for implementing models

3.2 Proposed methodology

MRI scans are collected from a variety of sources and given for preprocessing. The image's dimensions are then changed by the pre-processing layer. This model classifies AD into three categories and recognizes it. Leveraging the power of 3D convolutions and connection-wise attention mechanisms, our densely linked CNN architecture tackles the challenge of extracting meaningful features from complex 3D brain MRI scans. This robust and efficient network provides a novel approach to analyzing these rich datasets, paving the way for advancements in neuroscience and medical diagnosis. The suggested deep learning-based system model used MRI data to detect and classify disorders at an early stage [20-23]. MRI scans were among the raw training data that was collected. The image was converted from its original 96×120×96×3 dimensions via a pre-processing layer. The study explores applying transfer learning to detect Alzheimer's disease. It utilizes pre-trained deep learning models, modifying their final layers to adapt to this specific task. The proposed paradigm is illustrated in full in Figure 2.

We suggested ResNet (Residual Network) as a solution to the problem of vanishing gradients that occurs during deep convolutional network training. ResNet introduces skip connections, which use an identity function to bypass nonlinear transformations. This one-of-a-kind architectural feature allows for the training of much deeper networks with less computational effort. The benefit of ResNet is that it avoids the vanishing gradient problem that plagues traditional networks by propagating gradients from one layer to the next. Furthermore, densely linked convolutional networks, a novel connectivity pattern, were introduced [24]. This pattern enhances interlayer communication even further, contributing to improved network performance. ResNet capitalizes on the network's potential by reusing features, rather than relying solely on extremely deep or wide architectures to enhance representational power. This approach results in streamlined models that are easier to train and more efficient in terms of parameter usage. Additionally, it has been demonstrated that these feature maps effectively integrate information from previous layers, thereby increasing input variation and enhancing overall network efficiency. Consider a convolutional network being applied to a single image, X₀. The network consists of L layers, where l indexes the layer and H_l(.) denotes a non-linear transformation implemented by each layer. As a result, the feature maps from all previous layers, including X₀, X₁, … and X_l, are input to the lth layer.

X_l=H_l [(X_l−1, X_l−2, X_l−3, …, X₀)]            (1)

This network was built using a time-saving strategy that took visual attention into account. This work employed a convolutional neural network architecture with a connection-wise attention mechanism, allowing for flexible feature map integration through weighted summation. Weights for this summation were learned automatically during training, enabling the model to optimize feature representation for the task at hand. By focusing on the most useful information, this has made the network simpler and more effective [25-28]. The i^th layer of the convolution neural network received a weighting coefficient W in accordance with Eq. (2), where Wi represented an attention vector composed of i-1 elements. Formula 3, in which the feature maps from the j^th layer and H_l were represented by x_j (1<j<l-1). was a non-linear transformation that connection-wise focused on the layout of the lth layer.

W_i=[w_i−1, i, w_i−2, i, …, w₂, i, w₁, i]              (2)

x_l=H_l(w_l−1, x_l−1)+w_l-2, x_{l− 2} … + w₁, x₁                    (3)

Figure 2. Basic architecture of proposed methodology

3.png

Figure 3. Sample architecture for implementing models with transfer learning

There were four distinct layers in the network. First, there was the layer that the network received the picture patches through called the input layer. The second kind of layer was a convolutional one, which used both the input images and the learnt filters to generate feature maps for each filter.

3.3 Transfer learning for classification

Leveraging transfer learning, the proposed approach achieved 3-way AD classification as depicted in Figure 3.

When we have a large dataset from which model learn all parameters, we can switch to transfer learning. A trained network, such as Resnet50v2, is used as a starting point for learning a new task. After training on ImageNet, the Resnet50v2 model was used ADNI dataset. The frozen fully connected layers produced 890 features and outputs for 3 classes, necessitating a transfer learning approach. To accommodate four-class categorization, the model's architecture underwent modification. The final layers were replaced with a new fully connected layer, a SoftMax layer, and an output layer specifically designed for multi-class handling. The network was then trained using a dataset of magnetic resonance images and optimized training parameters [29]. After training, the model's accuracy was evaluated to assess its effectiveness in making correct classifications. To measure model performance and guide training, loss was calculated using the Cross-Entropy function. This function ensures that the model's output dimensions match the number of classes being classified. X denotes the feature space, and P(X) denotes the associated marginal probability [30-34]. In P(X), X={x1, x2, ..., xn}, where n denotes the number of input images. The domain is represented mathematically.

Domain $=\{\mathrm{X}, \mathrm{P}(\mathrm{x})\}$

Within two distinct domains, the ways in which features were distributed and represented differed significantly. To formalize a specific task within a domain, a set of potential labels (W) and a prediction function (f(.)) were employed.

Task = {W,f(.)}

The model's prediction function (f(.)) was trained on features extracted from the data, enabling it to make predictions on unseen test data. The proposed framework involved two domains: a target domain (Domain_p) and a source domain (Domain_q). Data points in the source domain with label w_si were designated as x_si, while those in the target domain with label w_ti were designated as x_ti. The target domain and the source domain is formulated as follows:

$Domain_p=\left\{\left(x_{t 1}, w_{t 1}\right),\left(x_{t 2}, w_{t 2}\right), \ldots.\left(x_{t i}, w_{t i}\right)\right\}$               (4)

$Domain_q=\left\{\left(x_{s 1}, \mathrm{w}\right),\left(x_{s 2}, w_{s 2}\right), \ldots .\left(x_{s i}, w_{s i}\right)\right\}$              (5)

Transfer learning shines as a powerful technique for building predictive models (f(.)). It leverages insights gleaned from past tasks and domains (source activities and domain) to efficiently train the model and accurately predict labels for new data points (x). f(x) was represented mathematically as $f(x)=P(W / X)$.

Algorithm:

• Input

P(Y), Y={y1, y2, ..., yn}: Probability distribution of samples in the dataset, where Y represents the collection of samples.

• Pre-Training

For each sample in the dataset:

• Utilizing a pre-trained source domain (Ds) network with embedded knowledge.
• Preparing target domain (Dt) training and validation sets for model adaptation.
• Initiating knowledge transfer via training and validation on these sets.

End for

• Fine-Tuning

For each feature f(y):

• Perform model customization for the target domain via fine-tuning of designated layers, targeting {Y, P(y)}.
• Optimize target task performance through further fine-tuning utilizing the training dataset (Dt).
• Conduct model evaluation on unseen images using the test dataset (Dt) to gauge categorization efficacy.

End For

• Output

The fine-tuned model achieved a high level of accuracy in the classification of test dataset images.

3.4 Transfer learning for pretrained models

3.4.1 InceptionResNetV2

Inception-ResNet-V2 framework is built around the Residual Inception Block. each block is followed by a meticulous dimensionality check. After each block, an 11-convolution filter expansion layer ensures accurate input depth representation before summation. To maintain harmony, batch normalization is selectively applied to conventional layers only. This intricate network elegantly accepts 113×137×113 pixel inputs and orchestrates a symphony of 164 layers. At the heart of this innovation lies the Residual Inception Block (RIB), a masterful ensemble of diverse convolutional filters and residual connections [35]. This architecture takes advantage of residual connections to combat severe network deterioration and quicken the training process. Since there were no tuning parameters in this core design, Max Pooling was performed to reduce overfitting in the convolutional structure by increasing the correlation between feature importance and label category [36, 37]. This means that max Pooling outperforms the Flatten method in terms of parameter efficiency. To safeguard against overfitting and promote model generalization, a Dropout layer is strategically injected, wielding a constant value of 0.8.

$\sigma(x)_i=\frac{e^{x_i}}{\sum_{j=1}^k e^{y_i}}$            (6)

The SoftMax activation function, applied to the dense layer, transformed its outputs into probability distributions across K classes, as specified in Eq. (6), which in this case is e=2.718, the dense layer was activated using the SoftMax activation function.

$w^{\prime}=w-\alpha \times \nabla\left(w ; x^{(i)} ; y^{(i)}\right)$               (7)

Throughout backpropagation, we optimized with Stochastic Gradient Descent (SGD), an iterative method. In the give Eq. (7), where w stands for weight, $\alpha$ for learning rate, and $\nabla\left(w ; \quad x^{(i)} ; \quad y^{(i)}\right)$ for the gradient to weight, input, and output/label, respectively.

3.4.2 Proposed ResNet50V2 with TL

ResNet50v2, a star performer in the computer vision world, sits alongside champions like VGG16, DenseNet121, Xception, and MobileNetV2. Built upon vast datasets of diverse images, these pre-trained models offer their expertise through transfer learning algorithms, even with limited data and resources. In this study, we leverage a large medical image dataset and perform transfer learning with ten distinct pre-trained weights derived from ResNet50v2. ResNet50v2, a Convolutional Neural Network (CNN) boasting 50 layers, forms the backbone of our exploration. Figure 4 unveils its architecture, alongside our fine-tuning setup for transfer learning.

ResNet50v2 architecture features a series of convolutional layers, starting with an initial layer using 64 kernels, a stride of 2, and a 7×7 filter, followed by 3×3 pooling. It then stacks multiple sets of convolutional layers, each set containing three layers: 1×1 convolutions, 3×3 convolutions, and a final 1×1 convolution that increases channel depth. This pattern repeats with progressively larger kernel numbers and more repetitions through the network. Max pooling and hidden layers combining convolutions, batch normalization, and ReLU activations further refine the features. Notably, the original fully connected layer with 1000 out-features is replaced with a group of fully connected layers to enhance the model's performance. To adapt ResNet50v2 for three-class dementia classification (non-demented, mild, moderate, very mild), the original final layer is replaced with a custom dropout scheme. This involves selecting the first 2048-feature layer with a 50% chance of inclusion, followed by a ReLU layer and another dropout layer with the same probability. Finally, a final fully-connected layer with 4 outputs maps features to the specific dementia classifications. Notably, this study explored transfer learning, utilizing 10 pre-trained ResNet50v2 models from diverse medical image datasets to optimize performance for this specific task.