Ayush Giri
| Prashna Thapa | J. Saira Banu* | Shashwat Poudyal | Baibhav Rijal | Siddhant Karki
© 2024 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
Precise segmentation of brain tumors from MRI scans remains a formidable challenge, driven by the critical demand for accuracy in medical imaging. To surmount this challenge, our paper introduces the Federated ResUHybridNet—a cutting-edge methodology that harmonizes the resilience of ResNet with the precision of U-Net. Nestled within the sophisticated realm of federated learning, this innovative architecture fosters collaborative model training, optimizing the training process and steadfastly upholding stringent data privacy standards. The methodology employs a Federated learning framework for collaborative model training across multiple hospital nodes. It features the ResUHybridNet architecture, combining the deep feature learning of ResNet with the detailed segmentation capabilities of U-Net. This integration optimizes brain tumor segmentation by synergizing the strengths of both architectures. Furthermore, the decentralized ResUHybridNet model undergoes fine-tuning by leveraging the local data of each individual participating hospital. The study is dedicated to the segmentation of brain tumors using 3D MRI scans as the imaging modality. The dataset employed encompasses 3D volumetric data, enhancing the depth and spatial understanding crucial for the evaluation of the Federated ResUHybridNet architecture. Overall, the Federated ResUHybridNet model significantly advances data privacy measures through federated learning and achieves optimized model performance. These contributions mark a notable stride towards enhancing brain tumor diagnosis standards and refining subsequent treatment strategies.
deep learning, brain tumour segmentation, federated learning, ResUHybridNet, data privacy
Brain tumor segmentation, essential for precise diagnosis and treatment planning in medical imaging, is pivotal for guiding interventions that can significantly impact patient outcomes. Despite strides in deep learning, challenges persist in handling diverse datasets [1], ensuring privacy, and maintaining consistency across healthcare institutions. However, as with any asset, there's a flip side. The primary challenge addressed is the accurate segmentation [2] of brain tumors in medical imaging, coupled with the imperative need for stringent data privacy measures. This dual concern necessitates innovative solutions that balance precision in diagnosis with robust data protection. The very nature of centralizing data – placing large amounts of sensitive information in one location or under one management umbrella [3] – can inadvertently make it an attractive target for malicious intents. As the intricacies of data management and security grow in complexity, so does the vulnerability landscape. It's not just about unauthorized access; there are concerns about data integrity, mishandling, and potential misuse. This has led many in the healthcare sector, especially those leading institutions with vast data holdings, to tread cautiously. The hesitancy to share or pool data could impede the kind of groundbreaking research these centralized systems were designed to foster in the first place.
Dive into the world of medical imaging, and you're immediately met with a tapestry of innovations, challenges, and aspirations [4, 5]. Among the myriad tasks under this umbrella, the segmentation of brain tumors is especially critical. This process is not just a technical challenge; it's a linchpin in the diagnosis and subsequent treatment of neurological conditions. An accurate segmentation can lead to precise treatment plans, better patient outcomes, and reduced treatment times. Conversely, inaccuracies could have grave ramifications.
However, even as advancements in deep learning usher in promising methodologies, challenges remain. Handling vast and dense datasets, ensuring data privacy, and maintaining consistent model performance across healthcare institutions each come with its set of questions [6]. Furthermore, the variability in data – stemming from different imaging technologies, patient demographics, and even subtle differences in institutional practices – adds another layer of complexity. Addressing these challenges is not just a matter of refining algorithms; it's about understanding the multifaceted nature of medical data and the ecosystems in which they exist [7]. The primary contribution of the paper lies in the fusion of ResNet and U-Net architectures, resulting in the creation of ResUHybridNet. This hybrid model leverages the feature extraction capabilities of ResNet, known for its deep residual learning, and the exceptional segmentation performance of U-Net. By synergistically combining these architectures, ResUHybridNet achieves a holistic approach to brain tumor segmentation, capturing both high-level features and fine-grained details simultaneously. Section 2 deals with necessities of accurate brain tumor segmentation Section 3 mentions enhancing model accuracy and generalization through federated learning framework, Section 4 explains how the data privacy and security can be enhanced using the federated learning framework, Section 5 mentions about Transforming Segmentation modalities. Section 6 clearly mentions the existing works, Section 7 dives into the proposed methodology and finally includes the Results and Conclusions section.
The precision required in brain tumor segmentation not only underscores its critical role in the medical landscape but also serves as the backbone for drawing insightful conclusions and formulating detailed treatment strategies [8]. Its significance extends across diverse medical applications, including the intricate demands of surgical procedures, the specificities of radiation therapy, and the ongoing vigilance required for disease monitoring. The unequivocal need for accurate tumor detection is paramount, not only uplifting patient care standards but also significantly influencing the trajectory of medical research.
Pioneering advancements in brain tumor segmentation hold the potential to reshape the landscape of patient care, contributing to more effective treatment plans and improved outcomes [9]. The integration of federated deep learning models introduces a promising avenue for collaborative and decentralized research paradigms, suggesting a future where such approaches might become the norm.
The ongoing convergence of traditional medical techniques with modern algorithmic advancements prompts a call for sustained interdisciplinary research efforts. This intersection provides a fertile ground for exploring innovative methodologies, optimizing existing techniques, and ultimately advancing the field of brain tumor segmentation to new heights. As technology continues to evolve, the collaborative exploration of these frontiers will be essential for unlocking breakthroughs that stand to revolutionize medical diagnostics and treatment strategies.
In the realm of medical imaging, particularly with brain tumor segmentation, the pursuit of model accuracy is relentless. Every incremental improvement can translate into enhanced clinical outcomes and patient safety. While achieving high accuracy on training datasets is commendable, the true challenge lies in ensuring that such models generalize well across diverse and unseen datasets. Models that excel in controlled environments but falter in real-world scenarios can have detrimental consequences.
Several factors contribute to the quest for optimal accuracy and robust generalization. One is the diversity of data sources. For instance, the variances in imaging devices, techniques, and protocols across different hospitals and regions can introduce nuances that models need to recognize and handle. Another significant aspect is the incorporation of domain knowledge. Leveraging insights from experienced radiologists and neurologists can bridge the gap between algorithmic precision and practical utility [10].
Figure 1. Overview of federated learning
Furthermore, the rise of hybrid models like ResUHybrid Net in conjunction with Federated learning opens new horizons. Such architectures harness the strengths of multiple neural network designs, promoting a synergistic improvement in performance [11]. Federated learning, on the other hand, allows for decentralized training on data sources, fostering model generalization without compromising data privacy as shown in Figure 1. Moreover, the pursuit of model accuracy extends beyond algorithmic prowess. Ethical considerations and interpretability play pivotal roles in the deployment of these models in real-world healthcare settings. Striking a balance between technological advancement and ethical responsibility is essential for building trust in the reliability and utility of these advanced medical imaging techniques. As the landscape of medical imaging evolves, the collaboration between data scientists, medical professionals, and ethicists becomes increasingly vital. Interdisciplinary efforts are crucial for addressing the multifaceted challenges in brain tumor segmentation, paving the way for innovations that can revolutionize diagnostic accuracy and patient care on a global scale.
The secure segmentation of brain tumors is ushering in a transformative era in the healthcare industry, providing accurate insights that empower medical professionals to make informed decisions regarding surgeries, radiation therapy, and disease monitoring. This precision not only enhances patient care but also accelerates medical research through the application of deep learning models. The integration of federated learning with deep learning emerges as a powerful approach to address the challenges associated with brain tumor segmentation, fostering collaborative partnerships across healthcare sectors and reshaping the landscape of diagnosis and treatment [12].
The protection of sensitive medical data is paramount in ensuring patient privacy and maintaining the integrity of healthcare systems [13]. The proposed methodology plays a crucial role in tackling this challenge through the application of federated learning. By design, federated learning allows model training to occur locally on individual hospital nodes without the need for raw data to be centralized. Moreover, federated learning enables collaborative model training across multiple nodes, creating a robust and generalized model without exposing individual patient data. This not only enhances data privacy but also addresses concerns related to data security and unauthorized access. By fostering a collective approach to model development, the proposed methodology mitigates the risks associated with centralized repositories, aligning with evolving data protection regulations and ethical standards.
In summary, the protective measures embedded in the proposed methodology, specifically through federated learning, serve as a safeguard for sensitive medical data. This not only aligns with ethical considerations in healthcare but also establishes a foundation for collaborative advancements in brain tumor segmentation, ultimately redefining standards in diagnosis and treatment while preserving patient privacy.
In medical research, the constant advancement of AI models relies on diverse datasets, yet patient confidentiality poses a persistent challenge. Federated learning emerges as a groundbreaking solution, reshaping the landscape of medical imaging research, especially in studying brain tumors. This innovative approach facilitates seamless data sharing across institutions without compromising patient privacy. The intrinsic value of federated learning lies in its capacity to expedite research, enabling collaborative analysis of diverse datasets. This not only expedites the development of highly accurate brain tumor segmentation models but also significantly deepens our understanding of various brain diseases [14].
The paramount advantage of federated learning is its potential to usher in a new era of swift and precise diagnosis and treatment strategies, promising a fundamental shift in the paradigm of patient care. Equally critical is its unwavering commitment to upholding patient confidentiality and data security, mitigating ethical concerns inherent in medical research. Beyond these essential safeguards, federated learning champions inclusivity, resource efficiency, and continuous model improvement. The approach not only advances the domains of neurology, neuroscience, and medical imaging but also fosters a collaborative spirit among researchers and medical institutes.
The development of a federated learning model for brain tumor segmentation signifies not just a technological breakthrough but a transformative force in medical research and patient care. Healthcare professionals in neurology, neuroscience, and medical imaging can anticipate not only advancements in their respective fields but also the dawning of a new era characterized by teamwork, shared data, and progressive collaboration in the pursuit of improved healthcare outcomes.
Deep learning techniques have made remarkable progress and have proven effective in medical image analysis, especially in brain tumor classification. Among these techniques, convolutional neural networks (CNNs) have emerged as a prime example, demonstrating the ability to learn truly complex features from raw data.
6.1 CNN-based approaches
Zikic et al. [15] utilized a simple neural network with 2 convolutional layers and 2 fully-connected layers, optimizing with a log loss function. Achieved competitive results with a relatively simple neural network but may lack the depth for more complex feature learning.
Pereira et al. [16] designed specific architectures for HGG and LGG segmentation, training FCNNs on 2D patches and refining results with CRF as recurrent neural networks.Notable for addressing specific glioma types and achieving state-of-the-art performance on the BRATS 2013 dataset., but the complexity of architecture might impact computational efficiency.
Kamnitsas et al. [17] introduced DeepMedic, utilizing multi-scale patches for 3D CNN methods. Introduced DeepMedic, showcasing the importance of multi-scale patches for capturing local and global information in 3D CNN methods. The downside was that this method may include computational complexity and resource-intensive training.
Isensee et al. [18] designed a modified U-Net with residual blocks and deep supervision, leading to improved brain tumor segmentation results. Interpretability and computational efficiency due to the increased model complexity is a challenge.
6.2 Federated Learning (FL) approaches
Sheller et al. [19] evaluated FL alongside other data-private collaborative learning methods, including institutional incremental learning and cyclic institutional incremental learning, within the context of brain tumor segmentation.While FL mitigates institutional biases, challenges may arise in terms of communication overhead and coordination across different institutions, and the method's performance may be influenced by the heterogeneity of data sources.
Li et al. [20] extended FL to brain tumor segmentation using a DNN and the BraTS dataset, emphasizing privacy-preserving techniques contributing to the emerging field of collaborative learning. However, its limitations may include communication overhead and potential challenges in maintaining model performance with decentralized training.
Dou et al. [21] leveraged FL to detect chest CT abnormalities in COVID-19 patients, achieving remarkable generalization performance. Demonstrated FL's success in detecting chest CT abnormalities in COVID-19 patients, showcasing its potential in diverse medical contexts. Remarkably, the best generalization performance achieved an accuracy of 95.66% using an FL model. The FLED-Block model, which combines various neural network architectures, achieved an impressive 98.2% accuracy in predicting COVID-19 outcomes using Federated Learning (FL). It utilized capsule networks for feature extraction, employed extreme learning machines (ELM) for classification, and integrated blockchain technology for secure data retrieval.
Guo et al. [22] introduced a cross-site modeling platform utilizing FL for MR image reconstruction. Provided a cross-site modeling platform, addressing challenges of diverse scanners and acquisition protocols, enhancing the applicability of FL. This study focused on aligning concealed features from different sub-sites with those from the main site, yielding promising results across multiple datasets. Potential heterogeneity in data quality and the need for careful consideration of cross-site variations poses a challenge.
Our work builds on state-of-the-art methodologies by incorporating the strengths of CNN-based approaches, leveraging the effectiveness of ensemble methods for improved robustness, and incorporating privacy-preserving techniques such as federated learning. We aim to address limitations observed in prior works, particularly in terms of computational efficiency and generalization, by proposing an innovative hybrid model that synergistically combines the strengths of these approaches. Our methodology aims to advance the field by providing a more comprehensive and efficient solution for accurate brain tumor segmentation while considering privacy concerns and diverse data sources.
The research focuses on brain tumor segmentation using magnetic resonance imaging (MRI) data from five hospitals. Data preparation involves ensuring completeness and quality, with incomplete samples being removed. A specialized DataGenerator class facilitates data processing and augmentation. The ResUHybridNet model, combining ResNet and U-Net, is employed for segmentation, with a structured architecture, including encoder and decoder blocks. The federated learning setup involves multiple hospitals training local models, and a central server aggregating insights using Federated Averaging. Stratified sampling ensures equitable data distribution, and visualization tools validate this distribution. Hyperparameter tuning involves Grid Search, Random Search, and Bayesian Optimization to optimize the ResUHybridNet model's performance. The methodology emphasizes privacy, fairness, and autonomy in federated learning with medical data [23].
7.1 Data preparation and exploration
The research begins by collecting magnetic resonance imaging (MRI) data from five different hospitals, each of which is represented by a separate directory. In this dataset, each patient sample includes various MRI scans, such as flair, t1ce, t2, t1, and a segmentation mask, which are crucial for brain tumor segmentation. To ensure the quality and integrity of the data, the code performs data completeness checks for each sample, ensuring that all required MRI scans are present [24]. We iterate through each sample, verifying the presence of essential files and flagging incomplete samples. Incomplete samples, which lack any of the required MRI scans, are identified and removed from the dataset. To facilitate data processing, augmentation, and model input, a specialized DataGenerator class is implemented. This class is designed to preprocess and augment MRI data, allowing it to be efficiently fed into the deep learning model. Augmentation techniques include zooming, horizontal flipping, vertical flipping, and nearest-neighbor fill mode.
7.2 Model architecture
The model used in this research is the ResUHybridNet, an architecture designed for semantic segmentation of brain tumor regions in MRI scans. This model is structured with input dimensions of 128x128 pixels and classifies the pixels into four categories: "Not Tumor," "Necrotic/Core," "Edema," and "Enhancing." The architecture follows a hybrid design that combines elements of ResNet and U-Net [25]. It is defined using the Keras API and includes encoder and decoder blocks. These blocks consist of convolutional layers, batch normalization, activation functions, and residual connections. The model is compiled using categorical cross-entropy as the loss function and the Adam optimizer.
The encoder segment leverages the strengths of the ResNet architecture, renowned for its residual connections that optimize gradient flow, preventing the vanishing gradient problem, thus ensuring stable and effective training. The input is processed through a ResNet block with 64 filters. This results in a feature map, which is subsequently down sampled using max pooling. The processed data then proceeds to another ResNet block, this time with 128 filters, and is again followed by max pooling. Continuing the trend, the next block employs 256 filters, and, post-processing, the feature map undergoes max pooling.The final encoding ResNet block utilizes 512 filters. Following its processing, a max pooling operation further reduces the spatial dimensions of the feature map.
After encoding, the data reaches the bottleneck, a region where the deepest representations are learned. Here, a ResNet block with a hefty 1024 filters processes the feature map, ensuring extraction of the most intricate details essential for precise segmentation.
The decoder echoes the U-Net's design, celebrated for its expansive blocks that restore the spatial dimensions of the image while maintaining the learned features.First Up-sampling Block: Beginning with an up-sampling of the feature map from the bottleneck, it's concatenated with the feature map from the fourth ResNet block (512 filters) and then passed through convolutional layers.The resulting map is upsampled and fused with the third ResNet block’s feature map (256 filters) before undergoing convolutional operations.Similarly, the map is upscaled and combined with the second ResNet block’s outputs (128 filters) followed by convolutions.
Fourth Up-sampling Block: The last up-sampling step sees the map combined with the first ResNet block’s outputs (64 filters) and processed through convolutional layers.The model concludes with a convolutional layer that has filters equal to the number of classes (in this case, 4). It uses a softmax activation to ensure the output values range between 0 and 1 and can be interpreted as probabilities for each class.The ResU-HybridNet architecture is then compiled with a categorical cross-entropy loss and optimized using the Adam optimizer as shown in Figure 2. Alongside accuracy, multiple metrics, including MeanIoU and custom metrics like dice coefficients for different tumor regions, provide a comprehensive evaluation of the model's performance.
Figure 2. ResUHybridNet architecture
7.3 Federated learning setup and training loop
The research employs a horizontal federated learning setup in which multiple hospitals act as client nodes. Each hospital possesses its own dataset and trains a local model. These local models and the central server model follow the same ResUHybridNet architecture. The server model initializes with the same architecture as the client models.
The federated training loop is designed to enable iterative training over multiple rounds. In this research, 100 rounds are executed. During each round, the client models are trained on their local datasets using the DataGenerator class. After training, the client models' weights are captured and transmitted to the server. The server aggregates these weights using Federated Averaging to update its own weights. This ensures that the global model reflects insights from all client models.The server model is then evaluated on validation datasets from all hospitals. Evaluation metrics, including loss, accuracy, mean intersection over union (IoU), and dice coefficient, are computed. These metrics are aggregated and averaged across all hospitals to assess the performance of the federated model. Additionally, the training time for each round is recorded, providing insights into the time complexity of the federated training process [26].
7.4 Data distribution and visualization
The process of ensuring equitable data distribution across multiple hospital nodes is enhanced through the application of stratified sampling. Stratified sampling, in this context, aims to guarantee that each subset of data, whether it's for training, validation, or testing, has a representative fraction of samples from every hospital. This ensures that no hospital's data is underrepresented or overrepresented. To validate this even distribution and to provide a clearer understanding of the data structure, visualization tools are employed. These visual tools offer an intuitive perspective on how data from each hospital has been integrated, highlighting the distribution percentages, potential outliers, and any patterns that might emerge.
Moreover, in the realm of federated learning, ensuring each hospital's data is correctly loaded and preprocessed has deeper implications than just balanced training. It's a testament to the system's capability to respect the individuality, privacy, and autonomy of each institution. When dealing with medical data, especially from varied and heterogeneous sources, it's paramount that the system doesn't compromise on these principles. This not only builds trust among participating entities but also bolsters the reliability and credibility of the federated model's outcomes. In essence, the combination of stratified sampling and meticulous data visualization acts as a double-check mechanism as shown in Figure 3, ensuring that the principles of fair representation and privacy are upheld in a federated learning environment.
Figure 3. Visualization of ground truth mask
7.5 Model hyperparameter tuning and optimization
Hyperparameter tuning remains a pivotal step in optimizing machine learning models. The selection of optimal hyperparameters can significantly influence the performance of complex architectures, such as the ResUHybridNet in this study. The Grid Search method serves as an initial approach. This exhaustive technique involves defining a subset of the hyperparameter space. The model undergoes training and evaluation for each hyperparameter combination. After testing all combinations, the one yielding the best performance becomes the chosen set. Though comprehensive, the Grid Search method can demand high computational resources, particularly with vast hyperparameter spaces [27].
Following this, the Random Search technique is applied. Contrary to Grid Search, which evaluates every possible combination, Random Search randomly selects combinations for model training and evaluation. Empirical studies indicate that Random Search can attain comparable or even superior results to Grid Search in reduced time, especially when a limited number of hyperparameters substantially affect model performance [28].
Consequently, Bayesian Optimization is integrated into the process. This approach employs a probabilistic model to predict which hyperparameter combinations are likely to yield improved results [29]. By focusing on areas of the hyperparameter space that appear promising, Bayesian Optimization can efficiently identify optimal settings, often with fewer evaluations than traditional methods.
This methodology introduces several key advancements. The rigorous data preparation process ensures data completeness and quality, contributing to the reliability of the subsequent analysis. The implementation of the ResUHybridNet model, a hybrid architecture fusing ResNet and U-Net [30], showcases a thoughtful combination of strengths from both frameworks for more accurate segmentation. The federated learning approach, with multiple hospitals acting as client nodes, not only allows decentralized training but also prioritizes privacy, fairness, and autonomy in handling sensitive medical data. The incorporation of stratified sampling enhances data distribution equity across hospitals, crucial for model generalization. Visualization tools offer transparency and validation of the data integration process. Moreover, the hyperparameter tuning strategy, incorporating Grid Search, Random Search, and Bayesian Optimization, reflects a comprehensive effort to optimize model performance efficiently. This methodology stands out for its holistic approach, addressing technical challenges while upholding ethical principles in the context of medical data and federated learning.
In the realm of medical imaging, particularly brain tumor segmentation, Accuracy is a fundamental metric in evaluating the overall correctness of a segmentation model. In the case of brain tumor segmentation, accuracy measures the proportion of correctly classified pixels (both tumor and non-tumor) in relation to the total number of pixels. The accuracy of ResUHybridNet, standing at an impressive 0.987 as shown in the comparison analysis in Table 1, signifies an exceptionally high level of overall correct predictions [31]. This metric is crucial in medical imaging, where misclassifying pixels can have significant consequences [32]. The high accuracy of ResUHybridNet suggests its proficiency in providing precise and reliable segmentations, a vital characteristic in the context of medical diagnoses and treatment planning. When evaluated against established neural network models, including 3D U-Net, UNETR, and U-Net++, the superiority of ResUHybridNet was evident. The Dice coefficient, also known as the Sørensen-Dice index, is a metric specifically designed for assessing the spatial overlap between the predicted segmentation and the ground truth. In brain tumor segmentation, achieving a high Dice coefficient is crucial for accurately capturing the boundaries of tumors. A Dice coefficient of 0.79 for ResUHybridNet highlights its effectiveness in delineating tumor regions with a high degree of spatial overlap compared to the ground truth as shown in the comparative analysis in Table 1. The Dice coefficient's sensitivity to both false positives and false negatives makes it particularly relevant in medical image segmentation tasks, where precise delineation of tumor boundaries is paramount. The robust performance of ResUHybridNet in this metric emphasizes its ability to provide accurate and consistent segmentation results, essential for clinical decision-making and subsequent interventions.
A deeper examination of the ResUHybridNet reveals its strengths. By melding the resilience and robust feature extraction of Residual Networks (ResNets) with the holistic data capture ability of U-Net, this architecture manages to identify and segment even the most intricate details from brain tumor MRI scans. Such a design ensures both high precision and dependable consistency in the segmentation results.
However, the study did not stop at just creating a high-performing architecture. In the pursuit of refining the training process and addressing real-world data challenges, horizontal federated learning was integrated. When juxtaposing traditional neural network training methods with federated learning, some clear advantages emerged for the latter. Notably, federated learning streamlined the convergence process and reduced overall training time. These advantages are particularly beneficial when considering decentralized datasets or situations where patient data privacy is a priority.
Table 1. Comparison of accuracy and dice scores
|
Model |
Accuracy |
Dice Score |
|
3D U-Net |
0.985 |
0.78 |
|
UNETR |
0.983 |
0.77 |
|
U-Net++ |
0.978 |
0.72 |
|
ResUHybridNet |
0.987 |
0.79 |
8.1 Segmentation and predictions
The presented visualizations offer a meticulous examination between expert-annotated ground truth tumor segmentations and predictions generated by our advanced deep learning architecture. Initially, the Fluid-Attenuated Inversion Recovery (FLAIR) MRI slice is showcased, renowned for capturing hyper-intense regions, instrumental in delineating certain brain lesions. Adjacent to the FLAIR MRI slice is the expert-annotated segmentation, where diverse color overlays signify specific tumor sub-regions, offering granular insight into tumor anatomy and morphology. Following this, a composite representation amalgamates all tumor sub-regions, providing a comprehensive overview of the entire tumor structure and its segmented components.
Subsequent images highlight predictions for specific tumor sub-regions such as NECROTIC/CORE, EDEMA, and ENHANCING as shown in Figure 4. Each prediction demonstrates the capability of our proposed architecture to discern and delineate these intricate sub-regions. Notably, while some predictions closely align with the ground truth, highlighting the model's precision, others signify areas where further refinement might be necessary.The juxtaposition of ground truth against predicted segmentations serves not just as a testament to our model's accuracy but also underscores the complexities and variances inherent in brain tumor segmentation tasks, emphasizing potential avenues for enhancement in future iterations of the model.
8.2 Centralized vs federated training
In an analysis comparing centralized and federated training methods, several observations emerge. The Mean Intersection Over Union (IoU), a standard metric for evaluating model performance, was tracked over 100 rounds as shown below in Figure 5. Throughout the training, both methods displayed a steady increase in the IoU metric. While the centralized training method initially exhibited a marginally superior IoU, federated training eventually caught up around the 40th round, and subsequently, both methods maintained analogous performance trajectories.
When scrutinizing the convergence speed by evaluating loss values over 100 rounds, discernible trends surfaced. Centralized training exhibited a more abrupt decline in loss, indicating an initial faster convergence rate compared to its federated counterpart. This initial disparity underscores the efficiency of centralized training in swiftly adapting to the provided dataset, allowing for rapid adjustments in model parameters
However, as the training rounds progressed, a noteworthy evolution occurred: the loss values for federated training gradually approached and eventually converged with those of centralized training. This observation suggests that while centralized training offers expedited initial convergence benefits, federated training possesses the capability to attain commensurate performance benchmarks with extended training rounds. The diminishing gap in loss values between the two methods by the conclusion of the 100 rounds indicates that the federated approach, despite its initially slower convergence, can effectively adapt and refine its model parameters to achieve comparable performance to the more rapidly converging centralized training. This nuanced analysis emphasizes the dynamic nature of convergence in the context of training methodologies, shedding light on the trade-offs and benefits associated with both centralized and federated training approaches as seen in the convergence speed comparison in Figure 6.
(a) Classification of disease category
(b) Segmentation based on mask
Figure 4. Classification of tumors
Figure 5. Mean IoU comparison
Figure 6. Convergence speed comparison
The inception of ResUHybridNet marks a momentous stride in the domain of neural network designs optimized for medical imaging applications, addressing the challenges of accurate brain tumor segmentation from 3D MRI scans. It emphasizes precision within the complexities of diverse datasets and stringent data privacy standards, effectively tackling the primary problem of balancing accuracy and privacy in brain tumor segmentation. At its core, this network ingeniously melds the resilience and adaptability offered by ResNets with the superior segmentation prowess inherent to the U-Net architecture. This marriage of attributes has set new benchmarks, especially in the precise and critical task of brain tumor identification and mapping.
These results underscore the effectiveness of ResUHybridNet in revolutionizing the field of medical imaging, particularly in brain tumor identification. The fusion of ResNets and U-Net architecture has demonstrated its potential for high-precision segmentation, promising improved patient outcomes.
However, it's essential to acknowledge the limitations. While ResUHybridNet has shown promise in brain imaging, its adaptability and performance in other specialized medical imaging segments, such as cardiac imaging or musculoskeletal studies, remain unexplored. Additionally, further refinements are needed to ensure seamless integration into clinical diagnostic systems.
To unlock the full potential of ResUHybridNet, future work should involve in-depth exploration of its applicability in various medical domains. Collaborative global research partnerships can provide diverse datasets, enhancing the model's versatility and efficacy across different patient demographics and ethnic profiles. Additionally, a crucial aspect of future work involves gathering and incorporating patient feedback to maintain the human touch, comfort, and care quality at the heart of all endeavors. This forward-looking approach promises not only enhanced models but also superior patient care outcomes.
In an era where data privacy and decentralization have emerged as frontline concerns, especially in the sensitive realm of medical data, this strategy offers a promising pathway. It promises enhanced model efficacy without compromising on the crucial tenets of data security and patient confidentiality. The initial achievements with ResUHybridNet, while commendable, represent merely the tip of the iceberg. There exists a vast canvas of opportunities and challenges that await exploration. On the practical front, the real acid test would be to gauge how seamlessly ResUHybridNet dovetails into existing clinical diagnostic systems.
The Federated ResUHybridNet model holds promise for transforming brain tumor diagnosis and treatment planning in healthcare. Its applications extend beyond segmentation, impacting surgical procedures, radiation therapy, and ongoing disease monitoring. The federated learning framework ensures collaborative advancements in medical research without compromising patient data privacy, aligning with evolving healthcare data management practices.
A broader vision would also necessitate collaborative global research partnerships. Such collaborations could pool in diverse datasets, enhancing the model's versatility and ensuring its efficacy across varied demographic and ethnic profiles. Lastly, and perhaps most importantly, the end beneficiary of all these advancements is the patient. It's imperative to gather and incorporate their feedback, ensuring that while we scale the peaks of technological advancements, the human touch, comfort, and care quality remain at the heart of all endeavors. By venturing into these avenues, we don't just enhance a model; we inch closer to synergizing advanced computational capabilities with the ultimate goal of superior patient care outcomes.
The current work on federated learning for brain tumor segmentation in medical imaging shows promise but is not without limitations. Challenges include a potential lack of diversity in datasets, small sample sizes for certain tumor types, limited representation of rare tumors, variability in imaging protocols, ethnic and geographic biases, demands on computational resources and training time, logistical hurdles in implementing federated learning, concerns about model interpretability, and the need for continuous monitoring of long-term stability. Additionally, addressing regulatory and ethical compliance issues is essential. Overcoming these limitations through ongoing research, collaboration, and refinement of the federated learning framework is crucial to ensuring its applicability, robustness, and ethical soundness in real-world healthcare settings.
Future research in brain tumor segmentation should prioritize the integration of multi-modal imaging data, including PET and DTI, to bolster the Federated ResUHybridNet model's robustness. This integration is expected to provide a more comprehensive understanding of tumor characteristics, ultimately improving the model's accuracy and performance. Real-world clinical validation is crucial for assessing the model's effectiveness across diverse patient populations and clinical scenarios, ensuring its practical utility. Additionally, incorporating interpretability techniques, such as attention mechanisms, will enhance the model's transparency, fostering trust among healthcare professionals and contributing to its seamless integration into clinical practice. These advancements aim to elevate the model's performance and reliability in the dynamic landscape of medical imaging [33].
The authors would like to express their profound appreciation and gratitude to the Vellore Institute of Technology for their unwavering support and guidance. The institution's dedication to fostering research and innovation played a pivotal role in the realization of this work. The mentorship provided by the faculty and the conducive environment for scholarly discussions have been instrumental in shaping the direction and outcomes of this research. We are deeply thankful for the opportunity to collaborate and thrive within such a vibrant academic community.
|
Acronyms and abbreviations |
||
|
DeepMedic |
A 3D CNN approach introduced by Kamnitsas et al. |
|
|
U-Net |
A deep learning architecture for biomedical image segmentation |
|
|
FL |
Federated Learning |
|
|
CNN |
Convolution Neural Networks |
|
|
MRI |
Magnetic Resonance Imaging |
|
|
HGG |
High-Grade Giloma |
|
|
LGG |
Low-Grade Giloma |
|
|
FCNN |
Fully Convolutional Neural Networks |
|
|
CRF |
Conditional Random Field |
|
|
RNN |
Reccurent Neural Networks |
|
|
BRATS |
Brain Tumor Segmentaion Benchmark |
|
|
DNN |
Deep Neural Network |
|
|
CT |
Computed Tomography |
|
|
ELM |
Extreme Learning Machine |
|
|
FLED |
Federated Learning Enhanced Design |
|
|
IoU |
Intersection Over Union |
|
|
Subscripts |
||
|
2D |
Two Dimensional |
|
|
3D |
Three Dimensional |
|
|
MR |
Magnetic Resonance |
|
|
Symbols |
||
|
% |
Percentage |
|
[1] Beam, A.L., Kohane, I.S. (2018). Big data and machine learning in health care. JAMA, 319(13): 1317-1318. https://doi.org/10.1001/jama.2017.18391
[2] Obermeyer, Z., Emanuel, E.J. (2016). Predicting the future—big data, machine learning, and clinical medicine. The New England Journal of Medicine, 375(13): 1216-1219. https://doi.org/10.1056/NEJMp1606181.
[3] Davenport, T., Kalakota, R. (2019). The potential for artificial intelligence in healthcare. Future Healthcare Journal, 6(2): 94-98. https://doi.org/10.7861/fhj.2019-0004
[4] Liu, Y., Chen, P.H.C., Krause, J., Peng, L. (2019). How to read articles that use machine learning: Users’ guides to the medical literature. JAMA, 322(18): 1806-1816. https://doi.org/10.1001/jama.2019.16489
[5] Menze, B.H., Jakab, A., Bauer, S., et al. (2015). The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging, 34(10): 1993-2024. https://doi.org/10.1109/TMI.2014.2377694
[6] Chen, T.J., Son, Y.J., Park, A., Baek, S.J. (2022). Baseline correction using a deep-learning model combining ResNet and UNet. Analyst, 19(2022). http://doi.org/10.1039/D2AN00868H
[7] Topol, E.J. (2019). High-performance medicine: The convergence of human and artificial intelligence. Nature Medicine, 25(1): 44-56. https://doi.org/10.1038/s41591-018-0300-7
[8] Mo, Y., Wu, Y., Yang, X., Liu, F., Liao, Y., 2022. Review the state-of-the-art technologies of semantic segmentation based on deep learning. Neurocomputing, 493: 626-646. https://doi.org/10.1016/j.neucom.2022.01.005
[9] McKinney, S. M., Sieniek, M., Godbole, V., et al. (2020). International evaluation of an AI system for breast cancer screening. Nature, 577(7788): 89-94. https://doi.org/10.1038/s41586-019-1799-6
[10] Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.M., Larochelle, H. (2017). Brain tumor segmentation with deep neural networks. Medical image analysis, 35: 18-31. https://doi.org/10.1016/j.media.2016.05.004
[11] Wadhwa, A., Bhardwaj, A., Vivek Singh Verma, V.S. (2024). A review on brain tumor segmentation of MRI images. Magnetic Resonance Imaging, 61: 247-259, https://doi.org/10.1016/j.mri.2019.05.043.
[12] Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639): 115-118. https://doi.org/10.1038/nature21056
[13] Long, J., Shelhamer, E., Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, pp. 3431-3440. https://doi.org/10.1109/CVPR.2015.7298965
[14] Saha, A., Zhang, Y.D., Satapathy, S.C. (2021). Brain tumor segmentation with a multi-pathway ResNet based UNet. Journal of Grid Computing, 19, pp.1-10.
[15] Zikic, D., Ioannou, Y., Brown, M., Criminisi, A. (2012). Segmentation of brain tumor tissues with convolutional neural networks. In Proceedings - MICCAI Workshop on Multimodal Brain Tumor Segmentation, pp. 36-39.
[16] Pereira, S., Pinto, A., Alves, V., Silva, C.A. (2016). Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging, 35(5): 12401251. https://doi.org/10.1109/TMI.2016.2538465
[17] Kamnitsas, K., Ledig, C., Newcombe, V.F.J., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B. (2017). Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis, 36: 61-78. https://doi.org/10.1016/j.media.2016.10.004
[18] Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., Maier-Hein, K.H. (2019). No New-Net. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2018. Lecture Notes in Computer Science(). https://doi.org/10.1007/978-3-030-11726-9_21
[19] Sheller, M.J., Reina, G.A., Edwards, B., Martin, J., Bakas, S. (2019). Multi-institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2018. Lecture Notes in Computer Science(), vol 11383. Springer, Cham. https://doi.org/10.1007/978-3-030-11723-8_9
[20] Li, W., Wang, G., Fidon, L., Ourselin, S., Cardoso, M.J., Vercauteren, T. (2017). On the compactness, efficiency, and representation of 3D convolutional networks: Brain parcellation as a pretext task. In: Niethammer, M., et al. Information Processing in Medical Imaging. IPMI 2017. Lecture Notes in Computer Science(), vol 10265. Springer, Cham. https://doi.org/10.1007/978-3-319-59050-9_28
[21] Dou, Q., Chen, H., Jin, Y., Yu, L., Qin, J., Heng, P.A. (2020). Multi-level contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Transactions on Biomedical Engineering, 64(7): 1558-1567. https://doi.org/10.1109/TBME.2016.2613502
[22] Guo, P.F., et al. (2022). Auto-fedrl: Federated hyperparameter optimization for multi-institutional medical image segmentation. In European Conference on Computer Vision, pp. 437-455.
[23] KhoKhar, F.A., Shah, J.H., Khan, M.A., Sharif, M., Tariq, U., Kadry, S. (2022). A review on federated learning towards image processing. Computers and Electrical Engineering, 99: 107818. https://doi.org/10.1016/j.compeleceng.2022.107818
[24] Ambesange, S., Annappa, B., Koolagudi, S.G. (2023). Simulating federated transfer learning for lung segmentation using modified UNet model. Procedia Computer Science, 218: 1485-1496. https://doi.org/10.1016/j.procs.2023.01.127
[25] Swarnendu, G., Das, N., Das, I., Maulik, U. (2019). Understanding deep learning techniques for image segmentation. ACM computing surveys (CSUR), 52(4): 1-35. https://doi.org/10.1145/3329784
[26] McMahan, H.B., Moore, E., Ramage, D., y Arcas, B.A. (2016). Federated learning of deep networks using model averaging. arXiv preprint arXiv:1602.05629. https://doi.org/10.48550/arXiv.1602.05629
[27] Ronneberger, O., Fischer, P., Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234-241. https://doi.org/10.1007/978-3-319-24574-4_28
[28] Zhao, Y., Wang, J., Liu, Y., He, X. (2020). Federated learning with non-IID data. arXiv preprint arXiv:2006.00582. https://doi.org/10.48550/arXiv.1806.00582
[29] He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp. 770-778. https://doi.org/10.1109/CVPR.2016.90
[30] Zhang, Z., Jung, C., Makeig, S., Rao, B.D. (2017). Computationally efficient convolutive non-negative matrix factorization for the representation of large non-stationary signals. IEEE Transactions on Signal Processing, 65(21): 5757-5770.
[31] Konečný, J., McMahan, B., Yu, F.X., Richtárik, P., Suresh, A.T., Bacon, D. (2016). Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492. https://doi.org/10.48550/arXiv.1610.05492
[32] Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A.W.M., van Ginneken, B., Sánchez, C.I. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42: 60-88. https://doi.org/10.1016/j.media.2017.07.005
[33] Bergstra, J., Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13: 281-305.
