Laila Baloch
| Ahthasham Sajid | Christine Dewi | Henoch Juli Christanto* | Afia Zafar
© 2024 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
Medical imaging is essential in contemporary healthcare as it assists in the identification of diseases, development of treatment strategies, and ongoing monitoring of patients. Over the years, deep learning (DL)techniques have emerged as a transformative force in medical image reconstruction, enabling the generation of high-quality images from noisy, incomplete, or under-sampled data. This review paper provides a comprehensive survey of recent advancements and applications of deep learning methods in medical image reconstruction. The key challenges in medical image reconstruction include issues related to reconstruction accuracy, noise sensitivity, and data limitation. A variety of deep learning models and their combinations are suitable for medical image reconstruction due to their unique capabilities, such as spatial hierarchy capture, adversarial learning, and other features, which allow them to address the complexities and challenges associated with medical image reconstruction. The paper analyses the key contributions of DL-based approaches in different imaging modalities, including computed tomography (CT) and magnetic resonance imaging (MRI). DL techniques are enabling image reconstruction in specialized medical fields like neuroimaging and cardiac imaging, but practitioners face challenges in training complex models and understanding their results. Finally, future research directions are suggested to improve the key limitations highlighted in this survey study.
magnetic resonance, image reconstruction, imaging (MRI), Deep learning (DL), CNN, generative adversarial networks (GANs)
In the field of healthcare, Artificial Intelligence (AI) is offering significant advantages by enhancing productivity and efficiency through the automation of routine tasks. Moreover, it aids medical specialists in rapidly generating insights for early disease diagnosis, leading to valuable benefits. Medical image reconstruction plays an essential role in medical imaging because it aims to create high-quality and detailed images from raw data received by imaging systems like computed tomography (CT) and magnetic resonance imaging (MRI). This process is essential for precise diagnosis, treatment planning, and disease monitoring [1]. Several medical conditions require the use of high-quality reconstructed images for effective diagnosis, treatment planning, and monitoring. Advanced image reconstruction techniques using deep learning models have shown promising results in enhancing medical images. These models learn complex patterns and structures from large datasets, leading to better image reconstruction and clearer visuals for medical professionals. A variety of imaging modalities are available for the diagnosis and identification of diseases, including X-ray, CT, MRI, ultrasound, and positron emission tomography (PET). MRI scans are widely utilized in the clinical setting due to their ability to provide accurate disease diagnosis without exposing patients to harmful radiation. Despite their numerous benefits, MRI scans can still suffer from limitations that impact their performance such as sensitivity to various artifacts and noise, which can degrade image quality. To address these limitations, AI-based image reconstruction techniques are being integrated into MRI scans for denoising and artifact correction, enhancing the overall image quality and diagnostic accuracy [2]. The integration of AI holds the potential to enhance the accuracy, speed, and efficiency of MRI reconstruction. In CT and X-ray imaging, patients are exposed to ionizing radiation. Image reconstruction algorithms aim to achieve high image quality with the lowest possible radiation dose, ensuring patient safety without compromising diagnostic information [3].
1.1 Deep Learning-based Reconstruction
Deep learning algorithms can be used to reconstruct images from sparse and incomplete data. For example, CNNs can be trained on large datasets of MRI or CT scan images to learn the underlying patterns in the data, and then use this knowledge to reconstruct high-quality images from under-sampled data [1].
The main features of AI-based reconstruction models are described below:
Deep learning-based image reconstruction can enhance the speed and accuracy of medical imaging procedures, which is important in time-sensitive medical applications like emergency care. The requirement for huge volumes of training data, the computational intensity, and the possibility for overfitting are all challenges and limits of deep learning-based image reconstruction in medical applications. Despite these obstacles, deep learning-based image reconstruction has shown great promise in medical applications and is expected to play an important role in improving the accuracy and speed of medical imaging techniques in the future.
The majority of the research is focused on image pre-processing, classification, and exploring various applications of Convolutional Neural Network (CNN) techniques in medical imaging. Few studies reviewed the image reconstruction domain; therefore, we conducted a survey, analyzing various studies to present an updated and comprehensive synthesis of the latest advancements in the field. This review examined relevant publications published between 2018 and 2023. Moreover, it examines how deep learning techniques such as CNN, and adversarial networks are suitable for image reconstruction due to their unique capabilities, including spatial hierarchy capture, and adversarial learning, which help address the complexities associated with medical image reconstruction. Additionally, it highlights how DL have facilitated image reconstruction in specialized medical fields, including neuroimaging and cardiac imaging. The training of complex models can be challenging due to the limited availability of large, labeled datasets. Additionally, doctors are concerned about the difficulty of understanding deep learning results, as they require clear and understandable outcomes to make informed decisions. Overall, this study analyzed current advancements in this area and the associated major challenges. Additionally, the integration of DL-based approaches in medical images specifically MRI scans and CT has the potential to enhance the overall image quality and diagnostic accuracy by addressing limitations such as sensitivity to artifacts and noise. However, the lack of diverse and comprehensive datasets poses a significant challenge in training models that can successfully recreate images from unseen data, leading to overfitting issues. Recent studies in medical image reconstruction focus on enhancing computational efficiency through efficient architecture design, utilizing model compression techniques, and emphasizing integration with hardware accelerators like GPUs and TPUs, ultimately facilitating real-time processing in clinical applications.
This article aims to investigate the following objectives.
2.1 Overview of deep learning (DL)
Deep learning (DL), a subfield of machine learning (ML), is driven by the data processing mechanisms observed in the human brain. Deep learning does not rely on predetermined human criteria to function; instead, it leverages a vast amount of data to associate the provided input with specified labels. DL is planned to utilize various layers of algorithms (artificial neural organizations, or ANNs), every one of which gives a different understanding of the information that has been fed to them [3].
Figure 1. Difference between DL and ML
Traditional ML techniques require a series of steps to achieve a single classification task. The steps include pre-processing, feature extraction, wise feature selection, learning, and classification. However, the feature selection might degrade the performance of ML methods and prompt the wrong separation between classes. To overcome this issue, DL can mechanize the learning of features for a considerable length of time, dissimilar to traditional ML strategies. DL empowers learning also classification to be accomplished in a solitary shot. Figure 1 illustrates the difference between DL and ML ability, Robustness, Scalability, and Generalization of DL. DL has turned into an amazingly famous sort of ML calculation lately because of the immense development and advancement in the field of big data [4]. It is as yet in persistent advancement in regard to novel execution for several ML assignments and has simplified the improvement of numerous learning fields, such as image reconstruction, image segmentation, object detection, and image super-resolution [5]. Moreover, the performance of DL is far much better than that of human beings in carrying out different tasks such as image classification, medical decisions, speech recognition, price prediction, stock preference, weather prediction, tracking, and biometrics. During the imaging process, various factors such as electronic interference, motion, and inherent system noise can introduce undesirable distortions into the images. Image denoising techniques aim to enhance the quality of MRI images by effectively removing or minimizing these unwanted elements, thereby improving the accuracy and clarity of the diagnostic information. DL algorithms can be used to reduce the amount of noise in images, resulting in clearer and more accurate images, due to the adaptable models, approaches, and frameworks used to carry out image denoising tasks. So the next part of this section defines the approaches, models, and frameworks for image denoising. CNNs have gradually improved their performance in AI vision tasks over the years, surpassing human vision with a 5% error rate in Figure 2 below.
Figure 2. Performance of DL Models error rate
2.2 Frameworks for deep learning
Deep learning frameworks provide a set of tools for testing, training, and building deep neural networks using a high-level programming interface. Broadly utilized DL structures, like PyTorch, TensorFlow, Theano, Caffe, Chainer, and Keras refer to Figure 3 [6]. This section explained the frameworks used for deep learning and how they are used in various applications.
2.2.1 Tensorflow
Tensorflow was developed by Google’s Brain, it utilizes data flow graphs to process the data. Its major strength lies in its expansive high-order machine learning library that supports Python, C++, and R. TensorFlow excels in tasks such as denoising, classification, and object detection. One notable advantage is its extensive community support and integration capabilities. However, some users find its learning curve steep.
2.2.2 Theano
Theano was developed by University de Montreal, it supports a Python interface, which allows the users to optimize, evaluate and define mathematical expressions for dealing with neural networks. While versatile in applications like denoising and classification, Theano's main advantage lies in its ability to handle complex mathematical expressions effectively. However, the disadvantage of Theano was that it is slower than TensorFlow.
Figure 3. Frameworks for DL
2.2.3 Keras
Keras was developed by Francois Chollet, basically, It was written in Python language, which supports high-level neural network API. Keras run on top of Tensorflow and Theano. Its advantage lies in its simplicity and ease of use, making it an ideal choice for quick prototyping. Keras is known for its user-friendly design, facilitating rapid model development. However, its abstraction may limit fine-grained control in certain scenarios.
2.2.4 PyTorch
PyTorch was primarily built by Facebook’s AI Research lab (FAIR). Its advantages include seamless integration with Python, CUDA, and C/C++ libraries. PyTorch is favored for tasks ranging from image denoising to action recognition. Its disadvantage includes a slightly smaller community compared to TensorFlow.
2.2.5 Caffe
Yangqing Jia is the creator of Caffe, which stands for Convolutional Architecture for Fast Feature Embedding. The simplicity of the model makes it a superb option for jobs involving picture detection and classification. Nevertheless, Caffe can be seen as less adaptable for specific intricate network structures. Due to its ability to utilize both CPU and GPU, it is highly regarded for its rapidity and effectiveness.
2.2.6 Chainer
Chainer was created by Preferred Networks (IBM, Intel, Microsoft, and Nvidia) and written in Python. Chainer's unique feature is its dynamic computation graph, allowing for intuitive model design. It executes on top of the Numpy and CuPy Python libraries. However, it may not be as optimized for large-scale deployments as some other frameworks.
2.2.7 Apache MXNet
The deep learning framework called Apache MXNet focusees on efficiency and adaptability, allowing users to seamlessly combine imperative and symbolic programming. Its advantage lies in efficient memory utilization and scalable deployment options.
2.2.8 Deeplearning4j
Deeplearning4j is a comprehensive framework that intends to make deep learning application development on the Java Virtual Machine (JVM) easier. Its advantages include multiple components, including DL4J for generating MultiLayerNetworks and ComputationGraphs, ND4J for linear algebra operations, and Python4J for Python integration. However, its ecosystem may be less mature compared to Python-based frameworks.
2.2.9 CNTK
Microsoft Cognitive Toolkit (CNTK) is a free and open-source deep learning platform for building, training, and analysing neural networks. It supports a wide range of model types, such as feed-forward DNNs, CNNs, and RNNs/LSTMs. Its advantage lies in effective training, employs automatic differentiation and parallelization over several GPUs and servers. However, CNTK may be considered less user-friendly for beginners compared to some other frameworks.
2.2.10 Torch
Torch is an open-source ML library based in the Lua scripting language use for the creation of deep neural networks. It is one of the most popular platforms for deep learning research. Its advantage includes a dynamic computational graph and LuaJIT scripting language. Torch is favored for its flexibility, but its adoption may be limited due to the scripting language choice.
A deep learning framework may provide a wide variety of features; the choice depends on the researcher's specific needs and priorities. Factors such as flexibility, community support, ease of use, performance, and compatibility with existing tools and environments all play a crucial role in determining the most suitable framework for a particular research project. New frameworks could appear as the area of deep learning develops, and as per the need old ones might get updated and improved to increase their functionality.
To support medical diagnoses and propose treatments, a number of medical imaging methods, including radiography, MRI, PET, and CT scans, are often utilized in clinical settings. These techniques provide pictures of biological, anatomical, and physiological features. With the use of these technologies, doctors can diagnose and treat patients more accurately, almost eliminating the chance of a false positive. Similar to this, AI models are revolutionizing the medical field by helping patients as well as physicians. In academic research settings as well as clinical settings, it has the capacity to tackle difficult medical issues [7]. AI significantly aids in image reconstruction by utilizing the power of deep learning models to address various challenges such as low resolution, noise, missing information, and artifacts. The application of AI in image reconstruction spans diverse domains, including medical imaging, and real-time processing, contributing to the generation of visually compelling and informative images.
The MRI and CT scan are the most commonly used imaging techniques in medicine. The MRI scan has established itself as a reliable clinical tool due to its capacity to deliver precise illness diagnoses without subjecting patients to dangerous radiation. However, it may experience various problems that affect its performance. Many approaches have been used to improve the system's functionality and give patients comfort in order to meet these constraints. The fact that MRI scans take a long time to finish is one of the most difficult issues with them. The two most advanced techniques for speeding up the MRI scan are called CS and PI, however they also have certain drawbacks. As a result, it continues to confront difficulties caused by lengthy iterations and a slow rate of acceleration.
Conversely, deep learning (DL) has experienced a significant increase in its use in medical imaging. CNN-based DL models have been the basis for AI solutions that rely on imaging. These solutions that are led by deep learning aim to provide help for clinical decision-making and offer numerous benefits. Firstly, CNN-based DL models can reduce the number of parameters required while maintaining model quality. Secondly, they do not require manual feature engineering as they can automatically extract features from images. Additionally, the literature supports the use of CNN- based DL for image classification and recognition with high accuracy [2].
In recent times, various imaging modalities, including MRI, PET, and CT scan, have effectively integrated DL algorithms for image reconstruction. This has not only resulted in enhanced image quality but has also enabled real-time imaging. Medical imaging requires real-time imaging to prevent protracted delays and provide a greater rate of acceleration for dynamic functions. DL has also excelled in low-rank computer vision models, image classification, image segmentation, and denoising [8]. Furthermore, DL enhanced the performance of undersampled MR image reconstruction by a substantial margin and demonstrated tremendous potential for further improvement. DCNNs reconstruct images from compressed-sensing MRI and PI using distinct methods. The deep learning networks undergo training using sets of MRI k-space data that have been subsampled and entirely sampled, consisting of original and estimated images.
Additionally, inverse image processing limitations such as accelerated MRI reconstruction have been successfully addressed using DL techniques in the last few years. Multiple strategies of Deep Convolutional Neural Networks (DCNNs) are incorporated with CS and PI to restore MRI and mitigate the challenges associated with them. These algorithms are the state-of-the-art to make accessible more MRI data and achieve performance more efficiently [9]. However, few studies have applied local interpretable model-agnostic explanation (LIME) and Shapley additive explanation (SHAP) techniques along with CNN to enhance interpretability. Deep learning models have already demonstrated the capacity to alleviate the workload of medical professionals, enhance patient care, and decrease expenses. For instance, deep learning algorithms such as Resnet50 have been utilized to differentiate COVID-19 from X-rays in order to achieve a quicker diagnosis with a 97% accuracy rate. DL has been employed in several research to perform tasks such as identifying brain tumors in CT scans and analyzing MRI data to predict subtypes. However, a major challenge in the implementation of DL-based solutions in medical settings is the lack of interpretability and transparency of algorithm-driven decisions. Despite numerous efforts to develop explainable AI- based Clinical Decision Support Systems (CDSS), there remains a lack of a universal notion of explainability and excluded material [10].
MRI reconstruction based on deep learning is divided mainly into two categories: data- driven and model-driven. The data-driven approach uses the redundant information in the original input to learn the mapping relationship from input to output, enabling flexible and adaptive reconstruction processes, robustness in diverse imaging scenarios, and the potential to learn from large datasets. However, data-driven methods may necessitate substantial training data, exhibit limited interpretability, and face computational intensity, impacting real-time applicability. The model-driven approach, on the other hand, uses a pre-defined model to solve the reconstruction problem. This approach offers potentially more interpretable outcomes, suitability for scenarios with limited data, and computational efficiency, particularly in resource-constrained settings. Nevertheless, model-driven approaches may struggle to adapt to complex mapping relationships and introduce biases or assumptions that could affect reconstruction accuracy and generalizability. However, DL models necessitate a large number of datasets and extensive training. It can be time-consuming, and obtaining fully sampled data can be challenging due to limitations such as organ motion or signal decay in the physiological or physical systems. Some studies attempt to overcome the data scarcity problem by employing transfer learning, though a limited amount of fully sampled data is still needed for fine-tuning the pre-trained network [11].
On the other hand, to deal with the problem of smaller datasets in deep learning, several strategies have been explored, including data augmentation and synthetic data generation. These methods rely on manipulating data artificially to expand the dataset's size. Patch- based training can also help with the problem of tiny datasets by dividing each piece of data into smaller patches. This substantially increases the sample size in the dataset without altering it artificially. Similarly, Transfer learning approaches, which involve re-proposing or altering a pre-trained model with fine-tuning, can be used to tackle the problem of insufficient training datasets [7]. The network learns the weights of one task, which can be used in another as pre-trained weights, and the network can then be trained or fine-tuned for the new task. Furthermore, the choice of a loss function is critical when training a DL model to determine the reconstruction error between the model's prediction and the matching ground-truth images [9].
In general, image reconstruction is an inverse problem that facilitates the conversion of a mediocre copy back into the original ideal image. Image reconstruction is the process of introducing two-dimensional images into a computer and then modifying them to a more functional and constructive form for the human observer's exploration or refinement. Iterative reconstruction and analytical reconstruction are the two fundamental categories of reconstruction techniques (IR).
Iterative Reconstruction (IR). There are essentially two methods for restoring images. iterative reconstruction and analytical reconstruction. In the iterative method, the restoration question is limited to calculating a finite number of image values from a finite number of measurements.
Although iterative reversal looks to need more processing resources, the procurement process is capable of dealing with increasingly complicated models [12].
3.1 Algebraic Reconstruction Technique (ART)
ART is a well-known iterative technique for medical image reconstruction that solves equations in a linear system.
$P_1=W_{31} f_1+W_{31} f_1+W_{31} f_1 \ldots W_{31} f_1$ (1)
Eq. (1) deals with multilevel thresholding. Eq. (3) describes how ART determines the value of each pixel at each j-position by updating the projection values from Eq. (2) and applying a correction factor.
$P_1=\sum_{I=1}^N W_I l \mathrm{f}_i$ (2)
$f_j^{k=1}=f_j^k+\frac{P_i-\sum_{i=1}^N f_l^k W_i l}{\sum_{i=1}^N W_{i l}^2}$ (3)
where, f j stands for a pixel value at the j-position, wij for the jth pixel weighted ratio that the ith ray passes around, and k for the number of repetitions. Pi stands for the measured affect projection information.
3.2 Characteristics of AI in medical images
AI use in medical imaging is not restricted to disease identification; it also has the potential to improve other elements of medical imaging. AI algorithms, for instance, may be utilized to streamline the imaging procedure, leading to better picture quality, quicker scans, and less radiation exposure for patients. AI algorithms may also be used to automatically segment photos and highlight particular elements, which makes it simpler for medical professionals to recognize and analyse essential information. The application of AI in medical imaging can also lessen the workload on the medical staff, allowing them to spend more time with patients and come to more educated conclusions. In the end, this may lead to better patient outcomes and lower healthcare expenses [4].
However, using AI in medical imaging is not without its challenges. Verifying the accuracy of AI algorithms is one of the most challenging tasks since poor diagnosis can have disastrous consequences for patients. Furthermore, it's important to address ethical and legal issues, such as data privacy and the possibility that AI systems may discriminate against certain groups of people. A summary of key features of recently implemented deep learning models is found in Table 1. The table describes the literature on these features.
4.1 Peak Signal-to-Noise Ratio (PSNR)
Peak Signal-to-Noise Ratio (PSNR) is a common measure for evaluating the quality of medical image reconstructions. It determines the maximum potential power of a signal to the power of noise, which influences the effectiveness of the reconstructed image [13]. The performance of different algorithms and methodologies has been evaluated in the context of medical image reconstruction using PSNR. PSNR is favored in medical image reconstruction due to its sensitivity to subtle changes in pixel values, making it suitable for assessing the fidelity and effectiveness of reconstruction techniques crucial in medical diagnosis. For instance, an image reconstruction approach based on ℓ0 gradient reduction was developed in the field of limited angle computed tomography (CT) to reduce artefacts and improve image quality. This algorithm's performance was compared to that of other approaches using the PSNR, highlighting its benefits in terms of image quality [13].
PSNR has also been used to evaluate the performance of various image reconstruction procedures for medical imaging modalities such as MRI and CT. For instance, the software-equivalent PSNR values for MRI and CT images in a study on energy-efficient high-fidelity image reconstruct ion with memristor arrays for medical diagnosis were 40.21 dB and 22.38 dB, respectively. The reconstructed pictures were verified to match the criteria for medical diagnosis using the PSNR values [14].
4.2 Structural similarity index (SSIM)
The structural similarity measure (SSIM) was used to assess the quality of images rebuilt using deep learning-based image reconstruction in limited-angle computed tomography (CT) [15]. The SSIM calculates the similarity between the reconstructed picture and the original image by taking brightness, contrast, and structural modifications into consideration. [16]. It is preferred when assessing structural changes in images. These factors making it suitable for evaluating the perceptual quality and overall similarity. It provides a more perceptually meaningful metric than PSNR.
The findings showed that the deep learning-based reconstruction approach outperformed standard reconstruction methods in terms of SSIM values, indicating improved image quality [15].
Table 1. Covers the literature of key features of deep learning models
|
Sr.no |
Reference |
Applied DL Model |
Contribution |
Advantages |
Disadvantages |
|
1. |
[17] |
Deep residual-constrained reconstruction (DRCR) framework for LDCT imaging using learnt convolutional sparse coding (LCSC). |
To further confine the image update process, the LCSC network is used for feature map and filter updating. |
Low dose computed tomography (LDCT) can greatly reduce radiation dose damage. |
LDCT decreases projection signal-to-noise ratio (SNR) and degrades reconstruction image quality. |
|
2. |
[18] |
The low-rank tensor aided k-space generative model (LR-KGM) is a parallel image reconstruction model. |
For learning, convert low-rank information into high-dimensional previous knowledge. The multi-channel data was particularly constructed into a big Hankel matrix. |
Reduce the amount of training samples, which are then compacted into a tensor for the preceding learning step. |
When the singular value thresholding is set to a high value, the reconstruction result will not have a substantial local error. |
|
3. |
[19] |
Focuses on a Deep Learning (DL) model that has been optimised to increase picture quality and reconstruction from undersampled images. |
To improve the reconstructed picture quality, the model is combined with the self-attention, normalisation, and data consistency layers. |
With better reconstruction images, a model with data consistency achieves a reasonable solution. |
The data samples are insufficient to improve the model's performance. |
|
4. |
[20] |
High- Resolution MR Image Restoration |
Deep learning for quicker MRI using sub-Nyquist sampling methods to reduce k-space data. |
High-resolution MR images were obtained from under-sampled k-space data. Uniform subsampling was used in the phase-encoding direction, and a tiny amount of low-frequency k-space data was added. |
Only 29 of the k- space data can generate high- quality images as effectively as standard MRI reconstruction with fully sampled data |
|
5. |
[21] |
Transfer learning for MR image reconstruction |
An approach using transfer learning to supplement scarce training data in MR image reconstruction. |
The results show that transfer learning may be used in MRI reconstruction. Can overcome the issue of low training data. |
Theoretical analysis is still needed to explain the underlying mechanisms. |
|
6. |
[22] |
Nonlinear Feature Extraction |
A model for extracting nonlinear characteristics from visual pictures and collecting correlations between fMRI voxel activity. |
Robust in capturing correlations, useful for fMRI recordings |
Need to incorporate RNN to explore dynamic vision reconstruction. |
|
7. |
[23] |
End-to-end reconstruction model for fMRI |
An end-to-end model for reconstructing fMRI data. |
Extracts nonlinear features from fMRI recordings Robust in capturing correlations among voxel activities. |
Does not decode all of the visual information that the brain can decode. |
|
8. |
[24] |
AUTOMAP |
A deep learning framework for MR image reconstruction used automated transform by manifold approximation. |
Accurate compared to conventional methods Recasts image reconstruction as a supervised learning task Ability to learn reconstruction transforms for various MRI. |
Computationally intensive. |
|
9. |
[25] |
Faster MRI Reconstruction with Sub- Nyquist Sampling |
Reduces k-space data for faster MRI reconstructions. |
Improved speed compared to conventional methods. |
The algorithm requires theoretical study to describe it, and it is quite difficult. |
|
10. |
[4] |
Under sampled Dynamic Cardiac MRI |
A reconstruction method for MRI data |
2x quicker than compressed sensing methods |
Not applicable to parallel imaging. |
|
11. |
[26] |
DNN model for image reconstruction |
Deep neural networks (DNN) are used to rebuild images from subsampled MRI data. |
Image denoising and super-resolution are possible applications. It makes use of some picture attributes. |
Does not explicitly exploit all image properties.
|
|
12. |
[8] |
Deep learning model for fMRI reconstruction |
A deep learning model for recreating perceived stimuli from fMRI brain responses |
Suitable for the development of new neuroprosthetic devices |
Not all aspects of brain responses are explicitly exploited |
|
13. |
[22] |
End-to-end framework for super- resolution MR reconstruction |
Reconstructed pictures of high quality from noisy, low-resolution clinical MRI scans An end-to-end modal |
Produces high-quality images Efficient and straightforward. Can handle low-resolution. clinical scans |
May not explicitly exploit all image properties. |
|
14. |
[27] |
Pre-trained DNN for Image Reconstruction |
A nonlinear feature extraction model for visual imagery |
Capable of capturing correlations among voxel activities of fMRI recordings |
Inadequate in exploring the reconstruction of dynamic vision |
|
15. |
[28] |
Variational Model using DL |
A model that integrates the mathematics of variational models with deep learning. |
Outperforms standard reconstruction algorithms. |
Various choices of error metrics still need to be studied. |
|
16. |
[29] |
Denoising and Data Consistency Enforcement |
A method for denoising and ensuring data consistency during picture reconstruction. |
Due to the decrease of trainable parameters, it does not require a large amount of training data. |
Different error metrics still need to be investigated. |
|
17. |
[30] |
MR Image Reconstruction using DL Modal |
Assigning lower weights to noisy training images in the weighted loss function |
Outperforms standard reconstruction algorithms. Improved image quality of the reconstruction. |
Quality of reconstruction may still depend on the presence of noise. |
|
18. |
[31] |
KIKI-net |
Proposed cross-domain CNNs, outperforms conventional algorithms with K-net and I-net. |
Superior to single-domain CNNs with tissue-structure restoration and removal of aliasing artifacts. |
Only applicable up to a reduction factor of 3 to 4 based on variable- density Cartesian under sampling. |
|
19 |
[32] |
Present a novel picture reconstruction approach from dispersed data based on multigrid relaxation of the Poisson equation and convolutional neural networks (CNN). |
Develop the image's reconstruction issue as a Poisson equation with irregular boundary conditions, then present a fast multigrid approach for solving it, and lastly improve the rebuilt image. |
The use of multigrid and CNN algorithms ensures that the output picture resolution has no effect on reconstruction speed. |
Unable to use the neural network on generic photos. |
|
20. |
[33] |
Provides a hardware technique for implementing image reconstruction tasks on disparate hardware platforms. |
The processor system (ARM) and the programmable logics are inextricably linked. |
Improve overall system performance and resource utilization. When using iterative and non-iterative methods, the suggested heterogeneous hardware architecture reach picture reconstruction speeds of 24 and 1700 frames per second, respectively. |
Multiple types of hardware components can increase the cost of the system. synchronization overhead leads to inefficiencies and performance bottlenecks. |
4.3 Mean Squared Error (MSE)
Mean Squared Error (MSE) is a common metric used to quantify the quality of image reconstruction or any type of regression task. This metric gauges the mean of the squared disparities between anticipated values and real values. Within the sphere of medical image reconstruction, MSE holds the potential to gauge the fidelity of the reconstructed image against the original. A diminished MSE value denotes superior reconstruction, implying proximity between pixel values in the reconstructed and original images. Conversely, an elevated MSE value signifies amplified disparity between the reconstructed and source images. A lower MSE value indicates better alignment between predicted and ground truth pixel values, signifying superior reconstruction quality [34]. A study [35] highlight the need for a nuanced approach, coupling MSE with advanced deep learning techniques to address the unique challenges posed by medical images.
While MSE is a useful metric, it can struggle to identify perceptual distinctions that are important in medical imaging, such as minor variations in textures, edges, or forms. In certain circumstances, more complex metrics or even perceptual models that take into account human perception may offer a more thorough assessment of image quality. Metrics such as the Perceptual Index (PI) consider factors like color, texture, and spatial structure, aligning with human-centric evaluations [36]. Advanced models like Generative Adversarial Networks (GANs) create not only quantitatively similar but also perceptually realistic images, making them valuable in tasks requiring high-quality and visually convincing results and metrics such as Fréchet Inception Distance (FID) and Perceptual Loss leverage sophisticated frameworks to ensure not only quantitative similarity but also perceptual realism [37]. FID provides a comprehensive assessment, considering both visual quality and diversity, essential in tasks where the variety of generated images is crucial [38]. Additionally, metrics based on natural image statistics ensure that reconstructed images not only resemble real images but also capture the statistical regularities observed in the natural environment.
These metrics and models are selected based on the specific goals of the image reconstruction task, emphasizing human perception, realism, and the inherent properties of real-world scenes. The utilization of more complex metrics and perceptual models, such as GANs and FID is increasingly important in medical image reconstruction to address the limitations of traditional metrics and effectively capture the intricate details present in medical images.
This section presents a key open research challenges as future work directions in the field of medical image reconstruction.
5.1 Data availability and reconstruction quality
Data availability and quality pose significant challenges in medical image reconstruction. The limited availability of labeled and high-quality medical image datasets hinders the development and evaluation of reconstruction algorithms [39, 40]. Large-scale labelled dataset acquisition is costly and time-consuming, making it challenging to efficiently train deep learning models [40]. Additionally, due to the low frequency of abnormal results, medical imaging datasets frequently exhibit class imbalance. This mismatch makes it difficult to develop models that can precisely rebuild and categorize anomalous results [41].
Furthermore, noise, artefacts, and differences in imaging methods can all affect the quality of medical image data [42]. These variances may have an impact on the process of reconstruction's correctness and dependability [42]. Additionally, privacy and ethical concerns around the sharing of patient data for research purposes might restrict the availability of varied and comprehensive datasets [41].
Researchers have investigated methods like data augmentation, generative adversarial networks (GANs), and partial supervision to improve the training process and the caliber of reconstructed pictures in order to overcome these issues [40, 41]. These methods aim to overcome the restrictions of small training datasets, broaden the diversity and volume of data that is accessible, and enhance the generalization of models [40].
5.2 Interpretability and explainability
Medical picture reconstruction presents challenges and constraints due to its interpretability and explainability. Although deep learning models have performed remarkably well at reconstructing images, they are sometimes viewed as "black boxes" since their decision-making process is opaque [40]. Concerns concerning trust, accountability, and the capacity to comprehend and confirm the results are brought up by this lack of interpretability [43].
Black box models for medical picture reconstruction are difficult to comprehend, which prevents their widespread use in clinical settings. To trust these models' accuracy and assure patient safety, healthcare practitioners need explanations and justifications for the judgments they make [2]. In addition, regulatory and ethical reasons require for accountability and openness in medical decision-making procedures [43]. It has been suggested to use methods like LIME (Local Interpretable Model-Agnostic Explanations) to faithfully and comprehensibly explain any classifier's predictions [44]. With the use of these techniques, the characteristics and patterns that contribute to the reconstruction process may be better understood, and the findings can be validated [44, 45].
5.3 Generalization to unseen data
In medical image reconstruction, generalizing to unknown data is a substantial difficulty. The capacity to train models that can successfully recreate pictures from unseen data is hampered by the lack of varied and comprehensive datasets [46, 47]. Over fitting is a problem with deep learning models that occurs when they get too adept at remembering the training data and struggle to generalize successfully to new and unknown input [48]. When used in real-life scenarios, this constraint might result in poor performance and inaccurate outcomes [49]. Researchers have suggested a number of approaches to overcome this problem. Even with little training data, adversarial counterfactual augmentation may be used to create powerful pictures that enhance the performance of downstream tasks [47]. It has been applied successfully in medical image reconstruction to address limited dataset challenges. In scenarios with scarce labeled data, this technique generates diverse yet realistic synthetic images to enhance model training. Notably, it has been employed in improving the performance of deep learning models even with minimal training data, ensuring robust reconstruction outcomes [Machine learning and deep learning approach for medical image analysis: diagnosis to detection]. Additionally, explicit uncertainty measurements added to the models can better generalize and reflect the inherent ambiguity in medical pictures. Techniques such as Bayesian approaches introduce explicit uncertainty quantification, aiding models in handling unseen data more effectively [50]. Effective methods for evaluating the generalization potential of reconstruction algorithms include cross-dataset comparison and testing on previously undiscovered categories [48].
5.4 Computational complexity
Medical image reconstruction's computational complexity is a serious hurdle in the area. Because of the high cost of forward and adjoin operators and the complexity of hyper parameter selection, traditional iterative reconstruction techniques can be computationally costly. Because of their complexity, these algorithms may be difficult to implement in real-world circumstances. To solve this problem, scholars have looked into other techniques. Deep learning-based methods, such as CNNs, have demonstrated promise in terms of lowering computing complexity while maintaining reconstruction performance. These techniques take advantage of GPUs' parallel processing capabilities to achieve quick reconstruction rates, even for big-picture sizes [46].
Furthermore, developments in computing technology have led to the viability of iterative reconstruction approaches for clinical applications. While considering processing economy, the application of Bayesian iterative algorithms and statistical iterative reconstruction approaches has proven considerable increases in picture quality [51].
Compressed sensing and generative adversarial networks (GANs) have also been developed to minimize computational complexity in picture reconstruction. These approaches take use of sparsely and picture structure to enable effective reconstruction with fewer measurements or lower computing needs [52]. The computational complexity of medical picture reconstruction creates practical deployment and real-time processing issues. However, advancements in deep learning, iterative algorithms, compressed sensing, and GANs offer potential solutions to reduce computational requirements while maintaining reconstruction quality [53].
Efficient architecture design involves the development of novel model architectures such as EfficientNet, MobileNet, and Squeeze-and-Excitation Networks, tailored for medical imaging tasks to balance computational efficiency and reconstruction performance [54]. Model compression techniques, including knowledge distillation and pruning, aim to reduce the size of deep learning models without compromising their reconstruction capabilities, enabling faster and more resource-efficient deployment [55] Integration with specialized hardware accelerators, such as GPUs and TPUs, has been emphasized to enhance the computational speed of medical image reconstruction models, crucial for real-time processing in clinical applications [56].
5.5 Integration with clinical workflow and decision support
The integration of image reconstruction with clinical workflow and decision support is a complex research challenge [57]. The fastMRI challenge, which aimed to reduce MR examination times, provided researchers with a large-scale dataset of MRI scanner raw data from a clinical patient population. The challenge primarily focused on supervised machine learning approaches, leading to advancements in image reconstruction and identifying remaining obstacles for clinical adoption. Addressing various aspects is crucial for integrating image reconstruction with clinical workflow and decision support. The evaluation of reconstruction methods involves quantitative image metrics and evaluation by a panel of radiologists [57]. Deep learning-based approaches can be integrated with reconstruction to enable complex post-processing tasks, such as radionics, which can improve diagnostic, prognostic, and predictive accuracy [58]. The development of deep learning-based classification frameworks supports clinical decision-making and addresses challenges like over fitting [59]. Additionally, accurate 3D reconstruction of face models is essential for providing reliable feedback for clinical decision support.
The study [50] shows that Through deep learning-based reconstruction, intricate post-processing tasks such as radiomics are facilitated, enabling the extraction of quantitative features from medical images. This process enhances diagnostic, prognostic, and predictive accuracy, offering valuable insights crucial for informed clinical decision-making. However, the integration process faces challenges, including the imperative need for model interpretability and justifiability. Ensuring compliance with regulatory requirements for accountability and addressing potential biases in clinical decision-making adds complexity to the integration effort.
5.6 Multi-modal image reconstruction
Multi-modal image reconstruction in medical imaging is a challenging and evolving research area. Multiple imaging modalities, such as MRI, CT, and PET, can give complimentary information and increase diagnostic accuracy when used together. In various fields, compressed sensing (CS) techniques have been developed for multi-modal image reconstruction, making use of picture sparsity [60]. CS allows for the reconstruction of high-quality pictures from sparsely sampled data, hence lowering acquisition time and increasing efficiency [61].
In multi-modal image reconstruction, dictionary learning has been presented as a method for adaptive sparsifying transform. The reconstruction algorithm may successfully eliminate aliasing and noise by learning the sparsifying lexicon, resulting in better picture quality [62].
Furthermore, future research should focus on the integration of multi-modal image reconstruction with clinical workflow and decision support systems. By providing complete and complementary information from several imaging modalities, this integration can improve the accuracy and efficiency of diagnosis and treatment planning [60]. The researcher should focus on compressed sensing, dictionary learning, and integration with clinical workflow and decision support systems.
5.7 Addressing issues of data bias and fairness
Addressing data bias and fairness problems in image reconstruction is an important open scientific subject. Addressing issues of data bias and fairness in picture reconstruction is a significant scientific concern. Data bias may be caused by a variety of factors, including an uneven representation of specific demographics or diseases in the training data. Fairness refers to the fair treatment of persons from various demographic groups as well as the avoidance of discriminating effects. Several ways may be used to reduce data bias and assure fairness in picture reconstruction. One strategy is to carefully select varied and representative datasets covering a wide range of people and diseases [63, 64]. This can aid in reducing bias and ensuring that reconstruction models generalize effectively across patient groups. Another method would be to include fairness-aware algorithms and approaches in the picture reconstruction process. These strategies seek to identify and minimize biases in the reconstruction process, guaranteeing equal outcomes for all persons [65]. Differential privacy strategies can also be used to safeguard patient confidentiality while allowing for data sharing and analysis [66]. Furthermore, the transparency and interpretability of image reconstruction models can aid in the identification and mitigation of biases. Potential biases can be identified and corrected by providing explanations and insights into the decision-making process of the models [65].
This study provides a detailed evaluation of the transformational influence of deep learning-based approaches in the domain of medical picture reconstruction. We looked into how the power of artificial intelligence may be combined with cutting-edge imaging technologies, such MRI, CT scans, and PET scans, to significantly improve clinical diagnosis and patient care. How these methods offer vital information on biological, anatomical, and physiological structures, assisting in precise diagnosis and efficient treatment. Traditional medical imaging workflows are being transformed by the use of deep learning models in this field. One important issue we studied was the efficiency and speed of medical picture reconstruction. Despite being time-consuming, conventional techniques like MRI scans are trustworthy. On the other hand, deep learning algorithms can dramatically speed up the imaging process. MRI scans may be completed more quickly by using methods like Parallel Imaging (PI) and CS. The assessment, however, also highlighted the difficulties and restrictions that come with these approaches. By providing real-time imaging and improving overall picture quality, deep learning (DL) techniques have enormous potential for overcoming these constraints.
Convolutional Neural Networks (CNNs) were highlighted while discussing the various deep learning modalities. These networks, which have outstanding skills in feature extraction, classification, and identification, have become the cornerstone of imaging-based AI systems. Additionally, CNN-based DL models eliminate the requirement for manual feature engineering, which accelerates the algorithm development process. The review discussed the drawbacks of DL-based medical picture reconstruction in addition to outlining its achievements. Since deep learning models are sometimes complicated interpretability and transparency have arisen as important challenges. Their inclusion into clinical processes, where accountability and explainability are crucial, is hampered by this lack of transparency. The debate emphasized the significance of approaches like LIME and SHAP, which aim to improve the interpretability of DL-based judgements by increasing trust and confidence in algorithmic results.
The paper also emphasizes how important it is to evaluate deep learning models. Performance measures like PSNR, SSIM, MSE, GAN, and FID were proposed as instruments to evaluate the calibre of reconstructed pictures quantitatively. These measures give academics and industry professionals unbiased standards to compare various algorithms against when making judgement calls. The intricacy and diversity of the area are the open research problems. As significant obstacles, generalization to unknown data, data availability and quality, and computational complexity developed. These difficulties have an effect on both algorithm development and the incorporation of these methods into clinical settings. Multidisciplinary collaboration including AI specialists, medical practitioners, and legislators is necessary to address these difficulties. Finally, clinical validation is necessary to demonstrate the practical usefulness of DL architecture. Therefore, future studies should focus on validating the performance of the various proposed architecture on a large-scale clinical dataset. Overall, this review serves as a valuable resource for researcher of medical imaging. By consolidating insights from various methodologies, it provides a basline for researchers, healthcare practitioners, and industry professionals to development novel reconstruction techniques, and ultimately improving diagnostic accuracy and patient care. Healthcare practitioners can make better decisions through these advanced and efficient reconstruction methods. In turn, patients benefit from expedited and accurate diagnoses, improving treatment outcomes and overall healthcare experiences.
Medical image reconstruction is a critical process for accurate diagnosis and treatment planning in healthcare. Deep learning techniques, such as Generative Adversarial Networks (GANs), autoencoders, and Convolutional Neural Networks (CNNs), have shown great promise in improving the accuracy and efficiency of medical image reconstruction. However, challenges in terms of data availability, interpretability, generalization, and computing efficiency still remain. Overcoming such challenges is critical to realizing the full promise of deep learning-based medical image reconstruction. In this study, we anticipate the development of more sophisticated algorithms that address the challenges and deliver improved results. The problems of data availability and quality in medical image reconstruction need the development of novel ways to overcome the lack of labeled data, class imbalance, and variability in picture quality. Techniques like data augmentation, GANs, and partial supervision may be used to improve the training process and the quality and reliability of reconstructed medical images. Techniques like adversarial counterfactual augmentation, uncertainty estimation, and cross-dataset assessment can help increase the generalization and reliability of reconstructed pictures. Image reconstruction integration with clinical processes and decision support systems will be critical in ensuring that these technologies have a significant influence on patient care. In the context of ethical and responsible medical imaging practices, the pursuit of justice, accountability, and transparency in algorithmic judgments is a continuing effort. Further improvements in deep learning techniques, as well as improved collaboration among medical professionals, researchers, and technologists, are expected in the future years. By tackling the problems and developing the procedures, we are on the verge of witnessing a new era of medical imaging that will benefit both patients and healthcare practitioners.
[1] Yaqub, M., Jinchao, F., Arshid, K., Ahmed, S., Zhang, W., Nawaz, M.Z., Mahmood, T. (2022). Deep learning-based image reconstruction for different medical imaging modalities. Computational and Mathematical Methods in Medicine. https://doi.org/10.1155/2022/8750648
[2] Nazir, S., Dickson, D.M., Akram, M.U. (2023). Survey of explainable artificial intelligence techniques for biomedical imaging with deep neural networks. Computers in Biology and Medicine, 156: 106668. https://doi.org/10.1016/j.compbiomed.2023.106668
[3] Zhang, Z., Seeram, E. (2020). The use of artificial intelligence in computed tomography image reconstruction - A literature review. Journal of Medical Imaging and Radiation Sciences, 51(4): 671-677. https://doi.org/10.1016/j.jmir.2020.09.001
[4] Ahishakiye, E., Van Gijzen, M.B., Tumwiine, J., Wario, R., Obungoloch, J. (2021). A survey on deep learning in medical image reconstruction. Intelligent Medicine, 1(3): 118-127. https://doi.org/10.1016/j.imed.2021.03.003
[5] Almotairi, K.H., Hussein, A.M., Abualigah, L., Abujayyab, S.K., Mahmoud, E.H., Ghanem, B.O., Gandomi, A.H. (2023). Impact of artificial intelligence on COVID-19 pandemic: A survey of image processing, tracking of disease, prediction of outcomes, and computational medicine. Big Data and Cognitive Computing, 7(1): 11. https://doi.org/10.3390/bdcc7010011
[6] den Bakker, I. (2017). Battle of the Deep Learning frameworks — Part I: 2017, even more frameworks and interfaces. Towards Data Science. https://towardsdatascience.com/battle-of-the-deep-learning-frameworks-part-i-cff0e3841750.
[7] Yaqub, M., Jinchao, F., Ahmed, S., Arshid, K., Bilal, M. A., Akhter, M.P., Zia, M.S. (2022). Gan-tl: Generative adversarial networks with transfer learning for mri reconstruction. Applied Sciences, 12(17): 8841. https://doi.org/10.3390/app12178841
[8] Song, Y., Zheng, S.J., Li, L., Zhang, X., Zhang, X.D., Huang, Z.W., Chen, J.W., Wang, R.X., Zhao, H.Y., Chong, Y.T., Shen, J., Zha, Y.F., Yang, Y.D. (2021). Deep learning enables accurate diagnosis of novel coronavirus (COVID-19) with CT images. IEEE/ACM transactions on computational biology and bioinformatics, 18(6): 2775-2780. https://doi.org/10.1109/TCBB.2021.3065361
[9] Feng, C.M., Yan, Y., Fu, H., Chen, L., Xu, Y. (2021). Task transformer network for joint mri reconstruction and super-resolution. In Springer, Cham (Ed.), Lecture Notes in Computer Science, Springer International Publishing, pp. 307-317. https://doi.org/10.1007/978-3-030-87231-1_30
[10] Gaur, L., Bhandari, M., Razdan, T., Mallik, S., Zhao, Z. (2022). Explanation-driven deep learning model for prediction of brain Tumour status using MRI image data. frontiers in genetics, 13: 448. https://doi.org/10.3389/fgene.2022.822666
[11] Ben Yedder, H., Cardoen, B., Hamarneh, G. (2021). Deep learning for biomedical image reconstruction: A survey. Artificial Intelligence Review, 54: 215-251. https://doi.org/10.1007/s10462-020-09861-2
[12] Du, W., Rao, N., Liu, D., Jiang, H.X., Luo, C.S., Li, Z.W., Gan, T., Zeng, B. (2019). Review on the applications of deep learning in the analysis of gastrointestinal endoscopy images. IEEE Access, 7: 142053-142069. https://doi.org/10.1109/ACCESS.2019.2944676
[13] Yu, W., Zeng, L. (2015). ℓ0 Gradient minimization based image reconstruction for limited-angle computed tomography. PLoS ONE, 10(7): e0130793. https://doi.org/10.1371/journal.pone.0130793
[14] Zhao, H., Liu, Z., Tang, J., et al. (2023). Energy-efficient high-fidelity image reconstruction with memristor arrays for medical diagnosis. Nature Communications, 14(1): 2276. https://doi.org/10.1038/s41467-023-35806-0
[15] El‐Rewaidy, H., Neisius, U., Gao, B., Qin, Q., Li, J.M., Zhou, Y., Yao, P., Xi, Y., Lin, Y.D., Qian, H., Wu, H.Q. (2020). Deep complex convolutional network for fast reconstruction of 3D late gadolinium enhancement cardiac MRI. NMR in Biomedicine, 33(7): e4312. https://doi.org/10.1002/nbm.4312
[16] Lee, D., Choi, S., Kim, H.J. (2019). High quality imaging from sparsely sampled computed tomography data with deep learning and wavelet transform in various domains. Medical Physics, 46(1): 104-115. https://doi.org/10.1002/mp.13258
[17] Liu, J., Zhang, T., Kang, Y., Wang, Y., Zhang, Y., Hu, D., Chen, Y. (2023). Deep residual constrained reconstruction via learned convolutional sparse coding for low-dose CT imaging. Biomedical Signal Processing and Control, 85: 104868. https://doi.org/10.1016/j.bspc.2023.104868
[18] Zhang, W., Xiao, Z., Tao, H., Zhang, M., Xu, X., Liu, Q. (2023). Low-rank tensor assisted K-space generative model for parallel imaging reconstruction. Magnetic Resonance Imaging, 103: 198-207. https://doi.org/10.1016/j.mri.2023.07.004
[19] Sarvari, A.V.P., Sridevi, K. (2023). An optimized EBRSA-Bi LSTM model for highly undersampled rapid CT image reconstruction. Biomedical Signal Processing and Control, 83: 104637. https://doi.org/10.1016/j.bspc.2023.104637
[20] Ghodrati, V., Shao, J., Bydder, M., Zhou, Z.W., Yin, W.T., Nguyen, K.L., Yang, Y.L., Hu, P. (2019). MR image reconstruction using deep learning: Evaluation of network structure and loss functions. Quantitative imaging in medicine and surgery, 9(9): 1516. https://doi.org/10.21037/qims.2019.08.10
[21] Lin, Z., Zhang, X., Guo, L., Wang, K., Jiang, Y., Hu, X.Y., Huang, Y., Wei, J., Ma, S., Liu, Y., Zhu, L., Zhuo, Z.Z., Liu, J., Wang, X.Y. (2019). Clinical feasibility study of 3D intracranial magnetic resonance angiography using compressed sensing. Journal of Magnetic Resonance Imaging, 50(6): 1843-1851. https://doi.org/10.1002/jmri.26752
[22] Lopez-Gamundi, P., Yao, Y.W., Chong, T.T., Heekeren, H.R., Mas-Herrero, E., Marco-Pallarés, J. (2021). The neural basis of effort valuation: A meta-analysis of functional magnetic resonance imaging studies. Neuroscience & Biobehavioral Reviews, 131: 1275-1287. https://doi.org/10.1016/j.neubiorev.2021.10.024
[23] Hay, L., Duffy, A. H. B., Gilbert, S.J., Grealy, M.A. (2022). Functional magnetic resonance imaging (fMRI) in design studies: Methodological considerations, challenges, and recommendations. Design Studies, 78: 101078. https://doi.org/10.1016/j.destud.2021.101078
[24] Koetzier, L.R., Mastrodicasa, D., Szczykutowicz, T.P., van der Werf, N.R., Wang, A.S., Sandfort, V., van der Molen, A.J., Fleischmann, D., Willemink, M.J. (2023). Deep learning image reconstruction for CT: Technical principles and clinical prospects. Radiology, 306(3): e221257. https://doi.org/10.1148/radiol.221257
[25] Sayalı, C., Badre, D. (2019). Neural systems of cognitive demand avoidance. Neuropsychologia, 123: 41-54. https://doi.org/10.1016/j.neuropsychologia.2018.06.016
[26] Zhang, H.M., Dong, B. (2020). A review on deep learning in medical image reconstruction. Journal of the Operations Research Society of China, 8(2): 311-340. https://doi.org/10.1007/s40305-019-00287-4
[27] Prajapati, B.G., Paliwal, H., Patel, J.K. (2022). Nanoparticle pharmacokinetic profiling in vivo using magnetic resonance imaging. In Pharmacokinetics and Pharmacodynamics of Nanoparticulate Drug Delivery Systems, Springer International Publishing, pp. 399-416. https://doi.org/10.1007/978-3-030-83395-4_22
[28] Westbrook, A., Lamichhane, B., Braver, T. (2019). The Subjective value of cognitive effort is encoded by a domain-general valuation network. The Journal of Neuroscience, 39(20): 3934-3947. https://doi.org/10.1523/JNEUROSCI.3071-18.2019
[29] Schüller, C. B., Kuhn, J., Jessen, F., Hu, X. (2019). Neuronal correlates of delay discounting in healthy subjects and its implication for addiction: an ALE meta-analysis study. American Journal of Drug and Alcohol Abuse, 45(1): 51-66. https://doi.org/10.1080/00952990.2018.1557675
[30] Soder, H. E., Cooper, J. A., Lopez-Gamundi, P., et al. (2021). Dose-response effects of d-amphetamine on effort-based decision-making and reinforcement learning. Neuropsychopharmacology, 46(6): 1078-1085. https://doi.org/10.1038/s41386-020-0779-8
[31] Vassena, E., Deraeve, J., Alexander, W.H. (2020). Surprise, value and control in anterior cingulate cortex during speeded decision-making. Nature Human Behaviour, 4(4): 412-422. https://doi.org/10.1038/s41562-019-0801-5
[32] Erzar, B., Lesar, Ž., Marolt, M. (2023). Fast incremental image reconstruction with CNN-enhanced poisson interpolation. https://doi.org/10.24132/CSRN.3301.24
[33] Cui, Z., Sun, C., Yang, P., Wang, H. (2023). A heterogeneous hardware scheme for accelerating the image reconstruction process of electrical tomography. Measurement Science and Technology, 34(10): 105902. https://doi.org/10.1088/1361-6501/acdff1
[34] Triwijoyo, B. K., Adil, A., Marzuki Mataram, J.I. (2021). Analysis of medical image resizing using bicubic interpolation algorithm. Jurnal Ilmu Komputer, 14(2): 20-29. https://doi.org/10.24843/JIK.2021.v14.i01.p03
[35] Xu, Y., Li, Y., Shin, B.S. (2020). Medical image processing with contextual style transfer. Human-centric Computing and Information Sciences, 10(1): 1-16. https://doi.org/10.1186/s13673-020-00251-9
[36] Rebecq, H., Ranftl, R., Koltun, V., Scaramuzza, D. (2021). High speed and high dynamic range video with an event camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(6): 1964-1980. https://doi.org/10.1109/TPAMI.2019.2963386
[37] Kelkar, V.A., Gotsis, D.S., Brooks, F.J., Prabhat, K.C., Myers, K.J., Zeng, R., Anastasio, M.A. (2023). Assessing the ability of generative adversarial networks to learn canonical medical image statistics. IEEE Transactions on Medical Imaging, 42(6): 1799. https://doi.org/10.1109/TMI.2023.3241454
[38] Ding, K., Ma, K., Wang, S., Simoncelli, E.P. (2021). Comparison of full-reference image quality models for optimization of image processing systems. International Journal of Computer Vision, 129(4): 1258-1281. https://doi.org/10.1007/S11263-020-01419-7/FIGURES/19
[39] Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A.W.M., van Ginneken, B., Sánchez, C.I. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42: 60-88. https://doi.org/10.1016/J.MEDIA.2017.07.005
[40] Haque, S., Haque, A. (2021). 3N-GAN: Semi-supervised classification of x-ray images with a 3-player adversarial framework. arXiv preprint arXiv:2109.13862.
[41] Shin, H.C., Tenenholtz, N.A., Rogers, J.K., Schwarz, C.G., Senjem, M.L., Gunter, J.I., Andriole, K.P., Michalski, M. (2018). Medical image synthesis for data augmentation and anonymization using generative adversarial networks. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11037: 1-11. https://doi.org/10.1007/978-3-030-00536-8_1/COVER
[42] Mohandass, D., Janet, J. (2011). A novel image retrieval technique for enhanced telemedical applications. TISC 2011 - Proceedings of the 3rd International Conference on Trendz in Information Sciences and Computing, Chennai, India, 46-50. https://doi.org/10.1109/TISC.2011.6169082
[43] Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D. (2018). A survey of methods for explaining black box models. ACM Computing Surveys, 51(5). https://doi.org/10.1145/3236009
[44] Ribeiro, M.T., Singh, S., Guestrin, C. (2016). ‘Why Should I Trust You?’: Explaining the Predictions of Any Classifier. NAACL-HLT 2016 - 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Demonstrations Session, pp. 97-101. https://doi.org/10.18653/v1/n16-3020
[45] Hasenstab, K.A., Huynh, J., Masoudi, S., Cunha, G.M., Pazzani, M., Hsiao, A. (2023). Feature interpretation using generative adversarial networks (FIGAN): A framework for visualizing a CNN’S learned features. IEEE Access, 11: 5144-5160. https://doi.org/10.1109/ACCESS.2023.3236575
[46] Jin, K.H., McCann, M.T., Froustey, E., Unser, M. (2017). Deep convolutional neural network for inverse problems in imaging. IEEE Transactions on Image Processing, 26(9): 4509-4522. https://doi.org/10.1109/TIP.2017.2713099
[47] Xia, T., Sanchez, P., Qin, C., Tsaftaris, S.A. (2022). Adversarial counterfactual augmentation: Application in Alzheimer’s disease classification. Frontiers in Radiology, 2. https://doi.org/10.3389/fradi.2022.1039160
[48] Thai, A., Stojanov, S., Upadhya, V., Rehg, J.M. (2021). 3d reconstruction of novel object shapes from single images. In 2021 International Conference on 3D Vision (3DV), London, United Kingdom, pp. 85-95. https://doi.org/10.1109/13DV53792.2021.00019
[49] Ghesu, F.C., Georgescu, B., Mansoor, A., et al. (2021). Quantifying and leveraging predictive uncertainty for medical image assessment. Medical Image Analysis, 68: 101855. https://doi.org/10.1016/j.media.2020.101855
[50] Rana, M., Bhushan, M. (2023). Machine learning and deep learning approach for medical image analysis: diagnosis to detection. Multimedia Tools and Applications, 82(17): 26731-26769. https://doi.org/10.1007/S11042-022-14305-W
[51] Thibault, J.B., Sauer, K.D., Bouman, C.A., Hsieh, J. (2007). A three-dimensional statistical approach to improved image quality for multislice helical CT. Medical Physics, 34(11): 4526-4544. https://doi.org/10.1118/1.2789499
[52] Li, W., Zhu, A., Xu, Y., Yin, H., Hua, G. (2022). A fast multi-scale generative adversarial network for image compressed sensing. Entropy, 24(6): 775. https://doi.org/10.3390/E24060775
[53] Tachella, J., Altmann, Y., Mellado, N., McCarthy, A., Tobin, R., Buller, G.S., Tourneret, J.Y., McLaughlin, S. (2019). Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers. Nature communications, 10(1): 4984. https://doi.org/10.1038/s41467-019-12943-7
[54] Üzen, H., Turkoglu, M., Aslan, M., Hanbay, D. (2023). Depth-wise squeeze and excitation block-based efficient-unet model for surface defect detection. Visual Computer, 39(5): 1745-1764. https://doi.org/10.1007/S00371-022-02442-0/FIGURES/13
[55] Schlemper, J., Caballero, J., Hajnal, J.V., Price, A.N., Rueckert, D. (2018). A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Transactions on Medical Imaging, 37(2): 491-503. https://doi.org/10.1109/TMI.2017.2760978
[56] Spagnolo, F., Corsonello, P., Frustaci, F., Perri, S. (2023). Design of a low-power super-resolution architecture for virtual reality wearable devices. IEEE Sensors Journal, 23(8): 9009-9016. https://doi.org/10.1109/JSEN.2023.3256524
[57] Knoll, F., Murrell, T., Sriram, A., Yakubova, N., Zbontar, J., Rabbat, M., Defazio, A., Muckley, M.J., Sodickson, D.K., Zitnick, L., Recht, M.P. (2020). Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge. Magnetic resonance in medicine, 84(6): 3054-3070. https://doi.org/10.1002/mrm.28338
[58] Adler, J., Lunz, S., Verdier, O., Schönlieb, C.-B., Öktem, O. (2018). Task adapted reconstruction for inverse problems. Inverse Problems, 38(7). https://doi.org/10.1088/1361-6420/ac28ec
[59] Wang, Z., Fan, Z., Sun, L.L., Hao, Y., Gay, H., Thorstad, W., Wang, X.W., Li, H. (2023). Deep-supervised adversarial learning-based classification for digital histologic images. In Medical Imaging 2023: Digital and Computational Pathology, 12471: 483-493. https://doi.org/10.1117/12.2654402
[60] Ravishankar, S., Bresler, Y. (2011). MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Transactions on Medical Imaging, 30(5): 1028-1041. https://doi.org/10.1109/TMI.2010.2090538
[61] Lustig, M., Donoho, D., Pauly, J.M. (2007). Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine, 58(6): 1182-1195. https://doi.org/10.1002/MRM.21391
[62] Dewi, C., Christanto, H.J. (2022). Combination of deep cross-stage partial network and spatial pyramid pooling for automatic hand detection. Big Data and Cognitive Computing. https://doi.org/10.3390/bdcc6030085
[63] Yan, L., Huang, W., Wang, L., Feng, S., Peng, Y., Peng, J. (2021). Data-enabled digestive medicine: A new big data analytics platform. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 18(3): 922-931. https://doi.org/10.1109/TCBB.2019.2951555
[64] Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M. (2017). Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2097-2106. https://doi.org/10.1109/CVPR.2017.369
[65] McCall, C.J., DeCaprio, D., Gartner, J. (2022). The measurement and mitigation of algorithmic bias and unfairness in healthcare AI models developed for the CMS AI health outcomes challenge. medRxiv, 2022-09. https://doi.org/10.1101/2022.09.29.22280537
[66] Beaulieu-Jones, B.K., Wu, Z.S., Williams, C., Lee, R., Bhavnani, S.P., Byrd, J.B., Greene, C.S. (2019). Privacy-preserving generative deep neural networks support clinical data sharing. Circulation: Cardiovascular Quality and Outcomes, 12(7): e005122. https://doi.org/10.1161/CIRCOUTCOMES.118.005122
