Rui Chen | Dongsheng Li | Fen Zhu | Xisheng Yan | Hua Wang | Chaokun Guan | Xuan Ke | Lingjun Zeng | Jie Zhao*
© 2024 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
Heart failure, a condition characterized by a decline in cardiac pumping capacity, necessitates precise assessment of cardiac function due to its systemic impact on blood circulation. Dynamic cardiac ultrasound imaging serves as a crucial tool for evaluating left ventricular function. The quality of these images directly influences the accuracy of diagnostics and the effectiveness of treatments. Existing cardiac ultrasound image processing technologies face limitations in enhancing details, noise reduction, and capturing dynamic information. This study introduces a novel image processing technique that integrates visual attention mechanisms and generative adversarial networks (GAN) to enhance the details of dynamic cardiac ultrasound images. Additionally, it employs an algorithm based on dynamic contour models for image segmentation and assessment of left ventricular function. The application of these techniques aims to improve the processing quality of cardiac ultrasound images, enabling more accurate assessments of left ventricular function and providing more effective support for the diagnosis and treatment of heart failure patients.
heart failure, cardiac ultrasound imaging, image processing technology, visual attention mechanisms, generative adversarial networks (GAN), dynamic contour models, left ventricular function assessment
In contemporary medical diagnostics, precise assessment of cardiac function is crucial for the treatment and management of patients with heart failure. Heart failure is a severe clinical condition involving a decline in cardiac pumping capability, affecting systemic blood circulation [1-3]. Dynamic cardiac ultrasound imaging, a non-invasive and cost-effective diagnostic method, can visually display the cardiac motion state and is valuable for assessing left ventricular function [4, 5]. However, traditional cardiac ultrasound images often face limitations in resolution and capturing dynamic details due to equipment and technological constraints, necessitating advanced image processing technologies to enhance diagnostic accuracy and efficiency.
Although cardiac ultrasound imaging is widely used in clinical diagnostics, the accuracy of assessing left ventricular function in heart failure patients remains a challenge. The function of the cardiac left ventricle directly influences patient prognosis, and precise functional assessment can guide clinical treatment decisions, such as drug selection and dosage adjustments [6-8]. Therefore, developing new cardiac ultrasound image processing technologies to better analyze and interpret image data holds significant clinical importance for enhancing diagnostic accuracy and timeliness [9-12].
Current cardiac ultrasound image processing methods have made certain advancements but still have many deficiencies. For example, many existing algorithms struggle to effectively handle noise in images and are not sufficiently precise in capturing cardiac details in dynamic images, limiting their effectiveness in actual clinical applications [13-16]. Moreover, these methods often overlook visual attention information in images, which is crucial for accurately identifying critical cardiac areas [17, 18]. Thus, researching and developing new image processing algorithms is particularly urgent.
This paper proposes a new dynamic cardiac ultrasound image processing technology, consisting of two innovative research parts. First, a detail enhancement algorithm for dynamic cardiac ultrasound images based on visual attention mechanisms and GAN, which can significantly improve the clarity and dynamic representation of images, thereby better revealing the cardiac functional status. Second, a technique for dynamic cardiac ultrasound image segmentation and left ventricular function assessment based on dynamic contour models, which not only improves segmentation accuracy but also enhances the precise measurement of left ventricular function parameters. Through the application of these two technologies, this study aims to advance the field of cardiac ultrasound image processing, providing more precise diagnostic support for heart failure patients, thereby optimizing treatment strategies and improving patient quality of life.
In the diagnosis and treatment of heart failure patients, accurate assessment of left ventricular function is crucial as it directly relates to the formulation of treatment plans and improvement of prognosis. The dynamic cardiac ultrasound image detail enhancement algorithm proposed in this paper, based on visual attention mechanisms and GAN, improves image quality and highlights critical cardiac structures. This makes dynamic cardiac ultrasound images clearer and more accurate in displaying the cardiac dynamics. The application of this technology not only effectively reduces uncertainties during the diagnostic process but also provides more precise structural and functional information of the heart. This assists physicians in accurately assessing the left ventricular function of heart failure patients, thereby guiding personalized treatment strategies.
2.1 GAN
The enhancement of dynamic cardiac ultrasound images involves dealing with the heart's motion patterns and subtle changes in cardiac structures, which is crucial for capturing and enhancing details related to cardiac motion. This paper applies GAN to the enhancement of dynamic cardiac ultrasound image details, focusing on converting unclear or low-quality ultrasound images into high-quality, detailed images. In this process, the generator model takes a random vector A representing the latent image feature space as input, following a probability distribution $o_e(a)$, where $\kappa$ represents a continuous or discrete feature space. Through training, the generator model $o_\phi(a)$ learns to simulate the distribution of actual ultrasound images $o_e(a)$, thereby generating samples similar to real ultrasound images. Figure 1 shows the schematic diagram of the GAN structure used.
Figure 1. Schematic diagram of the GAN structure used
In the application of detail enhancement in dynamic cardiac ultrasound images, the core of using GAN lies in processing the complex grayscale distribution and detailed features of cardiac motion in dynamic cardiac ultrasound images through a deep generative model. This method involves an adversarial process between a generator and a discriminator, where the generator attempts to create synthetic images from random noise vectors C that are as close as possible to real ultrasound images. These noise vectors C follow a predefined distribution o(c) and are transformed into image data through a deep neural network $h: C \rightarrow \kappa$, aiming to make the transformed images $o(c)$ approximate the distribution of real cardiac ultrasound images $o_e(a)$. Compared to infrared high dynamic image detail enhancement, dynamic cardiac ultrasound image detail enhancement not only deals with the complexity of image grayscale but also precisely captures and enhances key details of cardiac motion, such as the thickness of the ventricular walls and the dynamic changes of the cardiac valves, which have a direct impact on correctly interpreting the cardiac functional status and guiding clinical decisions.
(1) Discriminator
In the algorithm, the key task of the Discriminator F(a,θ) is to distinguish whether the input sample a is sourced from the real cardiac ultrasound image distribution $o_e(a)$ or produced by the generator model $o_\phi(a)$. The output of the discriminator ranges between 0 and 1, where 1 indicates the sample is a real image, and 0 indicates it is a generated image, essentially assessing the probability of the sample being a real cardiac ultrasound image as $o(b=1 \mid a)$. Correspondingly, the probability that the sample originates from the generator is $o(b=0 \mid a)=1-F(a, \theta)$.
$o(b=1 \mid a)=f(a, \theta)$ （1）
This setup is particularly important in dynamic cardiac ultrasound image enhancement, as compared to infrared high dynamic imaging, cardiac ultrasound involves more complex dynamic physiological structures, such as the dynamic changes of the ventricular walls and valves. This requires the discriminator not only to recognize the authenticity of the image but also to assess the naturalness and physiological accuracy of the dynamic details within the image.
The discriminator optimizes its performance through the cross-entropy loss function, which effectively promotes the authenticity and accuracy of the generated images, thus supporting the precise assessment of left ventricular function in heart failure patients. Given a sample (a,b), with b={1,0} indicating whether the sample is from the real distribution $o_e(a)$ or the generator model $o_\phi(a)$, the parameters of the generator and discriminator are represented by $\phi$ and $\theta$ respectively, with the loss function expressed as:
$\begin{aligned} & \underset{\theta}{M N}\left(R_a\left[\begin{array}{l}b \log o(b=1 \mid a) \\ +(1-b) \log o(b=0 \mid a)\end{array}\right]\right) \\ & =\underset{\theta}{M A X}\binom{R_{a \sim o_e(a)}[\log F(a, \theta)]}{+R_{a^{\prime} \sim o_e^{\prime}\left(a^{\prime}\right)}\left[\log F\left(a^{\prime}, \theta\right)\right]} \\ & =\underset{\theta}{M A X}\binom{R_{a \sim o_e(a)}[\log F(a, \theta)]}{+R_{c \sim o(c)}[\log (1-F(H(c, \varphi), \theta))]}\end{aligned}$ (2)
(2) Generator
In the algorithm, the main objective of the Generator $H(c, \phi)$ is to produce synthetic images from a random noise vector z that are indistinguishable from real cardiac ultrasound images, thereby deceiving the discriminator into incorrectly identifying their origin. In this process, the target function for the generator is empirically chosen to provide good gradients to facilitate effective learning and adjustments during model training. Especially in dynamic cardiac ultrasound image enhancement, consideration is given to the spatiotemporal details of the images, such as the subtle movements and structural changes during the cardiac pumping cycle. This requires the generator not only to produce visually authentic static images but also to capture the authenticity of these physiological dynamics.
$\begin{aligned} & \underset{\varphi}{\operatorname{MAX}}\left(R_{c \sim(c)}[\log F(H(c, \varphi), \theta)]\right) \\ & =\underset{\varphi}{M I N}\left(R_{c \sim o(c)}[\log (1-F(H(c, \varphi), \theta))]\right) \\ & \end{aligned}$ (3)
Choosing to use the $\log (b)$ function, where gradients are smaller near the value of 1 , may lead to reduced training efficiency, especially when the discriminator perceives the generated images to be almost indistinguishable from real images, i.e., $(1-H(c, \phi), \theta) \rightarrow 1$. Therefore, selecting a target function suitable for the characteristics of dynamic cardiac images is crucial for enhancing training feedback and boosting the model's generative capabilities, ensuring the generated cardiac images are not only visually accurate but also realistically dynamic, thus effectively supporting the precise assessment of left ventricular function in heart failure patients.
(3) Training
In the algorithm, the training process often requires special attention as it can be unstable. To ensure that the generator and discriminator reach a balance, the training of the discriminator should not be too strong initially, as this might prevent the generator from learning sufficiently, nor too weak, as this could cause the images produced by the generator to lack realism. During training, a key practice is that the discriminator is usually updated multiple times for each single update of the generator. This strategy ensures that the discriminator has adequate discerning capability before starting to train the generator, ensuring that the generator receives effective feedback and improves its generative power. In the application of dynamic cardiac ultrasound image detail enhancement, this is particularly important because the ultrasound images need to capture the dynamic changes and subtle features of the heart. An overly simplistic training process could make it difficult for the generator to produce sufficiently realistic images. The number of times the discriminator is updated during training is a hyperparameter, depending on the specific application scenario and the required quality of generation. This training method helps ensure that the GAN generates ultrasound images that are both realistic and sufficiently detailed for assessing left ventricular function in heart failure patients.
2.2 Network model based on visual attention mechanism and GAN
This study, based on visual attention mechanisms and GAN, has developed a specialized algorithm for dynamic cardiac ultrasound image detail enhancement. Its network structure incorporates a U-Net-based generator and a Patch-based discriminator. The schematic diagram of the network structure is shown in Figure 2. In this setup, the generator receives dynamic cardiac ultrasound images as input and outputs enhanced detail images along with corresponding saliency maps, which highlight key areas within the cardiac images, thereby guiding physicians to notice potential abnormalities or significant physiological information. The generator includes two decoders: one for enhancing cardiac images and another specifically for generating saliency maps. By introducing short connections between the encoder and the cardiac image enhancement decoder, this design aims to maintain the integrity of image details and enhance the flow of information from lower to higher layers. The saliency map decoder is attached to each layer of the cardiac image enhancement decoder, but they do not share weights, ensuring that the generation of saliency maps is focused on key visual features of the image. The role of the discriminator is to analyze the differences between images produced by the generator and real cardiac ultrasound images, further pushing the generator to produce more accurate and diagnostically valuable images.
Specifically, a generator combining ResNet and U-Net architectures has been used to improve image quality and accelerate network convergence. Notably, ResNet's residual blocks are embedded into the bottleneck part of the U-Net structure to effectively capture and refine the high-level semantic information in cardiac ultrasound images, aiding in better depicting the complexity of cardiac dynamics. The addition of four ResBlocks expands the receptive field, thus enhancing the perception quality of cardiac structural details. The strength of the U-Net structure lies in each encoder block being directly copied to a decoder part of the same size, with short connections promoting information flow and reducing the problem of gradient vanishing, thus maintaining the edges and low-level details of the image. Additionally, a saliency map decoder is introduced for saliency prediction based on the visual attention mechanism, performing pixel multiplication of the two outputs of the generator, highlighting areas crucial for the assessment of left ventricular function. The discriminator employs a PatchGAN structure, which allows it to evaluate specific sections of the image rather than the whole, providing a more refined true/false determination.
Figure 2. Schematic diagram of the network structure for dynamic cardiac ultrasound image detail enhancement algorithm
To ensure effective and stable convergence of the algorithm, the loss functions are set in stages within the algorithm. In the first stage, only the generator is trained, using L1 loss to optimize pixel accuracy, essentially optimizing the generator for Peak Signal-to-Noise Ratio (PSNR), aimed at initially improving the overall quality of the image to ensure that the generated images maintain high structural consistency with real images. This stage lays the foundation for the stability of subsequent GAN training, as the optimized generator can produce samples closer to real cardiac ultrasound images. Suppose the enhanced images and saliency maps output by the generator are represented by H1(a) and H2(a), and their corresponding real samples by b1 and b2. The product of corresponding elements of the matrices is represented by the symbol ○. The L1 loss and the region of interest L1 loss function expressions for the first stage are:
$M_{U E}=R\left[\left\|H_1(a)-b_1\right\|_1\right]$ (4)
$M_{E P U}=R\left[\left\|H_1(a) \circ H_2(a)-b_1 \circ b_2\right\|_1\right]_1$ (5)
In the second stage, the GAN architecture is introduced, using cross-entropy loss to train the discriminator, as well as combining L1 loss and GAN loss to optimize the generator, to improve the realism and dynamic detail expression of the generated images, especially the precise depiction of cardiac dynamics such as the motion of the ventricular walls and valve function. Assuming the generator is represented by H and the discriminator by F, the loss functions for the generator and discriminator are:
$M_H=\frac{1}{2} R\left[\left(F\left(H_1(a)\right)-1\right)^2\right]$ (6)
$M_F=\frac{1}{2} R\left[\left(F\left(b_1\right)-1\right)^2\right]+\frac{1}{2} R\left[\left(F\left(H_1(a)\right)\right)^2\right]$ (7)
To further enhance the visual quality of the images, convolutional layer outputs are used as semantic information. Assuming that the feature map of the $m$-th layer from a pretrained CNN model is represented by $\theta_m(*)$, the definition of the perceptual loss is as follows:
$M_o=R\left[\left\|\theta_m\left(H_1(a)-\theta_l\left(b_1\right)\right)\right\|_1\right]$ (8)
The total loss function for the generator in the second stage can be represented as:
$L O S S=M_{U E}+\eta_{E P U} M_{E P U}+\eta_H M_H+\eta_o M_o$ (9)
Compared to traditional ultrasound image segmentation, cardiac ultrasound image segmentation faces more challenges, mainly due to the dynamic changes in cardiac structures and the inherent noise and low contrast characteristics of ultrasound images. Especially when assessing the left ventricular function in heart failure patients, accurately identifying the boundaries and dynamic changes of the left ventricle is crucial. Thus, in the task of dynamic cardiac ultrasound image segmentation for left ventricular function assessment, this study focuses on how to effectively integrate image gradient information with regional information to enhance segmentation accuracy and the ability to resolve complex cardiac structures. For this purpose, this study proposes combining gradient-related local edge information with region-based segmentation techniques to explore whether this fusion method can improve segmentation results, thereby generating a more precise left ventricular model. Figure 3 shows a schematic diagram of a four-level pyramid.
Figure 3. Four-level pyramid schematic diagram
Specifically, this paper employs a multi-resolution theory-based dynamic contour model that uses Gaussian pyramid techniques to process dynamic cardiac ultrasound images for more precise detection and segmentation of the left ventricular edges. Initially, a set of Gaussian pyramid images is established for the cardiac ultrasound image, starting from the top layer’s low-resolution image, using the segmentation model to preliminarily determine the approximate edges of the left ventricle. These initial edges are then transferred to the next level of higher-resolution images through 2x interpolation, serving as the initial curves for the evolving contours. Subsequently, the segmentation model continues to evolve and fine-tune the curves, moving step-by-step downwards, each time refining and adjusting the contours to fit the higher resolution image features, until the bottom layer of the pyramid, i.e., the original image resolution, is processed.
3.1 Top layer image processing
Compared to high-resolution original images, the edges in the top layer images are more precise and unaffected by internal brightness irregularities, which is particularly important for dynamic cardiac ultrasound images, as cardiac images often exhibit blurring and dynamic changes due to ongoing heart movement. In handling the lowest resolution image edges within the proposed Gaussian pyramid-based dynamic contour model, the process starts by creating a low-resolution image at the top of the Gaussian pyramid structure. After Gaussian filtering of this image, the image gradient function $\left|\nabla I^{\wedge}\right|$ is used to generate an edge map. This edge map provides a clear visual indication at a lower resolution, showing the main edge positions of the left ventricle. The obtained low-resolution edge map provides a stable starting point for the subsequent segmentation process, which then, through continuous refinement and interpolation steps, progressively restores to higher resolutions, constantly optimizing and adjusting the segmentation contours until the precision of the original image is reached.
The segmentation algorithm starts with the top layer image—the lowest resolution image in the Gaussian pyramid—using a simplified model to preliminarily identify the contours of the left ventricle. Specifically, this paper introduces the level set method, which uses an implicit function $\phi$ called a level set to represent curves or interfaces in the image, continually updated during the segmentation process. This step establishes the rough positions as the foundation for the entire segmentation process, providing a starting point for the next step of refinement. The update iteration formula is as follows:
$\begin{aligned} & \frac{\partial \theta}{\partial s}=\omega \cdot \sigma(\theta) \cdot D I\left(\frac{\nabla \theta}{|\nabla \theta|}\right)- \\ & \sigma(\theta) \cdot\left[\eta_1\left(U-z_{I N}\right)^2+\eta_2 U-z_{\text {OUT }}{ }^2\right]\end{aligned}$ (10)
To enhance the model's edge-capturing capability and address common issues of noise and blurring in cardiac ultrasound images, this method introduces a Gradient Vector Field (GVF) as an external force, represented by $N \nabla \theta$. GVF is based on edge information and can create an extensive capture domain, allowing the model to attract contours from a distance towards the true edges, not just those areas near the initial contours. Applying this external force on the lowest resolution top-layer image effectively pulls the initially identified contours towards more accurate edge positions, which is particularly important in dynamic images where the edges continuously change due to the beating of the heart. Assuming the positive constant is represented by $\varepsilon$, the calculation formula is:
$\begin{aligned} & \frac{\partial \theta}{\partial s}=\omega \cdot \sigma(\theta) \cdot D I\left(\frac{\nabla \theta}{|\nabla \theta|}\right)-\varepsilon \cdot N \cdot \nabla \theta \\ & -\sigma(\theta) \cdot\left[\eta_1\left(U-z_{I N}\right)^2+\eta_2\left(U-z_{\text {OUT }}\right)^2\right]\end{aligned}$ (11)
In practical applications, compared to the external force term $\nabla h \nabla \theta$ in the traditional Geodesic Active Contour (GAC) model, the newly introduced external force term $N \nabla \theta$ provides a broader range of edge attraction. This means that after incorporating this external force into the level set model, even if the initial contour setup is not close enough to the true edges, it can still be effectively attracted to the correct position. After processing the top layer image, refinement through each layer of the Gaussian pyramid uses the results from the previous layer as the initial condition for the next layer, gradually evolving towards higher resolution layers. This process, from coarse to fine segmentation, not only ensures accurate edge capture but also accommodates the complex needs of assessing left ventricular function in dynamic cardiac ultrasound images.
3.2 Lower layer image processing
In the task of dynamic cardiac ultrasound image segmentation aimed at assessing left ventricular function, processing the lower layer images based on the Gaussian pyramid structure requires adjusting the segmentation methods to accommodate the finer details and challenges of uneven brightness. The rough contours obtained from the top layer image are passed down to the next layer using a 2x interpolation method, by which time the contours are already closer to the true edges, but the lower layer images, due to higher resolution, display more details and local brightness irregularities. Therefore, the global dynamic contour model used on the top layer image is no longer suitable at this stage primarily because it is not fit for fine local adjustments and cannot handle uneven brightness issues. In processing the lower layer images, models more suited for local detail adjustments are typically used, such as edge-based segmentation models that employ local-specific edge guiding force fields, which can more effectively handle detail issues at higher resolutions and reduce computational burden.
In this specific task of dynamic cardiac ultrasound image segmentation, the GAC model is chosen to process the lower layer images of the pyramid. Similar to its traditional application, the GAC model primarily uses image gradient information to guide the evolution of contours in this application. However, directly using the GAC model in cardiac ultrasound images may lead to the contours evolving towards incorrect edges, especially in areas where the edges are weak or blurred. Therefore, for accurate contour tracking of the left ventricle, especially in high-resolution images, it is necessary to appropriately adjust the traditional GAC model to ensure that the contours can accurately capture the dynamic boundaries of the left ventricle. Assuming the gradient characteristic function on image $U(a, b)$ is $h(U)=1 / 1+\left|\nabla\left(H_\delta * U\right)\right|^2$, represented by $h$. The main force dragging the contour to the target edge in the GAC model is represented by the term $\nabla h \nabla \theta$, and the reconstructed GAC level set iteration equation is given by:
$\frac{\partial \theta}{\partial s}=h \cdot \chi \cdot|\nabla \theta|+\nabla h \cdot \nabla \theta$ (12)
To enhance segmentation precision and overcome difficulties caused by the inherent noise and edge blurring of ultrasound images, this method introduces a prior shape constraint energy in the lower layer image processing workflow. This energy setting is based on the observation of maintaining approximate contour consistency from the top to the bottom layers in the image pyramid. By introducing this shape energy in the level set form, a stable shape constraint can be provided for the contour during the evolution process, preventing the contour from deviating too far from the actual boundaries of the left ventricle. This step is particularly important as it helps the model overcome issues of mis-segmentation that may arise from solely relying on local gradient information of the image, especially given the high precision requirements for edge localization in dynamic assessment of left ventricular function. Assuming the sigmoid-type function $T(\theta)=r^\theta-r^{-\theta} / r^\theta+^{-\theta}$ is represented by $T(\theta)$, and the level set function interpolated from the previous layer of the pyramid is represented by $\theta_0$, the level set expression for this energy is:
$R_2=\frac{1}{2} \int_{\Psi}\left(T(\theta)-T\left(\theta_0\right)\right)^2 f a f b$ (13)
3.3 Lower layer image processing
In the segmentation process at the bottom layer of the pyramid, the final evolution of the contour is accomplished by combining several different influencing factors. Initially, the basic shape of the curve is controlled using $\int_{\Psi}|\nabla G(\theta)| d a d b$ to ensure the contour's smoothness and continuity. Following that, according to the GAC model, image gradient information is used to attract the contour to the target edge, refining the contour to the true boundaries of the left ventricle. Lastly, the prior shape energy function is minimized using the steepest descent method to further optimize the contour, ensuring it conforms not only to the image edge information but also to the pre-defined shape of the left ventricle. Assuming the weighting coefficients are represented by $\beta_1, \beta_2$, and $\beta_3$, the final iterative function for θ on the bottom layer of the pyramid is given by:
$\begin{aligned} & \frac{\partial \theta}{\partial s}=\beta_1 \cdot \sigma(\theta) \cdot D I\left(\frac{\nabla \theta}{|\nabla \theta|}\right)+\beta_2 \cdot \nabla h(\nabla U) \cdot \nabla \theta \\ & +\beta_3 \cdot\left(T\left(\theta_0\right)-T(\theta)\right) \cdot\left(1-T^2(\theta)\right)\end{aligned}$ (14)
In the task of dynamic cardiac ultrasound image segmentation for left ventricular function assessment, the mathematical implementation of the dynamic contour model based on the Gaussian pyramid addresses specific cardiac imaging challenges such as image blurring caused by continuous heart motion and indistinct left ventricular boundaries. To optimize model performance and ensure convergence within a multi-resolution framework, a penalization term $\left(\nabla^2 \theta-\nabla(\nabla \theta /|\nabla \theta|)\right)$ is used to avoid reinitialization of the level set function, helping to maintain the stable evolution of the contour without losing accuracy. In the numerical implementation, spatial derivatives $\partial \theta / \partial a, \partial \theta / \partial b$ are calculated using the central difference method to achieve more precise spatial location information, while the time derivative $\partial \theta / \partial s$ is realized through forward difference. These methods are chosen to accommodate the dynamic characteristics of cardiac ultrasound images, ensuring rapid and precise tracking of the left ventricular boundaries across different cardiac cycles. This mathematical approach provides a solid computational basis for efficient and reliable assessment of left ventricular function, adapting to the complex variations in left ventricular boundaries in a dynamic environment.
3.4 Left ventricular function assessment
In the diagnosis and treatment of heart failure patients, the assessment of left ventricular function is directly linked to measuring cardiac pumping efficiency. Using dynamic cardiac ultrasound imaging technology, dynamic functional and structural changes of the left ventricle can be visually observed. Utilizing high-precision dynamic cardiac ultrasound image segmentation results, the following steps can be implemented to accurately assess left ventricular function:
(1) Left Ventricular Volume Measurement: Initially, by accurately determining the left ventricular boundaries through image segmentation, the volume of the left ventricle can be calculated at different cardiac cycle stages. End-diastolic volume (EDV) and end-systolic volume (ESV) are crucial parameters in assessing left ventricular function.
(2) Ejection Fraction (EF) Calculation: The EF is a core metric for evaluating left ventricular pumping capability, calculated as: EF=(EDV-ESV)/EDV*100%. A low EF value typically indicates impaired cardiac pumping function and is one of the key indicators of heart failure.
(3) Wall Thickness and Motion Analysis: Through segmentation results, the thickness variations of the left ventricular wall during the cardiac cycle and the wall motion speed can also be measured, which helps to assess the functional state of the cardiac muscle and detect potential local muscle motion abnormalities.
(4) Other Parameter Analysis: Specifically including stroke volume, cardiac output, peak ejection rate, peak ejection time, peak filling rate, peak filling time, overall longitudinal strain, overall circumferential strain, global radial strain, etc.
(5) Regional Function Assessment: Using segmentation technology, the left ventricle can be divided into regions, analyzing volume changes and wall motion in each area to identify functional impairments in specific cardiac regions.
In implementing these assessment steps, it is necessary to consider the patient's specific circumstances, including the uniqueness of the cardiac structure and the patient's overall health condition.
Table 1. Evaluation metrics for single frame dynamic cardiac ultrasound images by different image enhancement algorithms
|
Algorithm |
Evaluation Metric |
Sequence One |
Sequence Two |
|
SRAD |
TMQI |
0.9685 |
0.9015 |
|
FSITM |
0.9456 |
0.9564 |
|
|
DTCWT |
TMQI |
0.9856 |
0.9682 |
|
FSITM |
0.9485 |
0.8893 |
|
|
BM3D |
TMQI |
0.9454 |
0.8124 |
|
FSITM |
0.9587 |
0.9426 |
|
|
Total Variation Denoising |
TMQI |
0.9782 |
0.9784 |
|
FSITM |
0.9562 |
0.9623 |
|
|
The Proposed Algorithm |
TMQI |
0.9863 |
0.9789 |
|
FSITM |
0.9784 |
0.9762 |
Figure 4. Normalized mean change curves of images by different algorithms
In Table 1, we can observe the performance of different image enhancement algorithms on single frame images from dynamic cardiac ultrasound image sets. From the evaluation metrics TMQI and FSITM, it is evident that the algorithm proposed in this paper demonstrates superior performance on both sequences. Specifically, for Sequence One and Sequence Two, the TMQI scores for the proposed algorithm are 0.9863 and 0.9789 respectively, and FSITM scores are 0.9784 and 0.9762 respectively. Compared to other algorithms, such as SRAD, DTCWT, BM3D, and Total Variation Denoising, our algorithm not only performs best overall in TMQI scores but also shows higher image similarity and detail retention capability in FSITM scores. Notably, BM3D shows significantly lower TMQI scores on Sequence Two, only reaching 0.8124. The analysis clearly indicates that the dynamic cardiac ultrasound image detail enhancement algorithm based on visual attention mechanisms and GAN proposed in this paper has significant advantages in terms of detail clarity and image authenticity. By effectively enhancing image quality, this algorithm can better reveal cardiac functional status, which is crucial for accurately assessing left ventricular function in heart failure patients. Moreover, the high TMQI and FSITM scores of this algorithm suggest that its superior performance is not limited to image enhancement but also in maintaining original image features and enhancing image comparability, which is particularly important for clinical diagnosis and subsequent image analysis.
From Figure 4, it is apparent that different image processing algorithms have distinct effects on enhancing dynamic cardiac ultrasound images. We compared the normalized mean change curve data for algorithms such as SRAD, DTCWT, BM3D, Total Variation Denoising, and the algorithm proposed in this paper based on visual attention mechanisms and GAN. It is evident from the figure that the SRAD algorithm shows higher normalized means in most sequences, indicating a more pronounced enhancement effect, especially in the middle and later parts of the sequence, where it outperforms other algorithms. However, SRAD's mean fluctuates significantly, which may indicate unstable enhancement effects. DTCWT and BM3D demonstrate certain enhancement capabilities during processing but generally have lower means compared to SRAD and Total Variation Denoising. The Total Variation Denoising algorithm exhibits a more stable and consistent enhancement effect throughout the sequence, though its enhancement magnitude is not as pronounced as SRAD's. Compared to the aforementioned algorithms, the proposed algorithm typically has lower normalized means but shows smaller fluctuations and higher consistency. This is because the proposed algorithm employs visual attention mechanisms and GAN, which not only help maintain the naturalness and true representation of image details but also effectively avoid excessive enhancement that can lead to loss of detail. While numerically it might seem inferior to algorithms like SRAD, in reality, the proposed algorithm places a greater emphasis on enhancing image quality while maintaining the natural feel and diagnostically relevant details of the images. It can be concluded that although the proposed algorithm does not achieve the highest in normalized mean, it has a distinct advantage in terms of detail preservation and naturalness. By intelligently identifying and enhancing key areas of cardiac structure, the algorithm can better reveal the cardiac functional status.
Table 2 details the comparison of left ventricular function parameters between heart failure patients and healthy groups obtained through dynamic cardiac ultrasound imaging feature tracking technology. The table reveals significant statistical differences between the two groups in several key indicators. Particularly, the ESV is significantly higher in the heart failure group compared to the healthy group (57.15±18.32 ml vs 45.23±9.84 ml, P=0.0004), indicating impaired ventricular contraction function in heart failure patients. The EF also shows significant differences between the groups, with lower EF values in heart failure patients (55.23±9.47% vs 61.23±5.47%, P=0.001), further confirming reduced left ventricular pumping efficiency in heart failure. Additionally, global radial strain and circumferential strain are significantly lower than in the healthy group, suggesting decreased contractile force and deformability of ventricular myocardial fibers in heart failure patients.
Table 2. Left ventricular measurements: Dynamic cardiac ultrasound imaging feature tracking parameters
|
Parameter |
Heart Failure Group n=28 |
Healthy Group n=26 |
F/Z-value |
P-value |
|
EDV |
126.32±25.23 |
121.15±15.23 |
1.045t |
0.300 |
|
ESV |
57.15±18.32 |
45.23±9.84 |
3.023t |
0.0004* |
|
Stroke volume |
67.23(19.23) |
73.23(17.23) |
-2.235u |
0.023* |
|
Cardiac output |
4.72(1.40) |
5.32(1.00) |
-1.542u |
0.124 |
|
EF |
55.23±9.47 |
61.23±5.47 |
-3.326u |
0.001** |
|
Peak ejection rate |
426.23±104.15 |
478.23±104.23 |
-1.325t |
0.189 |
|
Peak ejection time |
112.48(37.00) |
112.48(81.00) |
-0.875t |
0.365 |
|
Peak filling rate |
398.23±99.58 |
436.12±114.58 |
-1.895u |
0.074 |
|
Peak filling time |
165.23(48.14) |
165.23(92.14) |
-0.068t |
0.936 |
|
PER /EDV |
3.36±0.77 |
3.89±0.72 |
-2.134u |
0.036* |
|
PER /EDV |
3.01±0.48 |
3.56±0.71 |
-3.216t |
0.021* |
|
Overall longitudinal strain |
-21.25(3.75) |
-21.25(3.65) |
-0.612u |
0.534 |
|
Overall circumferential strain |
-22.13±3.74 |
-24.13±2.74 |
4.045t |
<0.001** |
|
Global radial strain |
46.33±16.14 |
61.23±15.48 |
-2.895t |
0.005** |
Table 3. Correlation analysis between left ventricular function parameters and dynamic cardiac ultrasound image processing metrics in heart failure patients
|
|
Stroke Volume |
EF |
PFR/EDV |
PFR/EDV |
||||
|
r-value |
p-value |
r-value |
p-value |
r-value |
p-value |
r-value |
p-value |
|
|
Gradient Magnitude |
0.164 |
0.389 |
-0.134 |
0.445 |
0.224 |
0.224 |
-0.156 |
0.389 |
|
Edge Response |
-0.365 |
0.052 |
0.289 |
0.124 |
0.204 |
0.278 |
0.157 |
0.421 |
|
Edge Direction |
-0.312 |
0.114 |
-0.081 |
0.658 |
-0.073 |
0.705 |
-0.136 |
0.448 |
|
Edge Coherence |
-0.041 |
0.826 |
0.009 |
0.968 |
0.105 |
0.595 |
-0.221 |
0.236 |
|
Histogram Equalization Effect |
-0.325 |
0.091 |
-0.156 |
0.389 |
-0.154 |
0.415 |
0.061 |
0.754 |
|
Sharpening Index |
-0.369 |
0.052 |
-0.101 |
0.623 |
-0.071 |
0.725 |
0.078 |
0.658 |
|
Texture Heterogeneity |
-0.231 |
0.214 |
0.024 |
0.901 |
-0.135 |
0.456 |
0.072 |
0.721 |
|
Local Binary Pattern (LBP) Statistics |
-0.201 |
0.289 |
-0.028 |
0.874 |
-0.048 |
0.815 |
-0.105 |
0.618 |
|
Contour Change Rate |
-0.167 |
0.369 |
-0.256 |
0.178 |
-0.214 |
0.279 |
-0.389 |
0.041 |
|
Regional Expansion/Contraction Speed |
-0.189 |
0.324 |
0.189 |
0.356 |
-0.135 |
0.456 |
0.157 |
0.378 |
|
Shape Stability |
0.052 |
0.789 |
-0.033 |
0.854 |
-0.109 |
0.589 |
0.047 |
0.815 |
Table 4. Correlation analysis between left ventricular myocardial strain and dynamic cardiac ultrasound image processing metrics in heart failure patients
|
|
Overall Circumferential Strain |
Global Radial Strain |
||
|
|
r-value |
p-value |
r-value |
p-value |
|
Gradient Magnitude |
-0.046 |
0.811 |
0.017 |
0.921 |
|
Edge Response |
-0.489 |
0.007** |
0.189 |
0.325 |
|
Edge Direction |
0.136 |
0.478 |
-0.178 |
0.336 |
|
Edge Coherence |
0.154 |
0.426 |
-0.189 |
0.354 |
|
Histogram Equalization Effect |
0.000 |
0.989 |
0.002 |
0.987 |
|
Sharpening Index |
-0.124 |
0.51 |
-0.178 |
0.314 |
|
Texture Heterogeneity |
0.002 |
0.987 |
0.087 |
0.658 |
|
LBP Statistics |
0.128 |
0.521 |
-0.136 |
0.478 |
|
Contour Change Rate |
0.325 |
0.077 |
0.005 |
0.987 |
|
Regional Expansion/Contraction Speed |
-0.009 |
0.985 |
0.289 |
0.158 |
|
Shape Stability |
-0.084 |
0.658 |
-0.224 |
0.235 |
Table 5. Intra-and inter-observer variability in dynamic cardiac ultrasound
|
|
Intra-observer (n=12) ICC |
95% Confidence Interval |
Inter-observer (n=12) ICC |
95% Confidence Interval |
|
Global Longitudinal Strain |
0.915 |
0.702-0.985 |
0.856 |
0.548-0.968 |
|
Global Circumferential Strain |
0.925 |
0.723-0.978 |
0.925 |
0.756-0.978 |
|
Global Radial Strain |
0.935 |
0.789-0.989 |
0.936 |
0.806-0.987 |
Table 3 provides the results of the correlation analysis between left ventricular function parameters and dynamic cardiac ultrasound image processing metrics in heart failure patients. The table reveals that most image processing metrics do not show significant correlation with left ventricular function parameters, indicating that the relationship between cardiac function parameters and single image processing metrics might be complex, requiring a comprehensive evaluation using multiple metrics. For example, the correlation coefficient between edge response and EF is 0.289, although not statistically significant (p=0.124), the positive correlation suggests that enhanced edge response may be associated with better cardiac function status. Furthermore, the correlation coefficient for contour change rate with the last column metric (possibly a type of functional ratio or change rate) is -0.389, reaching statistical significance (p=0.041), suggesting that an increase in contour change rate might be associated with a decline in certain cardiac function parameters.
Table 4 presents the correlation analysis between left ventricular myocardial strain and dynamic cardiac ultrasound image processing metrics in heart failure patients. The analysis indicates that most image processing metrics show weak correlations with myocardial strain, displaying no strong statistical significance. However, it is noteworthy that edge response shows a strong negative correlation with overall circumferential strain (r-value=-0.489, p-value=0.007**), suggesting that enhanced edge response is associated with decreased circumferential strain, potentially indicating impaired myocardial function. Additionally, regional expansion/contraction speed, although not reaching statistical significance (p-value=0.158), shows a correlation coefficient (r-value=0.289) suggesting that increased expansion/contraction speed may be related to improvements in radial strain.
These findings highlight the potential application of the dynamic cardiac ultrasound image processing technology proposed in this paper in cardiac function assessment. Through detailed enhancement and precise image segmentation, this technology not only improves image clarity and dynamic representation but also enhances the ability to measure left ventricular function parameters, particularly in myocardial strain assessment. Although most image processing metrics do not show strong direct correlations with myocardial strain, this technology, by providing clearer and more accurate images, enables clinicians to interpret cardiac functional status and pathological changes more effectively.
Table 5 provides data on intra- and inter-observer variability in dynamic cardiac ultrasound imaging, showing the intraclass correlation coefficients (ICCs) and their 95% confidence intervals. The results demonstrate very high intra-observer consistency with ICCs of 0.915, 0.925, and 0.935 for global longitudinal, circumferential, and radial strain, respectively, ensuring the repeatability of measurements. The corresponding 95% confidence intervals further confirm the reliability of these estimates. Similarly, inter-observer consistency is also high, especially for global circumferential and radial strain, with ICCs of 0.925 and 0.936, respectively, and confidence intervals of 0.756-0.978 and 0.806-0.987, showing the consistency of evaluations among different observers. High observer consistency ensures the reliable application of these techniques in clinical settings, reducing variability between operators and making cardiac function assessment more accurate and consistent.
This paper introduces a novel dynamic cardiac ultrasound image processing technology, which by integrating visual attention mechanisms and GAN to enhance image details, and using dynamic contour models to improve image segmentation and assessment of left ventricular function. The main objective is to enhance image dynamic representation and measurement precision, thereby more accurately revealing cardiac functional status.
Experimentally, by comparing dynamic cardiac ultrasound images processed by different image enhancement algorithms, it has been found that the algorithm proposed in this paper, based on visual attention mechanisms and GAN, offers significant advantages in terms of detail clarity and dynamic representation. Through dynamic cardiac ultrasound image feature tracking analysis, this technology can effectively measure and distinguish left ventricular function parameters between heart failure patients and healthy individuals, demonstrating higher accuracy and consistency than traditional methods. Correlation analysis further validates the connection between image processing metrics and left ventricular function parameters in heart failure patients. Although the correlation for some metrics is not significant, key metrics such as edge response and contour change rate show clear correlations. By assessing intra- and inter-observer consistency, the reliability of this technology and its potential for clinical application have been confirmed.
The dynamic cardiac ultrasound image processing technology introduced in this study significantly enhances the detail representation and measurement precision of the images, providing a powerful tool for the diagnosis and treatment monitoring of heart failure patients. By improving image quality and assessment accuracy, the technology can more effectively reveal subtle changes in cardiac function, providing crucial decision support for clinicians.
Despite achieving a range of positive results, there are some limitations to this study. For example, the correlation between some image processing metrics and cardiac function parameters is not significant enough; future research could overcome this issue by integrating more image features and conducting deeper analysis. Additionally, the sample size of the study is relatively small, and future research should validate the effectiveness and reliability of these techniques in a broader patient population. Future studies could also explore applying these techniques to other types of heart diseases and further optimizing the algorithms to suit different clinical settings and needs, thereby comprehensively enhancing the diagnostic and therapeutic value of cardiac ultrasound imaging.
This paper was funded by the Scientific Research Project of Wuhan Municipal Health Commission (Grant No.: WZ21B01).
[1] Phan, P.T., Davies, J., Hoang, T.T., Thai, M.T., Nguyen, C.C., Ji, A., Zhu, K., Sharma, B., Nicotra, E., Hayward, C., Phan, H., Lovell, N.H., Do, T.N. (2024). Robotic cardiac compression device using artificial muscle filaments for the treatment of heart failure. Advanced Intelligent Systems, 6(3): 2300464. https://doi.org/10.1002/aisy.202300464
[2] Alarcón-Vieco, E., Martínez-García, I., Sequí-Domínguez, I., Rodríguez-Gutiérrez, E., Moreno-Herráiz, N., Pascual-Morena, C. (2023). Effect of coenzyme Q10 on cardiac function and survival in heart failure: An overview of systematic reviews and meta-analyses. Food & Function. 14: 6302-6311. https://doi.org/10.1039/D3FO01255G
[3] Hvid, R., Stuart, M.B., Jensen, J.A., Traberg, M.S. (2023). Intra-cardiac flow from geometry prescribed computational fluid dynamics: Comparison with ultrasound vector flow imaging. Cardiovascular Engineering and Technology, 14(4): 489-504. https://doi.org/10.1007/s13239-023-00666-2
[4] Green, L., Chan, W.X., Ren, M., Mattar, C.N.Z., Lee, L.C., Yap, C.H. (2023). The dependency of fetal left ventricular biomechanics function on myocardium helix angle configuration. Biomechanics and Modeling in Mechanobiology, 22(2): 629-643. https://doi.org/10.1007/s10237-022-01669-z
[5] Li, Y.Y., Craft, J., Cheng, Y., Schapiro, W., Gliganic, K., Haag, E., Cao, J.J. (2022). Optical flow analysis of left ventricle wall motion with real-time cardiac magnetic resonance imaging in healthy subjects and heart failure patients. Annals of Biomedical Engineering, 50(2): 195-210. https://doi.org/10.1007/s10439-022-02907-2
[6] Strohm, E.M., Callaghan, N.I., Ding, Y., Latifi, N., Rafatian, N., Funakoshi, S., Fernandes, I., Reitz, C.J., Di Paola, M., Gramolini, A.O., Radisic, M., Keller, G., Kolios, M.C., and Simmons, C.A. (2023). Noninvasive quantification of contractile dynamics in cardiac cells, spheroids, and organs-on-a-chip using high-frequency ultrasound. ACS Nano, 18(1): 314-327. https://doi.org/10.1021/acsnano.3c06325
[7] Zhang, Q., Samani, A., Peters, T.M. (2021). Image dynamic visualization and synchronization over Internet for distributed heart function diagnosis. Computerized Medical Imaging and Graphics, 88: 101850. https://doi.org/10.1016/j.compmedimag.2020.101850
[8] Okhovatian, S., Mohammadi, M.H., Rafatian, N., Radisic, M. (2022). Engineering models of the heart left ventricle. ACS Biomaterials Science & Engineering, 8(6): 2144-2160. https://doi.org/10.1021/acsbiomaterials.1c00636
[9] Han, L., Yang, Y. (2022). Image processing and imaging technology in cardiac medical surgery. Scientific Programming, 2022: 1-6. https://doi.org/10.1155/2022/3977582
[10] Ren, C., Liu, W. (2021). Cardiac ultrasound under speckle tracking technology based analysis of efficacy of respiratory rehabilitation on chronic obstructive pulmonary disease. Journal of Healthcare Engineering, 2021: 1-8. https://doi.org/10.1155/2021/7569908
[11] dos Santos, D.S., Ossenkoppele, B., Hopf, Y.M., Soozande, M., Noothout, E., Vos, H.J., Bosch, J.G., Pertijs, M.A.P., Verweij, M.D., de Jong, N. (2024). An ultrasound matrix transducer for high-frame-rate 3-D intra-cardiac echocardiography. Ultrasound in Medicine & Biology, 50(2): 285-294. https://doi.org/10.1016/j.ultrasmedbio.2023.11.001
[12] Bian, P., Zhang, X., Liu, R., Li, H., Zhang, Q., Dai, B. (2021). Deep-learning-based color doppler ultrasound image feature in the diagnosis of elderly patients with chronic heart failure complicated with sarcopenia. Journal of Healthcare Engineering, 2021: 1-8. https://doi.org/10.1155/2021/2603842
[13] Puvanasunthararajah, S., Camps, S.M., Wille, M.L., Fontanarosa, D. (2022). Combined clustered scan-based metal artifact reduction algorithm (CCS-MAR) for ultrasound-guided cardiac radioablation. Physical and Engineering Sciences in Medicine, 45(4): 1273-1287. https://doi.org/10.1007/s13246-022-01192-6
[14] Rindal, O.M.H., Bjåstad, T.G., Espeland, T., Berg, E.A.R., Måsøy, S.E. (2023). A very large cardiac channel data database (VLCD) used to evaluate global image coherence (GIC) as an in vivo image quality metric. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control. 70(10): 1295-1307. https://doi.org/10.1109/TUFFC.2023.3308034
[15] Bian, P., Zhang, X., Liu, R., Li, H., Zhang, Q., Dai, B. (2021). Deep-learning-based color doppler ultrasound image feature in the diagnosis of elderly patients with chronic heart failure complicated with sarcopenia. Journal of Healthcare Engineering, 2021: 1-8. https://doi.org/10.1155/2021/2603842
[16] Bianchini, E., Guala, A., Golemati, S., Alastruey, J., Climie, R.E., Dalakleidi, K., Francesconi, M., Fuchs, D., Hartman, Y., Malik, A.E.F., Makūnaitė, M., Nikita, K.S., Park, C., Pugh, C.J.A., Šatrauskienė, A., Terentes-Printizios, D., Teynor, A., Thijssen, D., Schmidt-Trucksäss, A., Zupkauskienė, J., Boutouyrie, P., Bruno, R.M., Reesink, K.D. (2023). The ultrasound window into vascular ageing: A technology review by the VascAgeNet COST action. Journal of Ultrasound in Medicine, 42(10): 2183-2213. https://doi.org/10.1002/jum.16243
[17] Xie, Y., Xie, Y., Chen, Y., Deng, C., Fang, L., Li, H., Sun, Z., Gao, T., Liu, J., Yuan, J., Li, Y., Jin, Q., Zhang, L., Xie, M. (2021). Ultrasound imaging of treatment-needed cardiac rejection with CD4-Targeted nanobubbles. Particle & Particle Systems Characterization, 38(8): 2100091. https://doi.org/10.1002/ppsc.202100091
[18] Caenen, A., Pernot, M., Nightingale, K.R., Voigt, J.U., Vos, H.J., Segers, P., D'Hooge, J. (2022). Assessing cardiac stiffness using ultrasound shear wave elastography. Physics in Medicine & Biology, 67(2): 02TR01. https://doi.org/10.1088/1361-6560/ac404d
