S. Hrushikesava Raju*
| S. Adinarayna | Vijaya Chandra Jadala | K. Yogeswara Rao | U. Sesadri | Y. Sreeraman
© 2025 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
Most traditional and semi-traditional methods consume time in the evaluation of flyover/bridge defects, and they incur aggregated costs for inspection checks. In recent cases, due to natural disasters such as floods and earthquakes, flyovers have collapsed because of a lack of strict monitoring using the latest methods and technologies. The kinds of defects that occur during flyover life are cracks, spalling, corrosion, weakening in structure, deformations in structural design, poor drainage, etc. To extend the life of a flyover/bridge, early detection of initial defects would avoid major damage. To handle detection efficiently, this paper proposes a hybrid deep learning model that integrates Faster Region-based Convolutional Neural Network (R-CNN), Long Short-Term Memory (LSTM), and Transfer Learning, along with IoT sensors. In this model, Faster R-CNN detects and localizes multiple defects in a single frame, LSTM processes temporal sequences of sensor data and predicts deterioration trends, and Transfer Learning uses MobileNetV3 for reduction in power consumption and latency, where pre-trained weights are used to enhance performance with limited annotated data. The specific sensors used are strain gauges (for load information), accelerometers (for inspecting vibrations), and temperature sensors (for monitoring materials contraction and expansion —which cause structural issues—and for continuous health monitoring). The defined hybrid model would enhance structural health monitoring using drones for image capture, sensors for measuring the health of specific regions, and high precision defect localization, it will also extend the infrastructure life by alerts and follow-up process adoption. This hybrid model ensures the longevity and safety of flyovers. The proposed hybrid approach is compared with existing models such as individual Convolutional Neural Networks (CNNs), Support Vector Machines (SVMs), Recurrent Neural Networks (RNNs), Random Forest, Transfer Learning, and Generative Adversarial Network (GANs), ultimately contributing to the extension of the flyover's lifespan.
defects, faster Region-based Convolutional Neural Network (R-CNN), Long Short-Term Memory (LSTM), Transfer Learning, health monitoring, sensors, informed decisions, accuracy
Many reasons and factors would impact the lifetime of the flyover. The civil project flyover construction with specific additions may result in a better life for the flyover. Flyovers, like any other infrastructure, can experience various types of defects that may compromise their structural integrity, safety, and functionality. Some common defects that can occur in flyovers are material deterioration, corrosion, spalling, settlement, erosion, joint and expansion joints, structural deformations, poor drainage, and insufficient maintenance.
The first significant issue is that cracks can develop on the concrete or asphalt surfaces of flyovers due to factors such as shrinkage, temperature fluctuations, excessive loads, or inadequate reinforcement. These cracks can range in size and severity, from superficial surface cracks to deeper structural cracks.
The second issue is spalling, which refers to the chipping, flaking, or breaking away of concrete or other construction materials from the surface of the flyover. It can occur due to factors such as corrosion of reinforcing steel, freeze-thaw cycles, or chemical reactions.
The third issue is that corrosion affects the reinforcing steel embedded within the concrete components of a flyover. Exposure to moisture, chloride ions (e.g., from deicing salts), or other environmental factors can lead to corrosion. Corrosion weakens the steel, causing cracks, spalling, and a reduction in the structural strength of the flyover.
The fourth issue is that settlement occurs when the flyover experiences uneven or excessive sinking of its foundation or supporting structure. It can occur due to factors such as soil consolidation, inadequate foundation design, or variations in soil conditions. Settlement can result in misalignment, increased stresses, and potential instability of the flyover.
The fifth issue is erosion, through which flyovers located near water bodies or areas with poor drainage systems are susceptible to erosion. Inadequate erosion control measures and water flow can gradually erode the soil beneath the flyover, compromising its foundation and stability over time.
The sixth issue is Joint and Expansion denotes Joint Issues through which flyovers often incorporate joints or expansion joints to accommodate structural movements caused by temperature variations or deformations. Issues can arise from joint failure, displacement, or insufficient maintenance, leading to a rough ride quality, increased noise, and reduced structural performance.
The seventh issue is structural deformations. Deformations can occur in flyovers due to factors such as excessive loading, settlement, foundation issues, or inadequate structural design. These deformations may manifest as excessive bending, twisting, or sagging of structural elements, compromising the stability and load-carrying capacity of the flyover.
The eighth issue identified is poor Drainage, through which inadequate or malfunctioning drainage systems can result in the accumulation of water on the surface of the flyover. Prolonged exposure to water can lead to increased deterioration, including corrosion, spalling, or erosion of the flyover's materials.
The ninth issue is insufficient maintenance, through which neglecting regular maintenance, inspections, and timely repairs can exacerbate existing defects and contribute to the overall deterioration of the flyover. Regular maintenance of activities, including inspections and prompt repairs, is essential for preserving the structural integrity and ensuring the safety of flyovers.
Addressing these defects requires thorough inspection, accurate assessment, and appropriate repair or rehabilitation measures. Regular inspections, adherence to quality construction practices, and proactive maintenance protocols are crucial for preventing defects and ensuring the long-term safety and functionality of flyovers. The deep learning approaches have proven to be effective in detecting defects during the construction of flyovers. Various deep learning techniques are employed for this purpose, ensuring accurate and reliable defect detection. The various deep learning techniques used are Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Generative Adversarial Network (GANs), Transfer Learning, Object detection methods, ensemble methods, etc.
The first approach is that CNNs are widely utilized for image-based defect detection. By training a CNN on a diverse dataset of annotated images showcasing different types of defects in flyover construction, the model can learn to identify and classify defects with a high level of accuracy. This includes detecting cracks, spalling, corrosion, and other visual irregularities observed in images or videos of the construction process.
The second approach is that RNNs are particularly suitable for analyzing sequential data, such as time-lapse videos or sensor data collected during flyover construction. RNNs can capture temporal dependencies and identify anomalies or deviations from expected patterns. They play a significant role in monitoring the construction process and detecting any irregularities or defects that may arise.
The third method is GANs consist of a generator network and a discriminator network. They are useful for generating synthetic samples that closely resemble normal flyover construction, while the discriminator network learns to differentiate between real and synthetic samples. By comparing the generated samples with real construction data, GANs can identify discrepancies and potential defects in the construction process.
The fourth approach is Transfer Learning, which involves utilizing pre-trained deep learning models on large datasets and fine-tuning them for the specific task of defect detection in flyover construction. By leveraging knowledge learned from a related domain, such as general object recognition, Transfer Learning can expedite model training and enhance detection accuracy for flyover construction images.
The fifth detection method is the object detection models, such as Faster Region-based Convolutional Neural Network (R-CNN), Single Shot Multibox Detector (SSD), or YOLO, which are employed to detect and localize specific defects in the images of flyover construction. These models excel at identifying the presence and precise location of defects, enabling targeted inspection and remediation efforts.
The sixth approach is ensemble methods combine multiple deep learning models to improve overall performance. By aggregating predictions from multiple models, ensemble methods enhance accuracy and robustness in defect detection for flyover construction.
Table 1. Considered methods in Flyover defect detection
|
Method |
Description |
Advantages |
Limitations |
|
Faster RNN |
Objects are detected using the region proposal concept with an RNN. |
Effective regions' defect identification and ensuring accuracy. |
Large datasets to be trained, and computation-intensive. |
|
Transfer Learning |
Pre-trained models are used for efficient feature extraction. |
Improves performance and reduces the training time. |
Requires fine-tuning for specific diseases. |
|
Traditional CNN |
For classification, CNN is preferred. |
Basic feature extraction is simple using this. |
Accuracy is low when compared with advanced models. |
|
SVM (Baseline) |
SVM is preferred for the classification of defects. |
Supports small datasets and is interpretable. |
Performance Issues with large, complex data. |
|
YOLO (Comparison) |
For the detection of objects, YOLO is preferred. |
Suitable for real-time applications with fast inference. |
Accuracy issues for occluded defects. |
|
Faster R-CNN |
For the detection of regions of defects, CNN is preferred. |
Localization accuracy is better than YOLO. |
Higher resource consumption and slower than YOLO. |
|
Proposed Method (Faster R-CNN, LSTM, and Transfer Learning) |
Faster R-CNN+LSTM+ Transfer Learning for defect detection. |
Ensures speed & accuracy. |
GPU acceleration is required for fine-tuning. |
From Table 1, to assess the effectiveness of the models, one factor considered is the accuracy for the methods, such as the Proposed method (Faster R-CNN + LSTM + Transfer Learning) outperforms SVM, YOLO, and traditional CNN. The second factor is the speed at which YOLO is faster, but the proposed method offers a better speed-accuracy trade-off. The third factor is Data Efficiency, in which Transfer Learning reduces dependency on large, labeled datasets. The combination of Faster R-CNN, LSTM, and Transfer Learning involves handling very small defects as well as YOLO, SSD, with the use of pixel-level precision, determines the defect growth and fatigue life using LSTM, and addresses the varying lighting, textures using pre-trained weights, as well as processes the reduced the needed data.
There are many advancements in bridge or flyover defect detection. The most traditional methods, which experience human error, are time-consuming and labor-intensive. To address these issues, an automatic defect detection technique is needed, which makes use of deep learning, computer vision, and machine learning models. The significant research in CNNs, R-CNNs, and Transfer Learning would stimulate further research and advancement in providing better solutions. The demonstration of solutions is described in this context as follows: Li et al. [1] discussed the accuracy degradation using deep neural networks under varying texture and lighting. To improve adaptability and accuracy, a multi-branch attention mechanism with deformable convolution. The proposed model uses the YOLOv8 architecture that enhances speed and ensures high precision in the results. Teng et al. [2] discussed the defects or cracks in the bridge surface, using YOLOv2, YOLOv3, and Faster R-CNN for faster speed and automatically. Among these, YOLOv3 was identified as the best accuracy, speed, and precision provider to other methods. It makes use of a Variant of Gaussian, such as the White Gaussian ratio, in the detection process. Xu et al. [3] discussed the issues such as the defects missing, a lack of accuracy, and the presence of false positives. To overcome these challenges, YOLOv8s is preferred with the help of ODConv in layer, 4D kernel usage, CBAM for channel attention mechanisms and spatial features, and CARAFE as an up-sampling for feature reconstruction, and reducing blurs. Rishitha et al. [4] discussed the bridge health using IoT, traditional methods, and a CNN deep learning model. The bridge's health is monitored and assessed in terms of deterioration and aging attributes. This study provides information about the bridge to the society of people and military personnel to know the safety of the environment. Li et al. [5] discussed the severity of defects in the bridges using an auto-encoder approach. Based on the vehicle's passage over the bridge and its intensity of vibrations, to determine the severity of the defect. In this, significant activities like noises are noted using the frequency domain, damage indicators are extracted, and the bridge health is predicted using damaged indicators. According to the research by Qiu et al. [6], the issues of multiscale bridges, leakages n small bridges, and lack of detection accuracy are overcome by a combination of Bi-directional pyramid feature network, Convolutional Block Attention Module (CBAM), and decoupled head with YOLO head would improve detection accuracy in the classification of defects.
Fan et al. [7] discussed the issues raised over long-span bridges, using anomaly detection methods. There are various approaches, like monitoring, inspection, and analysis. During assessment, the dynamic and static structural features are extracted, which state the health condition of the bridge. Gan et al. [8] discussed the various cracks to be identified at the bottom of the bridge, and use faster R-CNN and BIM for processing and classification. The UAVs are used for a close assessment of automatic bottom bridge crack detection. This model's accuracy and recall are evaluated. Karluklu et al. [9] demonstrated the issues of bridges during non-maintenance, and are monitored using drones, computer vision, and a machine learning approach with a color detection technique. This study explored especially steel bridges or iron bridges, where defect information is extracted, then passed to Building Information Modeling (BIM) for assessment. As reported by Li et al. [10], the abnormalities are detected using a hybrid model of LSTM, Expectation-Maximization Gaussian Mixture Model (EM-GMM) in which the former assesses the forces, depletions, and the latter approach for the number of clusters, and predicts each cluster's probability of random error influence. The accuracy of this model is compared against existing methods and applications. Santos-Vila et al. [11] discussed the real-time monitoring of the bridge, which avoids future expensive maintenance costs and ensures safety. The model used would use machine learning and signal processing concepts to eliminate labelled data and allow training over structural details, which define the current state of the bridge. Miao et al. [12] discussed the bridge performance assessment and possible identification of deterioration points. The used hybrid model involves three methods such as temporal pattern attention, Hidden Markov model, and LSTM. The measures, such as accuracy, are improved, and bridges under 30 years are inspected; better results are obtained. Yamane et al. [13] demonstrated the two methods for the extraction of images, and improve accuracy by asking questions, with reasons. Those 2 methods are Structure from Motion (SfM), and Visual Question Answering (VQA). The set of questions asked, and analyzing the cause, then estimates would achieve better accuracy in defect detection as well as lead to minimum maintenance rather than sudden, expensive budget. Amirkhani et al. [14] addressed various deep learning models and vision-based models to classify and detect bridge defects from popular databases. The models represented in this study with benefits as well as limitations. The analysis of these models helps to design an effective integrated model for classification and detection. Hou et al. [15] discussed the issues in the detection of bridge surface defects, and improve accuracy using a deep learning model U2 Network with ResNet U blocks, and detect precise contours over the bridge using 3D point cloud. This study is limited to underwater bridge surface defect detection. Cui et al. [16] discussed techniques involved in monitoring and detection for cracks, and their severity, and their severity, and their severity, and bolts loosening, etc., are determined, and analyzed. This helps to detect early, and minimize the loss of damage, and avoid expensive budgets. Wang et al. [17] discussed issues and challenges in detecting the defects while monitoring of railway bridges in terms of three approaches. First, a combination of fiber optic sensing and a computer vision approach for effective monitoring. Second, a combination of machine learning such as CNN, and Transfer Learning for efficient defect detection. Third, a combination of real-time applications and digital twins is used for monitoring. The challenges are addressed while using these approaches.
Wu and Huang [18] discussed the challenges raised during monitoring of the bridges and tunnels. The benefits are highlighted when the defects are identified early. The damage identification methods, sensing technologies, and monitoring approaches would extend the life of the bridges. This study also helps to search for new solutions to overcome deformations of the bridge. Zhang et al. [19] demonstrated various techniques such as sensing approaches, devices, wireless transmission models, and alerting on noise or irregularity using data preprocessing and warning models. The challenges are addressed and given for future research refinements. Chakurkar et al. [20] found defects and cracks on roads using conventional computer vision methods and deep learning models. The advantages of deep learning techniques are highlighted against the traditional models. The roads with the texture of cement, asphalt require extensive research for future directions. Shen et al. [21] demonstrated weak structures in bridges and tunnels using various means of deformation detection techniques such as stochastic models, change detection in ground deformation, and most civilian public rock mountains. The summary of benefits, pitfalls, and their applications is discussed in this study. In regard to reference [22], it ensures resource optimization by maintaining local and global stability and achieves predictive load balancing using a variant of Particle Swarm Optimization. Based on reference [23], optimize the cloud data centers using a hybrid model as well as introduce a novel performance evaluation approach for balancing the load over the network. Based on reference [24], ensures accuracy and security in digital transactions using a hybrid model to identify fraudulent transactions in a digital era, and optimizes the accuracy as well as performance. The studies mentioned would stimulate a novel approach to be designed, and ensure better accuracy and robustness. Table 2 demonstrates the significant studies, and listed gaps along with methodologies applied in those.
Table 2. Significant methods in the defect detection
|
Reference Study |
Methodology |
Gap Identified |
|
Li et al. [1] |
Multi-branch attention mechanism with YOLOv8 addresses the accuracy reduction for varying textures. |
Limitations on nightmare conditions and not integrating with sensor data. |
|
Teng et al. [2] |
Uses YOLOv2, YOLOv3, and Faster R-CNN for crack detection. |
Failed to detect temporal defects and micro defects. |
|
Xu et al. [3] |
Uses a combination of OLOv8s with ODConv, CBAM, and CARAFE to reduce false positives and missed defects. |
Time-consuming, and failed to detect multi-scale defects. |
|
Rishitha et al. [4] |
Uses IoT and CNN for the identification of the aging of defects. |
Scarcity of Large labelled datasets. |
|
Li et al. [5] |
Uses autoencoders for defect identification over vehicle-induced vibrations. |
Frequency domain methods failed to address, and vibration data alone failed to assess defect severity. |
In the proposed model, the integration of Faster R-CNN, LSTM, and Transfer Learning would involve the key aspects such as identifying the defects like spalling, cracks, using Faster R-CNN, defect progression over time is tracked and alerted using LSTM, and ensuring performance by pre-trained weights using Transfer Learning ResNet-50.
In this section, four activities are demonstrated in one aspect mentioned in terms of module interaction as depicted in Figure 1 of the proposed hybrid system, such as data collection, data preprocessing, real-time monitoring, multi-modal fusion, hybrid model detection, and evaluation of effectiveness. The second aspect is demonstrated in terms of the flow of actions performed to assess the effectiveness of the hybrid model, as predicted in Figure 2, which involves the installation of sensors and CCTVs, data collection, preprocessing, and other related activities to detect the defects The third aspect is described in Table 3, in which modules are specified with their purpose and technologies needed. The fourth aspect demonstrates its functionality through a pseudo procedure, PS1.
Table 3. Modules, their responsibilities, and technologies used
|
Module |
Responsibility |
Technologies Needed |
|
Sensor Data Ingestion |
Use sensors for strain, vibration, and temperature data. |
Sensors like accelerometers, strain gauges, MQTT protocol |
|
Image Acquisition |
Capture high-res images/videos of flyover surfaces. |
CCTV, LiDAR, and drones |
|
Data Preprocessing |
Sensor data to be normalized, images to be augmented, and missing values to be replaced. |
Scikit-learn, Pandas, and OpenCV |
|
Visual Defect Detection |
Cracks/spalling to be identified using object detection. |
Faster R-CNN with ResNet backbone and Transfer Learning |
|
Sensor Analytics |
Structural anomalies are identified from sensor time series. |
LSTM |
|
Multimodal Fusion |
For total assessment, must combine sensor and visual data. |
Attention Mechanisms and TensorFlow |
|
Alert System |
Alert every critical defect. |
Twilio (SMS), REST APIs, and email |
|
Maintenance Dashboard |
Defects, repairs prioritization, and costs report. |
Django and Tableau |
Figure 1. Significant modules' interaction in the hybrid model for flyover defect detection
Figure 2. Flow of activities in the hybrid Faster R-CNN – LSTM model for flyover defect detection
From PS1, the hybrid model of LSTM, Faster RNN, and Transfer Learning takes input from sensors and visual images from CCTVs. Then, data is collected, and follows data preprocessing for clean data, in which Kalman filtering and min-max normalization for sensor data, while enhancing images, would use denoising, augmentation, and segmentation. The hybrid model processes sensor data with LSTM and attention mechanism, image data by faster RNN, and a pre-trained MobileNet for reducing training time, and cross-layer fusion for ensuring better classification accuracy. The hybrid model uses LSTM for abnormal vibrations, and Faster R-CNN for major defects identification, then fuses the predictions of the defect is severe, requiring high attention. The effectiveness is measured in accuracy, precision, and recall, and ensures better real-time flyover health monitoring and alerting of critical infrastructure.
The usage of MobileNet over other pre-trained models is less parameter consumption due to depth-wise separable convolutions, supporting edge deployment with less resource-constrained hardware environment, experiences optimistic fine tuning, and supports of cross layer fusion, avoiding computational bottlenecks.
In overall, firstly, data preprocessing is applied for structural dynamic changes using Kalman filtering and min-max normalization, localizing defects using denoising, augmentation and segmentation, then processing the images using Faster R-CNN, processing the sensor data using LSTM, fusion the features of LSTM, R-CNN using Cross-modal Transformer. In Transfer Learning using MobileNetV3, it ensures speed and simplifies computation on edge devices, as well as ensures deployment. The fusion using Query Key Value (QKV) would enhance context-aware defect detection, especially spatial-temporal relationships.
|
PS1: Flyover_defects_detection_using_Hybrid_ model(Hardware[][],dataset1[][], dataset2[][], Accuracy): Input: Hardware[][],dataset1[][], dataset2[][] in which Hardware denote sensors, CCTVs, Drones, Dataset1 denote data read by sensors, and other devices, dataset2 denote images Output: Accuracy computation |
|
Step 1: Initiate data collection from sensors and CCTV cameras, and deploy sensors 1.1 Strain gauges for load inspection, accelerometers for vibration measuring, temperature sensors for expansion or contraction, 1.2 Humidity sensors for induced corrosion, and Cameras for visual cracks. 1.3 Collect maintenance data Step 2: Initiate Data preprocessing 2.1 For sensor data, use Kalman Filtering for Noise removal, use min-max normalization for scaling sensor readings, and use imputation for handling missing data. 2.2 For visual image data, use denoise filters for image enhancement, use augmentation in terms of rotation, flipping, and brightness adjustment for images, and use segmentation to isolate defect regions. Step 3: Build a Hybrid model using LSTM, Faster R-CNN, and Transfer Learning 3.1 Apply LSTM for time-series sensor data 3.2 Use an attention mechanism for adjusting the weights for the weighted features 3.3 Use the pretrained model MobileNetV3 for freeze the initial layers 3.4 Use adaptations for extracting features relevant to flyover, and do fine-tuning 3.5 Cross-modal fusion, in which fused features are extracted from the LSTM and the CNN features 3.6 Use fused features based on the Query, Key, and Value concept. Step 4: Do training with the hybrid model 4.1 Use the Binary cross-entropy loss function for defects, such as present or absent 4.2 Use AdamW optimizer 4.3 Do regularization in which Dropout is applied to LSTM and Dense layers, and early stopping is identified there is more validation loss observed. Step 5: Checking the Inference pipeline 5.1 In sensor data, use LSTM and attention weights, alert when more than 80% vibration is observed. 5.2 In image monitoring, a faster RNN for fixing defect regions. 5.3 Fusion is applied to classify the defect, such as anomaly_score in LSTM if high, and defect score in Faster CNN is high, then the defect is severe. Otherwise, the defect is normal level. Step 6: Evaluate the effectiveness of the model using accuracy, precision, and recall. Accuracy=True Positives + True Negatives / Total Number of Cases Where Precision=True Positives (TP) / (True Positives (TP)+ False Positives (FP)) Recall (Sensitivity)=True Positives (TP) / (True Positives (TP)+False Negatives (FN)) |
From Figure 3, the system initiates the first hardware setup that includes sensors, CCTVs for data collection, and real-time monitoring, then preprocesses the data into quality data, then uses Faster R-CNN on images for defects and LSTM on temporal features, then fuses the significant features, then applies a hybrid model of Faster R-CNN and LSTM, then evaluates the accuracy that defines the effectiveness of the model.
Figure 3. Flow of modules in the hybrid Faster R-CNN- LSTM model
In this, the various methods are considered and their benefits, shortfalls, and use cases are demonstrated in Table 4. From which, rather than individual models, the proposed hybrid LSTM-Faster RNN-Transfer Learning would outperform in accuracy, as well as be scalable for large infrastructures like flyovers. The effectiveness of the models is demonstrated in Table 5 in terms of accuracy, precision, and recall. In this, the dataset is considered from online sources, such as SDNET2018: A concrete crack image dataset for machine learning applications, where defects such as spalling, cracks, and corrosion are classified as multi-class defects. Tested over the dataset in terms of accuracy, which denotes the total number of correctly predicted instances in out of all, precision denotes correctly predicted instances in total predictions (false predictions), recall denotes actual predictions (missed predictions), interpretability denotes how best explains the predictions, and robustness denotes very less impact on accuracy as well as resolution.
From Table 4, the hybrid model only performs better accuracy, and is suitable for small bridges to large flyovers. The evaluated values are demonstrated in Table 4, with better understandability and bias bias-free detection aspect.
Table 4. Overview of methods for flyover defect detection
|
Method |
Advantages |
Limitations |
Best Use Case |
|
Manual Inspection |
Skills and expertise are provided for humans to inspect. No training needed |
Subjective type, expensive, and slow |
Small-scale inspections |
|
Traditional ML (Support Vector Machines) |
Support smaller datasets and faster tabular sensor data |
Can’t support temporal data and is poor with image/video type |
Vibration anomaly detection (no images) |
|
Traditional ML (Random Forest) |
Handles missing data and extracts feature importance |
Overfits and weaknesses on sequential data |
Support for Corrosion risk prediction |
|
Transfer Learning (ResNet, VGG) |
Reuses pre-trained weights and ensures high accuracy |
Requires fine-tuning |
Support the quick deployment of visual cracking and spalling |
|
CNN-only |
Automatic defect detection for Good for image defects |
Large, labelled datasets, and ignores sensor data |
Support pure visual inspections |
|
RNN-only |
Support time-series patterns |
Struggles with vanishing gradient descent and spatial defects |
Health monitoring using sensor data |
|
LSTM-only |
Handles past states and long-term dependencies |
Weak on image processing, and heavy computations |
Fatigue monitoring using sensors |
|
GANs |
Synthetic data generation with augmentation handles increased datasets |
Training is unstable and yields unrealistic defects |
Rare defect identification |
|
Faster RNN + Transfer Learning + Sensors |
Supports Multimodal data with real-time and ensures high accuracy |
Higher computational cost |
End-to-end monitoring with early alerts |
|
Hybrid model (Faster R-CNN+ LSTM +Transfer Learning Mobile Net + IoT enabled sensors) |
Ensures the best accuracy and is scalable for large infrastructures |
High resource consumption and requires annotations |
Requires large-scale infrastructures like flyovers |
Table 5. Measures assessment over specific methods for flyover defect detection
|
Method |
Accuracy (%) |
Precision (%) |
Recall (%) |
Interpretability |
Robustness |
|
Manual Inspection |
80 |
85 |
80 |
High |
Medium |
|
SVM |
90 |
90 |
85 |
Medium |
Medium |
|
Random Forest |
92 |
94 |
90 |
High |
High |
|
Transfer Learning (ResNet/VGG) |
95 |
96 |
94 |
Low |
High |
|
CNN-only |
98 |
95 |
97 |
Low |
High |
|
RNN-only |
90 |
86 |
92 |
Medium |
Medium |
|
LSTM-only |
94 |
92 |
95 |
Medium |
High |
|
GANs |
82 |
86 |
80 |
Very Low |
Medium |
|
Faster RNN + TL + Sensors |
97 |
97 |
96 |
Medium |
Very High |
|
Hybrid Model (Faster R-CNN+LSTM+IoT) |
99 |
98 |
98 |
Medium |
Very High |
Figure 4. Effectiveness of models w.r.to accuracy and performance
Table 5 is demonstrated in Figure 4 for better end-user understanding of the effectiveness of the models. The false defect detection is reduced by the hybrid model.
Table 6. Inference times, and precisions over specific methods
|
Model |
mAP@0.5 |
Inference Time |
|
YOLOv8 (Xu et al. [3]) |
82.1% |
40 ms |
|
Faster R-CNN (ResNet50) |
97% |
62 ms |
|
Pure LSTM (Li et al. [5]) |
92% |
15 ms |
|
Hybrid Model (Faster R-CNN, LSTM, and TL) |
97.7% |
28 ms |
From Table 6, precision and inference times are evaluated, and observed that the proposed hybrid model (Faster R-CNN, LSTM, and TL) would perform better in implementation time than other models, such as YOLOv8, individual Faster R-CNN, and individual LSTM.
A unified framework is required that ensures accuracy and real-time detection over visual and sensor data deployed on mobile devices. A hybrid LSTM, Faster R-CNN, and Transfer Learning model ensures the LSTM approach over sensor data, Transfer Learning, and Faster R-CNN over image and visual data. The pre-trained MobileNetV3 provides a light-weight architecture and low-cost approach for defect detection, supporting real-time processing of edge devices. It makes the complex environment also simpler in terms of less parameter consumption, faster configured edge devices support, flexible resource constraint environment, and better fine-tuning. It is superior to heavier models like ResNet, and VGG models. After processing with LSTM, Faster R-CNN, and fusing the features using cross-layer attention, it would effectively fuse the sensor and image data for early defect detection over flyovers and avoid expensive future maintenance costs. The asset of this system is real-time monitoring using drones and CCTVs. This significant module results in flyover longevity and safety. The LSTM for anomaly identification, Faster RNN for defect regions fixing, then fusing these, would determine whether the defect is severe or normal, so that the safety of the flyover is ensured. In the future, still better to increase long-range dependency support and reduce annotations using advanced lightweight modeling and contrastive learning.
I am thankful to my team and to my organization Koneru Lakshmaiah Education Foundation, for providing the facilities and allow to use research labs under AI and ML domain.
[1] Li, T., Liu, G., Tan, S. (2024). Superficial defect detection for concrete bridges using YOLOv8 with attention mechanism and deformation convolution. Applied Sciences, 14(13): 5497. https://doi.org/10.3390/app14135497
[2] Teng, S., Liu, Z., Li, X. (2022). Improved YOLOv3-based bridge surface defect detection by combining high- and low-resolution feature images. Buildings, 12(8): 1225. https://doi.org/10.3390/buildings12081225
[3] Xu, W., Li, X., Ji, Y., Li, S., Cui, C. (2024). BD-YOLOv8s: Enhancing bridge defect detection with multidimensional attention and precision reconstruction. Scientific Reports, 14(1): 18673. https://doi.org/10.1038/s41598-024-69722-8
[4] Rishitha, P., Sai, U.V., Chaudhary, S.D., Anuradha, G. (2022). A comprehensive study on bridge detection and extraction techniques. In International Conference on Computer Vision, High-Performance Computing, Smart Devices, and Networks, Kakinada, India, pp. 325-336. https://doi.org/10.1007/978-981-99-6690-5_24
[5] Li, Z., Lin, W., Zhang, Y. (2023). Real-time drive-by bridge damage detection using deep auto-encoder. Structures, 47: 1167-1181. https://doi.org/10.1016/j.istruc.2022.11.094
[6] Qiu, M., Huang, L., Tang, B.H. (2023). Bridge detection method for HSRRSIs based on YOLOv5 with a decoupled head. International Journal of Digital Earth, 16(1): 113-129. https://doi.org/10.1080/17538947.2022.2163514
[7] Fan, Z., Tang, X., Chen, Y., Ren, Y., et al. (2024). Review of anomaly detection in large span bridges: Available methods, recent advancements and future trends. Advances in Bridge Engineering, 5(1): 2. https://doi.org/10.1186/s43251-024-00113-9
[8] Gan, L., Liu, H., Yan, Y., Chen, A. (2024). Bridge bottom crack detection and modeling based on faster R-CNN and BIM. IET Image Processing, 18(4): 664-677. https://doi.org/10.1049/ipr2.12976
[9] Karluklu, D., Rinaldi, C., Crognale, M., Figuli, L., Gattulli, V. (2025). Enhanced image processing and storage for defect evaluation in BIM of inspected steel bridges. Discover Civil Engineering, 2(1): 11. https://doi.org/10.1007/s44290-025-00164-5
[10] Li, S., Xin, J., Jiang, Y., Yang, C., Wang, X., Ran, B. (2024). A novel hybrid model for bridge dynamic early warning using LSTM-EM-GMM. Advances in Bridge Engineering, 5(1): 8. https://doi.org/10.1186/s43251-024-00119-3
[11] Santos-Vila, I., Soto, R., Vega, E., Fritz, A.P., Crawford, B. (2024). Anomaly detection on bridges using deep learning with partial training. IEEE Access, 12: 116530-116545. https://doi.org/10.1109/ACCESS.2024.3447577
[12] Miao, P., Zhou, C., Wu, Y., Hu, W., et al. (2025). Enhancing bridge performance assessment with a hybrid attention-based long short-term memory and hidden Markov model using sparse inspection data. Structures, 71: 107995. https://doi.org/10.1016/j.istruc.2024.107995
[13] Yamane, T., Chun, P.J., Dang, J., Okatani, T. (2024). Deep learning-based bridge damage cause estimation from multiple images using visual question answering. Structure and Infrastructure Engineering. https://doi.org/10.1080/15732479.2024.2355929
[14] Amirkhani, D., Allili, M.S., Hebbache, L., Hammouche, N., Lapointe, J.F. (2024). Visual concrete bridge defect classification and detection using deep learning: A systematic review. IEEE Transactions on Intelligent Transportation Systems, 25(9): 10483-10505. https://doi.org/10.1109/TITS.2024.3365296
[15] Hou, S., Shen, H., Wu, T., Sun, W., Wu, G., Wu, Z. (2025). Underwater surface defect recognition of bridges based on fusion of semantic segmentation and three-dimensional point cloud. Journal of Bridge Engineering, 30(1): 04024101. https://doi.org/10.1061/JBENF2.BEENG-7032
[16] Cui, C., Zhang, Q., Zhang, D., Lao, W., Wu, L., Jiang, Z. (2024). Monitoring and detection of steel bridge diseases: A review. Journal of Traffic and Transportation Engineering, 11(2): 188-208. https://doi.org/10.1016/j.jtte.2024.03.001
[17] Wang, Y.W., Ni, Y.Q., Wang, S.M. (2022). Structural health monitoring of railway bridges using innovative sensing technologies and machine learning algorithms: A concise review. Intelligent Transportation Infrastructure, 1: liac009. https://doi.org/10.1093/iti/liac009
[18] Wu, Z.S., Huang, H. (2021). Early monitoring of diseases and disasters of bridges and tunnels. In 6th International Conference on Smart Monitoring, Assessment and Rehabilitation of Civil Structures, Shanghai, China, pp. 11-40. https://doi.org/10.1007/978-981-99-3362-4_2
[19] Zhang, B., Ren, Y., He, S., Gao, Z., Li, B., Song, J. (2025). A review of methods and applications in structural health monitoring (SHM) for bridges. Measurement, 245: 116575. https://doi.org/10.1016/j.measurement.2024.116575
[20] Chakurkar, P.S., Vora, D., Patil, S., Mishra, S., Kotecha, K. (2023). Data-driven approach for AI-based crack detection: Techniques, challenges, and future scope. Frontiers in Sustainable Cities, 5: 1253627. https://doi.org/10.3389/frsc.2023.1253627
[21] Shen, N., Wang, B., Ma, H., Zhao, X., Zhou, Y., Zhang, Z., Xu, J. (2023). A review of terrestrial laser scanning (TLS)-based technologies for deformation monitoring in engineering. Measurement, 223: 113684. https://doi.org/10.1016/j.measurement.2023.113684
[22] Dey, N.S., Sangaraju, H.K.R. (2024). A particle swarm optimization inspired global and local stability driven predictive load balancing strategy. Indonesian Journal of Electrical Engineering and Computer Science, 35(3): 1688-1701. https://doi.org/10.11591/ijeecs.v35.i3.pp1688-1701
[23] Dey, N.S., Sangaraju, H.K.R. (2023). Hybrid load balancing strategy for cloud data centers with novel performance evaluation strategy. International Journal of Intelligent Systems and Applications in Engineering, 11(3): 883-908. https://doi.org/10.18280/ijisae.110318
[24] Allamudi, A.K., Raju, H.S. (2025). Hybrid machine learning for fraud detection: Balancing accuracy and security in digital transactions. International Journal of Safety and Security Engineering, 15(2): 331-337. https://doi.org/10.18280/ijsse.150214
