Developed Improved Weighed Quantum Particle Swarm Optimisation with Deep Convolutional Neural Network Algorithm to Improve the Automated Concrete Surface Defect Detection in Bridge Inspections

The novel method IWQPSO-DCNN proposed to prevent inaccurate information and/or data loss. A systematized and standardized recommended image capture technique is used for information collecting in this research. Integrating Gazebo with UAVs opens up a wide range of possibilities for simulating and testing UAV-related algorithms and applications [14-16]. Both simulators provide realistic environments where UAVs can be virtually deployed, allowing researchers and developers to experiment with various scenarios without the cost and risk associated with real-world testing. In these simulations, users can model different types of UAVs, such as quadcopters, fixed-wing aircraft, or even hybrid designs, along with their sensors, actuators, and control systems. Simulate tasks like navigation, path planning, obstacle avoidance, surveillance, and payload delivery in diverse environments. Gazebo being open-source and highly customizable is widely used in the robotics community for simulating UAVs. It offers a rich set of features, including physics engines for realistic flight dynamics, sensor simulation (e.g., cameras, LiDAR, GPS), and integration with Robot Operating System (ROS) for seamless development and testing of UAV control algorithms. It also supports UAV simulation and offers a user-friendly graphical interface for building complex robotic systems. It provides a variety of pre-built models and environments, making it easier to get started with UAV simulations [16].

1.png

Figure 1. The overall scheme of the proposed method

Figure 1 depicts the general layout of the proposed strategy, which is broken down into three phases. A quick and inexpensive Image Capturing and Geo-Tagging (ICGT) that can effectively handle intermediary operations was created to gather referencing information from photos concurrently [17, 18]. The proposed system is for rapidly identifying cracks and harm recognition in the subsequent phase of crack-damaged recognition for offline inspection images. Every image's DCNN outcomes are obtained using its geotagged place. To create a Global Bridge Damage Map (GBDM), the images are immediately projected onto a global location. Machines have been implemented for five distinct purposes in the context of infrastructure and buildings project inspection and observation: (a) preservation inspection; (b) quality control during construction examination; (c) Model built; (d) progress observation; and (e) security examination.

2.1 Dataset collection

Similar to photogrammetry, visual imaging focuses on the collection of images, videos, and other types of visual data. Typically, still image cameras, video cameras, smartphones, and other devices are used to obtain these. A sample of the information gathered and information gathered utilizing a UAV equipped with visual imagery technology is shown in Figure 2. Data collection within Gazebo involves capturing sensor readings, images, and telemetry data generated by the virtual UAVs during their missions. Annotations may also be added to the data to indicate target objects, waypoints, or other relevant information. Gazebo provides a powerful platform for generating high-fidelity UAV datasets that can facilitate the development and evaluation of robust machine-learning algorithms for UAV applications. An investigation showed that aerial images obtained from Gazebo through the UAV system for pavement fracture inspection were more reliable than those gathered from more conventional sources, such as static photos and crack depth measurement instruments [19-20]. The research used a five-stage inspection process that included damage proof of identity, drone-enabled bridge evaluation, data inspection, and location risk evaluation.

2.png

Figure 2. Equipment IR-UAV with Gazebo and its sample thermal data

2.2 Infrared thermal imaging

Equipment Gazebo with IR-UAV and its sample thermal data collected are shown in Figure 3. Due to the disruption of heat transfer caused by delamination areas above it will be determined to be hotter than similar areas above soundness concrete [19]. An infrared camera placed on a low-altitude airplane was used to capture thermal images of two concrete bridge decks. The underlying defect locations detected by the Gazebo with the IRT-UAV system have been verified by conventional methods including hammering sound and Half-Cell Potential (HCP).

3.png

Figure 3. Gazebo with UAV equipment model

Table 1 provides the dataset format and information necessary for the development, testing, and validation of the creation and inspection of the DIWQPSO-DCNN method. The essential elements required for creating and evaluating the DIWQPSO-DCNN algorithm are included in the data table structure. Table 2 shows the sample data and its description for developing the algorithm, every column comprises an image sample together with its distinctive characteristics, raw image information, defect kind, defect location, seriousness straight, metadata, beforehand processed information, recovered vectors of characteristics, and real-world labeling [21].

Table 1. Dataset description

Table 2. Sample datasets

2.3 Data acquisition

An instance of a sensor-embedded Gazebo with UAV and the information it collects is shown in Figure 3. The deflection of beams was examined using a Gazebo with a UAV system equipped with a reflecting prism, whose location was monitored by a laser-tracking complete station [22]. The ceiling effect aerodynamic examination, developed using the use of Computational Fluid Dynamics (CFD) was used to optimize the Gazebo with UAV architecture. The Gazebo with UAV made use of the effect of the ceiling to its advantage to make connections and carry out its surveillance tasks of data acquisition is shown in Figure 4.

Several processes, comprising image extraction, preliminary processing, extracting features, and labeling, are involved in the information collection method used in the development of the IWQPSO-DCNN [21, 22].

4.png

Figure 4. Sensor embedded in Gazebo with UAV for data acquisition based on contact

2.3.1 Image capture

High-resolution images of concrete surfaces are captured using drones or stationary cameras equipped with high-resolution and thermal imaging capabilities [23-26]. The following equation describes the resolution of the captured image:

Resolution $=\frac{\text { No. of Pixels }}{\text { Image Area }}$          (1)

2.3.2 Image pre-processing

Pre-processing steps are applied to the raw images to enhance quality. Normalization, resizing, and augmentation were done for pre-processing the data.

Normalization:

$X_{ {norm }}(i, j)=\frac{X(i, j)-\min (X)}{\max (X)-\min (X)}$          (2)

where, X(i,j) is pixel (x, y) intensity value; min(X) and max(X) are minimum and maximum intensity values in the image.

Resizing:

The images are resized to a standard size suitable for the DCNN, for example, 256 × 256 pixels.

Augmentation:

By applying techniques such as random rotations, flips, scaling, and noise addition to the concrete surface images, we can generate a more extensive and varied dataset from the limited available images. This augmentation helps in simulating various real-world conditions and defects, making the model more robust and capable of identifying defects under different scenarios and lighting conditions [23].

2.4 Feature extraction

Feature extraction is a crucial step in the process of building and optimizing machine learning models, particularly in fields such as image processing, natural language processing, and signal processing.

Convolution Operation:

$F_{x y}=\sum_{m=1}^M \sum_{n=1}^N X_{(x+m-1)(y+n-1)} \cdot K_{m n}$          (3)

where, X is the input image, K is the convolution kernel (filter) of size M × N, and F is the output feature map.

Activation Function (ReLU):

f(x) = max(0, x)          (4)

Max Pooling Operation:

$P_{x y}=\max _{m, n} F_{(x+m)(y+n)}$          (5)

where, P is the pooled feature map.

Labelling: Manual labelling of the images involves identifying and categorizing defects such as cracks, spalling, and scaling. Each defect is annotated with its location and severity [27, 28].

Bounding Box Annotation:

$\left(i_1, j_1, i_2, j_2\right)$          (6)

where, $\left(i_1, j_1\right)$ and $\left(i_2, j_2\right)$ are coordinates of the bounding box around the defect.

2.5 Dataset structuring

The labelled data is structured into a dataset suitable for training and testing the IWQPSO-DCNN.

Splitting of dataset

Training Set Size = 0.7 × Total Dataset Size

Validation Set Size = 0.2 × Total Dataset Size

Test Set Size = 0.1 × Total Dataset Size

Based on the discoveries made by the drones during the examination flights, some suggestions are offered for the systematization of collecting information for routine visual examinations. Certain Gazebo with UAVs might not be able to navigate some areas of a bridge, including the area between pillars or the elastomeric bearings pad. The bridge's laterals, the highest point, and bottom sections ought to be examined. Take images of a bridge check, starting with the first abutment and going through all of the intermediary columns. The bridge placement and flight sequencing will be ascertained by its measurements shown in Figure 5.

5.png

Figure 5. (a) Acquisition of image; (b) Abutments (A1 and A2) and piers (P1 and P2); and (c) deck and bridge location

2.6 Proposed system

The measurement of the crack width in images was based on these two techniques. The information used to train the crack recognition model included open-source fracture information, camera-equipped Gazebo in UAVs, and smartphones shown in Figure 6. To ascertain the precision of the proposed model and confirm the viability of the proposed approach, the depths of the derived outlines were determined and contrasted with the real values shown in Figure 7.

6.png

Figure 6. The process of Gazebo with UAV took photos of bridge

7.png

Figure 7. Operation of Gazebo with UAV bridge inspection

Figure 8 displays the ICGT's software foundation. During the inspection, the ICGT can be started or stopped remotely from a station using the command line utility service is part of the ROS core tool chain. It should be noted that these two jobs operate concurrently at 5 frames per second to synchronize the information from the sensors and the photos [29, 30]. The process keeps running cyclically until the central controller issues the instruction to "Stop”.

8.png

Figure 8. Process of ICGT software on bridge inspection

2.6.1 Feature points of measurement and planar marker

The fracture widths in the photos were measured using the scale method. Planar markings are difficult to install on reinforced concrete structures since the majority of them are raised motorways or span rivers. The features of the concrete surfaces were measured using a total location, and the resulting positions were utilized as a substitute for planar markings in the opposite calculation of the spatial distances shown in Figure 9.

9.png

Figure 9. Initialization of particle using back-and-forth path

Every particle is first given a random positioning and speed. It then moves by updating both the swarm's best position ($G_k$) and its best previous position ($P_k$). Let $i_k$ and $v_k$ represent a particle's location and velocity at generation k, accordingly. In the following generations, that particle's velocity and location are determined by:

$v_{k+1} \leftarrow w \cdot v_k+\varphi_1 \cdot r_1 \cdot\left(P_k-i_k\right)+\varphi_2 \cdot r_2 \cdot\left(G_k-i_k\right)$          (7)

$i_{k+1} \leftarrow i_k+v_{k+1}$          (8)

where, $r_1$ and $r_2$ are selected at random of an identical distribution in the interval [0, 1], w is the inertial coefficient, $\varphi_1$ is the cognitive coefficient, and $v_k$ is the social coefficient. According to Eqs. (7) and (8), the movement of a particle involves a trade-off between traveling in the direction of its best prior position, going in the direction of its path, or going in the direction of the swarm's optimal position. The coefficients of w, $\varphi_1$, and $\varphi_2$ establish the ratio among selections.

Algorithm: IWQPSO-DCNN

A hybrid model of IWQPSO-DCNN was proposed to enhance the computerized identification of surface concrete faults in bridge inspections. Using the optimized IWQPSO-DCNN model, the procedure includes feature selection, hyperparameter optimization, and fault identification.

Step 1. Initialization

{

1.1. Particle Initialization: Initialize a swarm of N particles, where each particle represents a potential solution with a set of hyperparameters and selected features for the DCNN. Randomly initialize the position X and velocity $V_x$ of each particle.

$I_x=\left(i_{x 1}, i_{x 2}, \ldots, i_{x D}\right)$          (9)

$V_x=\left(v_{x 1}, v_{x 2}, \ldots, v_{x D}\right)$          (10)

where, x = 1, 2, ..., N and D - dimensionality of a number of hyperparameters and features.

1.2. Quantum Initialization: Initialize the quantum position Q_x for each particle.

$Q_x=\frac{1}{\sqrt{2 \pi \sigma^2}} e^{\sigma-\frac{\left(i_x-\mu\right)^2}{2 \pi^2}}$          (11)

}

Step 2. Fitness Evaluation

{

2.1. Training the DCNN: Train the DCNN model with the hyperparameters and features represented by each particle. Use the training dataset to perform forward and backward propagation.

2.2. Fitness Function: The fitness function f can be defined as the negative of the validation loss or a performance metric like accuracy or F1-score.

f(I_x) = Validation Loss (DCNN(I_x))          (12)

}

Step 3: Update Particle Positions and Velocities

{

3.1. Velocity Update: Update the velocity of each particle using the quantum-inspired velocity equation.

$\begin{gathered}V_x^{t+1}=w \cdot V_x^t+c_1 \cdot r_1 \cdot\left(P_x-I_x^t\right)+c_2 \cdot r_2 \cdot\left(P_x-I_x^t\right) +c_3 \cdot r_3 \cdot\left(Q_x-I_x^t\right)\end{gathered}$          (13)

where, w is inertia weight; $c_1, c_2$, and $c_3$ are coefficients of acceleration; $r_1, r_2$, and $r_3$ are random numbers between 0 and 1; $P_x$  is local best position; G is global best position.

3.2. Position Update:

Update the position of each particle.

$I_x^{t+1}=I_x^t+V_x^{t+1}$          (14)

}

Step 4: Quantum Position Update

{

Quantum Update: Update the quantum position of each particle based on the new position.

$Q_x^{t+1}=\frac{1}{\sqrt{2 \pi \sigma^2}} e^{\sigma-\frac{\left(i_x^{t+1}-\mu\right)^2}{2 \pi^2}}$          (15)

}

Step 5. Iterative Optimization

{

5.1. Convergence Check: Repeat steps 2 to 4 until convergence criteria are met

5.2. Select Optimal Hyperparameters and Features: The global best position G represents the optimal hyperparameters and feature set for the DCNN.

}

Step 6. Final Model Training and Evaluation

{

6.1. Train and evaluate Final DCNN: Train the DCNN with the optimal hyperparameters and selected features on the entire training dataset.

}

The IWQPSO-DCNN method can increase the precision and dependability of automatic concrete surface detection of imperfections for bridge examinations by implementing this method to optimize the hyperparameters and feature choice process.

2.7 Crack identification using IWQPSO-DCNN model training

A one-stage approach uses packed forecasting, while a two-stage model uses sparse forecasting. Compared to a one-stage model, the prediction results of a two-stage model are sparser because a select search is used to choose an established amount of areas of interest shown in Figure 10.

10.png

Figure 10. Multi-stage model detector

The fracture is represented by the black line in the centre of the image, and the bounding boxes for object detection are represented by the black frames [31-33]. The bounding box that gave image measurements based on the notion that the image's top-left corner is its point of origin is shown in Figure 11. One might extract the bounding box range and carry out image enhancement shown in Figure 12.

11.png

Figure 11. Crack detected by crack

12.png

Figure 12. Bounding box output

2.8 Inspections form

Other kinds are chosen based on the substance of every single bridge element such as the abutment, section, deck, wires, parapet/handrail, or pavement must be used to express the inspection findings regarding every one of these elements. The following categories of substances have been established in this study to streamline the process for as long as possible: (M) masonry, (REW) reinforced earth walls, (C) concrete, (S) metal, and (CB) cables.

13.png

Figure 13. Image acquisition of Gazebo in UAV (a) first abutment (b) pier or column in parallel (c) pier or column in cylindrical manner

Figure 13 shows the column inspection done from the bottom up and in an anticlockwise direction on every side. To achieve this, a list of the most frequent damages has been established, ones that could impact all varieties of building materials and components and of which the Gazebo UAV operator needs to be aware when conducting the inspection.

The proposed pipeline can potentially be expanded to identify failures in structures such as overhead electrical lines and railroads. It may additionally adapt and harness the benefits of Gazebo Robot in UAVs. The integration of a UAV equipped with a Gazebo Robot simulation and utilizing an IWQPSO-DCNN significantly enhances automated concrete surface defect detection in bridge inspections. This approach improves defect detection accuracy, as the combination of IWQPSO- DCNN allows for precise tuning of neural network parameters, effectively identifying defects such as cracks and spalling. IWQPSO offers efficient optimization, improving convergence rates and reducing the likelihood of being trapped in local minima, common in traditional optimization techniques. The Gazebo Robot simulation environment enables the testing and validation of the UAV’s navigation and inspection capabilities in a realistic yet controlled setting, ensuring the system can perform real-time defect detection effectively. This combination results in a robust, accurate, and efficient solution for bridge inspection and maintenance [33-35].