Design of a Real-Time Monitoring and Early Warning System for Engineering Safety Hazards Using Image Analysis Technology

In the monitoring and tracking of engineering safety hazards, real-time performance and accuracy are two crucial factors. The traditional Mean Shift algorithm, known for its good computational efficiency and unsupervised nature, is often applied to object tracking tasks, especially for tracking smooth movements of objects in dynamic scenes. However, when directly using the Mean Shift algorithm, the color feature description of engineering safety hazards tends to be rough, making it susceptible to external factors such as background complexity and lighting changes. These limitations can lead to deviations in tracking under certain conditions, such as when the color of the target object is similar to the background or when the hazardous object undergoes deformation. This issue is particularly prominent in complex engineering environments, where lighting conditions are often uneven, hazardous targets frequently change shape, and various noise interferences exist. Therefore, improving the Mean Shift algorithm is key to enhancing real-time tracking and identification capabilities of engineering safety hazards.

To address these challenges, this paper combines Scale-Invariant Feature Transform (SIFT) with the Mean Shift algorithm, using Principal Component Analysis Scale-Invariant Feature Transform (PCA-SIFT) features to provide a more robust description of hazardous targets. The advantage of the PCA-SIFT algorithm lies in its scale and rotation invariance of key points in images, which is critical for handling complex engineering environments. During the engineering monitoring process, the hazardous target may change due to variations in camera angles, the scale of the hazard, or fluctuations in environmental lighting. PCA-SIFT can effectively enhance the feature description of the hazardous target, allowing it to remain stable in complex engineering backgrounds. Furthermore, when the scale of the hazardous target changes, the scale invariance of the PCA-SIFT algorithm ensures that feature points are not lost due to target deformation, enabling the Mean Shift algorithm to maintain precise target localization even under scale variations.

Specifically, this is implemented in five stages:

Stage one: Establishing the scale space and detecting stable key points

The core of this process is to represent and process the hazardous target at different scales through multi-scale analysis, addressing changes in the target's size, position, and viewing angle in engineering scenarios.

First, to construct the scale space, the system needs to perform multi-scale representation of the input engineering monitoring images. In practical applications, engineering hazards may be at different distances or angles and may undergo scaling or deformation due to changes in engineering machinery or the environment. Therefore, describing the target at a single scale often cannot guarantee its stability and consistency. To address this issue, a Gaussian pyramid is used to represent the image at different scales. Specifically, assuming that the spatial coordinates are represented by (a, b), and the scale space factor is represented by δ, the representation of image U(a, b) at different scales is the result of the convolution of this image with the Gaussian kernel function H(a, b, δ):

$M(a, b, \delta)=H(a, b, \delta) * U(a, b)$    (1)

By adjusting the value of δ, the image is smoothed to different degrees to meet the needs of different scales. The construction of the Gaussian pyramid enables the representation of image features at different resolutions, ensuring the stability of hazardous targets at various scales. H(a, b, δ) is the scale-variable Gaussian function, expressed as:

$H(a, b, \delta)=\frac{1}{2 \tau \delta^2} e^{-\left(a^2+b^2\right)} / 2 \delta^2$    (2)

Based on the construction of the Gaussian pyramid, the key points are further extracted using the Difference of Gaussians (DoG) method. Specifically, the system performs a difference operation on images at adjacent scales to obtain a DoG image, which can more accurately detect key points that remain stable across different scales. In engineering safety hazard scenarios, these key points typically correspond to significant features such as edges and corners of hazardous targets. Each pixel point in the DoG scale space is detected as a potential key point by comparing its extremum with its surrounding points. In this way, the system can identify key points in the image that remain prominent across different scales, which will serve as stable descriptions of the target in subsequent tracking processes. The DoG scale space is defined as:

$\begin{aligned} F(a, b, \delta) & =(H(a, b, \delta)-H(a, b, j \delta)) * U(a, b) =M(a, b, j \delta)-M(a, b, \delta)\end{aligned}$    (3)

Stage two: Accurate localization of key points and generation of PCA-SIFT descriptors

After constructing the scale space and detecting local extrema points, the next step is to accurately locate these extrema points to determine their precise position and scale. To improve localization accuracy, further optimization of the detected local extrema points is required. Specifically, by fitting a three-dimensional quadratic function to the DoG scale space, the curve fitting process allows for more accurate determination of the position and scale of the key points. The three-dimensional quadratic function fitting effectively smooths the variations of key points across different scales and positions, ensuring that the selected key points exhibit higher stability in practical engineering scenarios. In complex and variable engineering sites, environmental factors such as lighting and noise may result in edge response points and low-contrast points, which often lack stable features and are unsuitable for subsequent tracking. Therefore, it is necessary to filter the detected key points and eliminate those unstable edge response points and low-contrast points. Specifically, by calculating the principal curvature of the key points, it can be determined whether a key point is an edge point. Points with high principal curvature are typically edge points, which are prone to interference during subsequent tracking and should be removed. At the same time, by setting a contrast threshold, key points with contrast below the threshold are discarded to ensure that the remaining key points maintain high distinguishability and stability under varying lighting conditions. Specifically, the Taylor expansion of H(a, b, δ) is as follows:

$F(A)=F+\frac{\partial F^S}{\partial A} A+\frac{1}{2} A^S \frac{\partial^2 F}{\partial A^2} A$    (4)

Setting the first derivative of the above equation to 0, we can get the extrema points:

$A^{\prime}=-\frac{\partial^2 F^{-1}}{\partial A^2} \frac{\partial F}{\partial A}$    (5)

Substituting the extrema points into Eq. (4), we get:

$F\left(A^{\prime}\right)=F+\frac{1}{2} \frac{\partial F^S}{\partial A} A^{\prime}$    (6)

After obtaining the precise position and scale of the feature points, we further assume that the scale of each key point is represented by M, and compute the gradient magnitude and direction at (a, b) to ensure the rotation invariance of the descriptor:

$l(a, b)=\sqrt{[M(a, b+1)-M(a, b-1)]^2+[M(a+1, b)-M(a-1, b)]^2}$    (7)

$\varphi(a, b)=T A N^{-1}\left\{\left[\begin{array}{l}M(a, b+1) \\ -M(a, b-1)\end{array}\right] /\left[\begin{array}{l}M(a+1, b) \\ -M(a-1, b)\end{array}\right]\right\}$    (8)

To further enhance the stability and interference resistance of hazard tracking, this paper uses the PCA-SIFT algorithm to generate feature descriptors. Specifically, the neighborhood of the feature points is divided into 41×41 pixel blocks, and by calculating the gradients in the horizontal and vertical directions within a 39×39 range, a feature vector of size 3042 is formed. This vector provides a detailed description of the gradient changes around the feature points, offering rich local information. However, directly using high-dimensional feature vectors for matching requires significant computation, which is detrimental to real-time performance. To address this issue, the PCA method is employed to reduce the dimensionality of the feature vector. PCA is based on the covariance matrix of the variables, and by calculating the eigenvalues and eigenvectors of the covariance matrix, it compresses and extracts the information. Specifically, by selecting the top few principal components, these components cover most of the information in the feature vector, thus representing the original high-dimensional data with fewer dimensions. This not only significantly reduces the computational load and improves matching efficiency but also effectively preserves the main information of the feature points, improving the accuracy and stability of the matching process.

Stage three: Establishing the tracking feature model of engineering safety hazards

In this step, the initial frame model of the hazard target is determined based on the position and scale information of the key points. Suppose the first pixel's coordinates are denoted as s_u(s_ua, s_ub), meaning that in the initial frame, the exact position of each pixel can be determined through the feature vector description of the detected key points. Additionally, based on the description of the key point feature vector, the local features of each key point can be represented as a high-dimensional vector, thus forming a complete initial frame feature model.

The essence of tracking is to match the features of the initial frame with those of the current frame to achieve continuous monitoring of the hazard target. In the initial frame, the center position of the hazard target can be denoted as f₀, and in the current frame, the tracking algorithm determines the new center position of the target, f₁, by matching the feature vectors of the key points. By comparing the displacement and feature changes between these two center points, the movement of the target can be determined, and the target position is updated accordingly. This key point-based tracking method can cope with complex situations such as changes in target scale, angle, and partial occlusion in engineering scenarios. Let the total number of pixels be represented by v; the kernel function by j(||a||²), and the template radius by g. Functions y and σ are used to determine whether the color value of the u-th pixel belongs to the feature value i, with normalization coefficients Z and Z_g. The following expressions represent the feature models for initial and current frame target tracking based on key point feature vector descriptions:

$\hat{w}_i\left(f_1\right)=Z \sum_{u=1}^v j\left(\left\|\frac{s_u-f_0}{g}\right\|^2\right) \sigma\left[y\left(s_u\right)-i\right]$    (9)

$\hat{o}_i\left(f_1\right)=Z_g \sum_{u=1}^{v g} j\left(\left\|\frac{s_u-f_1}{g}\right\|^2\right) \sigma\left[y\left(s_u\right)-i\right]$    (10)

The two normalization coefficients satisfy the following conditions:

$Z=\frac{1}{\sum_{u=1}^v j\left(\left\|\frac{s_u-f_0}{g}\right\|^2\right)}$    (11)

$Z_g=\frac{1}{\sum_{u=1}^{v g} j\left(\left\|\frac{s_u-f}{g}\right\|^2\right)}$    (12)

Specifically, by using the scale information of the key points, the local feature regions of the hazard target can be calculated, and a complete target feature model is generated based on these regions. The feature models of the initial and current frames are compared through the matching of key point positions, scales, and feature vectors. When the features in the current frame meet a certain similarity threshold with those in the initial frame, the system can confirm that the target in the current frame is the hazard target that needs to be tracked. In this way, the system can continuously lock onto the hazard target in dynamic engineering environments. Even when environmental lighting changes, equipment interference occurs, or the hazard target changes position, the system can still achieve stable target tracking.

1.jpg

Figure 1. Region matching diagram

Stage four: Applying the improved Mean Shift algorithm for tracking

Mean Shift is an iterative optimization method based on density estimation, primarily used to find the position of the maximum local density by shifting the center of the target model. In engineering safety hazard tracking and identification, the Mean Shift algorithm completes tracking by calculating the region in each frame that is most similar to the hazard target in the previous frame. Figure 1 shows a schematic diagram of region matching. Specifically, at the starting position of each frame, i.e., the center f₁ of the hazard target in the previous frame, the system gradually adjusts the target position by comparing feature models, aiming to maximize the Bhattacharyya similarity coefficient between the current target and the hazard target model. Let l represent the number of features, and the target vectors of the initial and current frames be represented by ŵ_i and ô_i, respectively. The Bhattacharyya similarity coefficient is calculated as follows:

$\hat{\vartheta}(s)=\vartheta[\hat{o}(s), \hat{w}]=\sum_{i=1}^l \sqrt{\hat{o}_i(s) \hat{w}_i}$    (13)

In practical engineering hazard tracking, optimization of the Bhattacharyya similarity coefficient is achieved using Taylor expansion. Suppose the search in the current frame starts from the hazard center f₁ in the previous frame, then the similarity coefficient can be expanded using a Taylor series, and this approximation can be used to calculate the displacement of the hazard target during tracking. Since the displacement of the hazard target between consecutive frames is typically small, using Taylor expansion simplifies the computation and makes the tracking process more efficient. The Taylor expansion starting at f₁ is as follows:

$\begin{aligned} & \vartheta[\hat{o}(b), \hat{w}] \approx \frac{1}{2} \sum_{y=1}^l \sqrt{\hat{o}_y\left(\hat{b}_0\right) \hat{w}_y} +\frac{1}{2} \sum_{y=1}^{v g} \hat{o}_y(b) \sqrt{\frac{\hat{w}_y}{\hat{o}_y\left(\hat{b}_0\right)}}\end{aligned}$    (14)

In practical tracking, since the displacement of engineering safety hazards between consecutive frames is small, the above expression can be approximated as:

$\begin{aligned} & \vartheta[\hat{o}(b), \hat{w}] \approx \frac{1}{2} \sum_{y=1}^l \sqrt{\hat{o}_y\left(\hat{b}_0\right) \hat{w}_y} +\frac{Z_g}{2} \sum_{y=1}^{v g} q_u j\left(\left\|\frac{b-a_u}{g}\right\|^2\right)\end{aligned}$    (15)

where,

$q_u=\sum_{\omega=1}^l \sqrt{\frac{\hat{w}_y}{o_y\left(\hat{b}_p\right)}} \sigma\left[Y\left(a_u\right)-\omega\right]$    (16)

In the process of solving for the extreme value of the Bhattacharyya similarity coefficient, the system computes the Mean Shift vector, which represents the displacement vector of the hazard target center position in the current frame. By solving the gradient of the similarity coefficient, the system obtains the position update vector of the hazard target, and adjusts the target position accordingly. The Mean Shift vector for engineering safety hazard tracking is expressed as:

$L_{g, H}(s)=f_z-f_1=\frac{\sum_{u=1}^{v_g} H\left(\left\|\frac{s_u-f}{g}\right\|^2\right) q_u s_u}{\sum_{u=1}^{v_g} H\left(\left\|\frac{s_u-f}{g}\right\|^2\right) q_u}-f_1$    (17)

The Mean Shift vector represents the difference between the current target position and the center of the target feature density distribution. In each frame, the system calculates the current frame's Mean Shift vector based on the position from the previous frame, guiding the movement of the target position. As the Mean Shift vector iterates and updates continuously, the position of the hazard target gradually converges to the location of maximum density, thereby completing the tracking for the current frame.

Stage five: Describing and predicting the development trend of engineering safety hazards using linear prediction

In real-time tracking and identification of engineering safety hazards, using linear prediction methods to describe and predict the development trend of hazards can help the system maintain stable tracking even when the target is temporarily obscured or under other complex situations. This is primarily because the movement of safety hazards in practical scenarios is typically continuous. For example, some large machinery may cause equipment tilting or steel structure loosening during operation, creating safety hazards that do not instantaneously jump from one location to another but gradually develop. The trajectory often follows certain physical laws and has strong predictability. Linear prediction methods, based on this continuity characteristic, analyze historical movement data to predict the next movement direction and position. In tracking engineering safety hazards, when a target is temporarily obstructed, the system cannot directly acquire its position. However, the hazard's trajectory can be considered a smooth curve. Linear prediction can use this feature to reasonably infer the target's location, ensuring that the target is still effectively tracked when it reappears. Through linear prediction, even if the hazard temporarily disappears, the system can predict its next position using the historical trajectory, thereby reducing the risk of losing the target.

The six detailed steps for describing and predicting the development trend of engineering safety hazards using linear prediction methods are as follows: First, assume that the position of the engineering safety hazard can be represented by a function of time d(s), where s represents time. This position function describes the trajectory changes of the hazard target over continuous time. In practical engineering environments, the position of the hazard target is usually affected by various factors, such as slight movements in equipment operation or stress changes in building structures. The expression for d(s) is given as:

$B=\beta_0+\beta_1^s$    (18)

$B=\left[\begin{array}{ll}1 & s\end{array}\right] *\left[\begin{array}{l}x_0 \\ x_1\end{array}\right]$    (19)

To predict the position of the hazard target, linear approximation of its past trajectory is required. Specifically, a linear approximation equation can be constructed to describe the trend of the hazard target's position changes. Let the linear approximation equation be:

$\gamma_1=d\left(s_1\right)-B=d\left(s_1\right)-x_0-x_1 s_1$    (20)

To ensure the accuracy of the linear approximation equation, the mean squared error (MSE) must be used to evaluate the error between the approximation equation and the true motion trajectory. The MSE is calculated using the following formula:

$R\left(\gamma_u^3\right)=\sum_{u=1}^V\left[d\left(s_u\right)-x_0-x_1 s_i\right]^2$    (21)

To achieve the optimal prediction results, the parameters x and y in the linear approximation equation must be adjusted to minimize the MSE. By solving the objective function that minimizes the MSE, the optimal parameter values can be obtained, resulting in the best linear approximation equation. This process can be achieved using gradient descent or the least squares method:

$\left[\begin{array}{c}x_0 \\ x_1\end{array}\right]=\left[\begin{array}{l}\frac{\sum_{u=1}^V s_u^2 \sum_{u=1}^V d\left(s_u\right)-\sum_{u=1}^V s_u \sum_{u=1}^V d\left(s_u\right) s_u}{F} \\ \frac{\sum_{u=1}^V s_u^2 \sum_{u=1}^V d\left(s_u\right)-V \sum_{u=1}^V s_u \sum_{u=1}^V d\left(s_u\right) s_u}{F}\end{array}\right]$    (22)

where,

$F=V \sum_{u=1}^V s_u^2-\left(\sum_{u=1}^V s_u\right)^2$    (23)

Once the optimal linear approximation equation is obtained, it can be used to predict the position of the hazard target in future frames. Let j be the frame number at the current moment, then the position of the hazard target in the next frame, j+1, can be predicted using the following formula:

$\hat{d}((j+1) / j)=x_0+x_1 s_{j+1}=x_0+{ }^4 x_1$    (24)

By solving for x₀ and x₁ based on s_u and d(s_u), we obtain:

$\hat{d}((j+1) / j)=\frac{1}{3}[4 d(j)+d(j-1)-2 d(j-2)]$    (25)

This formula represents the prediction of the development trend of the engineering safety hazard.

In the experimental section of this study, the improved Mean Shift algorithm, combined with SVM, was used to identify and provide early warning of safety hazards at an engineering site. Figure 4 presents a schematic diagram of the experimental results, clearly showing the effectiveness of the proposed method in identifying various types of safety hazards in a complex engineering environment. From the figure, it is evident that the improved Mean Shift algorithm significantly enhances tracking accuracy. It is able to accurately locate and continuously track hazard objects, even in the presence of significant environmental interference and complex object movements at the site. The experimental results demonstrate that the proposed method can accurately classify and issue early warnings for different types of safety hazards.

4.jpg

Figure 4. Illustration of engineering safety hazard detection results

From the data in Figure 5, it can be seen that there is a significant difference in the identification and localization errors of engineering safety hazards between the pre-improvement and post-improvement Mean Shift algorithms at different frame numbers. Before the improvement, the error increases rapidly with the number of frames, especially after 20 frames, reaching a maximum of 18.5, indicating significant instability in localization, particularly in the later frames (after 45 frames), where the error continues to rise, making it difficult to maintain a low error value. In contrast, the post-improvement algorithm performs more stably, with the error consistently staying at a lower level. During the early frames (0 to 25 frames), the error fluctuates slightly, remaining below 2.5, and only increases slightly in the later frames, with the maximum error being 3.5, which is significantly lower than the peak error before the improvement. This shows that the improved Mean Shift algorithm exhibits better robustness at different frame numbers and can achieve more precise hazard localization in complex engineering site environments. Analyzing the data, the improved algorithm effectively addresses the issue of error accumulation during tracking that existed before the improvement, especially maintaining a stable error range even at high frame numbers. The pre-improvement algorithm is easily disturbed by external factors as the number of frames increases, leading to large error fluctuations and a sharp rise, which negatively impacts the overall tracking accuracy. On the other hand, the improved algorithm, through optimization of the target tracking strategy, significantly reduces identification errors and enhances the tracking accuracy and consistency of the system in complex environments.

5.jpg

Figure 5. Comparison of engineering safety hazard detection localization errors

Table 1. Comparison of correct identification rates for engineering safety hazard detection

From Table 1, it is evident that the improved algorithm proposed in this paper performed significantly better in identifying engineering safety hazards compared to the traditional Mean Shift-SIFT algorithm. When the actual target count was 100, the Mean Shift-SIFT algorithm detected 82 targets and correctly identified 54 of them, with an accuracy rate of 54%. In contrast, the improved algorithm detected 94 targets and correctly identified 91, achieving a much higher accuracy rate of 91%. This demonstrates that the proposed algorithm significantly enhances accuracy and detection capability, especially in complex engineering environments. The improved algorithm reduces missed detections and false positives by employing more precise target tracking. The higher identification accuracy rate ensures the early warning system can detect safety hazards more promptly and effectively, thus improving safety management at the engineering site.

Table 2. Performance comparison of different methods for early warning of critical states of engineering safety hazards

Table 2 shows a clear advantage of the improved method in identifying critical early warning states of engineering safety hazards compared to the traditional SVM approach. The average recognition rate for the traditional SVM method across the four states (“Normal,” “Focus,” “Alert,” and “Danger”) is 80%, with the “Danger” state showing the lowest recognition rate of 70%. This lower accuracy in high-risk scenarios can lead to delayed or inaccurate system responses to potential safety hazards. In contrast, the proposed method achieves an average identification rate of 95%, with perfect recognition in the “Focus” and “Alert” states (100% accuracy). In the critical “Normal” and “Danger” states, the recognition rates are improved to 90%, showing higher overall precision and reliability. Based on the above experimental data, the method proposed in this paper demonstrates significant advantages in identifying the early warning critical state of engineering safety hazards through the improved SVM. Traditional SVM performs poorly in identifying high-risk "dangerous states," often resulting in misjudgments, which leads to the system's untimely or inaccurate response to potential safety hazards. In contrast, the proposed method performs more consistently across all states, particularly exhibiting higher accuracy and robustness in critical states such as "Alert" and "Danger." It effectively reduces false alarms and missed detections, thereby improving the overall response capability and accuracy of the early warning system.

From the data in Figure 6, it can be seen that the method proposed in this paper, the K-Means algorithm, and the hierarchical clustering algorithm show differences in performance under different False Positives per Window (FPW) conditions. As FPW increases, the miss rate of all three algorithms shows a downward trend. At lower FPW values (e.g., 1.0E-05), the miss rates of the three methods are relatively close, with the proposed method having a miss rate of 0.53, K-Means 0.54, and hierarchical clustering 0.52. However, at higher FPW values (e.g., 1.0E-01), the miss rate of the proposed method decreases to 0.02, while K-Means is at 0.05, and hierarchical clustering even exhibits a negative value (-0.01), indicating that hierarchical clustering performs poorly under high FPW conditions. Overall, the proposed method maintains a stable decrease in miss rate as FPW increases, particularly showing remarkable performance in the medium FPW range (e.g., 1.0E-02), with a miss rate of 0.32, better than other methods. By comparing the DET curve data of different methods, it can be concluded that the proposed method is more effective in reducing the miss rate under higher FPW conditions, which is significant for the real-time monitoring of engineering safety hazards. Compared to K-Means and hierarchical clustering methods, the proposed method demonstrates stronger stability and robustness in overall identification accuracy and false positive rate control, especially showing better performance in the low to medium FPW range. In contrast, the hierarchical clustering method shows a negative miss rate under high FPW conditions, indicating its unsuitability for scenarios with high false alarm rates.

6.jpg

Figure 6. DET curve of different methods

Table 3. Warning time comparison of different methods

From the warning time data in Table 3, it can be seen that the method proposed in this paper has significantly lower warning times across all scenarios compared to the K-Means and hierarchical clustering algorithms. In multiple test scenarios (from a to h), the warning times of the proposed method mostly range between 1.2 seconds to 1.8 seconds, with the fastest warning time occurring in scenario e at 1.235 seconds, while the slowest is in scenario d at 1.885 seconds. In comparison, the K-Means algorithm has relatively fast warning times but shows greater overall fluctuations, with scenario g reaching 1.389 seconds and scenario f at 1.712 seconds. The hierarchical clustering method exhibits the worst warning time performance, consistently exceeding 200 seconds in almost every scenario, even reaching 278.14 seconds in scenario g. This indicates that the proposed algorithm not only achieves high recognition accuracy but also has a significantly faster warning response time in complex environments compared to other methods. By analyzing the data, it is evident that the proposed improved Mean Shift algorithm combined with SVM significantly enhances real-time performance in the warning system. Although K-Means shows some performance in identification accuracy, its warning times fall short compared to the proposed method, especially in scenarios d and f, where the warning times approach or exceed 1.7 seconds, affecting real-time responsiveness. The warning times of the hierarchical clustering method are extremely unstable, particularly in scenario g, where the warning time far exceeds other methods, indicating its poor applicability in real-time warning applications. In contrast, the proposed algorithm can respond more quickly across different scenarios, demonstrating higher practicality and robustness, thereby meeting the high requirements for real-time monitoring and warning of safety hazards in engineering sites.