Research on the Real-Time Detection Method for Image Processing – Based Civil Structure Crack

Hazard of crack is enormous [1-5] for civil projects. The reasons for cracks in civil structure mainly include load, temperature change, poor quality of construction materials and poor construction process quality. The problem breaks the original mechanical structure of civil project, affects durability of civil structure directly and further causes project safety quality problem. Therefore, the earlier the crack of civil structure is discovered, the better it will be, so that we could make repair, reinforcement and grouting timely [6-11]. With rapid development of image processing technology over the past several years, image processing-based crack detection for civil structure became a focus among scholars both at home and broad [12-14]. The research on rapid and accurate civil structure crack detection method has important research value.

A kind of high-calculation efficiency deep learning (DL) model for crack/non-crack real-time classification was discussed in Geetha and Sim [15], where scalable AI (XAI) was used to study “black box” nature of DL model raised. Integrated with image binarization and Fourier-based 1D DL model, the framework is used to detect and classify concrete crack/non-crack characteristics quickly. A weak supervision network to segment and detect asphalt concrete bridge deck crack was developed in Zhu and Song [16]. First, data was distinguished through encoder and unmarked data characteristics were highlighted, so as to make original data generate weakly supervised convergence starting point. Second, characteristics were classified through K-means cluster (KMC). Then, semantic segmentation was implemented against crack in bridge deck defect image under weak supervision. According to experiment results, the method proposed could yield remarkable and better segmentation effect in all six types of defects, in comparison to other methods reported in references. A kind of DL object detection framework was proposed in Zhou et al. [17] by combining texture characteristics and concrete crack data. Texture characteristics and pre-processed concrete data were combined to increase quantity of characteristic channels, so as to lower demand for model training model and improve training speed. With this framework, concrete crack detection can be realized even if there are a limited number of samples available. A kind of analysis method was proposed in Xu et al. [18] to check crack development of glass fiber reinforced polymer (GFRP) - sea sand concrete composite material. The method is exported by analyzing strain dissipation and shear lag behavior and crack can be recognized quantitatively by the proposed Fourier transform analysis model. As shown by result, forecast error is less than 10%.

In view of complicated construction site of civil project and complex and diversified civil structure, existing method cannot realize crack detection and geometric measurement accurately. The main reason lies in the failure to pre-process civil structure crack through image processing technology and to extract crack intersection point steadily. For complicated and diversified civil structure, overfitting or underfitting problem easily occurs once the existing method is used, resulting in the failure to extract effective crack characteristic information. Therefore, this article researches the image processing-based real-time detection method for civil structure. The civil structure crack image was smoothed with bilateral filtering algorithm in Chapter 2 of this article. In Chapter 3, civil structure crack edge was extracted with interpolation calculation – based subpixel edge detection algorithm, so as to obtain more accurate crack edge position. After finishing pre-processing and edge extraction of civil structure crack image, civil structure crack was detected based on end-to-end object detection network YOLO in Chapter 4. The effectiveness of the model built was verified according to experimental results.

In actual construction site of civil project, light-sensitive face of image acquisition equipment is easily disturbed by lighting or lights on its own light-sensitive unit. Since response signal of significant object edge in image varies gradually and the part with most significant change in gradual variation can be believed the accurate crack edge of civil structure. In this article, interpolation calculation - based subpixel edge detection algorithm was used to extract the crack edge of civil construction in order to obtain the position of crack edge more accurately.

Suppose the distance among upper photosensitive units at A direction is expressed as q and that among upper photosensitive units at B direction as f, it can be believed that crack detection precision of civil structure is affected by q and f. According to the interpolation calculation - based subpixel edge detection algorithm, the first step is to obtain an integer pixel level edge of civil structure. The second step is to implement interpolation calculation for each edge pixel of civil structure crack to obtain information of its sub-pixel edge. Suppose the interpolation point is expressed as a1 and discrete function value as b1, Formula 5 shows the expression formula of polynomial interpolating function as below:

$g(a)=\sum_{\substack{l=0 \\ m \neq l}}^m \frac{\left(a-a_0\right) \cdots\left(a-a_{m-1}\right)\left(a-a_m\right)}{\left(a_l-a_0\right) \cdots\left(a_l-a_{m-1}\right)\left(a_l-a_m\right)}\quad b_l$    $=\sum_{l=0}^m \prod_{\substack{i=0 \\ i \neq l}}^m \frac{a-a_i}{a_l-a_i} b_l$               (5)

Sobel operator was used to detect and determine the edge point (a_i, b_j) by taking the three points a_i-q, a_i and a_i+q as interpolation points, so as to further obtain position of sub-pixel edge E. Suppose the gradient function value is expressed as O_{(i, j)}, take the three points O_{(i-1, j)}, O_{(i, j)} and O_{(i+1, j)} successively in A direction of civil structure crack gradient and put information of various points into above formula, to generate the difference function E(a) in A direction, which can be expressed as:

$E(a)$$=\frac{\left(a-a_i\right)\left[a-\left(a_i+q\right)\right]}{\left[\left(a_i-q\right)-a_i\right]\left[\left(a_i-q\right)-\left(a_i+q\right)\right]} \quad O_{(i-1, j)}$ $+\frac{\left[a-\left(a_i-q\right)\right]\left[a-\left(a_i+q\right)\right]}{\left[a_i-\left(a_i-q\right)\right]\left[a_i-\left(a_i+q\right)\right]} \quad O_{(i, j)}$ $+\frac{\left(a-a_i\right)\left[a-\left(a_i-q\right)\right]}{\left[\left(a_i+q\right)-\left(a_i-q\right)\right]\left[\left(a_i+q\right)-a_i\right\rceil} \quad O_{(i+1, j)}$                    (6)

Derive E(a), to make sure d(e(a))/da=0. Suppose the pixel width in A direction is expressed as q and the condition O_{(i, j)}>O_{(i-1, j)}, O _{(i, j)}>O_{(i+1, j)} is satisfied, coordinate of sub-pixel A direction in the form of discrete difference can be obtained further through the formula below:

$A=a_i+\frac{O_{(i-1, j)}-O_{(i+1, j)}}{O_{(i-1 . j)}-2 O_{(i, j)}\quad+O_{(i+1, j)}} \times \frac{q}{2}$            (7)

1.png

Figure 1. Direction diagram of civil structure crack edge

Similarly, take three points O_{(i-1, j)}, O_{(i, j)} and O_{(i+1, j)} in B direction and put them into Formula 5, to obtain the expression formula of interpolation function E(b) in B direction as shown in the formula below:

$E(b)=\frac{\left(b-b_i\right)\left[b-\left(b_i+q\right)\right]}{\left[\left(b_i-q\right)-b_i\right]\left[\left(b_i-q\right)-\left(b_i+q\right)\right]} \quad O_{(i-1, j)}$$+\frac{\left[b-\left(b_i-q\right)\right]\left[b-\left(b_i+q\right)\right]}{\left[b_i-\left(b_i-q\right)\right]\left[b_i-\left(b_i+q\right)\right]} \quad O_{(i, j)}$$+\frac{\left(b-b_i\right)\left[b-\left(b_i-q\right)\right]}{\left[\left(b_i+q\right)-\left(b_i-q\right)\right]\left[\left(b_i+q\right)-b_i\right]} \quad O_{(i+1, j)}$                (8)

Derive E(b), to make sure d(e(b))/db=0. Suppose the pixel in B direction is expressed as f and the condition O_{(i, j)}>O_{(i-1, j)}, O _{(i, j)}>O_{(i+1, j)} is satisfied, coordinate of sub-pixel B direction in the form of discrete difference can be obtained further through the formula below:

$B=b_i+\frac{O_{(i-1, j)}\quad-O_{(i+1, j)}}{O_{(i-1 . j)}-2 O_{(i, j)}\quad+O_{(i+1, j)}} \times \frac{f}{2}$             (9)

2.png

Figure 2. Sobel operator template

Since the sub-pixel position of civil structure crack edge is obtained through derivation and calculation, gradient direction of civil structure crack edge is indefinite. Figure 1 shows the direction diagram of civil structure crack edge. To obtain gradient direction of civil structure crack edge point, Sobel operator was used in this article to roughly extract civil structure crack edge, and gradient direction of known edge pixel was used to approximately substitute gradient direction of unknown sub-pixel. Through interpolation in this direction, accurate sub-pixel position of civil structure crack edge was calculated and obtained. The specific algorithm implementation steps are as below:

STEP1: By combining the four Sobel operator orientation templates (see Figure 2), implement convolution calculation based on pixel-level edge point E₀ in civil structure crack image extracted via Sobel operator. Convolution calculation result is expressed as B_i;

STEP2: Take absolute value of B_i. Suppose template in the i^th direction is expressed as O_i. 3×3 field of edge point E₀ is expressed as E'₀, i.e. gradient value of edge point E₀ in the i^thdirection of B_i and gradient magnitude of 3×3 domain center point of E₀ are expressed as B_max, gradient magnitude for the maximum value E₀ can be obtained as per the formula below:

$\left\{\begin{array}{l}B_i=O_i \cdot E_0^{\prime} \\ B \max \left(\left|B_i\right|\right)_{\max }\end{array}\right.$           (10)

STEP3: Judge the gradient direction of this edge point based on plus or minus of corresponding B_i of B_max. If it is plus, it can be believed that gradient direction of this edge point is the same with template direction. If it is minus, it can be believed that gradient direction of edge point is inverse to template direction.

STEP4: Approximatively substitute gradient direction of unknown sub-pixel with gradient direction of edge point coarsely extracted and implement interpolation calculation in this direction. Suppose the adjacent two points of E₀ point in gradient direction are expressed as E_-1 and E₊₁, gradient values are O_-1(O_{(i-1, j)}, O_(i。j-1) and O₊₁(O_{(i+1, j)}, O_(i。j+1) and included angle between gradient direction and A-axis is expressed as β, the sub-pixel coordinate of civil structure crack edge point is expressed as:

$A=a_i+\frac{O_{(i-1, j)}\quad-O_{(i+1, j)}}{O_{(i-1, j)}-2 O_{(i, j)}\quad+O_{(i+1, j)}} \times \frac{q}{2} \sin \beta$            (11)

$B=b_i+\frac{O_{(i-1, j)}\quad-O_{(i+1, j)}}{O_{(i-1, j)}\quad-2 O_{(i, j)}+O_{(i+1, j)}} \times \frac{f}{2} \cos \beta$          (12)

The equation above satisfies O_{(i, j)}>O_{(i, j-1)}, O_{(i, j)}>O_{(i, j+1)}.

Dataset of civil structure crack image used in this article includes 11,529 pictures. See Figure 1 for the number and proportion of images of specific transverse crack (Category 1), longitudinal crack (Category 2), irregular crack (Category 3), block crack (Category 4) and spray paint crack (Category 5) (Table 1).

Table 1. Number and proportion of civil structure crack attributes

Table 2. Experimental result comparison of crack edge detection

Figure 4 shows the pictures about crack image comparison after bidirectional filtration processing. After finishing image pre-processing, this article designs comparison experiment in order to check validity of the raised extraction algorithm for civil structure crack edge. The test method could improve subpixel edge detection algorithm (Method 1) and Sobel edge detection algorithm (Method 2). The experimental results are shown in Table 2, from which it can be learnt that the result fluctuation is greater when measuring geometric quantity of crack width with crack edge obtained as per Method 2, while measurement with crack edge obtained as per Method 1 is more accurate and deviation amplitude is smaller in case of crack width measurement for civil structure sample crack, meeting the requirements for construction accuracy.

4.png

Figure 4. Comparison picture of crack images after bidirectional filtration processing

5.png

Figure 5. Influence of noise on crack edge detection performance

Figure 5 shows the results of comparison with other classic edge detections, including differential edge detection method (reference algorithm 1), Roberts operator (reference algorithm 2), Prewitt operator (reference algorithm 3), Lapras operator (reference algorithm 4), LoG operator (reference algorithm 5) and Canny operator ((reference algorithm 6). As shown by the figure, algorithm proposed in this article reaches its optimal status, when noise concentration is relatively low; in the meantime, with increase of noise concentration, the algorithm is always kept at high quality factor FOM and peak SNR (signal to noise ratio). It shows that lager non-maximum value area has stronger restraint against gaussian noise. For Sobel operator, peak SNR under gaussian noise and mean square error have better performance, but the quality factor FOM value isn’t as ideal as expected.

Reference models for performance comparison of civil structure crack detection model include Faster R-CNN, VGG-Unet, SCNN and SSD. See Table 3 for experimental results of all these algorithms. As shown in Table 3, model of this article will have higher detection accuracy rate and faster speed in self-made image dataset of civil structure crack. It was learnt through algorithm performance comparison, in comparison with other algorithms that are unsuitable for minor goal segmentation and have magnification operation oriented to characteristic pattern covered by candidate box, calculation quantity of model in this article is less, FPS is relatively high, accuracy rate reaches 95.42% and FPS value reaches 02, reaching the expected effect with respect to real-time detection task for civil structure crack in this article.

Table 3. Experimental result summary of different algorithms

6.png

Figure 6. MAP result

Figure 6 shows the indicator MAP value of detection accuracy ratio of civil structure crack detection model established in this article. The indicator is mean value of area AP value under accuracy rate – recall rate curve of various crack categories. Compared with AP value, description of MAP for image dataset range is more accurate. According to Figure 6, AP value both of Category 4 and 5 exceeds 0.9, AP value of Category 3 crack reaches 0.85 and AP value of the another two categories is lower than 0.8. The main reason is that these two kinds of cracks are usually attached with discontinuous and irregular distribution characteristics and length of part of cracks is very short. By combining AP value of existing different categories of caracks, MAP value of the whole dataset is obtained as 0.844, relatively ideal.