Vehicle Classification and Counting System Using YOLO Object Detection Technology

The convolutional neural network is a multi-level feedforward neural network. The convolutional neural network can automatically learn image features with high nuances, and can be identified on the graph without the function of hand-crafted features. The object detection task is composed mainly of three different algorithms: object localization, feature extraction and image classification. This study uses YOLO object detection that combines the original scattered object detection steps into a single neural network, predicts each bounding box through the features of the entire image, and simultaneously calculates the probability of each bounding box for each class. The object bounding box and the location of the center point are then obtained. YOLO detects objects from the entire image and end-to-end [25], trains and calculations, and also maintains high precision in real-time operations.

The YOLO object detection method is based mainly on the GoogLeNet [26] image classification model. It extracts the input image characteristics from the initial convolutional layer and predicts the probability of the full-connected layer output. Each square is predicted by the convolutional neural network to include the target frame. The YOLO detection method confidence index is the basis for the detection model output. Eq. (1) of the confidence index is as follows:

$\text { Confidence Score } \left.\sigma\left(t_{o}\right)=\operatorname{Pr} \text { (Object }\right) * \text { IoU }_{p r e d}^{\text {truth }}$        (1)

Each bounding box corresponds to a confidence score. If there is no object in the grid cell, the confidence score is zero. If there are objects in the square, the confidence score is the perdition bounding boxes and the real marker IoU value of the bounding boxes. There are two main methods for solving multi-scale problems in the past, one is the image pyramid, and the other is the convolution kernel pyramid. When two different objects are in close proximity, causing the center point of the two objects to share a set of grid cells, the problem of object overlap occurs. In order to solve the problem of overlapping multiple objects in an image, the author proposes an anchors mechanism to introduce anchors of different sizes and different aspect ratios as pre-defined default bounding boxes.

During the object detection process the same area in the image is easily covered by multiple bounding boxes. The non-max suppression algorithm solves this common problem. When bounding box prediction is repeated in object detection, IoU is used to define a threshold value to define the repeated region. A threshold value IoU ≥ 0.5 is the object that we want to focus on. The bounding box datum for all IoU comparisons is then identified, which is the bounding box with the highest probability of prediction in this repeating region. Bounding boxes with the IoU ≥ 0.5 of the base bounding boxes are then discarded. IoU refers to the intersection between the ground-truth bounding boxes and perdition's bounding boxes divided by the union of the two bounding boxes. The IoU calculation is shown in the ground-truth and predicted results shown in Figure 1. The red line is the correct result for the artificial mark. The green line is the result predicted by the algorithm. What IoU has to do is measure the algorithm accuracy in these two results. Therefore, IoU can be used directly as an important indicator of the detection model accuracy.

1.png

Figure 1. IoU accuracy calculation

2.png

Figure 2. Schematic diagram of Region of Interest

3.png

3.png

Figure 3. Image tracking area after image processing

Because the image detection process will get a new object list every time an object is detected in the object frame, the objects in the upper and lower frames cannot determine whether it is the same vehicle if the training weight is insufficient. In the travel direction a single object will result in multiple calculations. In order to avoid double counting it must be solved by object tracking. In the image processing procedure, the images of some objects can only be processed and analyzed through region of interest (ROI). The image ROI often outlines the areas to be processed in the form of boxes, circles, ellipses, irregular polygons, and the like. A part of the image is separated from the area for processing, which reduces the image processing analysis workload, improves the precision and reduces the processing time.

In the middle of the lane image, a count line is drawn perpendicular to the lane in the lane image. The thickness is 1 pixel. A thick line of 20 pixels is expanded on the upper and lower sides of the count line. The two thick lines cover the ROI1 and ROI2 regions of interest. The tracking action is started when a vehicle center point position enters ROI1. If the center point of the object enters ROI2 through the counting line in the next frame, the vehicle direction is obtained through the upper and lower frames and the number of vehicles in that direction is increased by one, as shown in Figure 2. It shows a schematic diagram of the area of interest. It is not necessary to calculate the vehicle flow and direction for the entire image during object tracking. Just add the drawn line segment and ROI area to the hidden layer and then judge and calculate when a vehicle passes this line or this area. After the center point of the object passes through the traced area drawn in the front, the number of objects is added. Figure 3 shows the image processing detection area after image processing.

This experiment recorded videos on straight-line bidirectional roads, T-shaped intersections and cross-type intersection. Three experimental conditions were recorded in 1 to 2 minutes of video and divided into two parts for testing, the first part is internal testing. The internal test is to sample five videos codenamed A1-A3 and B1 and C1 for training, and obtain the weight after training, and then use this weight to test the five videos with the classification and counting system of this study. The second part is the external test. The external test is in the un-retrained samples of the video. After the training weights, the external video data was tested with the classification and counting system of this study. Eleven films codenamed A4-A12 and B2 and C2 for external testing. The actual number of vehicles on the road was compared with the number of vehicles measured by the system. Three counting results were obtained using the Confusion Matrix, which is defined as shown in Eqns. (2) to (4) [27].

$P R=\frac{T P}{T P+F P}$       (2)

$\mathrm{RR}=\frac{T P}{T P+F N}$       (3)

$F_{m}=\frac{2 \cdot P R \cdot R R}{P R+R R}$       (4)

The number of positive examples detected correctly is called (True Positive, TP). The number of positive examples detected as positive examples is called (False Positive, FP), and the number of positive examples detected as negative examples is called (False Negative, FN). The precision rate (PR) is the ratio of the correct number of parameters retrieved by the system to the total number of parameters retrieved by the system. The precision rate is a measure ranging between 0 and 1 that indicates the detection accuracy rate relative to ground truth. The recall rate (RR) is the correct number of turns retrieved and the actual total number of vehicles. The recall rate is also a measure ranging between 0 and 1 that can be interpreted as the converse of the omission error.

The ratio of $F_{m}$  (F-measure, $F_{m}$ ) is a weighted harmonic average that takes into account both the precision and the recall. This is applied to an evaluation of the retrieval system effectiveness and can also be used to compare the difference in performance between different technologies or systems. The system detection and counting results for three vehicle internal testing classes under the videos codename A1-A3 are relatively stable. Table 3 shows the average counting results for three vehicle external testing classes under the videos codename A4-A12. The $F_{m}$ metric of 9 videos are all above 97%.

Table 3. Results of external test in straight-line bidirectional roads

Table 4. Results of external test in T-shaped intersection roads

Table 5. Results of external test in cross-type intersection

Table 4 shows the average counting results for three vehicle classes in the T-shaped intersection external testing under the videos codename B2. It can be seen that the system detection count results in 4 over counts and 1 undercount in 72 vehicles. It has been observed that due to the complexity of the road environment and the angle of recording, small objects are easily covered by large vehicles during detection. This may result in vehicle leakage. The case of multiple calculations occurs when an object passes through the zebra crossing and the background is complicated by the black and white stripes of the zebra crossing. When the object is detected, a jump in the object frame leads to multiple calculations. The counting system will then have multiple counts and misses. The rate and recall rate and F_m metrics are both decreased, but the precision rate remains at 94.74%. The recall rate remains at 98.63%. The F_m  metric also maintained good performance above 96.64%.

Table 5 shows the average counting results for three vehicle classes in the cross-type intersection external test under the videos codename C2. C1 can be seen that in the 55 vehicles five cars are missing from counting system. It was observed that the poles, wires and traffic lights created shadows on the road due to the sun, which caused the system to calculate higher leakage than the first two sections because of the intersection. This traffic environment was shot with an aerial camera. The image is easy to shake up and down slightly, and the counting line is fixed in the image. It is easy to generate multiple counts when shaking, so that the recall rate is maintained at 94.03%. In the case of a missed vehicle, the precision is only maintained at 100%. After the precision and recall average the F_m  metric is 96.92%.

Table 6. Average counting results for three classes of vehicle in index by road type

Table 6 shows the average counting results for three vehicle classes in the videos codename A1-A12, B1-B2 and C1-C2 index by road type. It can be seen straight line bidirectional road that in the 1944 vehicles, 32 vehicles are extra and 10 vehicles are missing from counting system. After the precision and recall average the F_m  metric is 98.93%. In the T-shaped intersection 137 vehicles, 32 vehicles are extra and 10 vehicles are missing from counting system. After the precision and recall average the F_m  metric is 98.21%. In the cross-type intersection 118 vehicles, 1 vehicle is extra and 5 vehicles are missing from counting system. After the precision and recall average the F_m  metric is 97.52%. In the three categories of roads, heavy vehicles and motorbike, the heavy vehicles do not have over counting and undercounting. According to observations of three roads, more complex routes and backgrounds are prone to systematic misjudgments. The detection performance of four models is summarized in Table 7. It can be clearly seen from the results that Yolov3-SPP has the best recognition rate effect in three road conditions of present system.

Table 7. Detection performance of four models