Adedayo S. Adebayo | Joshua Olusola | Rasaq A. Kazeem | Omolayo M. Ikumapayi* | Imhade P. Okokpujie | Ikenna D. Uchegbu
© 2022 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
Traffic congestion and management have posed a major challenge to many cities in the world. Intelligent traffic management system plays an important role in monitoring and enforcing traffic laws with reduced labor. This paper uses vehicle information recognition to identify unpermitted lane shunting at the University of Ibadan main gate. The vehicle recognition system captures three main details of the vehicle; its license plate, make, and colour to ensure the system, which is named UiScope, is robust enough. Machine learning and deep learning algorithms including Convolutional Neural Network (CNN), Support Vector Machine (SVM), and K-Nearest Neighbor (KNN) algorithms are used to train classifiers for vehicle make, license plate, and colour recognition. The captured details are uploaded on a Structured Query Language (SQL) database to create a blacklist of vehicles that are shunted. The querying of the database is used to determine the shunted vehicle. The success rate for plate identification is 92%, character segmentation is 87%, character recognition is 75%, and vehicle colour recognition is 78%.
traffic congestion, UiScope, license plate recognition, traffic management, vehicle colour recognition, Vehicle make recognition, SQL database, shunting
A major concern to both developed and developing countries is traffic management and this centres on the need to curb traffic congestion at road junctions and entry/exit gates of universities and other educational institutions [1-3]. Traffic congestion has a lot of adverse effects on road users depending on its severity and duration. The need for vehicle owners to get to their destination on time especially before and after a long day’s work is the prime reason for the advent of intelligent transportation systems. Traffic flow is occasionally heavy especially at the close of business in many areas [4, 5], as a result of motorists scrambling to enter or leave the campus either to pick up kids and wards from staff schools or on days when important events, like university inaugural lectures, are hosted on the campus; these lead motorists to leave lanes with longer queues to less busy lanes. In many cases, two-lane roads have certain traffic laws restraining vehicles to keep to a particular lane if they intend to negotiate a turn at an intersection [6]. This is to avoid a complete gridlock of the road at points of intersection. Despite the use of regulatory traffic signs to instruct road users on the permitted movements, some motorists still shunt. They drive on the prohibited lane with the hope of shunting back to the required lane just before the road intersection. This usually causes more traffic problems and concurrently leads to traffic congestion as shown in Figure 1.
To improve the traffic flow in such areas, an intelligent transportation system is used to monitor and identify vehicles taking such prohibited movement in no-shunting areas. A traffic surveillance camera is a major advancement in intelligent transportation systems. It has brought a lot of vision-based technologies that can provide information on traffic flow, perform vehicle counts, vehicle speed, and identify a stolen vehicle [7]. These technologies can only be made effective in enforcing traffic laws when vehicles are accurately identified.
Figure 1. Traffic congestion caused by unpermitted shunting on a two-lane road at the University of Ibadan exit gate
Vehicle identification can be carried out using major pointers including license plate numbers, make and colour [8]. Automatic license plate recognition system has raised several research interests across various license plate types in the world due to their distinctive characteristics. Though the license plate is a unique parameter for vehicle recognition, license plate recognition cannot be solely relied on for vehicle recognition systems since license plates can be damaged, covered, removed, or modified [9]. Also, since the vehicle is in motion, number plate recognition alone is not enough to spot and identify an erring vehicle that has violated the no-shunt order. For effective vehicle identification at the University of Ibadan (UI) main gate, therefore, a robust system called UiScope which is a combination of number plate recognition, vehicle make or brand and vehicle colour is hereby presented. Vehicle make and colour recognition systems are systems that can also be utilized to identify and classify vehicles. However, these independent recognition systems cannot be effectively used for real-time tracking due to their inadequacies. They are therefore combined as a robust system for effective vehicle recognition known as UiScope for traffic flow management at the University of Ibadan main gate.
In recent times, machine vision has brought about various studies of its application to traffic management, and image-based vehicle monitoring is one of the several aspects of traffic monitoring. Vehicle recognition systems are used in vehicle feature-based tracking. As a result of this, a lot of attention has been given to license plate recognition systems by the research community. A distinctive feature in this field is that research is restricted to a specific region, city, or country. Over the past decade, many studies have been proposed for a vehicle make recognition. Yu et al. [10] conveyed the development and operation of an effective RFID tag-based framework for automatically detecting and gathering data from running vehicles on roads. The design involves active reading equipment and electronic tags, the distribution of tracking ground stations, the development of a two-layered channel, and tracking software. The system will traffic flow statistics, support traffic monitoring, special vehicle tracking, and traffic scheduling on the Internet of Things. Duan et al. [11] proffered ISeeCarRecognizer, an automatic VLP identification system, to decode the registration plates of Vietnamese VLPs at traffic tolls. The system is composed of three parts: number plate segmentation, VLP detection, and number plate recognition. Arya and Suwardi [12] theorized conducting research and developing a software system that consists of a Number Plate Recognition System used in the Indonesian traffic system. The authors developed a system that can be used for traffic light surveillance, automated parking, vehicle surveillance, and electronic toll collection for law enforcement purposes, among other things. Also, Manzoor et al. [13] proposed an innovative and durable real-time model recognition and vehicle make system that can accurately identify vehicles. To symbolize the dataset, the technique extracts visual features from vehicle images and generates relevant features. Besides, Yang [14] applied deep learning theory to vehicle identification, creating a vehicle identification algorithm based on a detailed convolution neural network. The proposed technique outperformed the conventional CNN in-vehicle image analysis, according to experimental observations. Furthermore, Namazi et al. [15] invented different methods to enhance a traffic control system when there is a mix of modern vehicles and human-driven vehicles with varying degrees of autonomy. The results indicated that the algorithms can correctly determine the type of detected vehicle in the widely researched scenarios 95.21% of the time. Two proposed methods with an accuracy rate of 91.01% and 91.73% evaluate the accuracy of identifying the lane the detected vehicle is in.
Sivaraman and Trivedi [16] presented a comprehensive active-learning structure for on-road vehicle recognition and monitoring. This research builds tracking systems and vehicle recognition using an innovative intelligent approach. The results demonstrated that this framework produces an efficient, robust, and precise onboard tracking and vehicle recognition system. A secure in-vehicle ASR system called SIEVE was created by Wang et al. [17]. It can successfully differentiate between voice control coming from the driver, other commuters, or electronic sound systems. The study used a SIEVE working model and conducted a field experiment under various driving circumstances. In-vehicle ASR systems can be used to defeat a variety of adversarial voice commands, according to the results. Moreover, Clady et al. [18] addressed a tool for determining multiclass vehicle types (model and make) using aligned contour points. A technique for developing a model from multiple frontal images obtained was provided. This approach was evaluated on a realistic data set (830 images with 50 various vehicle categories), yielding comparable results for corresponding recognition frameworks with various feature options. In addition, Ran et al. [19] demonstrated a vision-based strategy for recognition and vehicle detection, tracking of multiple vehicles, and motion parameter calculation using a series of grayscale-captured images from a moving vehicle. The experimental results indicated a stable system for real-time vehicle recognition. According to Vançin and Erdem [20], a new sensor circuit was created to provide quick, innovative solutions in real time for a vehicle detection technique. A proposed classification method and magnetic trademark length criterion were used to accurately classify passing vehicles as minibusses, cars, trucks, and buses with a 95% accuracy rate. In their study, Jusoh and Mohamad Zain [21] concentrated on using FCC to test the efficacy and accuracy of the system's ability to recognize characters with the help of their features. The outcome demonstrated that 95% recognition accuracy is achieved when characters' features are combined with FCC (FCCwF).
Rad and Jamzad [22] implemented in Visual C++ a system for lane change detection, vehicle classification, and vehicle counting. The vehicle tracking system developed applied Kalman filter and background differencing techniques. Coifman et al. [23] developed a real-time vehicle tracking system that is effective even in the presence of partial occlusion. It applies feature-based tracking by capturing and tracking salient vehicle features such as window corners and bumper edges. Owamoyo et al. [24] developed a license plate recognition for Nigerian vehicles by applying histogram equalization and median filter for noise removal. License plate extraction is done by Sobel vertical edge detection and morphology operations while projection analysis and connected component analysis are used for character segmentation. Amusan et al. [25] proposed the use of neural networks in Nigerian vehicle license plate recognition. It uses a canny edge detection operator for plate detection and bounding box techniques for character segmentation. ANN and backpropagation algorithms are used for alphanumeric character recognition. Emami et al. [26] presented a recognition of vehicle make and model from back views using the location of license plate and taillights of the vehicle as the Region of Interest (ROI). Hierarchical classification is used by dividing the ROI and considering different weights of features across the various vehicle classes. In their work, Liu and Wang [27] proposed the use of a domain-general dataset to build and compare vehicle make recognition models trained on CNN using the transfer learning method. CaffeNet, GoogleNet, and VGGNet are the baseline models that were used to train the dataset which had 196 different classes.
Vehicle colour recognition has also attracted several interests in intelligent transportation systems. Chen et al. [28] used Feature Context (FC) with selected configuration to divide the images into subregions, and then a histogram is created for each subregion before it is learned using linear SVM. Not all values in the histogram are used to classify the vehicle colour, only the values clustered to form a codebook for the problem. In another work, Dule et al. [29] tried to tackle the vehicle colour recognition problems using a 2D histogram with ROI configuration as features and a Neural Network as the classifier [30]. It also used FC but differs from that of Chen et al. [28] in the number of ROI used. In summary, any system designed for vehicle recognition requires more than one approach for effective traffic monitoring. This work presents a robust system made up of License plate recognition, colour, and make of the vehicle all in one, for traffic monitoring at the exit gate of the University of Ibadan, Nigeria.
A typical traffic flow at the exit gate of Nigeria’s Premier University - the University of Ibadan, is used as a case study for this work. This new system developed has therefore been named UiScope. This is a system that comprises two installed cameras that capture “erring” vehicles; store the particulars like, colour, number plates and vehicle makes in a database, and apprehend the violators of the "no-shunt-order" for effective traffic management at the University of Ibadan main gate. Existing law on the two-lane road leading out of the University main gate restricts vehicles leaving the campus to the right lane while it permits others that intend to negotiate a turn to remain on the left lane. As a result of this, there is always a traffic gridlock at the rightmost lane as compared with the left lane due to the number of vehicles leaving the Campus at any given instance. This work attempts to create a system that would enforce the existing no-shunting law to ease traffic flow for vehicles leaving the campus through the university's main gate.
A vehicle recognition algorithm is installed on two smart cameras. One camera is installed at the beginning of the no-shunting region to capture details of vehicles on the left lane. These details are uploaded to a blacklist database server. This database is queried by the other camera installed at the main gate to detect vehicles blacklisted from appearing on that lane. The process flow chart of the UiScope is shown in Figure 2. The system is developed to perform license plate, make, colour recognition, and database querying. All the algorithms in the system are implemented using PYTHON version 3.6.8, a proprietary software developed by Python Software Foundation, and the configuration of the computer includes Intel® Corei5, 2.67 GHz processor, Windows 7 operating system with 8GB RAM.
3.1 License plate recognition
This system is developed to recognize alphanumeric characters in Nigerian license plates using Python software. This recognition is performed in three major stages: detection of license plate area, character segmentation using image projection, and character recognition using SVM. Before these stages are performed, a few image pre-processing techniques are carried out to improve contrast, reduce noise and reduce the processing time of the system.
Figure 2. Process flow chart of the algorithm
Figure 3. Image binarization using Otsu thresholding
Noise removal: This process filters random variations of brightness or colour information caused by weather conditions. Gaussian filter is used because it preserves the edges in the image more effectively than many other filters.
Image binarization: This suppresses greatly image noise by eliminating image details that are not in the ROI (see Figure 3). Otsu thresholding developed by Nobuyuki Otsu is used for this image binarization.
Plate detection: It is at this stage that image projection is carried out on the pre-processed image. The region of interest, which is the license plate, is the area with the highest density of contours. The top and bottom boundaries of the license plate are defined by horizontal projection.
Character segmentation: In this stage, the characters on the license plate are mapped out and segmented into individual images. It uses vertical projection to find the gaps between characters on a license plate. The Bounding box technique is then used to create blocks of characters. These blocks capture and isolate each alphanumeric character on the license plate.
Character recognition: Nigerian license plate comprises both alphabets and numbers arranged in a certain pattern. The pattern helps to eliminate less relevant characters in the license plates. Support vector machine is used to train two models: one for the alphabet characters recognition and the other for the number characters recognition. This is done to enhance the recognition accuracy since a structured pattern of alphabets and numbers exists.
3.2 Vehicle make recognition
The vehicle make recognition is performed by training a classifier using a convolutional neural network. The dataset is created using vehicle images sourced from the internet using Google images. This system is built in two main stages and is introduced as follows.
Building of dataset: The dataset is created using vehicle images sourced from the internet using Google images. It is initially cleaned up to remove irrelevant images such as vehicle interior, vehicle engine, or damaged vehicle images. It also comprises over 8000 images classified into eight different vehicle makes.
Building and training of neural network: This stage involves the creation of layers in the neural network. The transfer learning method is adopted to build and train the network to save iteration time and increase accuracy. ResNet-50 model is the convolutional neural network used for transfer learning.
3.3 Vehicle colour recognition
A vehicle colour recognition algorithm is also added to the system to make it robust enough to cater to various abnormalities. The major challenge to this recognition is the effect of weather conditions on the illumination of the vehicle's colour. The recognition is carried out by extracting the vehicle from the entire image and then using a machine-learning algorithm to train a classifier.
Vehicle detection: This is the process of isolating the vehicle from the background to acquire the necessary region of interest for recognition. An object detection algorithm developed by TensorFlow is used to build a vehicle detection model. The trained model is used to detect the possibility of a vehicle in an image and to determine the position of the vehicle(s) in that image.
Training of vehicle colour classifier: A colour classifier is trained using KNN to recognize the approximate colour of the extracted vehicle from the image. The classifier is trained using plain images of different colours. These images vary in illumination in other to cater for different shades of the vehicle colour.
3.4 Database querying
A database is created using SQLite to save the license plate, make, and colour details of vehicles captured by the camera. It also saves the time, day and date that the details are captured. A camera is connected to the database to save the details while another camera, installed at the point of intersection, is connected to the same database to check for a details match. Only the last 100 vehicle details are queried to save the computational time required to scan the entire database.
Barring defaulted vehicles
Identifying the erring vehicles is not enough to control the flow of traffic out of the university community but apprehending the violating vehicle gets necessary to warn other motorists against shunting. An automatic boom barrier is designed to bar the exit of defaulting vehicles. The mechanism which is driven by a servo motor is mounted by the smart camera at the gate. The automatic boom barrier communicates with the database server using a Modbus relay. Once the defaulting vehicle is identified at the main gate, the boom arm remains in a barring position to hinder the exit of that vehicle. The exploded view of the designed automatic boom barrier is shown in Figure 7.
Ticketing system
The hindrance of defaulting vehicles from exiting through the University's main gate will increase traffic congestion if the system does not put a check on the delay caused by defaulting vehicles. In order not to create another problem, a ticketing system with fees that appreciates with time is used as a penalty for defaulting vehicles. Counter and timer operations are inculcated into the system to manage the problem.
The outcome of character segmentation, vehicle colour recognition, and vehicle make recognition are shown in Figures 4, 5, and 6, respectively. Performance analysis for license plate recognition is performed across each of the three stages. Plate detection has a success rate of 92%, character segmentation 87% and character recognition 75%. The make recognition is tested across the eight different vehicle makes to obtain an accuracy of 82%. While vehicle colour recognition is with 78% success rate. The result obtained for the license plate recognition shows a good performance of horizontal and vertical projection in obtaining plate extraction and character segmentation. The transfer learning method used in training the model for vehicle make recognition also has a significant success rate. The vehicle colour recognition was performed on the vehicle extracted from the image. This provided a greater success rate than when the recognition was run on the entire image.
Figure 4. Result of the character segmentation
Figure 5. Results of vehicle colour recognition
Figure 6. Results of the vehicle make recognition
Figure 7. Exploded view of the designed automatic boom barrier
In this paper, a system known as UiScope has been proposed using vehicle recognition systems and database querying for managing unpermitted shunting on two-lane roads at the University of Ibadan main gate. The frontal view of the vehicle is the region of interest needed for the recognition systems. The analysis carried out on the system shows that it is robust enough to perform its desired objective. The proposed method will be effective in determining an unpermitted movement in a no-shunting region and capture the license plate, vehicle make and colour details of the defaulting vehicle. The recognition of three germane vehicle information in one system protects it against the inadequacies of the three recognition systems if done individually. The system also apprehends and punishes traffic offenders to warn other vehicles against unpermitted shunting. This will assist in the efficient management of traffic flow at the University of Ibadan main gate.
[1] Banerjee, D., Chakraborty, S.K., Bhattacharyya, S., Gangopadhyay, A. (2009). Appraisal and mapping the spatial-temporal distribution of urban road traffic noise. International Journal of Environmental Science & Technology, 6(2): 325-335. http://dx.doi.org/10.1007/BF03327636
[2] Fouracre, P., Dunkerley, C., Gardner, G. (2003). Mass rapid transit systems for cities in the developing world. Transport Reviews, 23(3): 299-310. http://dx.doi.org/10.1080/0144164032000083095
[3] Ghosh, K., Singh, N., Maitra, B. (2019). Traffic management during a big sport event: a case study in Kolkata City. Journal of the Eastern Asia Society for Transportation Studies, 13: 1873-1891. https://doi.org/10.11175/easts.13.1873
[4] Aljanahi, A.A.M., Rhodes, A.H., Metcalfe, A.V. (1999). Speed, speed limits and road traffic accidents under free flow conditions. Accident Analysis & Prevention, 31(1-2): 161-168. http://dx.doi.org/10.1016/S0001-4575(98)00058-X
[5] Geller, M.D., Sardar, S.B., Phuleria, H., Fine, P.M., Sioutas, C. (2005). Measurements of particle number and mass concentrations and size distributions in a tunnel environment. Environmental Science & Technology, 39(22): 8653-8663. http://dx.doi.org/10.1021/es050360s
[6] Hossain, M.M., Sun, X., Mitran, E., Rahman, M.A. (2021). Investigating fatal and injury crash patterns of teen drivers with unsupervised learning algorithms. IATSS Research, 45(4): 561-573. http://dx.doi.org/10.1016/j.iatssr.2021.07.002
[7] Tourani, A., Shahbahrami, A., Akoushideh, A. (2017). Challenges of video-based vehicle detection and tracking in intelligent transportation systems. Soft Computing, 1-9.
[8] Anandhalli, M., Baligar, V.P. (2017). Vehicle detection and tracking based on color feature. In 2017 International Conference on Recent Advances in Electronics and Communication Technology (ICRAECT), 240-248. http://dx.doi.org/10.1109/ICRAECT.2017.48
[9] Mutua, S.M. (2016). An automatic number plate recognition system for car park management (Doctoral dissertation, Strathmore University).
[10] Yu, M., Zhang, D., Cheng, Y., Wang, M. (2011). An RFID electronic tag based automatic vehicle identification system for traffic IOT applications. In 2011 Chinese Control and Decision Conference (CCDC): 4192-4197. http://dx.doi.org/10.1109/CCDC.2011.5968962
[11] Duan, T.D., Du, T.H., Phuoc, T.V., Hoang, N.V. (2005). Building an automatic vehicle license plate recognition system. In Proc. Int. Conf. Comput. Sci. RIVF, 1: 59-63.
[12] Arya, F., Suwardi, I.S. (2007). License Plate Recognition System for Indonesian Vehicles. In Bandung: Proceedings of the International Conference on Electrical Engineering and Informatics, Institut Teknologi Bandung.
[13] Manzoor, M.A., Morgan, Y., Bais, A. (2019). Real-time vehicle make and model recognition system. Machine Learning and Knowledge Extraction, 1(2): 611-629. http://dx.doi.org/10.3390/make1020036
[14] Yang, Y. (2020). A Vehicle Recognition Algorithm Based on Deep Convolution Neural Network. Traitement du Signal, 37(4). http://dx.doi.org/10.18280/ts.370414
[15] Namazi, E., Li, J., Mester, R., Lu, C. (2020). Identifying and counting vehicles in multiple lanes by using a low-cost vehicle-mounted sensor for intelligent traffic management systems. In International Conference on Hybrid Artificial Intelligence Systems, 598-611. http://dx.doi.org/10.1007/978-3-030-61705-9_49
[16] Sivaraman, S., Trivedi, M.M. (2010). A general active-learning framework for on-road vehicle recognition and tracking. IEEE Transactions on intelligent transportation systems, 11(2): 267-276. http://dx.doi.org/10.1109/TITS.2010.2040177
[17] Wang, S., Cao, J., Sun, K., Li, Q. (2020). {SIEVE}: Secure {In-Vehicle} Automatic Speech Recognition Systems. In 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020), 365-379.
[18] Clady, X., Negri, P., Milgram, M., Poulenard, R. (2008). Multi-class vehicle type recognition system. In IAPR workshop on artificial neural networks in pattern recognition, 228-239. http://dx.doi.org/10.1007/978-3-540-69939-2_22
[19] Ran, B., Liu, H.X. (1999). Development of vision-based vehicle detection and recognition system for intelligent vehicles. Transportation research record, 1679(1): 130-138. http://dx.doi.org/10.3141/1679-18
[20] Vançin, S., Erdem, E. (2017). Implementation of the vehicle recognition systems using wireless magnetic sensors. Sādhanā, 42(6): 841-854. http://dx.doi.org/10.1007/s12046-017-0638-4
[21] Jusoh, N.A., Mohamad Zain, J. (2011). Malaysian car plates recognition using freeman chain codes and characters’ features. In International Conference on Software Engineering and Computer Systems, 581-591. http://dx.doi.org/10.1007/978-3-642-22170-5_50
[22] Rad, R., Jamzad, M. (2005). Real time classification and tracking of multiple vehicles in highways. Pattern Recognition Letters, 26(10): 1597-1607. http://dx.doi.org/10.1016/j.patrec.2005.01.010
[23] Coifman, B., Beymer, D., McLauchlan, P., Malik, J. (1998). A real-time computer vision system for vehicle tracking and traffic surveillance. Transportation Research Part C: Emerging Technologies, 6(4): 271-288. http://dx.doi.org/10.1016/S0968-090X(98)00019-9
[24] Owamoyo, N., Fadele, A.A., Abudu, A. (2013). Number plate recognition for Nigerian vehicles. Academic Research International, 4(3): 48.
[25] Amusan, D.G., Arulogun, O.T., Falohun, A.S. (2015). Nigerian vehicle license plate recognition system using artificial neural network. International Journal of Advanced Research in Computer and Communication Engineering, 4(11): 1-5. http://dx.doi.org/10.17148/IJARCCE.2015.41101
[26] Emami, H., Fathi, M., Raahemifar, K. (2014). Real time vehicle make and model recognition based on hierarchical classification. International Journal of Machine Learning and Computing, 4(2): 142. http://dx.doi.org/10.7763/IJMLC.2014.V4.402
[27] Liu, D., Wang, Y. (2017). Monza: Image classification of vehicle make and model using convolutional neural networks and transfer learning.
[28] Chen, P., Bai, X., Liu, W. (2014). Vehicle colour recognition on urban road by feature context. IEEE Transactions on Intelligent Transportation Systems, 15(5): 2340-2346. http://dx.doi.org/10.1109/TITS.2014.2308897
[29] Dule, E., Gökmen, M., Beratoğlu, M.S. (2010). A convenient feature vector construction for vehicle color recognition. In Proceedings of the 11th WSEAS international conference on nural networks and 11th WSEAS international conference on evolutionary computing and 11th WSEAS international conference on Fuzzy Systems, 250-255. http://dx.doi.org/10.5555/1863431.1863473
[30] Sari-Ali, I., Chikh-Bled, B., Ikumapayi, O.M., Dib, Z., Lorenzini, G., Menni, Y. (2022). Energy transfer from an electrical network-connected synchronous machine. Journal Européen des Systèmes Automatisés, 55(5): 573-579. https://doi.org/10.18280/jesa.550502
