A New Rail Surface Defects Detection Approach Using 3D Laser Cameras Based on ResNet50

A New Rail Surface Defects Detection Approach Using 3D Laser Cameras Based on ResNet50

Yunus Santur Merve YilmazerMehmet Karakose Erhan Akin 

Department of Computer Engineering, Faculty of Engineering, Firat University, Elazig 23119, Tukey

Department of Computer Engineering, Faculty of Engineering, Munzur University, Tunceli 62000, Tukey

Corresponding Author Email: 
merveyilmazer@munzur.edu.tr
Page: 
1339-1345
|
DOI: 
https://doi.org/10.18280/ts.390427
Received: 
25 June 2022
|
Revised: 
1 August 2022
|
Accepted: 
10 August 2022
|
Available online: 
31 August 2022
| Citation

© 2022 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).

OPEN ACCESS

Abstract: 

Rail transportation systems, which are used as one of the most common means of transportation worldwide, should be regularly inspected to prevent accidents that may occur. The rail condition monitoring can be performed in high accuracy and real time using computer vision, deep learning algorithms today. In this study, a new deep learning based approach using 3D laser cameras for rail inspection is presented. In the proposed approach, two 3D laser cameras placed on a real train, seeing the rail line from the left and right surfaces were used. These data consisting of sensitive distance value constitute the input data of the ResNet50 transfer learning model. The training was carried out on Nvidia Cuda supported graphics processing units using ResNet50 Convolutional Neural Network. During the test phase, the operation speed and accuracy rate of the method was measured by repeating the process on real-time rail profiles. The accuracy rate was calculated as 94%. As a result a new approach is presented based on deep learning using 3D laser cameras for rail inspection is presented.

Keywords: 

railway inspection, deep learning, ResNet-50 architecture, rail surface defects, laser cameras

1. Introduction

Rail transportation systems are one of the most frequently used transportation and cargo transportation methods of today. It is necessary to periodically inspect the faults such as fracture, scoring, crack in the rail lines that form part of the rail transportation system. This inspection process is in the simplest way, by means of a specialist, by manual by walking, or using mechanical tracks [1]. However, this method is very laborious, it requires a long time and most importantly, it is a very subjective method de-pending on the knowledge of the expert. A second method used in the control of the rail lines is the inspection process, which is called contacted and is carried out with measuring instruments that contact to the rail line with a special device [2]. Although this method provides satisfactory accuracy, it is slow and not suitable for long distance inspection. Due to these disadvantages, rail lines are currently being carried out without contact by computer vision methods using cameras or three-dimensional laser cameras [3]. Images can be obtained by placing different types of cameras on the train [4], to an inspection car on the track [5], or on the drone [6].

Thus, the rail lines can be inspected in real time and with high accuracy. The biggest disadvantage experienced in the inspection of the computer vision based on the use of a digital camera is the detection of stains such as oil, dust or foreign substances on the rail surfaces as anomalies. For this reason, the model can produce False Positive (FP) results. In systems using 3D laser camera can less FP results compared to based on RGB cameras [7, 8]. The major disadvantage of this method is that its costs are much higher. In systems using digital cameras, light source is also needed to prevent disadvantage such as shadow and light inequality. However, light sources are not required in systems using laser cameras. The components that make up the control system can be mounted on a special test device or conveniently mounted on the rail vehicle [9].

In this study, a new approach is presented after analyzing the literature studies for computer vision based rail control. In the study, it was aimed to detect anomaly conditions such as fracture, pitting, puncture, abrasion and crack that may occur on the top and lateral surfaces of the rail line. A train moving on the rail line was used for the study. A total of two 3D AT C5-1600CS 3D laser cameras was used that see both surfaces of the rail line on the experimental setup. Frames containing 3D rail profiles obtained during the training phase was labeled as "healthy" and "faulty" and were trained using ResNet50. Frames containing rail profiles read during the test phase were classified.

The rail condition monitoring is based on the principle of detecting anomaly states in the components of the rail line given in Figure 1. Literature studies are especially in the form of detecting fractures, pitting and cracks on the rail line, missing fasteners and classification of these anomalies.

Figure 1. Rail line components

CNN was trained using the ResNet50 model in Transfer Learning to detect the rail surface defects non-contactly. The images used for training were acquired using 3D laser cameras. Thus, even small surface defects can be detected. Experimental results show that the proposed method can detect rail surface defects with high accuracy.

Railway inspection is examined under two headings as contact and non-contact in the literature. Contacted methods are carried out manually, with the help of mechanical devices or with ultrasonic devices. Rail inspection process based on computer vision can be performed quickly and with high accuracy, but oil and dust stains on the rail line can be perceived as anomaly, which can cause the system to produce FP results. Rail inspection operations performed using 3D laser cameras are fast and highly accurate but costly. These methods are given comparatively in Table 1.

Table 1. Classifications of the rail inspection methods

Methods

Advantages

Disadvantages

Contacted

Methods

Mechanic devices

Cheap

Very slow, Unsafe, Low accuracy

Ultrasonic devices

Fast

Slow, Increase existing damage

Contact-Free Methods

Computer Vision System (CVS)

Safe, Fast, High accuracy

Expensive, FP results

CVS with 3D-laser camera

Safe, Very fast, Very high accuracy

Expensive

Below is a brief literature summary about rail analysis in Table 2. Thomas et al. used ultrasonic measurement, a contacted method for detecting rail surface defects [2]. Similarly, studies on surface abrasion, fracture and cavity detection have been carried out in the rail inspection processes performed in contacted with ultrasonic devices [10-12]. Min et al. carried out 20 type fracture detection using a 5 km/h real-time inspection process using a special test device contacted on the rail line using computer vision [13]. After noise reduction and contrast improvement was made on the images taken from the rail line, malfunction determined as a result of morphological processes according to the threshold value.

Chen et al. proposed a semi-supervised algorithm to detect defects on ballastless surfaces. Assuming that there is no foreign body between the rail and the fastener, they used the Mask R-CNN algorithm to determine these regions. Foreign body detection in the determined regions was performed using the deep SVDD (Support Vector Data Description) algorithm, which was developed at a rate of 89.23% AUC [14]. The block diagram for proposed method is given in Figure 2.

Figure 2. Block diagram for proposed approach [14]

Santur et al. presented an adaptive approach using Inertial Measurement Unit (IMU) and Artificial Neural Network (ANN) to eliminate the blur effect caused by vibration in rail lines [15]. Rail lines produce a lot of vibration due to their physical structure. Due to these vibrations, blur effect occurs in the images taken from the rail line. With the data obtained from the IMU (Acceleremeter, Gyroscope and Magentometer), an ANN has been developed that adaptively learns the deblurring parameters from the ANN network.

Wu et al. proposed the RBGNet_LWLC + ME hybrid model to perform rail segmentation and detect rail surface defects in the railway. They determined the precision and recall rates of the method they proposed to be over 90% [16].

Yang et al. aimed to detect rail surface defects using Machine Vision and Neural Networks. They have shown that they can detect rail defects of different shapes and sizes with segmentation with an accuracy of 97.47% [17].

Singh et al. used an object detection model based on the YOLOv4 algorithm to detect railway sleepers using UAV images. They showed that their proposed method could detect railway sleepers with 92% accuracy, 99.10% recall, and 99.08% average accuracy (mAP) success rates [18].

Ye et al. stated that the accuracy and f1 score ratio of their proposed method to detect cracks in railway concrete blocks using a deep learning network STCNet is 99.54% [19].

Chen et al. presented template matching approach for detection fasteners in Pantograph catener systems made using CNN [20]. In the study, 40000 fasteners in 2000 catenary system were used along a 100 km railway.

Wei et al. used image processing and CNN architectures together in their proposed method for the detection of deficiencies and breakage errors in fasteners, which are important components of rail-ways. They showed that they could detect each image in 0.23 seconds and position and classify it with a mAP rate of 97.9% [21].

Mittal and Rao obtained 0.96 Precision on 9 and 30 hours of video data with the deep learning model using CNN to determine the track line geometry [22].

Ren et al. achieved 0.91 accuracy using CNN for a similar purpose [23]. In the study, heat map and Treshold were used to detect the defective area.

Santur et al. achieved 0.98 accuracy with CNN and RF models in the test environment for the detection of rail profiles on the upper and lateral surfaces [24, 25]. In the study, an experimental study was carried out in a laboratory environment using a 3D laser camera.

Molleda et al. performed an approach using laser camera with a 0.12 error to measure rail profiles during production [26].

Tu et al. detected rail defects and fastener defects using instance segmentation. detected rail defects with a detection rate of 98% and a classification accuracy of 93.5%. They determined fastener faults with a recall rate of 95.1% [27].

Raza et al. conducted an image processing based simulation study for detection of rail faults [28].

Karaköse et al. (2016) was detected rail geometry distortions with 0.88 Accuracy with an image processing based approach [29].

Santur et al. presented an approach that uses pipeline architecture for high-speed control of rail lines, with a special test device using computer vision approach, 72 km/h and 0.97 accuracy was achieved [30].

Table 2. Literature review

Monitoring

Method

Algorithms

Hardware

Performance

Rail surface fault diagnosis

Contacted

 

Geometric [10]

Ultrasonic device

0.91 Accuracy

Rail surface, wear and fracture diagnosis

Ultrasonic [10-13]

Eddy Current [2]

-

Ballastless surface defects

Contactless

 

Mask R-CNN and SVDD algorithm [14]

Line camera

0.89 AUC

Rail surface fault diagnosis

Morphological image processing [13]

RGB/Line

camera

5 km/h

0.88/0.94 Recall

Detect railway sleepers

YOLOv4 algorithm [18]

 

 

 

0.92 precision, 0.99 recall

Fastener detection

3D CVS [20]

 

3D laser

camera

0.98 Accuracy

Rail line geometry measurement

Deep learning [22]

RGB camera

0.96 Precision

Classifying of rail inspections

Deep learning [23]

RGB camera

0.91 Accuracy

Rail defects and fastener defects

Instance segmentation [27]

Line camera

Rail defects 0.98 Accuracy

Fastener faults 0.95 recall

Santur et al. achieved 0.98 accuracy with the approach using 3D laser camera [30, 31].

Franca and Vassallo (2020) aimed to identify sleepers and sleepers defects using image processing, heuristics and feature merging methods. Experimental results have shown that they can detect traverse types with an accuracy of 97% and traverse defects with an accuracy of 93% [32].

Li et al. and Santur et al. have proposed big data approaches for the integration of rail inspection using components such as IMU, GPS, camera, 3D camera, encoder [33, 34].

2. Proposed Method

In this study, a new approach for rail inspection is presented using deep learning based ResNet-50 architecture, whose block diagram is given in Figure 3. The proposed method works in four steps.

  • In the first step, profile data is obtained by laser triangulation from the rail line with compact 3D sensors calibrated on both sides of the rail line.
  • These data were labeled "healthy" and "faulty".
  • In the third step, the rail profiles measured in real time from the system were classified using the model learned.
  • In the final step, to measure the classification success of the model, the confusion matrix was drawn and the evaluation metrics were calculated.

2.1 Obtaining 3D profile from laser cameras

Images obtained with ccd / cmos cameras contain gray level or rgb data of an image in the x, y plane, these images are called two-dimensional (2D). If this image contains the distance information in the z plane instead of the gray level, a three-dimensional (3D) image is obtained. Stereo vision, time of flight camera and laser triangulation are used to obtain three-dimensional images. The 3D laser camera continuously takes pictures and makes use of the profile change in the laser line to draw the object in a computer environment. The main methods used to obtain three-dimensional images are given comparatively in Table 3 [35].

Figure 3. The proposed approach

Table 3. Comparison of 3D image acquisition methods

Method

Hardware

Major advantage

Major disadvantage

Stereo camera

Two calibrated cameras

Cheap

Accuracy depends on calibration precision

ToF camera

Infrared light source, phase detector and camera

Cheap

Distance data is not sensitive

Laser camera

Calibrated camera, encoder and laser line source

Distance data is very sensitive

Expensive

2.2 ResNet-50 architecture

ResNet-50 is a 50-layer convolutional neural network trained using millions of images from the ImageNet dataset. It allows the CNN architecture to work with more layers. Increasing the depth of neural networks makes training more difficult due to the vanishing gradient problem. ResNet tries to solve this problem by learning some residuals instead of features with residual learning. Thus, despite the large number of layers, ResNet-50 is low in complexity and easy to optimize [36-38].

2.3 CNN architecture

Convolutional Neural Networks (CNN) is a deep learning algorithm that works on the principle of feature extraction by passing the input images through the convolution and pooling layers. It extracts features by applying fxf filter to an NxN size input image [39]. Various CNN-based models such as AlexNet, VGGNet, GoogLeNet, ResNet MobileNet and EfficientNet have been developed for classification studies [40]. The general structure of the CNN architecture is given in Figure 4.

Figure 4. An overview of the CNN architecture

2.4 Evaluation

The accuracy of the proposed model was measured by calculating the confusion matrix (Table 4) and evaluation metrics (Eq. (1), (2), (3), (4)). The confusion matrix is used to measure classification success and evaluate performance. It allows measuring model success for each class label as well as measuring the overall success of the model. In the confusion matrix, True Positive (TP) symbolizes the number of data that is correctly classified as faulty, and True Negative (TN) is correctly classified as healthy. False Negative (FN) is the number of data classified as faulty although it is healthy and is also known as Type-2 error. False Positive (FP), on the other hand, gives the number of data classified as faulty although it is healthy and is also known as Type-1 error.

Table 4. Confusion matrix

 

Actual Class

 

Predicted Class

 

P

N

P

TP

FP

N

FN

TN

accuracy $=\frac{T P+T N}{T P+T N+F P+F N}$                    (1)

precision $=\frac{T P}{T P+F P}$                    (2)

recall $=\frac{T P}{T P+F N}$                    (3)

$f_1=2 \times \frac{\text { precision } \times \text { recall }}{\text { precision }+\text { recall }}$                    (4)

3. Experimental Results

In this study, deep learning based railway inspection was carried out by using two AT C5-1600CS 3D laser compact sensors positioned to the left and right of a rail line to see its lateral and upper rail surfaces. The ResNet-50 transfer learning model used for the training process was developed using the Nvidia cuda supported Keras library in the Python development environment. The Nvidia GTX 750 GPU used for the application has 2GB of memory and 384 CUDA cores. Two 3d laser cameras given in Figure 5. for application, computer with Nvidia GPU, encoder and power supply are mounted on a real train.

Figure 5. Experimental setup

 

Figure 6. 3D, 2D imaging and GUI

The compact sensor supports two modes, 2-D and 3-D. It has 1600 * 1088 * 16-bit gray resolution in 2-D mode, and in 3-D mode it can measure up to 1600 points in 16-bit resolution on x-axis per profile. 3D laser cameras have a resolution of 313x15 micron meters at 25 Khz, horizontal and vertical. It can measure between 700mm ± 250mm and 500mm angle of view. In 2D mode, 1600x1088 8/16 bit gray images can be taken. The cameras placed on the train have a distance of 60 cm [41]. Before starting the training process, calibration was performed according to the adaptive threshold values using the distance and angles of the 3D sensor rail line. In Figure 6, after the calibration, the gray level image obtained by combining the data from the two sensors is given. The image shows the top surface, lateral surfaces and bolts of the rail profile. Data outside the viewing distance and below the threshold value are read as 0 from the sensor vector and these data are cut out from both training and test data. It is possible to read large amounts of data in seconds with the 3D sensor. Since it is not possible to label each of these data with hand, profile frames are combined and labeled as "healthy" and "faulty" in each sample taken from the healthy and unhealthy parts of the rail line. It shows integrated data from all input devices in the developed GUI. The ResNet-50 convolutional neural network architecture was used for the training process, the RELU used for activation is a normalized output to the 0-1 range, which includes the probability that the sampled portion is faulty.

Figure 7. Dataset examples

Figure 8. Confusion matrix obtained with test data

In Figure 7, some sample rail images and 3D cloud data created with the 3D sensor are shown during the training of the rail control. The performance of the model was measured by plotting the confusion matrix given in the Figure 8 and calculating the evaluation metrics given in Eqns. (1), (2), (3) and (4). Achieved 94.2% accuracy, 99% precision, 94.1% recall and 96.5% f1. Railway tracks are the surface in contact with the train wheel, faults based on load and friction occur. Surface abrasions, breaks, cracks that start to occur in small sizes should be repaired before they grow and cause accidents. Taking ray images with a 3D laser camera makes it easy to detect faults, even small ones. In the proposed method, data was acquired using a laser camera to ensure accurate detection of small-sized faults. Thus, small-sized defects on the rail surface were detected, and dust and oil accumulations on the rail surface were prevented from being detected as a fault. This is the advantageous aspect of the proposed method compared to the existing rail surface defect determination studies in the literature. The comparison of the studies carried out for the detection of various rail defects and the proposed method is given in Table 5.

Table 5. Comparison of rail defect method results

References

Method

Model evaluation

[42]

Image processing and Convolutional Neural Network (CNN)

Recall: 92.54%

Precision: 92.08%

[43]

Partitioned edge features (PEF)

Recall: 92.03%

Precision: 88.49%

[44]

FCN-8 deep-learning network

Efficiency rate: 81%

Our method

3D laser cameras images with ResNet-50 Convolutional Neural Network

Accuracy: 94.2%

Precision:99%

Recall: 94.1%

F1-score: 96.5%

Shang et al. [42] showed that they detected rail surface defects at 92.54% recall and 92.08% precision success values using CNN. They tested the effects of using different loss functions and different classifiers such as SVM and softmax on model success. They showed that the softmax classifier gave better results in the proposed method. Ni et al. [43] experimentally performed rail region extraction, edge detection, detect contour filling in their proposed study to detect rail surface defects. Bojarczak and Lesiak [44] used a deep learning network implemented in the Tensorflow environment, such as FCN-8, to experimentally prevent the brightness of the images from affecting the segmentation success in their proposed study to detect rail surface defects. Thus, they were able to perform image segmentation with a success rate of 81%. In the proposed method in this study, the images to be used in the CNN network were produced using 3D laser cameras that provide high-sensitivity image acquisition.

4. Conclusions

The railway line should be checked periodically, defective components should be detected and repaired without delay. For this purpose, new methods have been developed by using deep learning and computer vision technologies to detect faults in different components with different data sets and different techniques in the literature.

In this study, rail profile fault detection was made by using two 3D laser cameras that view the left and right surfaces of the rail line at an angle of 45o. Thanks to the approach, the lateral and upper surfaces of the rail line can be controlled with the same process. The laser camera used has a sensitivity of 313x15 µm horizontally and vertically. Each profile is 1600x16-bit in size and takes up about 13 KB. Considering the process of reading the rail profiles, writing to disk and generating diagnostic results, 0.0025 seconds, 1 reading was performed. In other words, 800 profile data is processed per second. This means I/O and model testing for 570 Mbytes of data in a 1 minute test period. Considering that the test process takes 1 hour, the actual rail length, which is taken as an example with approximately 3 million rail profile data, is 446 meters.

In summary, a new method based on the ResNet-50 architecture, which is one of the transfer learning models in convolutional neural networks, is proposed. The accuracy of the method was increased by using the ResNet-50 transfer learning model, which al-lows the weights of pre-trained neural networks to be used. In addition, by obtaining the data using a laser camera in the method, even very small errors can be detected. The fault detection performance of the model was measured as 0.94 Accuracy and 0.96 F1 value. This shows that the proposed method can perform more sensitive fault detection compared to the models trained using physical cameras.

  References

[1] Li, Q., Zhong, Z., Liang, Z., Liang, Y. (2015). Rail inspection meets big data: Methods and trends. 2015 18th International Conference on Network-Based Information Systems, pp. 302-308. http://dx.doi.org/10.1109/NBiS.2015.47

[2] Thomas, H.M., Heckel, T., Hanspach, G. (2007). Advantage of a combined ultrasonic and eddy current examination for railway inspection trains. Insight-Non-Destructive Testing and Condition Monitoring, 49(6): 341-344. http://dx.doi.org/10.1784/insi.2007.49.6.341

[3] Resendiz, E., Hart, J.M., Ahuja, N. (2013). Automated visual inspection of railroad tracks. IEEE Transactions on Intelligent Transportation Systems, 14(2): 751-760. http://dx.doi.org/10.1109/TITS.2012.2236555

[4] Yanan, S., Hui, Z., Li, L., Hang, Z. (2018). Rail surface defect detection method based on YOLOv3 deep learning networks. 2018 Chinese Automation Congress (CAC), pp. 1563-1568. http://dx.doi.org/10.1109/CAC.2018.8623082

[5] Chandran, P., Asber, J., Thiery, F., Odelius, J., Rantatalo, M. (2021). An investigation of railway fastener detection using image processing and augmented deep learning. Sustainability, 13(21): 12051. http://dx.doi.org/10.3390/su132112051

[6] Yilmazer, M., Karakose, M., Aydin, I., Akin, E. (2022). Transfer learning based fault detection approach for rail components. 2022 26th International Conference on Information Technology (IT), pp. 1-4. http://dx.doi.org/10.1109/IT54280.2022.9743539

[7] Siddiqui, A.A. (2015). A new inspection method based on RGB-D profiling. Doctoral dissertation, Virginia Tech.

[8] Aytekin, Ç., Rezaeitabar, Y., Dogru, S., Ulusoy, I. (2015). Railway fastener inspection by real-time machine vision. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 45(7): 1101-1107. http://dx.doi.org/10.1109/TSMC.2014.2388435

[9] Feng, H., Jiang, Z., Xie, F., Yang, P., Shi, J., Chen, L. (2013). Automatic fastener classification and defect detection in vision-based railway inspection systems. IEEE Transactions on Instrumentation and Measurement, 63(4): 877-888. http://dx.doi.org/10.1109/TIM.2013.2283741

[10] Fan, Y., Dixon, S., Edwards, R.S., Jian, X. (2007). Ultrasonic surface wave propagation and interaction with surface defects on rail track head. NDT & E International, 40(6): 471-477. http://dx.doi.org/10.1016/j.ndteint.2007.01.008

[11] Rizzo, P., Bartoli, I., Marzani, A., Lanza di Scalea, F. (2005). Defect classification in pipes by neural networks using multiple guided ultrasonic wave features extracted after wavelet processing. Journal of Pressure Vessel Technology, 127(3): 294-303. http://dx.doi.org/10.1115/1.1990213

[12] Mariani, S., Nguyen, T.V., di Scalea, F.L. (2017). Air-coupled ultrasonic testing of rails: Experimental prototype and finite element simulations. Materials Evaluation, 75(12): 1497-1507.

[13] Min, Y., Xiao, B., Dang, J., Yue, B., Cheng, T. (2018). Real time detection system for rail surface defects based on machine vision. EURASIP Journal on Image and Video Processing, 2018(1): 1-11. http://dx.doi.org/10.1186/s13640-017-0241-y

[14] Chen, Z., Wang, Q., Yu, T., Zhang, M., Liu, Q., Yao, J., Wu, Y., Wang, P., He, Q. (2022). Foreign object detection for railway ballastless trackbeds: A semisupervised learning method. Measurement, 190: 110757. http://dx.doi.org/10.1016/j.measurement.2022.110757

[15] Santur, Y., Karaköse, M., Aydın, İ., Akın, E. (2016). IMU based adaptive blur removal approach using image processing for railway inspection. 2016 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 1-4. http://dx.doi.org/10.1109/IWSSIP.2016.7502729

[16] Wu, Y., Qin, Y., Qian, Y., Guo, F., Wang, Z., Jia, L. (2022). Hybrid deep learning architecture for rail surface segmentation and surface defect detection. Computer‐Aided Civil and Infrastructure Engineering, 37(2): 227-244. http://dx.doi.org/10.1111/mice.12710

[17] Yang, H., Wang, Y., Hu, J., He, J., Yao, Z., Bi, Q. (2021). Segmentation of track surface defects based on machine vision and neural networks. IEEE Sensors Journal, 22(2): 1571-1582. http://dx.doi.org/10.1109/JSEN.2021.3133280

[18] Singh, A.K., Dwivedi, A.K., Nahar, N., Singh, D. (2021). Railway track sleeper detection in low altitude UAV imagery using deep convolutional neural network. 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pp. 355-358. http://dx.doi.org/10.1109/IGARSS47720.2021.9554542

[19] Ye, W., Deng, S., Ren, J., Xu, X., Zhang, K., Du, W. (2022). Deep learning-based fast detection of apparent concrete crack in slab tracks with dilated convolution. Construction and Building Materials, 329: 127157. http://dx.doi.org/10.1016/j.conbuildmat.2022.127157

[20] Chen, J., Liu, Z., Wang, H., Núñez, A., Han, Z. (2017). Automatic defect detection of fasteners on the catenary support device using deep convolutional neural network. IEEE Transactions on Instrumentation and Measurement, 67(2): 257-269. http://dx.doi.org/10.1109/TIM.2017.2775345

[21] Wei, X., Yang, Z., Liu, Y., Wei, D., Jia, L., Li, Y. (2019). Railway track fastener defect detection based on image processing and deep learning techniques: A comparative study. Engineering Applications of Artificial Intelligence, 80: 66-81. http://dx.doi.org/10.1016/j.engappai.2019.01.008

[22] Mittal, S., Rao, D. (2017). Vision based railway track monitoring using deep learning. arXiv preprint arXiv:1711.06423.

[23] Ren, R., Hung, T., Tan, K.C. (2017). A generic deep-learning-based approach for automated surface inspection. IEEE Transactions on Cybernetics, 48(3): 929-940. http://dx.doi.org/10.1109/TCYB.2017.2668395

[24] Santur, Y., Karaköse, M., Akın, E. (2016). Learning based experimental approach for condition monitoring using laser cameras in railway tracks. International Journal of Applied Mathematics Electronics and Computers, (Special Issue-1): 1-5. http://dx.doi.org/10.18100/ijamec.270656 

[25] Santur, Y., Karaköse, M., Akın, E. (2016). Condition monitoring approach using 3D modelling of railway tracks with laser cameras. In International Conference on Advanced Technology & Sciences (ICAT’16), pp. 132-135.

[26] Molleda, J., Usamentiaga, R., Millara, Á.F., García, D.F., Manso, P., Suárez, C.M., García, I. (2016). A profile measurement system for rail quality assessment during manufacturing. IEEE Transactions on Industry Applications, 52(3): 2684-2692. http://dx.doi.org/10.1109/TIA.2016.2524459

[27] Tu, Z., Wu, S., Kang, G., Lin, J. (2021). Real-time defect detection of track components: Considering class imbalance and subtle difference between classes. IEEE Transactions on Instrumentation and Measurement, 70: 1-12. http://dx.doi.org/10.1109/TIM.2021.3117357

[28] Rizvi, A.R., Khan, P.R., Ahmad, S. (2017). Crack detection in railway track using image processing. International Journal of Advance Research, Ideas and Innovations in Technology, 3(4): 489-496.

[29] Karaköse, M., Yaman, O., Akin, E. (2016). Real time implementation for fault diagnosis and condition monitoring approach using image processing in railway switches. International Journal of Applied Mathematics Electronics and Computers, (Special Issue-1): 307-313. http://dx.doi.org/10.18100/ijamec.270627

[30] Santur, Y., Karaköse, M., Akin, E. (2017). A new rail inspection method based on deep learning using laser cameras. In 2017 International Artificial Intelligence and Data Processing Symposium (IDAP), pp. 1-6. http://dx.doi.org/10.1109/IDAP.2017.8090245

[31] Santur, Y. (2018). Real time diagnosis of 3d rail profile faults with computer vision using laser cameras. Ph.D. Dissertation, Fırat University, Elazığ.

[32] Franca, A.S., Vassallo, R.F. (2020). A method of classifying railway sleepers and surface defects in real environment. IEEE Sensors Journal, 21(10): 11301-11309. http://dx.doi.org/10.1109/JSEN.2020.3026173

[33] Li, Q., Zhong, Z., Liang, Z., Liang, Y. (2015). Rail inspection meets big data: Methods and trends. In 2015 18th International Conference on Network-Based Information Systems, pp. 302-308. http://dx.doi.org/10.1109/NBiS.2015.47

[34] Santur, Y., Karaköse, M., Akin, E. (2017). Big data framework for rail inspection. In 2017 International Artificial Intelligence and Data Processing Symposium (IDAP), pp. 1-4. http://dx.doi.org/10.1109/IDAP.2017.8090326

[35] Gunatilake, A., Piyathilaka, L., Kodagoda, S., Barclay, S., Vitanage, D. (2019). Real-time 3D profiling with RGB-D mapping in pipelines using stereo camera vision and structured IR laser ring. In 2019 14th IEEE Conference on Industrial Electronics and Applications (ICIEA), pp. 916-921. http://dx.doi.org/10.1109/ICIEA.2019.8834089

[36] Reddy, A.S.B., Juliet, D.S. (2019). Transfer learning with ResNet-50 for malaria cell-image classification. In 2019 International Conference on Communication and Signal Processing (ICCSP), pp. 0945-0949. http://dx.doi.org/10.1109/ICCSP.2019.8697909

[37] Mascarenhas, S., Agarwal, M. (2021). A comparison between VGG16, VGG19 and ResNet50 architecture frameworks for image classification. In 2021 International Conference on Disruptive Technologies for Multi-Disciplinary Research and Applications (CENTCON), pp. 96-99. http://dx.doi.org/10.1109/CENTCON52345.2021.9687944

[38] Atliha, V., Šešok, D. (2020). Comparison of VGG and ResNet used as encoders for image captioning. In 2020 IEEE Open Conference of Electrical, Electronic and Information Sciences (eStream), pp. 1-4. http://dx.doi.org/10.1109/eStream50540.2020.9108880

[39] Chauhan, R., Ghanshala, K.K., Joshi, R.C. (2018). Convolutional neural network (CNN) for image detection and recognition. In 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC), pp. 278-282.

[40] Lu, J., Tan, L., Jiang, H. (2021). Review on convolutional neural network (CNN) applied to plant leaf disease classification. Agriculture, 11(8): 707.

[41] Moolayil, J., Moolayil, J., John, S. (2019). Learn Keras for deep neural networks. Birmingham: Apress. pp. 1-192. http://dx.doi.org/10.1007/978-1-4842-4240-7

[42] Shang, L., Yang, Q., Wang, J., Li, S., Lei, W. (2018). Detection of rail surface defects based on CNN image recognition and classification. In 2018 20th International Conference on Advanced Communication Technology (ICACT), pp. 45-51. http://dx.doi.org/10.23919/ICACT.2018.8323641

[43] Ni, X., Liu, H., Ma, Z., Wang, C., Liu, J. (2021). Detection for rail surface defects via partitioned edge feature. IEEE Transactions on Intelligent Transportation Systems, 23(6): 5806-5822. http://dx.doi.org/10.1109/TITS.2021.3058635

[44] Bojarczak, P., Lesiak, P. (2021). UAVs in rail damage image diagnostics supported by deep-learning networks. Open Engineering, 11(1): 339-348. http://dx.doi.org/10.1515/eng-2021-0033