Ashish Kumar Singh | Yong-Hoon Kim*
© 2021 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
The development of advanced radar system for detection and classification of UAVs is an essential requirement for today’s societal security. Such intelligent system could able to analyze the received radar signal and extract relevant information by utilizing sophisticated algorithm. In this letter, the utilization of micro-Doppler signature (MDS) for classification of drones, using convolutional neural network (CNN) model has been presented. We have generated images of micro-Doppler signatures using W-band radar system and used it for classification purpose. In this work, phase stretch transform (PST) has been utilized for edge detection and enhancement of the micro-Doppler images, to generate the edge-enhanced micro-Doppler image (EMDI). The comparison based on classification performance of CNN with different input datasets shows that the EMDI based CNN model outperformed the micro-Doppler image (MDI) based model.
classification, radar signal processing, W-band, micro-Doppler imaging, deep learning
In past decades, researchers across the globe have been implementing and utilizing deep learning methods as classification techniques. Recently, several classification methods such as Linear Discriminant Analysis, Artificial Neural Network (ANN) and Support Vector Machine (SVM) based on micro-Doppler signatures for target classification have been presented [1-4]. Here, the SVM has a little better performance than the ANN. Lei [4] tested out the pattern recognition based on micro-Doppler using five types of classifier including k-Nearest Neighbour (k-NN), Bayes Linear Discriminant Classifier, and Bayes Quadratic Discriminant Classifier, for different motion classes such as rotation, vibration, coning and tumbling. If a target has well-defined features, one can easily distinguish target from the other. There have been some efforts made toward the classification of flying targets. Molchanov et al. [5] investigated the effectiveness of SVM and Naïve Bayes Classifier (NBC) for classification of birds and small UAVs. Researchers have also investigated the effectiveness of dual band radar system for the measurement of UAVs, such as using X and L bands [6], X and K bands [7], K and W bands [8].
Some researchers have also studied the classification of human activities based on micro-Doppler signature (MDS) using features like stride-rate, velocity of torso, Doppler bandwidth and Fourier coefficients [9-12]. Radar based target identification and classification involves extraction of MDS. Various techniques can be used for the extraction of MDS such as Empirical Mode Decomposition (EMD) algorithm [13, 14].
Classification of human activities using Deep Convolutional Neural Networks (DCNNs) based algorithm using micro-Doppler signatures have been investigated, and Kim et al. [15] achieve accuracy of 97% and 90%, respectively. Kim et al. [16] have performed classification of two types of drones using bi-static X-band radar based on Convolutional Neural Network (CNN), using image generated by merging cadence-velocity diagram (CVD) and MDS images. Rahman and Robertson [17] has used CNN for classification of drones and birds using radar micro-Doppler images. They utilized dual frequency band (in K band and W band) for measurements of MDS. Kang et al. [18] presented classification of drone’s elevation angle based on MDS using multiple polarization at Ku-band frequency.
Complexity and computation cost of the classifier can increase with the number of features. So, it is necessary to consider significant features and limit redundant features for performance optimization. The physical characteristics of the target can induce the micro-Doppler effect, which are produced by movement of target’s body parts. As a result, micro-Doppler signatures correspond to different types of target such as human, birds, animals, cars, drones, etc. are dissimilar. Thus, micro-Doppler image can be used for the classification target from various class. However, distinguishing and classifying targets of the same type, such as drones with different numbers of blades and/or rotors, is a difficult task.
In this work, the classification of drones using CNN based on micro-Doppler image is presented. Images were generated from the received radar signal, using three different types of drones. W-band radar system has been used for the measurement purpose. For edge detection, PST has been utilized and the extracted edges are overlaid with the micro-Doppler image. Therefore, the generated images have more features as compared to simple spectrogram images. In order to enhance the classification performance, edge-enhanced micro- Doppler image (EMDI) has been proposed and compare it with micro-Doppler image (MDI) based classifier. Color is also an important feature in the micro-Doppler image, it will useful in differentiating micro-Doppler signatures of various targets. EMDI will preserve color information of image as well as it has enhanced edges. Therefore, it can improve overall classification performance.
The rest of this article is organized as follows: Section 2 explains the proposed method for generating edge-enhanced micro-Doppler image using radar system. Section 3 describes the experiment setup using W-band radar system. Section 4 presents results for the classification of drones. Finally, conclusion of the work is given in Section 5.
Classification performance of the architecture depends upon both configurations as well as the input dataset. If the input object contains more distinguishable information, it will be easier to discriminate the objects from various classes. A new input image, edge-enhanced micro-Doppler image (EMDI) obtained by overlaying micro-Doppler image (MDI) with detected edges of MDI. Asghari and Jalali [19] propose Phase Stretch Transform (PST), one of the latest edge detection methods has been used for enhancing the features of micro-Doppler images and thus useful for the classification of a target. PST is based on pure phase operation, as compared to conventional edge detection technique which based on amplitude. Suthar et al. [20] demonstrated that PST has superior performance in feature extraction of low contrast images. Color is also significant in the micro-Doppler image, and it will helpful in distinguishing micro-Doppler signatures of various targets. EMDI will preserve color information of image as well as it has enhanced edges. Hence, it may improve classification performance of CNN. Figure 1 shows the processing flowchart for generating EMDI using edge detection. The noise addition step is optional and it has been used for analysis of the proposed method for various SNR levels. The PST is based on a nonlinear dispersive phase operation. The processing steps for edge detection using PST [19] is shown in Figure 2.
Figure 1. Processing flowchart of generating edge enhanced micro-Doppler image (EMDI)
Figure 2. Block diagram of phase stretch transform
The input image was first smoothened out by using a localization filter. Phase stretch operation is applied to the filtered image in frequency domain. The output phase image is represented by $Y[n, m]$ where n and m are spatial variable, and it is described as:
$Y[n, m]=$∡$\{I F F T 2\{\widetilde{K}[p, q] \cdot \tilde{L}[p, q]\operatorname{FFT} 2\{X[n, m]\}\}\}$ (1)
where, $X[n, m]$ is the input image, ∡{∙} is the angle operator and, p and q are spatial frequency variables. Here, p and q are the two-dimensional inverse Fast Fourier Transform and two-dimensional Fast Fourier Transform, respectively. $\tilde{L}[p, q]$ is the localization kernel function in the frequency domain and $\tilde{K}[p, q]$ is a warped phase kernel function:
$\widetilde{K}[p, q]=e^{j \cdot \varphi[p, q]}$ (2)
where, $\varphi[p, q]$ is a nonlinear freqency dependent phase function. Here, higher amount of phase applied to the image at higher frequency features. The phase profile may consist of positive as well as negative values. Threshold is essential in order to set the negative values to zero. For calculating the threshold level, phase histogram has been utilized. Output phase of the transform can be helpful in detecting the edges of the input image. Since, the sharp transitions such as edges and corners in image have higher frequency features. In the post-processing stage, thresholding and morphological operations are necessary to generate edges of the image, and also for further enhancement of features. PST based edge detection technique has been utilized for SAR image processing and biomedical image processing. PST can also be used to achieve super-resolution [21].
A digital band-pass filter was implemented to remove noise from unwanted frequencies of the sampled base-band input signal. We have used short-time Fourier transform (STFT) technique to generate the micro-Doppler images (MDI) from the filter data. Prior to edge detection step, the image (MDI) was converted into grayscale. Grayscale image reduces the computational complexity, while preserving the brightness of image. Threshold and tuning parameters were used for optimizing the overall classification performance. Noise addition step was optional and it was used only for testing of indoor case at various SNR levels. A typical example of generated EMDI image is shown in Figure 3.
Figure 3. Generating edge-enhanced micro-Doppler image (EMDI) using MDI and detected edges
In this work, W-band radar system operating at 92 GHz has been used for measuring the MDS of targets. The block diagram of our W-band radar system is shown in Figure 4.
Figure 4. Block diagram of W-band micro-Doppler radar
Radar system was operating at 92.16 GHz, using a 3.84 GHz continuous wave signal along with 24 times frequency multiplier. Radar system parameters are given in Table 1. The detailed description of our W-band radar system is given in [22]. It consists of a transmitter front-end having a 24 times frequency multiplier and an output drive amplifier, a receiver front-end consisting of two 2-stage low-noise amplifiers and a bandpass filter, and a down converter mixer with an amplifier for local oscillator (LO). Down-converted baseband signal has been acquired at 1.25 MSPS using NI USB-6259 DAQ device and, filtered it using a digital band-pass filter (100 Hz-40 kHz) for further signal processing and analysis.
Table 1. Radar system parameters
|
Parameter |
Value |
|
Carrier frequency |
92.16 GHz |
|
Waveform type |
Continuous wave |
|
Antenna type |
Horn |
|
Antenna gain |
23 dBi |
|
Transmitted power |
25 dBm |
|
PLL frequency |
3.84 GHz |
|
Sampling frequency |
1.25 MSPS |
We have performed several experiments for indoor and outdoor scenario (see Figure 5 and Figure 6). Three different drones as shown in Figure 7, were used for the measurement and MDS of all three drones were used for generating EMDI and MDI datasets for classification purpose. In case of indoor measurement, the target was position at a distance of 1.5 m from the radar system and at a height of 1.2 m. For outdoor measurement, drone was hovering at height of 3 m and the ground range of drone from the radar was 10 m.
Figure 5. Indoor experiment setup
Figure 6. Outdoor experiment setup
Figure 7. Photograph of targets: (a) quadcopter with 2-blade rotors, (b) quadcopter with 3-blade rotors and (c) hexacopter with 2-blade rotors
In this work, image classification technique using CNN architecture has been used for the classification of drones.
4.1 Datasets
Input dataset consists of images derived from micro-Doppler signatures of drone. A fixed sample length of 200 ms were used for generating the image frame, for all the cases. Based on MDI and EMDI images, two different datasets were used for training and testing purpose. In Figure 9, typical examples of input images for classification purpose using MDI and EMDI datasets are shown. Here, comparison of the classification accuracy of MDI and EMDI based methods for all cases are given. The classification performance of CNN was enhanced by using EMDI datasets. In general, large training data result in lower variance of parameter estimation, however large testing data will result in smaller variance in output performance. So trade-off between these two are important for any classification model.
Figure 8. Block diagram of noise addition to the input signal (SNR = 5 dB), with thresholding and tuning. EMDI without PST tuning is shown on the top right corner
The processing steps for noise addition to the received signal is shown in Figure 8, along with the effect of PST tuning. Additional cases have also been analyzed, by adding white Gaussian noise to received radar signal in order to attain SNR levels of 0 dB, 5 dB and 10 dB, as shown in Figure 10.
GoogleNet architecture as CNN was utilized for the classification of drones. Dataset for each case is first shuffled and then divided into two parts (85:15 ratio) for training and validation purpose, because of small dataset size. For indoor case, dataset of each drone contains 2100 images. Thus, total 6300 images have been used for the classification purpose. Using outdoor experiment, dataset of each hovering drone contains 1400 images. Here, batch sizes of 32 and 40 for training and validating to process at a time, respectively. The classification performance has been logged for training loss and validation accuracy after every 40 iterations and 500 iterations, respectively.
4.2 Classification results
The classification results for the validation accuracy and train loss for the both datasets are presented in Figure 11. Results are plotted for the total 3000 iterations. In case of EMDI, the resulting classification accuracy for indoor experiment is 100% after 1000 iterations. Whereas in case of MDI, it is 99.9%. Training loss for EMDI and MDI is 0.0006 and 0.00336, respectively. For further investigation, classification performance for signal with different SNR levels of 0 dB, 5 dB and 10 dB were analysed.
Results show that EMDI has better performance than MDI based method, except in case of 10 dB. There are fluctuations in the accuracy plot for 5 dB and 0 dB case, which stabilizes after 2500 iterations. Variation in the accuracy may results when new data samples are selected for validation purpose and, when the weights of CNN are not well fitted. Therefore, the CNN models further need to change weights in order to increase accuracy, which also results in the fluctuation of train loss curve. Smaller batch size is used in order to avoid overfitting the weights and increase the classification performance. It may possible when the testing dataset and training dataset are too much similar. Also, the accuracy and detection rate is limited by the number of input data samples. In case of outdoor experiments, the classification result achieved the validation accuracy of 92.7% and 89% for EMDI and MDI, respectively.
Figure 10. Micro-Doppler images of all three drones for SNR of 0 dB, 5 dB and 10 dB, after thresholding
Figure 11. Classification performance results of CNN using W-band radar system for indoor and outdoor experiments
The work presented utilized utilizes CCN model for drone classification using W-band Doppler radar. A Phase Stretch Transform (PST) technique was used for edge detection and, overlaid detected edges with the micro-Doppler image. Image reformatting was done by overlapping the input image with its detected edges. EMDI and MDI datasets were used for the classification of the drone and results indicate that EMDI has outperformed in terms of both training loss and validation accuracy. Here, effectively implemented radar system for the classification of drones based on GoogLeNet architecture, using enhanced image datasets. Future research possibility includes investigating for the cases of complex background clutter and MIMO system while, preserving micro-Doppler information.
Authors thank to Myunghwan Lee and Jin Mi Jung at Millisys Inc. for their support in hardware implementations.
[1] Seyfioğlu, M.S., Gürbüz, S.Z. (2017). Deep neural network initialization methods for micro-doppler classification with low training sample support. IEEE Geoscience and Remote Sensing Letters, 14: 2462-2466. https://doi.org/10.1109/LGRS.2017.2771405
[2] Wang, F., Skubic, M., Rantz, M., Cuddihy, P.E. (2014). Quantitative gait measurement with pulse-doppler radar for passive in-home gait assessment. IEEE Transactions on Biomedical Engineering, 61: 2434-2443. https://doi.org/10.1109/TBME.2014.2319333
[3] Hornsteiner, C., Detlefsen, J. (2008). Characterisation of human gait using a continuous-wave radar at 24 GHz. Advances in Radio Science, 6: 67-70. https://doi.org/10.5194/ars-6-67-2008
[4] Lei, J. (2005). Pattern recognition based on time-frequency distributions of radar micro-doppler dynamics. Sixth International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing and First ACIS International Workshop on Self-Assembling Wireless Network, Towson, MD, USA, pp. 14-18. https://doi.org/10.1109/SNPD-SAWN.2005.59
[5] Molchanov, P., Harmanny, R., De Wit, J., Egiazarian, K., Astola, J. (2014). Classification of small UAVs and birds by micro-doppler signatures. International Journal of Microwave and Wireless Technologies, 6(3-4): 435-444. https://doi.org/10.1017/S1759078714000282
[6] Palamà, R., Fioranelli, F., Ritchie, M., Inggs, M.R., Lewis, S., Griffiths, H. (2019). Measurements of Multistatic X&L Band Radar Signatures of UAVS. 2019 International Radar Conference (RADAR), Toulon, France, pp. 1-5. https://doi.org/10.1109/RADAR41533.2019.171389
[7] Zhang, P., Yang, L., Chen, G., Li, G. (2017). Classification of drones based on micro-Doppler signatures with dual-band radar sensors. In Proceedings of the 2017 Progress in Electromagnetics Research Symposium—Fall (PIERS—FALL), Singapore, pp. 638-643. https://doi.org/10.1109/PIERS-FALL.2017.8293214
[8] Rahman, S., Robertson, D.A. (2018). Radar micro-Doppler signatures of drones and birds at K-band and W-band. Scientific Reports, 8: 1739. https://doi.org/10.1038/s41598-018-35880-9
[9] Park, J., Javier, R., Moon, T., Kim, Y. (2016). Micro-doppler based classification of human aquatic activities via transfer learning of convolutional neural networks. Sensors, 16: 1990. https://doi.org/10.3390/s16121990
[10] Tahmoush, D., Silvious, J. (2009). Remote detection of humans and animals. IEEE Applied Imagery Pattern Recognition Workshop (AIPR 2009), Washington, DC, USA, pp. 1-8. https://doi.org/10.1109/AIPR.2009.5466303
[11] Tekeli, B., Gürbüz, S.Z., Yüksel, M., Gürbüz, A.C., Guldogan, M.B. (2013). Classification of human micro-doppler in a radar network. IEEE Radar Conference (RadarCon13), Ottawa, ON, Canada, pp. 1-6. https://doi.org/10.1109/RADAR.2013.6586080
[12] Molchanov, P.O., Astola, J.T., Egiazarian, K.O., Totsky, A.V. (2013). Classification of ground moving targets using bicepstrum-based features extracted from micro-doppler radar signatures. EURASIP Journal on Advances in Signal Processing. https://doi.org/10.1186/1687-6180-2013-61
[13] Oh, B., Guo, X., Wan, F., Toh, K., Lin, Z. (2018). Micro-doppler mini-UAV classification using empirical-mode decomposition features. IEEE Geoscience and Remote Sensing Letters, 15(2): 227-231. https://doi.org/10.1109/LGRS.2017.2781711
[14] Zhao, Y., Su, Y. (2020). The extraction of micro-doppler signal with EMD algorithm for radar-based small UAVs’ detection. IEEE Transactions on Instrumentation and Measurement, 69(3): 929-940. https://doi.org/10.1109/TIM.2019.2905751
[15] Kim, Y., Ha, S., Kwon, J. (2015). Human detection using doppler radar based on physical characteristics of targets. IEEE Geoscience and Remote Sensing Letters, 12: 289-293. https://doi.org/10.1109/LGRS.2014.2336231
[16] Kim, B.K., Kang, H., Park, S. (2017). Drone classification using convolutional neural networks with merged doppler images. IEEE Geoscience and Remote Sensing Letters, 14: 38-42. https://doi.org/10.1109/LGRS.2016.2624820
[17] Rahman, S., Robertson, D.A. (2019). Classification of drones and birds using convolutional neural networks applied to radar micro-Doppler spectrogram images. IET Radar, Sonar & Navigation, 14(5): 653-661. https://doi.org/10.1049/iet-rsn.2019.0493
[18] Kang, H., Kim, B.K., Park, J.S., Suh, J.S., Park, S.O. (2020). Drone elevation angle classification based on convolutional neural network with micro-doppler of multipolarization. IEEE Geoscience and Remote Sensing Letters. https://doi.org/10.1109/LGRS.2020.3030113
[19] Asghari, M.H., Jalali, B. (2015). Edge detection in digital images using dispersive phase stretch transform. International Journal of Biomedical Imaging. https://doi.org/10.1155/2015/687819
[20] Suthar, M., Asghari, H., Jalali, B. (2018). Feature enhancement in visually impaired images. IEEE Access, 6: 1407-1415. https://doi.org/10.1109/ACCESS.2017.2779107
[21] Ilovitsh, T., Jalali, B., Asghari, M.H., Zalevsky, Z. (2016). Phase stretch transform for super-resolution localization microscopy. Biomed. Opt. Express, 7: 4198-4209. https://doi.org/10.1364/BOE.7.004198
[22] Singh, A.K., Kim, Y. (2018). Automatic measurement of blade length androtation rate of drone using W-band micro-doppler radar. IEEE Sensors Journal, 18: 1895-1902. https://doi.org/10.1109/JSEN.2017.2785335
