Staircase Detection to Guide Visually Impaired People: A Hybrid Approach

Staircase Detection to Guide Visually Impaired People: A Hybrid Approach

Ahsan Habib Md. Milon Islam Muhammad Nomani KabirMotasim Billah Mredul Mahmudul Hasan 

Department of Computer Science and Engineering, Khulna University of Engineering & Technology, Khulna 9203, Bangladesh

Faculty of Computing, University Malaysia Pahang, Gambang 26300, Kuantan, Malaysia

Corresponding Author Email: 
nomanikabir@ump.edu.my
Page: 
327-334
|
DOI: 
https://doi.org/10.18280/ria.330501
Received: 
12 June 2019
|
Revised: 
5 August 2019
|
Accepted: 
10 August 2019
|
Available online: 
30 November 2019
| Citation

OPEN ACCESS

Abstract: 

Eyes and other visionary organs are essential to human physiology, due to their ability to receive and process subtle details to the brain. However, some individuals are unable to visually perceive things in the surroundings. These visually impaired people face various hinderances in the daily life, and cannot navigate themselves without guidance. To help them navigate around the surroundings, this paper develops a hybrid system for the detection of staircase and the ground, using a pre-trained model and an ultrasonic sensor. The proposed system consists of an ultrasonic sensor, an R-GBD camera, a raspberry pi and a buzzer fixed on a walking stick. During the detection process, staircase images are captured by an RGB-D camera and then compared with pre-trained template images. Finally, our system was applied to detect different staircase images under various conditions (e.g. dark and noise), and found to achieve an average accuracy of 98.73%. This research provides an effective aid to the visually impaired.

Keywords: 

staircase detection, visually impaired people, sensors, computer vision, faster R-CNN

1. Introduction

Approximately 253 million individuals are visually impaired according to the World Health Organization (WHO). Roughly 36 million individuals are blind among them and the rest 217 million individuals have different vision impairments. Among these people, approximately 80% people are 50 years or above in age [1].

As the world is progressing in science and technology at a faster rate, new systems are being developed every day to make daily living more comfortable. But the people who have physical disabilities, require more help than the ordinary people. Technology has been developed to provide solutions so that they can live around society just like other normal people. Nowadays, development of visually impaired assistive devices [2-7] has become a prominent area of research. One of the most challenging tasks [8] is to develop such systems with a user interface for the people with such physical disabilities. In the last decade, many works [9-17] have been done in this field to help visually impaired people in different directions.

To solve the travelling problems of visually impaired people, we developed a framework using a hybrid approach for staircase detection, in which the detection of staircase is done by a pre-trained model and also by an ultrasonic sonar sensor. For the pre-trained model, we collected 510 images with different size and categories of staircase (both upstairs and downstairs). We trained the images using Faster R-CNN (Region Convolution Neural Network) which is a promising object detection model. For the ultrasonic sensor, we used one sensor transversely to the stick to detect both upstairs and downstairs. Initially a reference value which denotes the ground is set and many readings are taken for better accuracy. Next an average of the measured values is computed and the decision is made based on the reference and average value as follows. When the average value is less than the reference value (ref>avg), then it is considered as upstairs; and when the average value is greater than the reference value (ref<avg), then it is downstairs. In the proposed system, we detect the staircase through camera using pre-trained model and send the response to the raspberry pi and integrate the result with the ultrasonic sensor. We set a condition for better accuracy that the ultrasonic sensor does not start to work until the pre-trained model detects it whether it is upstairs or downstairs.

The remainder of this paper is organized as follows. Section 2 introduces the state-of-the-art of staircase detection system. In Section 3, the proposed methodology of the developed system is described. The implementation is explained in Section 4 and the experimental results are given in Section 5. The user experience is discussed in Section 6, and last, the paper is concluded in Section 7.

2. Overview of the State-of-the-Art

Recently, several techniques have been developed to detect staircases with the evolution of technology to assist the visually impaired. The sensor-based staircase detection systems comprise of different sensors (e.g., ultrasonic sensors, gyroscopes, Kinect sensor and tactile sensor) to collect data from the real-world environment to detect the stairs. On the other hand, computer vision has become a prominent research area that helps to develop different assistive devices for the individuals. The vision-based staircase detection systems use cameras (RGBD, Stereo, Depth, etc.) to capture the real-world environment and utilize appropriate algorithms to extract the staircase from the images. The hybrid system uses both the sensors and cameras with processing devices. Different types of algorithms and sensors are used to detect stairs in case of hybrid approach. The works associated with this area can be outlined as follows.

Romić et al. [18] introduced a method for staircase detection to assist visually impaired people to detect stairs in an unfamiliar environment. The preprocessing step in the system uses a combination of the canny edge detector and close morphological operator. Here, binary images with highlighted edges of different obstacles in the image including staircases are generated by the canny edge detector. The system processes the data, and detects the staircase accurately, and then informs the user of the staircase scenario. It can detect within 2-4 meters and notify the user. However, the system cannot detect movement on a staircase and it cannot detect the staircase in a long range. Ponnada et al. [19] proposed a manhole and staircase detection system for visually impaired people that can help the individuals to avoid the manhole and staircase in the way of walking. The system used an ultrasonic sensor for the detection of manhole and staircase, and a computer-vision technique to detect the hindrances. The speeded up robust features (SURF) algorithm extracts the features from the real-time images, and bivariate Gaussian mixture model (BGMM) is used to detect the staircase and manhole. The main components used in the system are Arduino Kit, ultrasonic sensor, vibrator, HC-05 Bluetooth and camera. The proposed device is small and lightweight and is effective for the blind and deaf people as the system will give notification about the current position, and provide directions about the stairs. However, the system may fail in the dark or in a place with high sound. Moreover, the system fails to categorize the moving obstacles in case of staircase detection.

Pham et al. [20] developed a real-time obstacle detection system in indoor environment for visually impaired people that can aid them to avoid four types of obstacles. The obstacles are walls, doors, staircases and a residual class that covers loose obstacles on the floor. The system used the RGB-D camera to capture both RGB images and depth maps at a resolution of 640x480 pixels with 30 frames per second. The color and depth are combined to generate a point cloud that is commonly used to represent three-dimensional data in x, y and z geometric coordinates of an underlying sample space. Then it used voxel grid filter to down sample the point cloud, and random sample consensus (RANSAC) algorithm is used for detection in point cloud data. The main components used in the system are a personal computer, an RGB-D camera, a tongue display unit (TDU), a Kinect sensor. The proposed scheme accurately detects and warns of the nearby obstacles and notify visually impaired people using either audio feedback or sensory substitution device or vibration. However, the system is not reliable when the light intensity is high. The system is more expensive and heavier to wear and only able to detect obstacles in a small range. Wang et al. [21] developed an RGB-D staircase recognition system using unsupervised domain adaptation learning algorithm to assist visually impaired people to navigate independently in an unfamiliar environment. The system used domain adversarial neural network (DANN) to extract transferable features to reduce the distribution shift between two domains. They used deep convolution neural network to extract the features and unlike 3-channel RGB images, they used 4 channels (depth information to the fourth channel) as depth information has a better feature representation than R, G and B channels. The system achieved classification accuracy (100%) for labelled escalator staircase and poor classification accuracy (80.6%) for unlabelled stationary data distribution. Therib [22] implemented a smart blind stick for the detection of holes, obstacles and ponds using microcontroller to aid blind people for navigation. The designed stick used two ultrasonic sonar sensors, one moisture sensor, Arduino, buzzers, vibrator and three LEDs. One ultrasonic sensor is set in a measured angle to detect hole or stair in front of visually impaired at distance about 48 cm and one is to detect obstacles, person, or wall etc. at a small distance. The moisture sensor is used to measure the degree of soil moisture to detect ponds or wet areas and alert the blind person when the degree exceeds a measured level set in the first leg of the four-leg-stick. The system obtained an average accuracy of 86.11%. The proposed system fails to classify between upstairs and downstairs, and the detection range is small.

Huang et al. [23] developed a system to detect staircase based on projection-histogram. The proposed framework divided into six main steps. At first, they used directional Gabor filter and canny edge detector to eliminate the effect of light and detects the edges of stairs respectively on the staircase image. Then irrelevant edges are filtered out. At the third step, horizontal edges are connected. Then only those horizontal edges are considered which have increasing tendency. Next, they performed the horizontal projection histogram drawing. Finally, they connected the histogram corresponding to the first horizontal edge and the last horizontal edge, and used the obtained straight line as the standard for measuring other edges. However, the system fails to work in harsh environment like light, rainy etc. and fail to detect spiral staircase or staircase with turn.

Chun et al. [24] proposed a ground plane hazards detection for visually impaired people. The system used two ultrasonic sensors which are set at two different pointing angles. One sensor is worked for responding for increase in range reading like descending ramp, descending staircase, drainage, pothole, etc. and the other one is used to respond for decrease in range reading like ascending ramp, ascending staircase, etc. The developed system achieved an average success rate (approximately 90%) for each ground plane hazard. However, the designed system fails to detect spiral staircase.

3. Hybrid Approach for Staircase Detection

In the proposed system, the data are collected using ultrasonic sensor and camera. The system has two parts with each part detecting staircase in different ways. The collected data are processed in raspberry pi for staircase detection. The block diagram of the proposed hybrid system is illustrated in Figure 1. The system generates feedback signal with the presence of staircase and sends it to the users for notification. The camera continuously captures images and sends it to the trained model for detection of the staircase. If no stair is detected in the given image, then the model would mark the output as ground level. The ultrasonic sensor is attached to the walking stick in a fixed position and a reference value was set by measuring the distance from ground from that fixed position. By comparing with that reference, the raspberry pi gives its output. The buzzer was set on the walking stick to provide signal to the user that the device has detected staircase ahead. As it is mentioned before, our system has both camera and ultrasonic sensor, we combined the result from individual sensing components.

Figure 1. Block diagram of hybrid staircase detection system

The proposed system architecture is depicted in Figure 2. At first, the images are taken from a camera, then passes them to CPU to predict the result. Moreover, staircase is detected using ultrasonic sensor. Through Wi-Fi, the ultrasonic sensor data is passed to the raspberry pi and finally, the results from raspberry pi and the CPU are compared to make the decision which is provided as the output.

Figure 2. System flow of the proposed hybrid approach

3.1 Data collection and description

We have collected 510 images in order to conduct the research. Some of them are from Khulna University of Engineering & Technology campus that are captured through camera of a cell phone and rest are from google. The developed system used vision-based technology where we used a model of neural network [25-27] which is described below. The images are collected from our surroundings and google search so that the system cover all types of stairs.

3.2 Data preparation

As the system used the concept of the neural network, hence pre-processing is necessary as per the requirement of the pre-trained model. The captured images were in high resolution and the size was around 5 MB but for training those images, the processing is very time consuming. Hence, we resized the images and converted them into around 720X960 size which are less than 150 KB with no change of output. We partitioned the images into two parts, one portion is for training purpose and another portion is for testing.

We used “LabelImg” for labelling training images with “upstairs” and “downstairs” classes. When the training image contains the upstairs, it is labelled with “upstairs” and for downstairs, it is labelled with “downstairs”. The labelling of the images is illustrated in Figure 3. In Figure 3(a) as the image contains upstairs, it is labelled with “upstairs” and in case of Figure 3(b) it is labelled with “downstairs” as the image contains downstairs.

3.3 Faster-RCNN-Inception-V2-COCO model

Faster R-CNN technique is 10 times faster than Fast R-CNN technique and 250 times faster compared to R-CNN. Faster R-CNN [28] mainly consists of two neural networks - region proposal network (RPN) and detection network. RPN helps to find out the region of interest (ROI) as a bounding box on the picture. The outputs from both the RPN is used as an input in the Detection Network. This Detection Network finally gives the intended output if the image in the bounding is upstairs or downstairs or nothing. Faster R-CNN is comparably faster than other R-CNN technique because in the case of ROI it uses RPN, while Fast R-CNN uses selective search method.

At first an image is passed into convolutional layers which generates feature map of the image. This feature map is used into RPN. In the case of RPN, each pixel of the image is considered as an anchor and several bounding box or anchor box are generated using these anchors. Before passing the image to the Detection Network, these anchor boxes are filtered using Non-Maximum Suppression and sampling techniques. Moreover, these ROIs are labeled calculating intersection over union (IoU) and comparing IoU with ground truth boxes. To accurately locate the bounding box on the object the RPN use bounding box regression method. Finally, in the Detection Network there are a fully connected layer and linear regression layer to find out the classification label and the location of bounding box.

Figure 3. Labelling of images (a) upstairs (b) downstairs

The model is pre-trained and provides results when the real time image is sent in the model. For this, a high-speed processor was required for faster output. The training procedure of Faster-RCNN-Inception-V2-COCO model is depicted in Figure 4. We used “LabelImg”, a graphical image annotation tool, written in python with Qt graphical interface for labelling our training images. As the proposed system can classify both upstairs and downstairs, we provided two categories of images for labelling.

Figure 4. Training procedure of Faster-RCNN-Inception-V2-COCO model

4. Implementation of the Prototype

The implementation of the system could be divided into four parts as the different parts of the system had different challenges to overcome. The overall system was integrated with ultrasonic sensor, raspberry pi, buzzer and camera into a stick. The main challenge was to integrate these different parts into the walking stick so that the person who would use the device would have no complexity during usage and the main objectives could be achieved. The developed prototype for staircase detection system is shown in Figure 5. The description of the process of implementation is as follows.

Figure 5. Prototype of the staircase detection system

4.1 Integration of components

We have connected the ultrasonic sensor by using jumper cables. We used HC-SR04 ultrasonic Sensor where four pins needed to connect with the pie out of the five. We measured the distance of the ground after placing the sonar sensor on a fixed position with a fixed angle. With that fixed angle we measured a fixed distance for ground which had been set as threshold below which it would detect up stair and above the threshold the raspberry pi would detect up stair no matter what is ahead. The sensor had setback when it is set on the noisy places. Even with moving obstacles ahead the data would deviate quite a lot. To ahead this major setback we took 1 thousand data per second and found the mean to compare it with the threshold value rather than comparing each data. For future improvement we should take more data for getting the mean as the data of the sonar sensor deviates with just a minor change of noise ahead it to get more accurate result from the sensor.

First challenge that we have faced here was to connect the camera with processor while the camera needed to be placed on the stick on a fixed position. We placed the camera in the top of the stick and configured in such a way that the camera would give continuous images to the model when the device is turned on. Then the model would detect both the upstairs and downstairs in each frame continuously. We configured the processing device such that it would generate some kind of message which would be sent to raspberry pi which notify the users the presence of up or down stair is detected by the camera.

We set the raspberry pi as client and the processor device as server using the TCP connection method. With each frame, a message would be generated whether the model detects staircase or not. Hence, there were three types of messages. If staircase was not detected then the server would send the “ground” to the pi. For upstairs and downstairs, the following messages were sent “upstair” and “downstair”, respectively. The raspberry pi received these messages and stored. At the time, same type string was generated by the achieved mean value of ultrasonic sensor. When these messages are matched then buzzer would beep faster if upstairs was by both the systems and slower if down stair was detected. For ground there wasn’t any notification. We also used “espeak” library so that the user could also use headphone instead of the buzzer. The audio signal was sent to the users through headphone.

4.2 Training the model with dataset

We collected a total of 300 images in different buildings in the real world and around 210 images from google of staircases of different types so that the system would be able to detect staircases. We divided them as training set and testing set of 353 images and 157 images, respectively. The testing set consists of the around 30% of the total images. We resized the images to lower quality with a resolution of 720x960 pixel so that the training of the model could be faster. We labelled the staircase in the images so that we could extract the attributes of the staircase in a CSV files. Then we passed these images with the CSV files into the Faster-RCNN-Inception-V2-COCO model which would generate a neural network by the given data which would detect staircase when real time image is passed to the developed model.

5. Experimental Results

This section shows the experimental results of various upstairs and downstairs images. First, experimental setup is discussed. Next, we provide the results and discussion.

5.1 Experimental setup

For vision system, all experiments were done using Intel(R) Core (TM) i3-5005U CPU @ 2.00 GHz processor along with 4.00GB RAM and NVIDIA (940M). The higher accuracy rate is achieved with the higher resolution of images like 1920x1080 or more. But the training time is higher with the higher resolution. We converted the images with size 720x960 for training and found a satisfactory result.

5.2 Results and discussion

The model, Faster-RCNN-Inception-V2-COCO, was trained and there occurred some loss in each step. Normally loss starts at high and get lower as training progresses. We stopped our training until the loss consistently dropped below 0.05, which took around 600 steps. The loss may vary with the models. The loss curve during the training phase is shown in Figure 6.

In Figures 7-8, some output images of the developed system are presented. Figures 7(a)-(d) contain the images of upstairs which are corrected detected by the system. Figures 7(e)-(h) contain images of downstairs which are correctly identified by the system.

Figure 6. Total loss vs. Iteration

During simulation, sometimes the developed system failed to classify between upstairs and downstairs when the contrast of images is too high or low. On the other hand, in case of real time, there is no problem when the environment is shinny. However, the system may fail to classify upstairs and downstairs in low light environment. In Figure 8(a), the image with downstairs is detected as upstairs because of low contrast of the image, while in Figure 8(b), the image with upstairs is detected as downstairs for the same reason.

 

Figure 7. Resultant images; (a)- (d) Correctly upstairs detected; (e)-(h) Correctly downstairs detected

Figure 8. Resultant images with error; (a) Downstairs is detected as upstairs; (b) Upstairs is detected as downstairs

Table 1. Detection accuracy of staircases

 

Staircase (Actual)

Negative Sample (Actual)

Staircase (Predicted)

102 (TP)

0 (FP)

Negative Sample (Predicted)

2 (FN)

53 (TN)

 

Table 2. Accuracy of classification between upstairs and downstairs

 

Upstairs

Downstairs

Upstairs

55

2

Downstairs

4

41

 

Tables 1-2 provide the summary of the simulation results from the proposed staircase detection system. We used an evaluation tool - confusion matrix/table - which gives a popular representation of the performance measure of any classification models. The matrix shows the number of correctly and incorrectly classified examples, compared to the actual (true) outcomes in the test data. In Table 1, TP, FP, FN, TN indicate true positive, false positive, false negative, true negative, respectively. TP means a correct prediction of a label (existence of stairs), FP denotes a false prediction of a label. On the other hand, TN presents a correct prediction of the other label (non-existence of stairs). FN points out that we predict stairs but actually the image does not contain stairs. The proposed system achieves an accuracy of 98.73% in average and 102 out of 102 images for positive sample and 2 out of 55 images for negative sample are detected as staircase in the testing phase. Here, negative sample indicates the images that contain no staircase. The output of the simulation is listed in Table 1.

For staircase, we further classified whether stairs are either upstairs or downstairs. For the images that contain upstairs, 55 out of 57 images are correctly detected having upstairs and the rest are detected having downstairs. On the other hand, for the images which contain downstairs, 41 out of 45 images are accurately detected with downstairs and the rest 4 are detected containing upstairs. We achieved 94.12% accuracy rate for classification of the images with upstairs and downstairs. Table 2 provides the accuracy of the classification of upstairs and downstairs.

6. User’s Experience

The prototype was developed using a quite low processor compared to the available device in the market. Hence, the system has a high potential in the market. Furthermore, the overall system gave a good experience while testing with visually impaired. In the noisy places, the buzzer sound became faint and therefore, the users required to use microphone connected to the raspberry pi. The developed prototype had to connect to Wi-Fi and the proposed system should have its connection module making a connection between the processing device and raspberry pi. The wiring was also a bit exposed making the device vulnerable with the surrounding obstacles. The overall performance of the developed device was not completely achieved as expected due to the processing device. However, using a high-power processing device would provide the expected latency.

Figure 9. Prototype in use. (a) Downstairs (b) Upstairs

Table 3. Real time results using only RGBD camera

Stairs type

Environmental conditions

Total image

Correctly detected

Detection accuracy (%)

Upstairs

Normal light condition

21

20

95.24

Low light condition

17

15

88.23

Downstairs

Normal light condition

20

19

95

Low light condition

15

13

86.67

 
The user prototype is depicted in Figure 9 in which a blind person carries the developed prototype for proper navigation. The images without staircases, where the detection of staircases should not occur, accuracy rates are slightly lower and the resolution of images has also impact for detection. For real-time experiment using only RGBD camera sensor and in case of upstairs with normal light condition, the system achieved an accuracy rate of 95.24% (20 out of 21 images are correctly detected as upstairs) and for the environment with low light, the detection accuracy is 88.23% (15 out of 17 images are correctly detected as upstairs). On the other hand, in case of downstairs, environment with normal light condition, the developed system achieved an accuracy rate of 95% (19 out of 20 images are correctly detected as downstairs) and for the environment with low light condition, the detection accuracy is 86.67% (13 out of 15 images are correctly detected as downstairs).

In Table 3, real time results are listed using only RGBD sensor. For ultrasonic sensor, the developed system provides good results when there is no interruption in the surrounding environment. When the obstacles are present in the surrounding environment, the detection accuracy is comparatively lower. For real time experiment in case of upstairs with the normal environmental condition, using only an ultrasonic sensor provided a detection accuracy of 96% (24 out of 25 samples are correctly detected as upstairs) with the developed system. On the other hand, in noisy condition, the system achieved an accuracy rate of 84.21% (16 out of 19 samples are correctly detected as upstairs). In case of downstairs with normal environment, the detection accuracy of the developed system is 95.65% (22 out of 23 samples are correctly detected as downstairs), while with noisy environment, the system achieves an accuracy rate of 82.35% (14 out of 17 samples are correctly detected as downstairs). In Table 4, the real time results from the system using only ultrasonic sensor are recorded.

Table 4. Real time results using ultrasonic sensor

Stairs type

Environmental conditions

Number of test

Correctly detected

Detection accuracy (%)

 

Upstairs

Normal condition

25

24

96

Noisy condition

19

16

84.21

 

Downstairs

Normal condition

23

22

95.65

Noisy condition

17

14

82.35

7. Conclusion

In this research, we propose a method both for upstairs and downstairs detection approach using ultrasonic sensor and RGBD camera for visually impaired people. We used a pre-trained object detection model named Faster-RCNN-Inception-V2-COCO model to develop the system. The main purpose of this proposed system is to build a user-friendly and cost-effective guidance system. The developed system obtained the accuracy of 95.24% and 96% for upstairs detection in normal light condition using RGBD camera and ultrasonic sensor, respectively. With this system, a user receives real-time notifications of the surroundings via buzzer and microphone. The system has a potential to help visually impaired people in navigation and make their life much safer and easier. In our proposed device, we use a hybrid approach that combines the results of two different sensors so that the device could work in all possible cases. Still, there are some minor issues to be resolved, and resolving the issue can be a future work to make the device user-friendly and suitable for industrial production.

In the future, the developed system may incorporate more features such as path holes, obstacles, pedestrians, vehicles, water detection, etc. The obstacles in the environment is either dynamic or static. The movement direction of vehicles can be detected. All processing should be completed in a single micro-controller. For better accuracy in the Faster-RCNN model training phase, it is necessary to increase the number of images. More sensors like laser sensor, IR sensor, etc. can be added to increase for better obstacle detection.

Acknowledgment

This research was partially supported by Fundamental Research Grant Scheme (FRGS) No. RDU1901150 from the Ministry of Higher Education, Malaysia.

  References

[1] Bourne, R.R., Flaxman, S.R., Braithwaite, T., Cicinelli, M.V., Das, A., Jonas, J.B., Keeffe, J., Kempen, J.H., Leasher, J., Limburg, H., Naidoo, K., Pesudovs, K., Resnikoff, S., Silvester, A., Stevens, G.A., Tahhan, N., Wong, T.Y., Taylor, H.R. (2017). Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: A systematic review and meta-analysis. The Lancet Global Health, 5(9): e888-e897. https://doi.org/10.1016/S2214-109X(17)30293-0

[2] Islam, M.M., Sadi, M.S., Zamli, K.Z., Ahmed, M.M. (2019). Developing walking assistants for visually impaired people: A review. IEEE Sensors Journal, 19(8): 2814-2828. https://doi.org/10.1109/JSEN.2018.2890423

[3] Tapu, R., Mocanu, B., Zaharia, T. (2018). Wearable assistive devices for visually impaired: A state of the art survey. Pattern Recognition Letters. https://doi.org/10.1016/j.patrec.2018.10.031

[4] Bauer, Z., Dominguez, A., Cruz, E., Gomez-Donoso, F., Orts-Escolano, S., Cazorla, M. (2019). Enhancing perception for the visually impaired with deep learning techniques and low-cost wearable sensors. Pattern Recognition Letters. https://doi.org/10.1016/j.patrec.2019.03.008

[5] Real, S., Araujo, A. (2019). Navigation systems for the blind and visually impaired: Past work, challenges, and open problems. Sensors, 19(15): 3404. https://doi.org/10.3390/s19153404

[6] Cardillo, E., Caddemi, A. (2019). Insight on electronic travel aids for visually impaired people: A review on the electromagnetic technology. Electronics, 8(11): 1281. https://doi.org/10.3390/electronics8111281

[7] Anthierens, C., Groux, D., Hugel, V. (2018). Sensory navigation guide for visually impaired sea kayakers. Journal of Field Robotics, 35(5): 732-747. https://doi.org/10.1002/rob.21775

[8] Abdulkader, S.N., Atia, A., Mostafa, M.S.M. (2015). Brain computer interfacing: Applications and challenges. Egyptian Informatics Journal, 16(2): 213-230. https://doi.org/10.1016/j.eij.2015.06.002

[9] Meshram, V.V., Patil, K., Meshram, V.A., Shu, F.C. (2019). An astute assistive device for mobility and object recognition for visually impaired people. IEEE Transactions on Human-Machine Systems, 49(5): 449-460. https://doi.org/10.1109/THMS.2019.2931745

[10] Rahman, M.M., Islam, M.M., Ahmmed, S. (2019). “BlindShoe”: An electronic guidance system for the visually impaired people. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 11(2): 49-54. 

[11] Sahoo, N., Lin, H.W., Chang, Y.H. (2019). Design and implementation of a walking stick aid for visually challenged people. Sensors, 19(1): 130. https://doi.org/10.3390/s19010130

[12] Bai, J.Q., Liu, Z.X., Lin, Y.M., Li, Y., Lian, S.G., Liu, D.J. (2019). Wearable travel aid for environment perception and navigation of visually impaired people. Electronics, 8(6): 697. https://doi.org/10.3390/electronics8060697

[13] Zhang, X.C., Yao, X.Y., Zhu, Y., Hu, F. (2019). An ARCore based user centric assistive navigation system for visually impaired people. Applied Sciences, 9(5): 989. https://doi.org/10.3390/app9050989

[14] Islam, M.M., Sadi, M.S. (2018). Path hole detection to assist the visually impaired people in navigation. 4th International Conference on Electrical Engineering and Information & Communication Technology (iCEEiCT), Dhaka, Bangladesh, pp. 268-273. https://doi.org/10.1109/CEEICT.2018.8628134

[15] Kamal, M.M., Bayazid, A.I., Sadi, M.S., Islam, M.M., Hasan, N. (2017). Towards developing walking assistants for the visually impaired people. IEEE Region 10 Humanitarian Technology Conference (R10-HTC), Dhaka, pp. 238-241. https://doi.org/10.1109/R10-HTC.2017.8288947

[16] Zhang, X.C., Yao, X.Y, Zhu, Y., Hu, F. (2019). An ARCore based user centric assistive navigation system for visually impaired people. Applied Sciences, 9(5): 989. https://doi.org/10.3390/app9050989

[17] Khanom, M., Sadi, M.S, Islam, M.M. (2019). A comparative study of walking assistance tools developed for the visually impaired people. 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT), Dhaka.

[18] Romić, K., Galić, I., Galba, T. (2015). Technology assisting the blind-video processing based staircase detection. 57th International Symposium ELMAR, Zadar, Croatia, pp. 221-224.

[19] Ponnada, S., Yarramalle, S., TV, M.R. (2018). A hybrid approach for identification of manhole and staircase to assist visually challenged. IEEE Access, 6: 41013-41022. https://doi.org/10.1109/ACCESS.2018.2852723

[20] Pham, H.H., Le, T.L., Vuillerme, N. (2016). Real-time obstacle detection system in indoor environment for the visually impaired using microsoft kinect sensor. Journal of Sensors, 2016: 1-13. http://dx.doi.org/10.1155/2016/3754918

[21] Jing, W., Kuangen, Z. (2019). Unsupervised Domain Adaptation Learning Algorithm for RGB-D Staircase Recognition. arXiv preprint arXiv:1903.01212.

[22] Therib, M.A. (2017). Smart blinding stick with holes, obstacles and ponds detector based on microcontroller. Journal of University of Babylon, 25(5): 1759-1768.

[23] Huang, X., Tang, Z. (2018). Staircase detection algorithm based on projection-histogram. 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), Xi'an, China, pp. 1130-1133.

[24] Chun, A.C.B., Theng, L.B., WeiYen, A.C., Deverell, L., Mahmud, A.A., McCarthy, C. (2019). A ground plane hazards detection tool for the visually impaired. International Journal of Mechanical Engineering and Robotics Research, 8(1). https://doi.org/10.18178/ijmerr.8.1.146-156

[25] Le, T.H. (2011). Applying artificial neural networks for face recognition. Advances in Artificial Neural Systems 2011: 1-16. http://dx.doi.org/10.1155/2011/673016

[26] Sallam, A.A., Kabir, M.N., Ahmed, A.A., Farhan, K., Tarek, E. (2018). Epilepsy detection from EEG signals using artificial neural network. International Conference on Intelligent Computing & Optimization, Pattaya, Thailand, pp. 320-327. 

[27] Dwivedi, A.K. (2018). Artificial neural network model for effective cancer classification using microarray gene expression data. Neural Computing and Applications 29(12): 1545-1554. https://doi.org/10.1007/s00521-016-2701-1

[28] Ren, S.Q., He, K.M, Girshick, R., Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, 91-99