Obstacle Detection and Assistance for Visually Impaired Individuals Using an IoT-Enabled Smart Blind Stick

Interacting and feeling environment is difficult for visually disable persons. Visually disable person faces difficulties to contact with their surroundings, in physical movement and to identify obstacles coming in their way [1]. So, it’s a challenge moving from source to destination at different locations. Visually impaired persons could not move freely from one place to another place without their family’s or friend’s support [2, 3]. So they are dependent on their family in every way like physically as well as financially. For helping visually challenged persons, many systems have been developed, but with many limitations. From many decades the Researchers are trying to understand their need and spending time to develop a helping tool for them [4]. Later on, “Smart Stick” is found very essential to assist the visually impaired persons to alert them about obstacles, informing about locations and guiding them to avoid danger places [5, 6]. Traditionally, the cane used by the blind was constructed of wood; however, the aluminum cane has since supplanted it. As we get farther into the modern day and the age of automation, however, this walking blind stick will become increasingly ineffective in allowing the visually handicapped to maintain their freedom. So that, the proposed SBS able to detect potential obstructions, ultrasonic sensors are installed at strategic positions. Additionally, it features sensors that can detect obstacles in the user's route. The aforementioned sensors connect with the alarm system by sending a signal to an Arduino-programmed microcontroller. The alarm system alerts the user of impediments in the form of a buzzer and a vibrator. The battery-powered gadget is portable and easy to carry everywhere. The equipment can be relied on to provide accurate results because of how it was built.

Smart stick helps the blind persons to navigate freely, participating in social activities and make self-sufficient. It alerts the person with a voice enabled automatic switch both in public as well as private places [7]. The proposed system aims in supplying a real time electronic device for visually impaired persons using Vilo Jones algorithm and arduino. Viola Jones is a cutting-edge method for fast object identification that can run at a rate of 15 frames per second. It was the first system that could detect objects in real time. This tool or device contains a camera through which it will detect the obstacles images present in outdoor and indoor [8]. Ultrasonic Sensor that has been fitted on the smart blind stick calculates the distances from different objects and the stick [9]. If the obstacle object present in front of the stick is in range, then the name of the object will be played by the speaker or the ear phone that has been fitted to the micro controller of the smart blind stick.

The smart blind stick proves really helpful to detect the objects on the way and easy to carry as it is light in weight [10]. But there is limitation in its range because of its own size. The tool provides mobility to the visually challenged person, make their travelling easy and make them independent physically as well as financially [11]. So, easy navigating tool is like an informer about environment and surroundings to visual impaired people. The tool is equipped with many pre-existing algorithms and programs that will inform the state of the surrounding or object nearby i.e. whether there are any solid objects or objects in fluid state along with its distance from the user [12]. The proposed system provides a complete sense of vision with help of some devices like ultrasonic sensor, water sensor, voice playback board, raspberry pi and speaker.

This paper proposes a new well known technology BLE which is popularly known as Bluetooth Low Energy to measure surface distance that lies among different IoT devices and mobile devices with some minimal errors. In this paper, we have taken the help of BLE which is suitable for connectivity with very low power consumption and proximity. Some IoT devices have facilities of self -correcting parts in order to rectify some errors collected from different environmental factors. When it comes to wireless connections, Bluetooth Low Energy (BLE) is your best bet for keeping in touch over relatively short distances.  To an extent, BLE's ability to facilitate inter-device communication is analogous to that of Wi-Fi. Battery life is prioritized over fast data transfer speeds in BLE's intended use cases. Bluetooth LE compliant beacons get the job done swiftly without depleting the battery because the quantity of data you need to transfer to a visitor's smartphone is small. In order to conserve power and increase data transfer rates, the Bluetooth BLE communication system as a whole uses 40 frequency channels, each of which is 2MHz apart.

In our surrounding we may have different types of tools and techniques which are either fixed or portable. Smart phones, wearable devices like smart watches are portable and easy to carry at home, office, street, etc.  where sensors, appliances, smart TV, etc. are fixed either at home or office [7, 13]. BLE is one of the most trusted technologies among the IoT devices because of its very low consumption of power, that proves very less than a new version Bluetooth, and proximity can be provided between transmitter and receiver [8, 14]. Even nowadays it’s also used in our smart phones as a popular technology.

1.1 Motivation

The rapid development of technology has made it possible for people to simplify and improve their way of life. Blind people have trouble with simple things like getting around town, seeing friends and family, and completing other routine errands. In light of this serious problem, we propose the use of "smart sticks," which enable their users to stroll without worrying about bumping into other people or stationary objects. It's an upgrade on the standard blind stick, as it serves as a walking companion by alerting the blind person, via a headset attached to his phone, of impending dangers including water, walls, stairs, or muddy terrain. This method helps the visually impaired navigate their environment by acting as a guide, much like a white cane. A water sensor and ultrasonic sensor will be attached to a white cane to monitor environmental conditions. Water detection sensors can tell if there is a puddle, while ultrasonic sensors use the reflected ultrasonic waves to identify obstructions in its path.

1.2 Problem Statement

The previously designed smart blind sticks have a number of limitations like it may not detect the state of an object like whether the object is in solid or liquid state. Some blind sticks were unable to give the accurate distance of the object from the visually impaired person using PIC16F690 microcontroller and Electronic Travelling Aid (ETA). So by considering the above issues, the blind stick is updated to the enhanced versions by giving the state of the object with the accurate distance calculation through versatile IoT devices and Viola Jones algorithm. The blind stick is equipped with an Arduino UNO, Bluetooth, a soil moisture indicator, and three ultrasonic sensors. The Smart Blind Stick is equipped with sensors that can automatically detect obstacles in the user's path, and it also monitor environmental conditions.

1.3 Contributions of this work

This paper reviews the advanced technologies used to enhance the working of a smart blind stick. New methods are used to bring accuracy in calculating the distance as well as the size of the objects, the visually impaired person will be going to face. IoT peripherals are used to improve the hardware capability. The wireless sensors of IoT plays a vital role in sensing, collecting and processing the data collected. The data collected are filtered and necessary actions being taken based on the codes given. An updated a stick with cutting-edge innovation and a fresh approach to implementation have been created as an affordable interface for the visually impaired.

The ultrasonic sensor is built into the walking cane to help the visually impaired avoid collisions. of ensure people may travel freely and safely, the system will inform the blind person of potential dangers in the form of a speech module on a daily basis. Confidence in one's footing is increased when using this blind stick. The microcontroller (an Arduino Uno R3) will take the raw sensor data and convert it into the brief pulses needed to power the speech module. This tool will be invaluable in helping people get past their obstacles and into a better life.

The paper has been organized as follows. Section 2 gives the literature review of the paper which broadly describes the methodologies followed by existing systems. Section 3 elaborates the overall study of the paper and the proposed system. Section 4 describes the methodologies followed in this paper. Section 5 gives the result analysis of the implemented system. Finally, the paper is concluded in Section 6.

This paper proposes a self-debugging system which is used to examine the distance as well as finding and correcting the errors if found while replacing or adding reference distance. In this system we have both measuring device i.e. target beacon as well as self-debugging system. The target beacon is implemented in order to calculate or measure the most appropriate distance among different measuring devices present within the network i.e. the target beacon and self-debugging beacon which were being placed at a particular point having constant distance. In order to measure or evaluate the most appropriate distance in collaboration with RSS among all the measuring devices, there will be requirement of an extra self-debugging beacon. The destination beacon signal is received by self-debugging beacon and the strength of the received signal at the target beacon (scPower) gets published. The proposed system is designed to use the strength of the signal from destination beacon (power p_x) and from self-debugging beacon scPower by measuring device. We have also discussed the drawback that the power p_x do not have any impact on user’s environment as the value is fixed and therefore power p_x takes over scPower in order to bring more stability and accurateness.

The formation of smart blind stick can easily implement the calculation of accurate distance and it could provide greater flexibility in navigation [33]. The devices and peripherals that will be used in this system could have the capability of judging the obstacles nearby and take necessary steps.

3.1 Proposed work

The proposed system is an advance blind stick with improved navigation for visually disabled person. The innovative smart blind stick is designed for people who are visually challenged to ease up their lives through this advanced technology. The innovative smart blind stick is consisting of different types of sensor to detect different obstacles like water, light, etc. along with large and small objects as shown in Figure 1. Here also the proposed system uses ultrasonic sensor to detect obstacles through ultrasonic waves. After sensing an obstacle sensor passes the data to microcontroller for further processing. It receives the data calculate the distance to check how close the obstacle is? It does nothing if obstacle is not so close otherwise the buzzer will alert with a sound about the obstacle. It alerts with different sound for different types of obstacles. Hardware and mechanical parts are embedded as a part of complete device. If we consider recent technologies then many systems take the help of embedded systems, so small size microprocessors with low power consumption technologies are manufactured as a mandatory component with low cost. The top of the hardware is building with intelligence mechanism by taking all possible sensors and existing networks embedded in a unit to manage both available resources and augmented functionalities provided by a network.

The embedded system designs intelligent techniques to manage power consumption. In ordinary embedded system microprocessors uses external chips for memory and peripheral interface circuit where as in modern embedded system, special microcontroller having storage capacities are used. The processors could be general purpose for certain class of computation or it could be special purpose which is customized for some specific task. So Digital Signal Processor or DSP help to standardize the processors. The heavy production of those embedded systems brings an economical growth whereas the system engineers could design embedded system for specific task with reduced size and cost, with improved reliability and performance. The block diagram of the proposed system is given in Figure 2.

11.png

Figure 1. Detection of large and small obstacles

Generally, in the SBS is used to detect any type of objects by using the Viola-Jones algorithm. It is highly potent, and it has shown to be particularly useful in real-time face detection. While our algorithm takes an eternity to train, it is extremely fast at detecting objects in real time.

The programme, given an image (the technique works on grayscale images), examines numerous tiny sub-regions in an effort to locate a face. Because an image may have numerous faces of varying sizes, it must examine a wide range of relative positions and proportions. In their method, Viola and Jones used Haar-like features to identify external objects.

Digital images with Haar-like properties are often utilized for object recognition. Some features of the human face are universal, such as the fact that the area around the eyes is always darker than the surrounding pixels and the area around the nose is always lighter. By adding up the values of the pixels in both regions and comparing the results, we can easily determine which zone is lighter or darker. The total pixel value in the darker area will be less than the total pixel value in the brighter area. If one side is brighter than the other, it could be the outer edge of an eyebrow, and if the center is glossier than the sides, it could be a nose. To do this, we can use features inspired by the work of Haar to decipher the various facial aspects.

2.png

Figure 2. Block diagram of proposed system

3.2 Component description

a) Ultrasonic Sensor: Itis used to detect obstacles within a set threshold limit and alert the person through beep speech through speaker if any obstacle detected within that range [34]. The sensor provides a variety of sound pattern and speech to identify different types of obstacles and different directions (Top, Middle, Pit and Water). The ultrasonic sensor emits sound scopes which is inaudible to human ears with frequency lying in ultrasonic spectrum (>20kHz). The sound waves are used to detect the nearby obstacles and the detector is used to detect the reflected sound waves through the help of ultrasonic sensors.

The sound waves that have the frequencies above the human hearing are considered as the ultrasonic waves [35]. We come across a number of applications like detectors to measure proximity, movements, level of liquids etc. which are being used to produce, detect, process and execute the ultrasonic signals. Ultrasonic Ranging Module HC - SR04 is used for detecting the obstacles found in the front of the person and the distance has the range from 2cm to 400cm, that is depcited in Figure 3.

HC-SR04 consist of

• 4 pins-ground
• Vcc
• Trigger and
• Echo

There are two opening –

i. Transmitter: The device that is used for signal transmission from source to destination.

ii. Receiver: The device is used to receive the reflected signals that were sent as a high frequency ultrasound waves by the sender or transmitter.

3.png

Figure 3. Ultrasonic sensor for SBS

b) Infrared Sensor: Only front obstacle detection is not enough so obstacles on the sides are detected through IR sensors, that shows in Figure 4. Firstly, the obstacles are detected and the data is forwarded for calculation according to the formula: us/58=centimetres or uS/148=inch. The condition and direction of the obstacles are checked through this calculation.

4.png

Figure 4. Infrared sensor for SBS

This device has the capacity to detect any obstacle found nearby within a range of 700nm to 1mm. The output totally depends on the type of infrared rays received by the IR sensors. Output provide for comparative circuit as the variation cannot be analysed. But we may have two predictions:

• The LED fitted on the microcontroller does not glow if any signals are not received by the IR receivers and thus the comparator device goes low.
• The LED starts glowing if it receives any signal and thus the outputs also get higher.

c) Arduino UNO R3: This device is used to alert by operating the buzzer through sending signal after detecting the distance of obstacle. It instructs the buzzer to beep for one time which indicates obstacle on the left side, two times if obstacles are found on the front side and three times if the obstacles are found on the right side [36]. The vibration alert is also facilitated through vibrator connected parallel along with the buzzer for vibration sensation.

Arduino UNO is a motherboard or we can say it is a microcontroller which works with the concepts and technologies of ATmega328. It consists of 20 pins out of which there are about 16 digital pins responsible for input and output operations, 6 analog pins which are responsible for taking the inputs [37]. This motherboard also contains power jack combined with Quartz crystal technology of about 16MHz. The microcontroller is also facilitated with Reset buttons along with ICSP header. Arduino UNO R3 proves very user friendly to work with which is shown in Figure 5.

We can connect this microcontroller either directly with the computer USB cable or AC-DC adapter or battery with 5V operational voltage.

5.png

Figure 5. Arduino UNO R3 for SBS

Features of Arduino UNO:

• 5V- Voltage required for different operations
• 7V to 12V- Voltage required for giving the input to different peripherals
• 14 pins- responsible for carrying out different input and output operations digitally
• 06 pins- responsible for taking the inputs in analog format
• 40mA- responsible for giving DC current to each pin for input and output operations
• 50mA- responsible for giving DC current to 3.3V pin for different operations
• 32KB- Flash or secondary memory to hold the data
• 2KB- Static RAM for carrying out different executions
• 1KB- Electronically Erasable PROM for microcontroller start up
• 16MHz- clock speed required for different executions

d) Light sensor: This device is used to alert about the darkness or brightness of a place to the blind person, that is depcited in Figure 6. Also helps to detect whether it is night or day [38].

6.png

Figure 6. Light sensor for SBS

e) Moisture sensor: It is used as a part of the device used to detect if there is any water pits or water puddles, that is shown in Figure 7. If any obstacle found the sensor sends the signal to microcontroller and it sends signal to buzzer to alert the person through sound [39].

7.png

Figure 7. Moisture sensor for SBS

8.png

Figure 8. RF transmiter and receiver for SBS

9.png

Figure 9. Water sensor for SBS

f) RF transmitter and receiver: 433 MHz RF transmitter and receiver modules are considered as RF Modules, that depcits in Figure 8. In order to transfer the data between two microcontrollers, we require transmitter and receiver [40]. There is no power consumption by the transmitter during logic zero which in turn reduces the transmission of the carrier frequencies which lowers the power consumption impact on the batteries. So when there is any requirement of transmission, the logic zero changes to one where the carrier frequency operates with a power supply of 3V along with 4.5mA. Serial transmission method is used to transmit the data from sender to receiver [41, 42].

g) Water sensor (Robodo REL_35): To see the presence of water and sound an alarm to inform the visually impaired, water detectors are electrical devices that detect the presence of water, that depcits in Figure 9. Liquid leaks, precipitation, water levels, and other types of water can all be monitored [43]. A leak, spill, flood, or rain may be detected by an Arduino-connected water sensor.

Table 2. General characteristics of different sensors

In Table 2, the general characteristics of several sensors are shown.

Different sensors like Ultrasonic sensors or infrared sensors or PIC18F46K80 model or ISD1932 model sensors are tested individually as well as installed together. Studies of ‘ultrasonic sensors' and its reflections on different obstacles are very important, since ultrasonic sensors operate on the echo principle. To start the measurement process, the microcontroller sends a pulse of around 10µs which is considered as very high level pulse to the pin(T1) triggering the sensor. After receiving the pulse, an ultrasonic signal of around 40 KHz and 450µs (T2) is sent by the sensor and then it will wait to record the output of rising edge through the port (T3) ranging from around 150µs to 25ms and it totally depends on the distance being measured by the sensors. During the absence of obstacles (no reflected signal), it waits for around 38 milliseconds approximately before starting transmission again.

Ultrasonic distance sensors detect obstacles based on the concept of TOF popularly known as Time of Flight - the result is a pulse that in two experiments has been conducted using different obstacles and decides how long it takes to travel from the source to the destination which is depicted in Figure 11.

11a.png

(a)

11b.png

(b)

Figure 11. Implementation of Smart Blind Stick (SBS)

After testing the performance of different ultrasonic sensors in the lab, the results were compared with different simulation evaluations and calculations. This comparison shows the difference between the calculation value and the measurement value of an ultrasonic sensor's analog voltage. After that the error value that is calculated is as follows:

For Small range (5.5cm) - 1mV to 6mV.

For Medium range (75.2cm) - 9mV to 23mV.

For high range (250cm-400cm) - 30mV to 47mV.

In order to test the efficiency of the smart stick, we have done experiment in real world by taking the real blind people anonymously. In the first round we have taken various numbers of obstacles and we carried out around two experiments. In this experiment, we took around six blind people to test on the use of smart stick out of which we have taken two trained blind people and four untrained blind people. So, in order to make the experiment more effective, we have taken a testing area where we have taken various obstacles of different sizes and then we allowed a blind person to walk through it. This experiment carried out by involving two trained blind people on smart stick and four non trained blind people. Through experiment the walking speed of the user is recorded. Training and untrained participants are measured by the time they take to successfully walk through obstacles.

Taking into consideration of the above calculations, we observed that the trained people average travelling speed is about 0.81 m/s and the untrained people average speed of travelling is about 0.40 m/s and comparing the value with the sighted people average travelling speed is about 1.5m/s.

Using this method, it was shown that the training resulted in twice as much travel speed as well as increased user confidence in avoiding obstacles. Comparisons should be made with available and current technology. As part of evaluating the performance of a proposed guidance system, a number of factors should be considered, such as detecting the range, time consumed to respond the objects and consumption of power.

Detecting the range is considered as the first parameter. An obstacle-finding device that can cover obstacles between 0 and $2 \mathrm{~m}$ is considered low range, between 2 to $4 \mathrm{~m}$ is considered medium range, and while above $4 \mathrm{~m}$ is considered high range. If we will consider the second parameter then we will observe the response time that a sensor takes to respond as per the following:

• Fast responding- 0ms to 100ms
• Medium Responding- 100ms to 200ms
• Slow Responding- 200ms and above

The consumption of power is considered as the third parameter and it will reflect the amount of time the system will work by the help of battery or without direct current or recharge. The following power consumption are taken into consideration:

• Low Power- 0.5W or less
• Medium Power- 0.5W to 1W
• High Power- 1W and above

The comparison of the working potential and the result gained of ultrasonic sensors are mentioned in Table 3. The proposed model is giving better object detections in both indoor and outdoor, that is tested in various range of distance (cm). So that, this approach shows best performance that various traditional models, which is depicted in Table 1. 

Table 3. Comparison result of ultrasonic sensors

The calculated analog values of the sensors are represented in the graph in Figure 12 and the comparison of the calculated values and measured values are represented through graph in Figure 13.

Figure 13 shows 1 black line and 1 red line. The black line represents calculated analog values of the sensors and the red line represents measured analog values of the sensors.

12.png

Figure 12. Graph to represent actual analog values of the sensors

13.png

Figure 13. Graph to represent the comparison between the actual analog values and the measured values of the sensors

5.1 Discussion

In order to be considered portable, the device must be light in weight and easy to wear for an extended period as opposed to being non-portable. According to our results, we performed better than the literature from the past 5 years. Several Ultrasonic Sensors, PIC Microcontroller 16F877A, and Voice Alerts for Visually Impaired Persons would provide a quick response while navigating through the testing area whenever they will come in contact with any type of obstacles approximately within an area of about 70 cm. We will take the help of around three sensors for guiding the visually impaired people through the smart stick.

The device is integrated with 16F877A microcontroller along with speakers and vibrating devices to detect the obstacles or obstructions within a range of 2 meters. Similarly, ultrasonic devices and adjustable belts for blind people consists of five sensors, audio recorder of APR9600 model and secondary memory for audio playback devices along with the earphone fitted to the AT89S52 model microcontroller.