Effectiveness of a Multifunction Virtual Keyboard with Electrooculography Signals for People with Disabilities

The rapid advancement of modern technology has driven the development of various innovative solutions aimed at improving people's daily activities, particularly enhancing the quality of life of individuals with disabilities. Assistive technology plays a crucial role in addressing functional limitations and promoting higher levels of independence [1-8]. People with disabilities are defined as individuals with physical, mental, sensory, or intellectual limitations that may hinder full participation in social and functional activities [9-12]. In some cases, individuals may experience multiple disabilities, a combination of more than one type of limitation that significantly impacts adaptive capacity and level of independence [13]. Globally, the demand for assistive technology continues to increase. According to the World Health Organization (WHO), approximately 1.3 billion people, representing 16% of the global population, live with some form of disability [14]. Furthermore, more than 2.5 billion individuals require at least one assistive device, and this number is projected to reach 3.5 billion by 2050, though the need remains largely unmet [15]. This situation underscores the urgency of developing assistive systems that are not only innovative but also accessible, affordable, and adaptive. In this context, the human-computer interaction (HCI) paradigm plays a crucial role in ensuring that technological systems are designed in alignment with the capabilities, limitations, and interaction characteristics of users [16-20].

One promising direction in HCI is the utilization of biosignals as an intuitive and non-invasive communication channel between humans and machines [21-29]. Among these biosignals, electrooculography (EOG) has emerged as a powerful and widely adopted modality due to its ability to capture eye movement activity through the electrical potential difference between the cornea and retina [30, 31]. Eye movements, such as horizontal, vertical, and blinking, generate distinctive electrical patterns that can be reliably detected and offer several advantages, including low acquisition cost, high temporal resolution, and fast response [5, 10, 11, 31-35].

Recent research trends indicate a significant shift towards the integration of machine learning and deep learning techniques in EOG-based systems to improve classification performance and noise robustness. Techniques such as convolutional neural networks (CNNs), hybrid models, and adaptive algorithms demonstrate robust capabilities in handling complex, noisy, and user-dependent biosignals [36-44]. A recent systematic literature review analyzing 59 studies published between 2019 and 2025 confirms this trend, with approximately 61% of eye movement-based virtual keyboard systems adopting artificial intelligence approaches, with deep learning models being the dominant choice due to their ability to model nonlinear relationships in biosignals. However, most studies still rely on limited-scale datasets dominated by healthy participants in controlled laboratory settings, thus limiting generalizability to real-world conditions [45].

While numerous studies have demonstrated significant progress in the development of EOG-based systems, most existing approaches still focus on a single function and are developed in isolation, such as text-based communication systems, mobility control, or environmental interaction. Virtual keyboard systems have achieved high levels of accuracy and efficiency through the implementation of adaptive layouts and asynchronous interaction mechanisms [19, 20, 24-28, 46]. Meanwhile, mobility systems have demonstrated reliable control performance using machine learning-based classification approaches [1, 47], including EOG-based robotic device control applications for assistive tasks [48]. Intelligent environmental systems have also provided promising usability results in real-world scenarios [4]. However, most of these systems remain single-function solutions that operate separately without integration into a unified platform. This fragmentation poses significant limitations in real-world applications, especially for users with severe disabilities who require seamless interaction across multiple activities, such as communication, environmental control, and digital connectivity. Furthermore, many previous studies have focused more on improving classification accuracy while paying less attention to system integration, usability, and long-term operational stability. These limitations reveal a clear gap in the development of EOG-based HCI systems that are not only algorithmically accurate but also integrated, multifunctional, and ready for use in real-world contexts.

To address these gaps, this study proposes a multifunctional virtual keyboard system based on EOG signals that integrates communication and electronic device control within a unified interface. Unlike prior studies that develop isolated solutions, the proposed system enables users to perform multiple tasks simultaneously within a single platform. The main contributions are fourfold. First, the system integrates an EOG-based virtual keyboard with a real-time communication platform via the Telegram API, enabling users to compose and send messages using eye movements. Second, it combines text input, messaging, and device control within a unified system. Third, a hybrid classification approach is developed by combining polarity-based detection with the K-Nearest Neighbors (KNN) algorithm to distinguish signals with similar characteristics. Although prior studies have demonstrated the dominance of CNNs in biosignal classification, the selection of KNN in this study is driven by the alignment between data characteristics and system requirements. In particular, the limited-scale dataset inherent in experimental settings, a common characteristic in biosignal-based research, renders deep learning models suboptimal, as they typically require large volumes of data to achieve robust generalization and mitigate overfitting [49]. In contrast, KNN, as a computationally efficient non-parametric method, provides stable performance under limited data conditions without requiring complex training procedures [50]. Furthermore, the structured and distinguishable nature of EOG signals enables reliable classification using feature- and distance-based approaches without the need for deep hierarchical feature extraction [51, 52]. Therefore, KNN is adopted as a practical and efficient baseline model, aligned with both the data characteristics and real-time operational requirements. Fourth, this study extends beyond algorithmic performance by incorporating a comprehensive user-centered evaluation involving both healthy participants and individuals with disabilities, including long-term reliability and usability analysis. The system is designed for real-world deployment, emphasizing computational efficiency and real-time operation on resource-constrained devices. The results indicate that the proposed system achieves high performance across multiple evaluation dimensions, including accuracy, reliability, and usability, while maintaining stable operation over extended use. These findings underscore the importance of transitioning from isolated, algorithm-focused approaches toward integrated system design in next-generation assistive technologies.

This research uses an applied experimental approach with the aim of developing a multifunctional virtual keyboard system based on EOG signals. This system is designed to be able to carry out two main functions simultaneously, namely sending digital messages through integration with the Telegram API and controlling microcontroller-based electronic devices.

Figure 1. Block diagram of research stages

The system was developed as a laboratory prototype intended for individuals with motor and multiple disabilities who also experience speech disorders, so they cannot use fingers or voice commands as input media. The stages of this research include: (1) problem identification and literature study (2) design of an EOG-based virtual keyboard system (3) acquisition and preprocessing of EOG signals (4) classification of eye movements (5) integration of the multifunctional system (6) implementation and functional testing (7) performance evaluation and analysis of results (8) conclusions and recommendations. These stages can be seen in Figure 1.

3.1 Problem identification and literature review

The initial stage of the research began with identifying the needs of users with motor and speech disabilities through literature studies and observations of existing assistive systems. The results showed that most previous EOG-based keyboard systems were still limited to one-way typing functions and had not been integrated with device control systems or digital communication platforms. Based on this gap, this research focused on designing a multifunctional biosignal-based system that could perform two functions in a single interface and was easy to use.

3.2 Electrooculography-based system design

Figure 2 shows a diagram of the overall system design, which combines two main, integrated components: hardware and software.

Figure 2. Overall block diagram of an electrooculography (EOG)-based virtual keyboard system

The hardware part is designed to capture, amplify, and convert EOG signals into digital data ready for processing. Meanwhile, the software part is responsible for processing the data, displaying a virtual keyboard interface, and sending commands to electronic devices and messages via the Telegram application. The integration of these two components allows the system to work as a whole, detecting eye activity, translating it into commands, and realizing communication and control functions for electronic devices for users. To explain each component shown in the diagram in more detail, the following outlines the system's design stages in a structured manner. The first stage is the electrode, which detects electrical signals from eye activity and transmits them to the EOG circuit. The initial signal is then amplified and filtered to allow only relevant EOG signals to be transmitted to the NI USB-6008 data acquisition device. The signal is then converted into digital data, which is processed by a computer or laptop, which also serves as a user interface for operating a multifunction virtual keyboard. Command output from this keyboard is used to control electronic equipment, while the graphical interface is developed using Visual Studio. To support communication functions, the system utilizes the Telegram application, integrated through an API.

3.3 Multifunctional virtual keyboard design

The virtual keyboard in this study functions to send messages and operate electronic devices. The virtual keyboard's display is constructed using a morphological chart consisting of a main menu display, a general word display, a message sending display, a keyboard display, and a display for operating the electronic device. The following is a morphological chart for each virtual keyboard display.

Participants evaluated several layout options based on:

a. Visibility: ease of seeing and recognizing the keys.

b. Visual comfort: comfort of the colors used for the eyes.

c. Ergonomics: comfort in reaching the desired keys.

The result of most participants' choices was determined as the final design for the virtual keyboard. Each interface component was designed based on the most preferred configuration, as summarized in Tables 2-6.

Table 2. Morphological chart of the main menu display

Table 3. Morphological chart of the common words display

Table 4. Morphological chart of the messaging display

Table 5. Morphological chart of the keyboard display

Table 6. Morphological chart of the electronic equipment control display

3.4 Eye movement classification

A software-based classification framework was developed to interpret EOG signals using polarity features and the KNN algorithm. The system translates eye movements, left, right, up, down, and blink, into navigation commands for the virtual keyboard. Polarity-based classification is applied to directly distinguish left, right, and downward movements due to their distinct signal polarities. However, upward gaze and blinking exhibit similar polarity patterns; therefore, a K-NN classifier is employed to differentiate these signals. The overall classification process is illustrated in Figure 3.

Figure 3. Signal classification flowchart

3.4.1 Signal acquisition and preprocessing

The dataset used in this study consists of 1,400 labeled EOG signal samples collected from 10 participants, including 8 healthy individuals and 2 users with mild motor disabilities, representing a heterogeneous user group. Each participant contributed multiple samples across different eye movement classes. EOG signals were acquired using a two-channel configuration, with electrodes positioned below the right eye (Ch1) and on the left outer canthus (Ch2), while a reference electrode was placed on the nasal bridge and the ground electrode on the forehead.

The analog signals were amplified using an instrumentation amplifier (gain = 1000) and filtered using a 0.1-30 Hz band-pass Butterworth filter (4th order) to remove baseline drift and electromyographic noise. The signals were sampled at 1 kHz using an NI USB-6008 interface and normalized using Min-Max scaling. This preprocessing pipeline ensures consistent signal representation across participants and reduces inter-subject variability.

All data were collected under controlled laboratory conditions to ensure signal stability. External noise factors, such as varying lighting conditions and unrestricted head movement, were not explicitly incorporated. While this setup enables controlled evaluation of the proposed method, it may limit generalizability to dynamic real-world environments.

3.4.2 Thresholding and polarity determination

A dynamic thresholding procedure was applied based on participant-specific baseline signals. The baseline amplitude was recorded during a 5-second resting period, and the threshold was computed as:

$\mathrm{T}=\mu_{\mathrm{rest}}+\sigma_{\mathrm{rest}}$

where, $\mu_{\text {rest}}$ and $\sigma_{\text {rest}}$ represent the mean and standard deviation of the resting signal, respectively. The threshold values typically ranged from + 0.45 to + 0.55 V. Eye movements exceeding ± T were classified as significant events.

Polarity analysis was then used to directly classify left, right, and downward movements based on the sign differences between channels Ch1 and Ch2. This approach enables fast and computationally efficient classification for signals with distinct polarity characteristics, reducing the need for more complex models.

3.4.3 K-Nearest Neighbor classification for upward gaze and blinks

Since upward gaze and blinking exhibit similar polarity patterns, a KNN classifier is employed to distinguish these two classes. Three features are extracted from the EOG signals: maximum amplitude, area under the curve (AUC), and polarity direction. Euclidean distance is used as the similarity metric. The optimal K value is determined through experimental evaluation using K = 1-5 on a validation set. While K = 1, 2, and 4 achieve perfect accuracy, these configurations are more prone to overfitting, particularly for K = 1. In contrast, K = 3 achieves 99.75% accuracy with improved generalization, as supported by confusion matrix analysis. Therefore, K = 3 is selected as the optimal parameter to balance accuracy and robustness.

The dataset is randomly shuffled and divided using a hold-out strategy into 1,000 training samples (500 upward gaze and 500 blinks) and 400 testing samples (200 upward gaze and 200 blinks), corresponding to an approximate 71:29 split. Both sets are derived from the same participants but are separated to ensure evaluation. This classification approach operates on preprocessed signals to ensure consistent feature representation across users. Although filtering and thresholding help mitigate noise interference, the system is evaluated under controlled conditions; therefore, robustness to dynamic noise remains a limitation and an important direction for future work.

3.5 Multifunctional system integration

This system integrates communication and electronic device control within a single virtual keyboard interface. The communication function is implemented using the Telegram API, while environmental control is achieved using an ESP32-based WiFi relay module for operating household devices such as lights and fans. The ESP32 is selected due to its real-time processing capability, low power consumption, and integrated WiFi connectivity, making it suitable for embedded IoT-based assistive systems. Compared to Arduino and Raspberry Pi, it offers a more balanced trade-off between computational efficiency, connectivity, and system complexity.

Communication between modules is performed via asynchronous serial and WiFi protocols, achieving an average latency of 30-60 ms, which is sufficient for real-time interaction. The system operates reliably for single-device and small-scale multi-device control under controlled conditions, although scalability and long-term stability in larger deployments remain areas for future work. This integrated design enables seamless operation as both a communication tool and an electronic device controller. Figure 4 illustrates the system operation sequence.

Figure 4. Electronic equipment operating circuit

3.6 Implementation and functional testing

The proposed system was implemented and evaluated in a controlled laboratory environment involving ten participants (eight healthy individuals and two users with mild motor disabilities). These participants are the same individuals involved in the dataset acquisition process described in Section 3.4. During testing, participants were instructed to execute predefined tasks using eye-gaze interaction.

Functional testing was conducted to verify the integration and synchronization of system components, from EOG signal acquisition to interface responsiveness. The evaluation focused on three core functionalities: text input, message transmission via Telegram, and control of electronic devices through the virtual keyboard interface.

System performance was assessed using three task-based criteria: accuracy, reliability, and ease of use. Accuracy was measured as the proportion of correctly executed commands during task completion, with any incorrect command classified as an error. Reliability was evaluated by observing the consistency of system responses across repeated task executions at different time intervals, including the occurrence of delays, glitches, or unexpected behavior. Ease of use was assessed by measuring the time required for participants to learn the system and by observing any difficulties or need for assistance during operation.

All tasks were performed repeatedly, and system responses were systematically recorded, including successful executions, errors, response delays, and abnormal behavior. These observations formed the basis for subsequent performance evaluation.

3.7 Evaluation

The evaluation was conducted using a task-based observational approach to assess the performance of the proposed EOG-based multifunctional virtual keyboard system. Participants were required to perform a series of predefined tasks, including character selection, text input, message transmission via Telegram, and control of electronic devices using eye-gaze interaction.

During task execution, system performance was evaluated based on three variables: accuracy, reliability, and ease of use. Accuracy was defined as the system’s ability to correctly interpret eye movements and execute the intended commands. Reliability referred to the consistency of system performance across repeated trials under similar conditions. Ease of use was defined as the level of effort required to learn and operate the system, evaluated through learning time, user dependency, and observed interaction difficulties.

Throughout the evaluation, system responses, interaction stability, and operational consistency were continuously monitored. Any errors, delays, or unexpected behaviors were recorded and analyzed to assess overall system performance. The evaluation results were used to determine the system’s effectiveness in supporting communication and electronic device control through eye-gaze interaction, particularly for users with disabilities. Additionally, it should be noted that the limited number of participants and the dominance of healthy subjects may introduce potential bias, which could affect the generalizability of the results to broader real-world scenarios.

This study addresses a key gap identified in the literature, namely the fragmentation of EOG-based systems across isolated functional domains such as communication, mobility and interaction, and environmental control systems. While prior studies demonstrate high classification accuracy and strong task-specific performance, their limited integration restricts real-world applicability. Accordingly, this study shifts the focus from algorithm-centric performance toward an integrated HCI system capable of supporting multiple user needs within a unified platform.

In communication systems, EOG-based virtual keyboards have reached high algorithmic maturity, including near-perfect detection accuracy and improved efficiency through asynchronous input and adaptive layouts [20, 25, 28]. However, as shown in Table 10, these systems remain limited to text input and lack environmental control capabilities. Similarly, mobility and interaction systems achieve high control accuracy and low error rates [18, 47], while environmental control systems demonstrate strong usability and real-time responsiveness [4]. Despite these strengths, all remain task-specific and lack cross-functional integration, reinforcing structural separation in existing EOG-based solutions. In parallel, algorithm-centric studies improve classification performance using machine learning and deep learning methods [21, 36-44], yet are largely confined to signal classification without extending to complete HCI system implementation. This confirms that algorithmic performance alone is insufficient without functional integration.

Table 10. Comparative analysis of existing electrooculography-based systems and the proposed approach based on functional integration

Several studies also emphasize usability without adequate quantitative validation. Although gaze- and blink-based interfaces are often described as intuitive, the absence of consistent performance metrics limits reproducibility and objective evaluation [24, 45]. This underscores the need for balanced evaluation combining usability with measurable performance indicators. The findings of this study indicate that the primary challenge in modern EOG systems is no longer classification accuracy, but system-level integration across multiple interaction functions. The proposed system addresses this gap by integrating communication and environmental control within a single platform while maintaining competitive performance in accuracy, reliability, and usability. These results demonstrate that high performance can be achieved alongside functional integration, highlighting system-level design, usability, and adaptability as key requirements for real-world assistive technologies. To further illustrate this limitation, Table 10 presents a comparative analysis of existing EOG-based systems based on functional integration. As shown in Table 10, most existing systems remain task-specific, lacking integration across multiple interaction domains, thereby reinforcing the fragmentation identified in the literature.

Although the proposed system demonstrates strong performance under controlled laboratory conditions, several challenges may arise in real-world deployment. Variations in lighting conditions and electrical interference may affect signal quality and classification stability. While filtering and thresholding help mitigate noise interference, their effectiveness under highly dynamic conditions requires further investigation. In addition, the evaluation was conducted on a relatively small sample of 10 participants, which may not fully represent the diversity of end users, particularly individuals with varying levels of motor impairment, potentially affecting generalizability. Future work will therefore focus on improving robustness through adaptive signal calibration and advanced noise-handling techniques, as well as conducting large-scale evaluations involving more diverse user populations.