Amal Hameed Khaleel*
| Thekra Abbas | Abdul-Wahab Sami Ibrahim
© 2025 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
Artificial intelligence has become a vital component of contemporary technology. Despite advancements in computer technology, Arabic virtual keyboards face challenges in layout efficiency and gaze-based control due to the unique characteristics of the Arabic language. The virtual keyboard is an effective input mechanism for human-computer interaction systems. This paper suggests an Arabic virtual keyboard application system that utilizes artificial intelligence and can be controlled using eye movements. The user interacts with the system by utilizing the camera output displayed on the screen, while the webcam serves as an input device. The study proposed a novel method called Distinct Frequency-Alphabetical for virtual keyboard layout. Hence, it studied how to design a keyboard for typing Arabic text using an algorithm for gaze directions in real-world scenarios. The empirical results of the proposed system have produced better results than the previous systems. It achieves this with an average typing rate of 18 characters per minute and 4 words per minute. The testing of the proposed system received positive feedback from several users; it achieved a NASA-TLX score of 8% and 87.5% on the system usability scale.
virtual keyboard, eye gaze tracking, HCI, Arabic keyboard layout, assistive technology
Recently, virtual keyboards have often been used for augmentative communication. The layout and design are two important characteristics of any on-screen keyboard [1]. Assistive technology creates a basis for proposing flexible solutions designed to facilitate the lives of individuals [2]. Computer vision, as a discipline, focuses mainly on human-computer interaction (HCI) through visual perception [3]. The human eye intuitively interprets the communication and interaction of people, from which it can obtain information on the surrounding environment and respond appropriately [4, 5].
Monitoring eye activity is known as eye gaze tracking (EGT). Eye gaze analysis may also aid in comprehending human behaviour, attention, and other cognitive functions [6].
The keyboard system suggested supports Arabic due to two characteristics [7-9]: 1) Arabic is the fifth most common language in the world, and about 300 million people speak it. 2) Arabic is a globally utilized language and is extensively taught in educational institutions and professional environments. On the other hand, typing in the Arabic language faces several challenges:
•The Arabic language is characterized by a rare feature among the world's writing systems: a cursive writing system.
•The Arabic language is characterized by a special feature: allography. The shapes of 15 of the 28 letters differ depending on their location within the word (beginning, middle, end, or isolated). Therefore, 22 letters connect to the next letter, while 6 letters connect to the previous letter.
•The Arabic language contains diacritical marks above the letters.
•The direction of writing in Arabic is from right to left.
The keyboard is a commonly used kind of text input modality, and its usability has a significant influence on the overall usability of a system. One of the primary objectives of human-computer interaction is to ensure that the system is user-friendly and easy to use. In other words, allowing the user to successfully do a job in a manner that is both safe and effortless, while also ensuring a high level of convenience and protection [10-12]. There are several keyboard layouts, but the most common are arranged as follows [13]:
The QWERTY Layout is considered the typical keyboard layout in the Western world. The QWERTY keyboard layout was initially created to tackle difficulties with mechanical typewriters. One issue with the printing mechanism was that the key slugs would easily get stuck if a key was pushed before the preceding one had returned. Sholes addressed this issue by experimenting with the most common English two-letter sequences, digraphs, and assigning the most frequent pairs to opposing sides of the key layout. This was done to avoid mechanical issues. Sholes' remedy was sound and significantly minimized the key jam. So, the Sholes design opened the way for the QWERTY layout to be the "universal" layout [14, 15].
Alphabetic Layout (ABC Layout) is useful from a cognitive standpoint. An alphabetical arrangement can minimize the time unskilled typists spend searching for characters to write.
Frequency Layout is the frequency of letters that often determines the location of a key in a keyboard layout using statistical methods. The Arabic language has 28 alphabet letters that do not appear in equal numbers in a text [16].
To improve the arrangement, it used Fitts’ Law because it is a perfect model of human psychomotor behaviour, which was developed in 1954. Fitts’ law establishes how many words per minute can be typed using the new layout and improves it if necessary. The key arrangement, row weight, and hit direction are critical when improving typing speed and accuracy in a single-finger keyboard, which may be a finger, stylus, eye, etc. [12]. Fitts’ Law is a kind of model that will correctly predict how long a human user will take to access a target during movement. It can be used in designing a layout of keyboard layout to model how long it will take for a human to correctly access a key. It is also used to simulate the time taken for movement and to determine the difficulty of a target selection task, which is referred to as an index of difficulty (ID). Normally, it is given as Eq. (1).
$I D=\log _2 \frac{D}{w}$ (1)
where, D is the distance between the current position and the target. On the other hand, W refers to the dimension or width of the target item. Hence, it is evident that the user's performance and accuracy are directly linked to the design of the key layout. Therefore, people exhibit superior performance while using a larger-sized computer keyboard compared to smaller ones.
Popular designs for virtual keyboards are three: rectangular, circular, and hexagonal, as the QWERTY keyboard, Hooke’s keyboard, and ATOMIK keyboard, respectively [10].
The contributions to this paper are as follows: 1) New strategy for keyboard layout. 2) New design for the virtual keyboard. 3) To control the keyboard, use the gaze-eye directions only with the webcam.
The organization of this paper is as follows: Section 2 summarizes the related works. Methodology of Virtual Keyboard in Section 3. The discussion of analysis and evaluation of results in Section 4. Finally, in Section 5, the conclusions.
This section summarizes the most important studies on the majority of Arabic virtual keyboards. However, augmentative and alternative communication support in Arabic is limited (eye typing only exists in very few commercial programs, e.g., Grid, MyTobii, iWriter) [17]. The iWriter proposed in the study [7] facilitates sight typing, which is useful for communicating in Arabic, providing a way of communication for people with severe motor impairment. Preliminary use tests show a good layout and size for the key to make a selection with the gaze. The user design approach ensured that the system meets the needs of its target audience, making it a suitable option for Arabic communication.
In previous years, Benabid Najjar [17] described a proposal to develop an Arabic keyboard for single-pointer input devices such as mobile phones and gaze-controlled interfaces. In Arabic, the key layout is optimized depending on the movement duration of the pointer and the frequency of letter change. This study intends to increase typing speed and user experience, particularly for those with impairments. In another study [18], Hosny provided an efficient Arabic keyboard layout for single-pointer applications while eliminating inefficiencies. A novel genetic algorithm (GA) was created based on a quadratic assignment problem, integrating crossover and mutation procedures. The experimental findings indicated that simple swap mutation outperformed other approaches, while hybridizing GA with simulated annealing did not increase solution quality or processing time.
Over the years, developed research, such as AlSabah et al. [19], described a novel way of typing Arabic characters and diacritics with a regular keyboard layout. It enables users to switch between two alphabets by pressing the left shift and alt keys, giving versatility and convenience of usage. Each letter key displays the first alphabet's default diacritic, making it easier to add diacritics. The backspace key removes diacritics from each letter, giving users more control over the text they enter. The approach seeks to increase typing efficiency and accuracy in Arabic, making it more accessible to non-specialist users. Also, in 2021, Benabid Najjar et al. [20] proposed an optimization system for the arrangement of keys on the Arabic keyboard for applications that employ a single-pointer input device. It used three methods for layout: common (QWERTY), genetic algorithm (GA), and simulated annealing (SA). Eye tracking was used to evaluate the usability of the optimized layouts. The results revealed by calculating the overall parsed distance when entering a particular text show that the optimized layout (simulated annealing) exceeds the common layout (QWERTY) in terms of anticipated typing speed. However, the usability study revealed that participants' familiarity with the common layout of the keyboard influences typing speed, although there are no significant variances. In another study in 2021, Qtaish et al. [21] created a better Arabic keyboard layout (KL), which was based on a detailed examination of letter distances, frequencies, and probabilities of Arabic letters and bi-grams. A vast corpus of five million words was generated for the development process. The revised KL was tested and compared to the present KL in terms of letter frequency, finger travel distance, hand and finger stress, bi-gram frequency, row distribution, and most frequently used words. The upgraded KL proved to be more efficient than the present KL. Samanta et al. [22] proposed a method for developing virtual keyboards for Indian languages, addressing the complexity of large character sets and inflections. The method was tested with Bengali, Hindi, and Telugu and proved effective in dealing with these languages and can be applied globally.
Recently, Gizatdinova et al. [23] describe two vision-based interfaces (VBIs) that improve performance and user experience during character-based text entry with an on-screen virtual keyboard. Head-based VBI uses head motion to control the computer cursor and mouth-opening movements to select keyboard keys. Gaze-based VBI uses gaze to indicate the keys and an adjustable dwell for key selection. Emile Tatinyuy et al. [24] introduced a new hierarchical strategy for optimizing key selection on virtual keyboards using eye gazing. The suggested approach uses eye movements to split the regular QWERTY keyboard into smaller sections. The search area is narrowed by picking halves consecutively, and within each region, keys are illuminated one at a time for choosing.
The keyboard is the primary input device in computer systems. Recently, the emergence of the field of human-computer interaction and the rapid increase in the number of mobile devices and touch screens have led to the development of virtual keyboards. In this paper suggest a new design and layout of the virtual keyboard as follows:
3.1 Virtual keyboard design
The virtual keyboard is designed based on four important factors (shape of keyboard, number of keys, size of keys, and color of keyboard) that can be explained in the following:
a) Shape of keyboard
Research on virtual keyboards emphasizes the importance of a layout that conforms to natural hand postures and minimizes user fatigue, as ISO 9241-410:2008 (Ergonomics of human-system interaction - Design criteria for physical input devices) [25]. The rectangular shape of the proposed keyboard was used to make it easy to identify the four directions accurately and minimize user fatigue. Also, the rectangular arrangement of the key layout is a conventional layout of a standard keyboard.
b) Number of keys
The virtual keyboard research emphasizes the importance of fewer keys (with predictive text or multi-tap input), as shown in ISO 9241-400 series, to balance functionality and simplicity [26]. When creating the eye-keyboard, the focus was on employing the largest size of the window to be visible to the user, emphasizing ease of movement in all four directions by minimizing the maximum inter-key distance and reducing the number of keys. As a result, the number of proposed keys is 24 (6×4). Experiments have shown that this design is best for moving the cursor easily on the keyboard, by allowing the user to move their sight in four directions easily.
c) Size of keyboard
The small keys lead to higher error rates on touchscreens, as shown in ISO 9241-410 [27]. The proposed virtual keyboard contains two primary components: the initial component (eye-keyboard) displays a total of 24 command keys. The second component (Eye-Board) is an input text screen that allows the user to view the output text in real-time. It allows the printing of thirty to fifty words. The size of the first component (eye-keyboard) is 1200×150 pixels, and the size of the second component (the eye board) is 200×800 pixels. The concept is to augment the quantity of keys in each row to reduce the ambiguity of the keyboard, while simultaneously preserving a sufficiently large key size to facilitate choice. So, the important and prominent keys have been made larger based on Fitts’ Law, which indicates that the closest and largest key is the one that will be easier to handle and will take less time to reach. Therefore, the size of the primary key is 200×300 pixels, while the size of the rest of the keys is 200×200 pixels. Therefore, the size of the major and minor keys in the proposed keyboard is 4.5 and 3.5 cm, respectively. In addition, the letters that appeared most frequently were positioned in the lower row for eye-user convenience, while the less common characters were placed in the upper row.
d) Color of keyboard
It is well-acknowledged that colors have an impact on the sense of sight. So, choosing a color that is visually soothing and pleasant is advisable. ISO 9241-112 (Presentation of information) advises using color to enhance contrast and readability [28]. The RGB color details adopted for the keyboard interfaces are light green, white, and dark charcoal.
3.2 Keyboard layout
The significance of designing an optimized keyboard layout is to enhance text entry performance to the greatest extent possible. This is because (1) eye typing can be considerably slow due to the dwell time threshold that imposes a limit on the maximum typing speed, and (2) it can also result in eye fatigue and discomfort when engaged in prolonged computer-related tasks due to the frequent pattern of eye movements. The keyboard layout uses four techniques (QWERTY, Alphabetic, Frequency, and Proposed Layout (Distinct Frequency-Alphabetical)):
a) QWERTY layout
This work used the QWERTY layout to exploit the user’s previous computer knowledge. The QWERTY layout is divided into two parts, as shown in Figure 1.
Figure 1. Divided QWERTY keyboard
In the suggested Arabic keyboard layout, the letter (لا) was replaced with the letter (ذ), and the letters (د, ج, ط) were added in the fourth row on part 1 of the keyboard, as well as modified letters (أ, إ). Hence, the five basic harakat were added on the fourth row on part 2 of the keyboard, as seen in Figure 2.
Figure 2. QWERTY keyboard layout
b) Alphabetic layout
The alphabetical layout is used when the user is well-skilled in the knowledge of alphabetical character sequences, as shown in Figure 3.
c) Frequency layout
An optimized layout that arranges letters based on the frequency of character transitions would minimize the distance between the most frequently paired characters, decrease the distance travelled by the eyes when reading a given text, reduce the time required to locate the next key, and ultimately enhance the overall typing speed.
Figure 3. Alphabetical keyboard layout
Figure 4. Frequency keyboard layout
Figure 5. Arabic letter frequency distribution
Letter frequency refers to the average number of times each letter of the alphabet appears in the language. In Arabic layout, the alphabet comprises 36 letters (28 main letters and eight modified letters). The sequence of letters from most frequent to least common, which was generated from 1,374,698 words, is seen in Table 1 and Figures 4 and 5.
Table 1. Arabic letter frequency distribution in decreasing order
|
Letter |
Frequency (%) |
Letter |
Frequency (%) |
|
ا |
12.835 |
إ |
1.27 |
|
ل |
11.81 |
ذ |
1.225 |
|
ن |
7.43 |
ج |
1.115 |
|
م |
7.3 |
ة |
1.065 |
|
و |
6.65 |
ى |
1.035 |
|
ي |
6.5 |
ص |
0.835 |
|
ه |
4.785 |
خ |
0.775 |
|
ب |
4.07 |
ش |
0.685 |
|
ر |
3.975 |
ث |
0.65 |
|
ع |
3.425 |
ز |
0.5 |
|
ت |
2.895 |
ض |
0.475 |
|
أ |
2.825 |
ط |
0.44 |
|
ف |
2.74 |
ء |
0.395 |
|
ك |
2.605 |
غ |
0.35 |
|
ق |
2.41 |
ئ |
0.32 |
|
د |
2.24 |
آ |
0.305 |
|
س |
2.145 |
ظ |
0.22 |
|
ح |
1.555 |
ؤ |
0.145 |
d) Distinct Frequency-Alphabetical Layout
The layout optimization process for a virtual keyboard involves arranging the keys and decreasing the typing time to allow the user to type with maximum efficiency. In this paper, a novel algorithm (Distinct Frequency-Alphabetical) suggests utilizing frequency and the alphabetical order of letters together. The system employed to produce the optimized layout involved arranging letters based on a high frequency and, based on alphabetic arrangement, if the frequency of letters was equal. This was done to prevent significant gaps between characters and reduce the time required to navigate the text. This study calculates Arabic letter frequency using the Quran and known Arabic books, including the first seven volumes of Ibn Katheer's series The Beginning and The End, the book of Sirah in The Sealed Nectar of Almubarakfuri, and the book of The Masterpiece of the Brides for Al-Shuri [29].
The effect of the mathematical representation $\sqrt{t f}$ or $(1+\log (t f))$ The statistical calculation process is better than using frequency (tf) alone, so it is used. Since the frequency of distinct letters affects the quality of the results, the following equation was proposed.
$t f^{\prime}=1+\log \left(\frac{t f}{\frac{s d t f}{s s t f}}\right)$ (2)
where, $t f$ is the frequency of the letter, $t f^{\prime}$ is the new frequency of the letter, dtf is the distinct frequency of the letter, sstf is the sum of distinct frequencies of letters, and $s s t f$ is the sum of frequencies of all letters. In Arabic, the sdtf from Table 1 is 52.5 and $s s t f$ is 100.
The proposed Eq. (2) is a normalized and log-scaled frequency, obtained by normalizing tf using the ratio of global statistics, compressing the output using log-scaling, and finally offsetting the result to avoid zero or negative values.
This equation is meant to reward letters that appear more often than average (via putting them on the first part of the virtual keyboard) and penalize letters with overly rare appearances (via putting them on the second part of the virtual keyboard).
The Arabic language, the highest frequency value found in Table 1 is (12.835: ا), and the lowest frequency value is (ؤ :0.145); therefore, the average is (≈ 6.49). So, the frequency values whose value is 6 and above can be considered distinct. Thus, the distinct letters are six ((ا ,ل, ن, م, و, ي.
Finally, The Distinct Frequency-Alphabetical of Arabic letters are shown in Table 2.
Table 2. Arabic letter frequency distribution in decreasing order
|
Letter |
Frequency (tf’) |
Letter |
Frequency (tf’) |
|
ا |
2.39 |
إ |
1.38 |
|
ل |
2.35 |
ذ |
1.37 |
|
ن |
2.15 |
ج |
1.33 |
|
م |
2.14 |
ة |
1.31 |
|
و |
2.10 |
ى |
1.29 |
|
ي |
2.09 |
ص |
1.20 |
|
ه |
1.96 |
خ |
1.17 |
|
ب |
1.89 |
ش |
1.12 |
|
ر |
1.88 |
ث |
1.09 |
|
ع |
1.81 |
ز |
0.98 |
|
ت |
1.74 |
ض |
0.96 |
|
أ |
1.73 |
ط |
0.92 |
|
ف |
1.72 |
ء |
0.88 |
|
ك |
1.70 |
غ |
0.82 |
|
ق |
1.66 |
ئ |
0.78 |
|
د |
1.63 |
آ |
0.76 |
|
س |
1.61 |
ظ |
0.62 |
|
ح |
1.47 |
ؤ |
0.44 |
Table 2 shows the results of the Distinct Frequency-Alphabetical algorithm in Arabic, and the sequence of letters is as follows:
ا , ل , م , ن , و , ي , ه , ب , ر , ع , ت , ف , أ , د , س , ق , ك , ح , ذ , ج , ة , إ , ص , ى , خ , ش , ث , ز , ض , ط , غ , ء , ئ , آ , ظ , ؤ
The Distinct Frequency-Alphabetical of the proposed keyboard is shown in Figure 6.
Figure 6. Distinct frequency-alphabetical keyboard layout
|
Algorithm 1. Distinct frequency-alphabetical algorithm |
|
Input: Frequency of letters Output: New order of the letters on the keyboard Begin Step 1: Calculate the probability of frequency by using the equation (2). Step 2: Multiply the result of tf´ by 10, and take the integer part of the frequency value. Step 3: Sort the letters in descending order based on their frequency value, but if two letters have the same frequency, they are ordered based on the alphabetical sequence. Step 4: Partition the resultant letters from the sequence into two segments. The segment with the greatest numerical value is positioned in the first set of keys and the other in the second set. Step 5: The letters with the highest frequency are positioned in the bottom part of the keyboard, and the least valuable letters are positioned in the upper part of the keyboard*. End. |
In the proposed keyboard, the weights of the lower rows are higher than the upper rows, so that they can be accessed in a short time. Therefore, the letters with the highest frequency are placed in the lower rows and vice versa.
3.3 Principle of virtual keyboard working
The mechanism for moving and pressing a key is based on using the results of the Di-EyeNet algorithm [30] to detect the direction of the gaze-eye. The Di-EyeNet algorithm of the CNN model, that used as a secure CNN system [31, 32], is explained as follows:
|
Algorithm 2. Di-EyeNet |
|
Input: Eye image Output: Direction of eye gaze Begin Step 1: Resize the image to (64×64×3), and the feature map to (7×7×512). Step 2: Create a first block that has: a) Two convolutional layers that use 128 (5×5) filters with a stride of 2. // to collect spatial information while reducing the dimension of the output feature maps. b) Two Max-pooling layers that reduce the feature map size by (2×2) using ReLU activations. Step 3: Add a dropout layer to the network with 45% probability. // to reduce the spatial dimensions associated with the feature maps by half after block 1. Step 4: Create a second block which consists of: a) One convolutional layer b) One Max-pooling layer with (2×2) filters. Step 5: Add a dropout layer to the network with 8% probability. Step 6: The last block output is routed to a flattened layer, which is subsequently routed to a 128-D Fully Connected (FC) layer. Step 7: Add a dropout layer to the network with 8% probability. // to limit the number of trainable parameters while retaining great performance. Step 8: Connect the FC layer to a single SoftMax layer, which determines the identification of six eye directions. // directions are (left, right, top, down, straight, and closed-look) End. |
The virtual keyboard's working stages are illustrated in Figure 7.
The user's eye gaze is measured to determine the direction of key activation, while the eye blink detection results in the act of pushing the desired key and typing the letter.
The virtual keyboard operates on the idea of sequential illumination of keys based on gaze direction, in which each key lights up individually. The key was pushed if the requirement for detecting eye blinks aligned with the specified activation time for a certain key. It is crucial to immediately offer the user effective feedback on their command choices to prevent them from diverting their attention to the typing board (Eye-board) to check its content. The user receives audio feedback as an acoustic beep after successfully executing a command to go from one menu to another. Furthermore, when selecting a letter, the key is visually emphasized by changing its color, and the letter is audibly uttered. To achieve this, it changed the letter's background to a highlighted color while keeping the letter itself black. For example, if the letter 'ل' is illuminated at a certain moment and the eye blinks at that time, the letter ' ل' will be inputted, and a letter sound will be emitted to indicate that a letter has been entered. This visual and auditory stimulus prompts individuals to be proactive, enabling them to anticipate the subsequent character.
Figure 7. Flow chart of the visual keyboard working
3.4 Content keyboard
The proposed Arabic virtual keyboard consists of two menus, each consisting of the following:
The first menu has 20 keys representing the Arabic alphabet (ا - ف) and three fundamental keys (سطر, فراغ, حذف), in addition to one key for navigating to the following menu.
The second menu has 15 keys representing the Arabic alphabet (ق -ة ), as well as three fundamental buttons (سطر, فراغ, حذف). Additionally, it includes five keys for basic harakat and one key for navigating to the previous menu. The Arabic eye-keyboard shown in Figure 8.
(a) First menu
(b) Second menu
Figure 8. Arabic eye-keyboard
In this section, the results are presented and discussed according to different types of measures.
4.1 Evaluation measures
The evaluation of the proposed virtual keyboard’s design was made using both objective and subjective measurements, which account for the primary influence of the users’ performances in their interactions with novel applications.
4.1.1 Objective evaluation measures
(1) Characters Per Minute (CPM)
It is used as a main measure for the entry rate of the text. Eq. (3) [22, 33]
$\mathrm{CPM}=\frac{No.of \ character }{Time \ taken } \times 60$ (3)
(2) Word Per Minute (WPM)
It is used as a measure for the entry rate of the text, which is calculated using Eq. (4) as follows [22, 28]
$W P M=\frac{ No.of \ word }{ Time \ taken } \times 60$ (4)
(3) Character Error Rate (CER)
The calculation of the character mistake rate is as follows in Eq. (5):
$C E R=\frac{S+D+I}{N}=\frac{S+D+I}{S+D+C}$ (5)
where, S is for the count of substitutions, D is for the count of deletions, I is for the count of insertions, C is for the count of correct characters, and N is the total number of characters that are present in the reference, which is equal to the sum of S, D, and C.
(4) Word Error Rate (WER)
Eq. (6) calculates the word mistake rate as:
$W E R=\frac{S+D+I}{N}=\frac{S+D+I}{S+D+C}$ (6)
where, S is for the count of substitutions, D is for the count of deletions, I is for the count of insertions, C is for the count of correct words, and N is the total number of words that are present in the reference, which is equal to the sum of S, D, and C.
(5) Total Error Rate (TER)
It's calculated using the average of the character error rate and word mistake rate, as in Eq. (7).
$T E R=\left(\frac{C E R+W E R}{2}\right) * 100$ (7)
(6) Keystrokes Per Character (KSPC)
It refers to a calculation that determines the average number of keystrokes required to input each character of a given text [34], as expressed by Eq. (8)
$\mathrm{KSPC}=\frac{No.of\ keystrokes }{ No.of\ Character }$ (8)
4.1.2 Subjective evaluation measures
(1) System usability scale (SUS)
SUS is a ten-item Likert scale that comprehensively assesses subjective usability, learnability, and satisfaction with the system [35].
(2) NASA Task Load Index (NASA-TLX)
NASA-TLX is a widely used, objective, multidimensional instrument to gauge the subjective level of effort needed to judge a system operation's effectiveness and/or other features [36, 37].
4.2 Experimental setup
Several settings must be installed before starting to implement the experiments, which are as follows:
a) Apparatus and materials
The developed system was implemented on an MSI laptop of “15 in.." which had an Intel R Core TM i7-11800H 11th Gen CPU with a processing speed of 2.30 GHz. The laptop had a RAM of 16 GB and the resolution of the screen is 1920×1080 pixels. Besides, the laptop was attached to an NVIDIA GTX GEFORCE graphics card, and the Windows 11 operating system version was installed on it.
b) Participants
The proposed work included a wide range of ages, including youthful, middle-aged, and elderly participants, as shown in Table 3.
Table 3. Information of the participants
|
Gender |
Age (year) |
Total |
||
|
15-19 |
20-29 |
30-45 |
||
|
Female |
2 |
3 |
5 |
10 |
|
Male |
1 |
7 |
2 |
10 |
c) Sentences of experiments
All sentences used in the experiments on the Arabic keyboard are from famous Arabic phrases [38, 39].
4.3 Experiment analysis
Many experiments were conducted, and their results were compared at each stage of the proposed system. These experiments started on 1/4/2024 and end on 1/11/2024.
4.3.1 Comparison of keyboard layouts
Keyboard design has a significant impact on user performance. It can alleviate health issues such as eye fatigue, tension, and strain that might potentially impact user productivity. The primary constraint that the new layout could encounter is the potential resistance from users to embrace a different arrangement of keys, even though the existing layouts are not optimal.
Consequently, an assessment of the user-friendliness of the optimized layouts was carried out utilizing eye-tracking technology. Assessing the usability is crucial to guarantee the efficiency and user contentment of the interface design. The most significant criteria for assessing the system are accuracy and speed, but they are opposites, as increasing speed reduces accuracy. An emphasis was placed on studying the effect of reducing the estimated time it takes to move at the actual pace at which users type. This dissertation used four layouts (QWERTY, Alphabetical, Frequency, and Distinct Frequency-Alphabetical).
In Table 4, the results are the average speed for five experiments that were conducted on three sentences of different lengths.
It is clear from the results that the distinct alphabetical repetition method is the best, followed by the simple repetition method. The most frequently used letters on the first panel of the virtual keyboard system result in faster and more comfortable typing for the user. As for the alphabetical design and QWERTY layout, it was found that they showed a slight difference in typing speed. In the first experiments, the alphabetical layout was the best, as inexperienced typists relied on alphabetical advice because they did not know the QWERTY layout. However, tests after short training of the typists show that the alphabetical order loses its advantage, and users rely on their memory to determine the locations; simply memorizing the locations of the keys appears to be a better strategy than relying on alphabetical knowledge.
Moreover, there is a statistically significant difference in typing performance (WPM and CPM) across layouts for each word based on values of ANOVA p-value; also, Table 4 shows the Distinct Frequency-Alphabetical layout consistently outperforms others.
Table 4. Comparison of keyboard layouts
|
Sent. |
QWERTY |
Alphabetical |
Frequency |
Distinct Frequency-Alphabetical |
|
||||
|
WPM |
CPM |
WPM |
CPM |
WPM |
CPM |
WPM |
CPM |
p-value |
|
|
عين |
1.23 |
6.17 |
1.44 |
7.21 |
3.24 |
16.21 |
4.49 |
18.44 |
< 0.001 |
|
مرحبا |
1.52 |
7.57 |
1.82 |
9.09 |
2.23 |
11.12 |
2.85 |
14.64 |
0.002 |
|
لوحة مفاتيح |
0.007 |
0.07 |
0.007 |
0.08 |
0.008 |
0.086 |
0.63 |
0.09 |
< 0.001 |
4.3.2 Comparison of the typing experiments
The word "مرحبا" was tried printing several times, as seen in Table 5. It has been shown that the results of the measures improve with each new experience. This is due to the user memorizing the letter's location and knowing how to use eye movements better to reach the letter to be printed.
The experiments were conducted for four consecutive weeks.
Table 5. Comparison of the typing experiments
|
Trials |
WPM |
CPM |
TER |
|
1 |
1.39 |
6.96 |
83 |
|
2 |
1.96 |
9.81 |
60 |
|
3 |
2.26 |
11.3 |
20 |
|
4 |
3.01 |
15.06 |
0 |
Table 5 demonstrates a positive correlation between the number of trials and the rise in WPM and CPM. Due to the individual's lack of familiarity with the virtual keyboard, the early attempts resulted was low WPM and CPM rates but a large TER score. Nonetheless, as the individual becomes used to the keyboard, their WPM and CPM rates rise with down TER progressively.
4.3.3 Comparison of eye typing by lengths of the word
Different word lengths were used to determine the effect of increasing the number of letters in a word on both measures (WPM and CPM), as shown in Table 6.
Table 6. Comparison of eye typing by lengths of the word
|
Words |
Length of word |
WPM |
CPM |
Time |
|
عين |
3 |
3.57 |
10.71 |
0.28 |
|
مرحبا |
5 |
2.85 |
14.64 |
0.35 |
|
مفاتيح |
6 |
2.17 |
13.04 |
0.46 |
It is noted that time is directly affected by the length of the word, as it increases with the increase in the number of letters in the word to be typed. In contrast, the value of the WPM and CPM metrics is inversely proportional to the length of the word, as shown in Figure 9.
Figure 9. Comparison of result eye-typing by length of the word
4.3.4 Comparison of eye-typing by lengths of the Arabic sentences
Comparisons were made to typing on the Arabic keyboard using five sentences, and Table 7 shows the results.
Table 7. Comparison of eye-typing by length the Arabic sentences
|
No. |
Sentences |
No. Word |
No. Char. |
WPM |
CPM |
Time |
|
1 |
مرحبا |
1 |
5 |
1.53 |
7.64 |
0.69 |
|
2 |
البحر ازرق |
2 |
10 |
2.11 |
10.54 |
0.95 |
|
3 |
الله هو الأمان |
3 |
14 |
1.83 |
8.54 |
1.64 |
|
4 |
العلم نور والجهل ظلام |
4 |
21 |
0.73 |
3.83 |
5.48 |
|
5 |
كل ما هو جديد جميل |
5 |
18 |
1.32 |
5.63 |
3.02 |
It is clear from Table 7 that the value of the two measures (WPM and CPM) decreases with the increase in the number of words of approximately equal length because the user's ability to write decreases rapidly over time. On the other hand, if the length of the words is unequal in the sentences, the sentence with the highest letters is the one with the highest scale values, as in sentences 4 and 5.
4.3.5 Comparison of typing with harakat
Table 8. Comparison of typing with/without harakat
|
Sentence |
Without Harakat |
With Harakat |
||||
|
WPM |
CPM |
Time |
WPM |
CPM |
Time |
|
|
قصّةُ القمرْ |
0.76 |
4.56 |
2.63 |
0.4 |
2.57 |
5.06 |
|
حُفِظَ الدرسَ |
0.67 |
4.36 |
2.98 |
0.33 |
2.0 |
6.0 |
Table 8 shows that the speed of typing with harakat is low because the movements are located in the second part of the virtual keyboard, and this needs to be moved every time a movement is to be typed.
4.3.6 Comparison of the keystrokes per character
KSPC is a metric that quantifies the level of precision by accounting for the additional effort required to rectify errors. Optimally, the Keyboard Stroke Per Character (KSPC) value should be 1.00, signifying that each keystroke produces a single character. When participants rectify errors when entering data, the error rate becomes 0%, but the KSPC value exceeds 1. Table 9 shows the effect of the KSPC on the time taken to type the word" مرحبا ".
Table 9. Comparison of the keystrokes per character
|
WPM |
CPM |
KSPC |
Time |
ACC.% |
Error Rate% |
|
0.15 |
0.75 |
5.6 |
6.71 |
100 |
0 |
|
0.53 |
2.64 |
2.6 |
1.9 |
100 |
0 |
|
0.66 |
3.31 |
2.2 |
1.51 |
100 |
0 |
|
0.72 |
3.59 |
1.4 |
1.39 |
100 |
0 |
|
0.91 |
4.57 |
1 |
1.1 |
100 |
0 |
Table 9 demonstrates an inverse correlation between the number of trials and the number of keystrokes required to input each character (KSPC). As the user becomes used to the suggested virtual keyboard, the KSPC decreases by decreasing the rate of mistakes and correcting typing words. Also, it is noted that as the value of the KSPC scale increases, time increases directly with it. On the other hand, the accuracy measure and error rate remain constant and do not change. Therefore, the KSPC measure can be considered a more accurate measure.
4.3.7 Comparison of keyboard characteristics
The comfort of the user in terms of flexibility and ease of use is the new influential factor in measuring the importance of the proposed system. The characteristics of the virtual keyboard, including shape, color, size, and number of keys, and even an alteration in the arrangement of keys, can substantially boost typing efficiency. Therefore, the efficiency of the virtual keyboard typing system can be measured through two metrics:
a) System Usability Scale
The proposed system achieved 87.5 % on the system usability scale, as seen in Figure 10. These results surpass the results of previous systems, as shown in Table 10. Such an achievement can be attributed to using the proposed layout (Distinct Frequency-Alphabetical) in the keypad design, which makes learning easier.
Figure 10. Average scores of the SUS scale
Figure 10 shows the changes in the SUS scale of 87.5%. It shows that the participants liked using the system very much and said that the system is simple and easy to use, causing little complexity. The system works smoothly and is very well integrated. The system has no violations and does not require technical support, but it consumes some time. In contrast, the participants think they can learn the system quickly. So, the user feels confident when using it because he does not need additional things to be able to use it.
Table 10. SUS comparison of the proposed system and the earlier systems
|
Ref. |
SUS of Ref. |
|
[40] |
87 |
|
[34] |
75.4 |
|
Proposed |
87.5 |
It can be noted from the table above that the proposed system does not carry any psychological or physical burden because it does not use any devices that are installed on the user’s body. Accordingly, the user feels comfortable when using this system, unlike previous systems.
b) NASA Task Load Index
The experiments were conducted for eight consecutive weeks (two trials per week). The NASA-TLX index was calculated statistically for eight weeks, and the proposed system achieved an average score of 8 %, which means it only puts a small load on the user. These results surpass the results of previous systems, as shown in Table 11, and Figure 11 shows the average scores of the NASA-TLX.
Figure 11 shows the changes in the NASA-TLX scale. Mental demand decreases over time, while physical demand is small from the beginning and decreases more over time. Also, the initial effort is relatively low and decreases with time. In contrast, the time demand decreases significantly over time and is inversely proportional to performance, which increases considerably over time. As for frustration, its value ranges between up and down, but in the end, it decreases significantly.
Table 11. NASA-TLX comparison of the proposed system and the earlier systems
|
Ref. |
NASA-TLX of Ref. |
|
[40] |
18 |
|
Proposed |
8 |
Figure 11. Average scores of the NASA-TLX
4.4 Limitations for the study
There are several limitations to this study, and although the proposed method addresses most of them, they still affect the accuracy and improvement of the results.
(1) Participant demographics bias
The age distribution or digital literacy of participants might be skewed (e.g., mostly young or tech-savvy users), which limits generalizability to a broader population (e.g., elderly, low-literacy users). So a virtual keyboard was designed that can be typed on simply, so that everyone can use it.
(2) Webcam resolution
Low-resolution webcams can negatively affect the accuracy of gaze tracking and input precision. This introduces variability in performance that may not reflect the layout’s true efficiency. Therefore, it is preferable to use high-resolution webcams.
(3) Environmental lighting conditions
Eye-tracking accuracy is highly sensitive to ambient lighting. Fluctuations in brightness could interfere with gaze direction detection, impacting user performance during testing. Although the proposed keyboard can work under varying lighting conditions, it works best in brightly lit areas.
(4) Hardware constraints
The system was tested using general-purpose webcams, which may not represent optimal hardware conditions. Performance could differ significantly with specialized eye-tracking devices. This will certainly be better, but this will lead to high costs that users may not be able to afford.
(5) Fixed keyboard size
The study points out that the key sizes in the virtual Keyboard are fixed, which may not be visible when used on small tablets and mobile phones.
In this study, we have introduced the Distinct Frequency-Alphabetical layout, which is the best among the layouts (QWERTY, Alphabetical, and Frequency) that made the proposed virtual keyboard have good speed despite only using a webcam. The short typing time is the main factor that affects the increase in typing speed measurements. But in reality, the location of the letters on the keyboard, as well as the length of the word, the number of words in the sentence, and the presence of the harakat affect the amount of typing time. In addition, the shape and number of keys, in addition to the color and size of the keyboard, affect the subjective measurements, as the appropriate size and color that is comfortable for the eye help increase the user's comfort.
The KSPC scale can be considered the most important scale and an alternative to the accuracy scale to measure accuracy concerning the time taken to type, as it calculates the number of modifications made by the person to write the word correctly, and the TER scale can be considered the most important scale and an alternative to the error scale to measure the amount of error according to the type of error (deletion, replacement, extra). Hence, the proposed keyboard system was obtained at TER=0 and KSPC=1. The keyboard virtual system is necessary and important for entering data and interacting with the computer, so this system can be developed with some suggestions that can be added in the future by developing the proposed system by adding the most frequent words of the user in their typing to the system dictionary. In addition to proposing using a Raspberry Pi to store the proposed program, he is then connecting it to a display screen, making it possible to use the proposed keyboard without a laptop.
[1] Prabhu, V., Prasad, G. (2011). Designing a virtual keyboard with multi-modal access for people with disabilities. In 2011 World Congress on Information and Communication Technologies (WICT), pp. 1133-1138. https://doi.org/10.1109/WICT.2011.6141407
[2] Tamilselvi, R., Sarguna, R. (2017). Eye tracking based virtual keyboard for disabled people. International Journal of Advanced Research Trends in Engineering and Technology, 4(11): 331-334.
[3] Chakraborty, P., Roy, D., Rahman, M.Z., Rahman, S. (2019). Eye gaze controlled virtual keyboard. International Journal of Recent Technology and Engineering, 8(4): 3264-3269. https://doi.org/10.35940/ijrte.d8049.118419
[4] Gomide, R.D.S., Loja, L.F.B., Lemos, R.P., Flôres, E.L., et al. (2016). A new concept of assistive virtual keyboards based on a systematic review of text entry optimization techniques. Research on Biomedical Engineering, 32(2): 176-198.
[5] Dahmani, M., Chowdhury, M.E., Khandakar, A., Rahman, T., et al. (2020). An intelligent and low-cost eye-tracking system for motorized wheelchair control. Sensors, 20(14): 3936. https://doi.org/10.3390/s20143936
[6] Kumar, D., Sharma, A. (2016). Electrooculogram-based virtual reality game control using blink detection and gaze calibration. In 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Jaipur, India, pp. 2358-2362. https://doi.org/10.1109/ICACCI.2016.7732407
[7] Al-Wabil, A., Al-Issa, A., Hazzaa, I., Al-Humaimeedi, M., Al-Tamimi, L., Al-Kadhi, B. (2012). Optimizing gaze typing for people with severe motor disabilities: The iWriter Arabic interface. In Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility, Boulder, Colorado, USA, pp. 261-262. https://doi.org/10.1145/2384916.2384983
[8] Boudelaa, S., Perea, M., Carreiras, M. (2020). Matrices of the frequency and similarity of Arabic letters and allographs. Behavior Research Methods, 52(5): 1893-1905. https://doi.org/10.3758/s13428-020-01353-z
[9] Aknuranda, I., Syawli, A., Setiawan, B.D. (2020). Comparative evaluation of usability between QWERTY-based Arabic and non-QWERTY-based Arabic keyboard layout: Empirical evidence. Journal of Information Technology and Computer Science, 5(2): 177-193. https://doi.org/10.25126/jitecs.202052199
[10] Lystbæk, M.N., Pfeuffer, K., Grønbæk, J.E.S., Gellersen, H. (2022). Exploring gaze for assisting freehand selection-based text entry in ar. Proceedings of the ACM on Human-Computer Interaction, 6(ETRA): 1-16. https://doi.org/10.1145/3530882
[11] Thurner, S., Hanel, R., Liu, B., Corominas-Murtra, B. (2015). Understanding Zipf's law of word frequencies through sample-space collapse in sentence formation. Journal of the Royal Society Interface, 12(108): 20150330. https://doi.org/10.1098/rsif.2015.0330
[12] Gobena, D.L., Seifu, A. (2018). A comparison of QWERTY and alphabetical layout on small handheld devices. In International Conference of Design, User Experience, and Usability, Las Vegas, NV, USA, pp. 614-622. https://doi.org/10.1007/978-3-319-91803-7_46
[13] Buzing, P. (2003). Comparing different keyboard layouts: Aspects of qwerty, dvorak and alphabetical keyboards. Delft University of Technology Articles, 901-910.
[14] Rempel, D. (2008). The split keyboard: An ergonomics success story. Human Factors, 50(3): 385-392. https://doi.org/10.1518/001872008X312215
[15] Agyepong, E., Buchanan, W.J., Jones, K. (2018). Detection of algorithmically generated malicious domain. In 6th International Conference of Advanced Computer Science & Information Technology, pp. 13-32. https://doi.org/10.5121/csit.2018.80802
[16] Islam, R., Rahman, S., Sarkar, A. (2021). Computer vision based eye gaze controlled virtual keyboard for people with quadriplegia. In 2021 International Conference on Automation, Control and Mechatronics for Industry 4.0 (ACMI), pp. 1-6. https://doi.org/10.1109/ACMI53878.2021.9528213
[17] Benabid Najjar, A. (2013). Toward an optimized arabic keyboard design for single-pointer applications. In Proceedings of the 15th Annual Conference Companion on Genetic and Evolutionary Computation, San Antonio, Costa Rica, pp. 1717-1718. https://doi.org/10.1145/2464576.2480784
[18] Hosny, M.I., Alswaidan, N., Najjar, A.B. (2014). An optimized single-finger Arabic keyboard layout. In 2014 Science and Information Conference, pp. 321-328. https://doi.org/10.1109/SAI.2014.6918206
[19] Alsabah, S.S.S., Alrifai, J., Aly, M. (2016). U.S. Patent No. 9,298,277. Washington, DC: U.S. Patent and Trademark Office. https://patents.google.com/patent/US9298277B1/en.
[20] Benabid Najjar, A., Al-Wabil, A., Hosny, M., AlRashed, W., Alrubaian, A. (2021). Usability evaluation of optimized single-pointer arabic keyboards using eye tracking. Advances in Human‐Computer Interaction, 2021(1): 6657155. https://doi.org/10.1155/2021/6657155
[21] Qtaish, A., Alshudukhi, J., Al-shaibani, B.A.M., et al. (2021). An improved Arabic keyboard layout. Science International, 31(1): 5-15.
[22] Samanta, D., Sarcar, S., Ghosh, S. (2013). An approach to design virtual keyboards for text composition in Indian languages. International Journal of Human–Computer Interaction, 29(8): 516–540. https://doi.org/10.1080/10447318.2012.72848
[23] Gizatdinova, Y., Špakov, O., Tuisku, O., Turk, M., Surakka, V. (2023). Vision‐based interfaces for character‐based text entry: Comparison of errors and error correction properties of eye typing and head typing. Advances in Human‐Computer Interaction, 2023(1): 8855764. https://doi.org/10.1155/2023/8855764
[24] Emile Tatinyuy, V., Noumsi Woguia, A.V., Ngono, J.M., Fono, L.A. (2024). Multi-stage gaze-controlled virtual keyboard using eye tracking. PLoS One, 19(10): e0309832. https://doi.org/10.1371/journal.pone.0309832
[25] Zhai, F., Fu, D., Du, S., Ge, K., et al. (2002). What is China doing in policy-making to push back the negative aspects of the nutrition transition? Public Health Nutrition, 5(1a): 269-273. https://doi.org/10.1079/PHN2001303
[26] MacKenzie, I.S., Zhang, S.X. (1999). The design and evaluation of a high-performance soft keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Pittsburgh Pennsylvania, USA, pp. 25-31. https://doi.org/10.1145/302979.302983
[27] Parhi, P., Karlson, A.K., Bederson, B.B. (2006). Target size study for one-handed thumb use on small touchscreen devices. In Proceedings of the 8th Conference on Human-computer Interaction with Mobile Devices and Services, Helsinki, Finland, pp. 203-210. https://doi.org/10.1145/1152215.1152260
[28] Tognazzini, B. (2003). First principles of interaction design (Revised & expanded). https://www.asktog.com/basics/firstPrinciples.html.
[29] Madi, M. (2010). A study of Arabic letter frequency analysis. http://www.intellaren.com/articles/en/a-study-of-arabic-letter-frequency-analysis.
[30] Khaleel, A.H., Abbas, T.H., Ibrahim, A.W.S. (2024). A novel convolutional feature-based method for predicting limited mobility eye gaze direction. International Journal of Advances in Intelligent Informatics, 10(2): 220-238. https://doi.org/10.26555/ijain.v10i2.1370
[31] Abdullah, R.M., Alazawi, S.A.H., Ehkan, P. (2023). SAS-HRM: Secure authentication system for human resource management. Al-Mustansiriyah Journal of Science, 34(3): 64-71. https://doi.org/10.1007/978-3-030-62696-9_4
[32] Al-Tai, M.H., Nema, B.M., Al-Sherbaz, A. (2023). Deep learning for fake news detection: Literature review. Al-Mustansiriyah Journal of Science, 34(2): 70-81. https://doi.org/10.1007/978-3-030-62696-9_4
[33] Roux, T.B., Rouvier, M., Wottawa, J., Dufour, R. (2022). Qualitative evaluation of language model rescoring in automatic speech recognition. In Interspeech, Incheon, South Korea, pp. 3968-3972. https://doi.org/10.21437/Interspeech.2022-10931
[34] De Rosa, M., Fuccella, V., Costagliola, G., Adinolfi, G., et al. (2020). T18: An ambiguous keyboard layout for smartwatches. In 2020 IEEE International Conference on Human-Machine Systems (ICHMS), Rome, Italy, pp. 1-4. https://doi.org/10.1109/ICHMS49158.2020.9209483
[35] Khaleel, A.H., Abbas, T.H., Abdul-Wahab, S.I. (2023). Enhancing human-computer interaction: A comprehensive analysis of assistive virtual keyboard technologies. Ingenierie des Systemes d'Information, 28(6): 1709-1717. https://doi.org/10.18280/isi.280630
[36] Whittington, P., Dogan, H. (2016). Smartpowerchair: Characterization and usability of a pervasive system of systems. IEEE Transactions on Human-Machine Systems, 47(4): 500-510. https://doi.org/10.1109/THMS.2016.2616288
[37] Rajanna, V., Hansen, J.P. (2018). Gaze typing in virtual reality: Impact of keyboard design, selection method, and motion. In Proceedings of the 2018 ACM symposium on eye tracking research & applications, Warsaw, Poland, pp. 1-10. https://doi.org/10.1145/3204493.3204541
[38] Al-Attas, S.M.N. (1991). The concept of education in Islām: A framework for an Islamic philosophy of education. International Institute of Islamic Thought and Civilization.
[39] Holes, C. (2004). Modern Arabic: Structures, Functions, And Varieties. Georgetown University Press.
[40] Meena, Y.K., Cecotti, H., Wong-Lin, K.F., Prasad, G. (2019). Design and evaluation of a time adaptive multimodal virtual keyboard. Journal of Multimodal User Interfaces, 13(4): 343-361. https://doi.org/10.1007/s12193-019-00293-z
