On a Moving Target Selection Model in Virtual Reality Based on Decision Trees

User goal selection models are dominated by Fitts's Law, a behavioural model of human movement and in particular user goal selection, which has been widely used in human-computer engineering and HCI. In 1992, Dr Fitts proposed the original version of Fitts's Law inspired by Shannon's Theorem 17 in informatics [1]. Mckenzie and Buxton [2] proposed that Fitts's Law can be used for prediction and measurement. Table 1 shows the most used versions of Fitts's law and its variants until 2003. Goldberg et al. [3] compared the performance of models 1, 3 and 4 in Table 1. The results showed that the square-root model proposed by Meyer et al. [4] was more suitable for fixed W and A targets except the most difficult targets (with large A/W). In addition, several studies have extended Fitts's Law. Zhang et al. [5] investigated a control theory that could be an alternative to Fitts's Law based on an improved crossover model. Tang et al. [6] proposed that the performance of the previous trial had a predictable effect on the next trial when Fitts's Law was used in measuring. Bertucco et al. [7] investigated quick pointing action. At the level of motor planning, they tested the relationship between movement time (MT) and index of difficulty (ID). The results showed that due to different moving parts, changes in movement distance may lead to changes in the relation between MT and ID. Bernhaupt et al. [8] argued that in terms of MT, the shortest MT should be considered in addition to the average time. Glazebrook et al. [9] showed that thanks to a more efficient online control process, MTs to the last target location in an array are shorter than predicted by Fitts's Law. Petrič et al. [10] showed that Fitts's Law may not be applicable to cyclic impact tasks.

Table 1. The pre-2003 Fitts models

Table 2. Fitts's Law extended models

Table 2 shows the current extended models of Fitts's Law. Cha and Myung [11] proposed a variant of Fitts's Law equation (model 1 in Table 2) that can be applied in a 3D environment. Based on empirical data, the model fits better than previous models for 3D tasks. Murata and Fukunaga [12] proposed model 2 in Table 2, focusing on a 3D eye-gaze input system, and the results showed that the model fits the experimental data well and predicts pointing times for various HCI tasks more effectively. The above two models demonstrate the feasibility of introducing Fitts's Law into a 3D environment.

Dynamic target selection is a hot topic in the field of HCI research, and the extension to virtual reality scenarios is also important. However, most of the current researches focus on static targets selection [13-16]. In recent years, for the problem of moving target selection, some scholars have enhanced pointing by extending the alternative targets or setting up more easily achievable agents for each target [17].

An improved moving target prediction model is proposed based on the use of target size and distance to predict targets, considering the problem of head pose.

Predictions of expected targets were introduced by computing relevant features for head and hand targets within a time window, which can be generalised to different dynamic targets selection tasks, using the Ortega scoring function [18] as a benchmark to evaluate the accuracy of feature prediction.

(1) Apparatus and participants

The apparatus and participants are the same as in the previous section.

(2) Design

In each sub-experiment, the experimental sphere appeared 5m in front of the participant, 0.3m below the initial head pose of the participant.

Sub-experiment 1

In sub-experiment 1, there was only one experimental sphere in the experimental scenario and the experimental variables were the radius of the sphere and the sphere position (left: x-axis -0.5, middle: origin position, right: x-axis 0.5). Five trials were conducted for each of the six scenarios (30 trials in total). The aim of sub-experiment 1 was to familiarise participants with the experimental and to acquire the dynamic target task.

Sub-experiment 2

In sub-experiment 2, there are two spheres in each trial (sph₁, sph₂), the experimental variable is the radius of the spheres, and the relative positions of the spheres shown in Figure 2. In each trial, the two spheres move simultaneously at a constant speed of 1.5m/s for 5 trials for each of the 12 scenarios (60 trials in total).

(3) Procedure

During the experiment, each participant was asked to stand in a designated circle in the experimental scene, facing the experimental wall, and complete relevant target selection tasks. In the course of the experiment, a horizontal set of spheres appeared in front of the participant and began to move in the positive direction of their z-axis. Each sphere has the same texture and colour and was scaled according to the sphere's radius.

The participant had to reach out and select the target, but not to step out of the designated circle. Each target automatically disappeared when hit by the participant or when it got 0.5m past the participant's head on the z-axis. Once all the spheres disappeared, the trial was over. Auditory and tactile feedback is used to assist the participant in the selection process. Only when a sphere was hit, a sound would be played in the experimental scene and the handle would twitch slightly.

During each experiment, the task completion time, the head pose, the handle pose, the target sphere position and the collisions that occurred between the handle and the sphere are recorded in an application log.

(4) Analysis of experimental data

Based on each completed experiment, an inter-participant analysis was performed. Trials where the participant didn’t hit a sphere were discarded. We calculated the relationship between the position of the sphere and the participant's head pose and handle pose. We also calculated the average of these measurements over a time window to assess the target sphere predicted by the different feature sets.

a. Positioning relative to the participant's head

The head vector $(\vec{T})$ is calculated from the participant's head orientation $\left(Q_h\right)$. Considering that the corresponding head vector is $(0,0,-1)$ if the participant has not turned his/her head, the unit vector $-\widehat{m}$ needs to be rotated in the direction of the head to calculate $\vec{T}$.

$\vec{T}=\operatorname{rot}\left(-\hat{m}, Q_h\right)$      (15)

Then calculate the position of the sphere in the head pose (P_h, Q_h)

$P_{i h}=P_i-P_h$      (16)

The dot product between the normalized vectors is calculated as follows.

$d o t_i=\hat{T} \times \hat{P}_{i h}$      (17)

The dot product is a normalized scalar that is easy to interpret, and the closer it is to 1, the more aligned the user's head orientation is with the sphere. The dot product difference is calculated as follows.

$\Delta d o t=d o t_1-d o t_2$      (18)

This quantity is used to evaluate the position of the user's head relative to the two spheres. The closer its value is to 1, the more aligned the user's head is with sph₁. The closer its value is to -1, the closer the user's head is to sph₂. 0 indicates that the user's head is in the middle of the two spheres. Finally, we calculated the handle-sphere distance D_i and the distance difference ΔD.

$D_i=\left|P_w-P_i\right|$      (19)

$\Delta D=D_1-D_2$      (20)

Similar to the dot product difference, the distance difference is used to assess the position of the user's handle with respect to the sphere: Greater than 0, the larger it is, the farther the handle is from sph1; Less than 0, the smaller it is, the farther the handle is from sph2; A value equal to 0 means that the handle is equidistant from both spheres.

b. Distance score

To verify the validity of the proposed features, their prediction accuracy was compared with the distance scoring function proposed by Ortega et al. [18] According to his method, all the targets are in an ordered list of increasing distance values, with the order denoted by j. The score of each N closest targets are increased as follows.

$d S_j(\mathrm{t})=d S_j(\mathrm{t}-1)+(\mathrm{N}-\mathrm{j}) \Delta \mathrm{t} ; \quad(\mathrm{j}<\mathrm{N})$      (21)

All the other targets will have their scores decreasing.

$d S_j(\mathrm{t})=d S_j(\mathrm{t}-1)-(0.9 \mathrm{~N}) \Delta \mathrm{t} ; \quad(\mathrm{j} \geq \mathrm{N})$      (22)

$d S_j(\mathrm{t}) \geq 0$      (23)

Since there are two spheres in the experimental environment, hence N=1. This only requires increasing the fraction closest to the target. The decay rate in equation 22(0.9N) is much higher than in the Ortega formula (0.5N). This is because the selection task requires a large amount of time to wait for the target to appear, so the great majority selection behaviour occurs at the end of the experiment.

c. Choice of time window

Since the data are not completely consistent for each movement, Δdot and ΔD are averaged over a time window rather than using discrete values. As the analysis is performed post-hoc, ideally the time window should start before the start of the arrival movement, with the user then specifying their intention and movement, and then end before reaching the target. Within the scope of this study, a graphical analysis of the point graph was carried out as time passed to heuristically determine the appropriate window.

The value of Δdot gradually approaches 0 from the initial non-zero, between 0 and 1, indicating that they were choosing between two spheres; after 1 second Δdot started to diverge here, indicating that the participant's head was facing one of the spheres. The user's choices between two spheres clustered above and below 0. Based on this, 1.5s was chosen as the end time of the window. These times corresponded to 42.5% and 63.8% of the 5th percentile of selection times (2.35s).

Within this time window, the average dot product difference $\overline{\Delta d o t}$ and the average handle-sphere distance $\overline{\Delta D}$ were calculated. The scoring function was also run through the time window. Depending on the final score, each target i is marked as 0 or 1.

(5) Results

Single and combined feature sets were estimated to predict the probability that the participant would choose the first sphere. The J48 classifier was used to evaluate the feature set because it can select the simplest decision tree based on the input attributes to generate easily interpretable rules. The performance of this research model can be simply assessed by calculating the prediction accuracy. Table 4 shows that both single and combined feature sets outperformed both chance and frequency predictions. The 95% confidence intervals for the accuracy of the two classifiers were:

$\left(\right.pre_1-\left.pre_2\right)\pm z_{95}$$\sqrt{\begin{aligned} & \frac{\operatorname{pre}_1\left(1-p r e_1\right)}{n u m E x p} \\ & +\frac{\operatorname{pre}_2\left(1-p r e_2\right)}{n u m E x p}\end{aligned}}$     (24)

Although the model generated using the feature $\overline{\Delta d o t}$ didn’t significantly outperform the model generated using $\overline{\Delta D}$, the latter was not practical as it generated a more complex tree with lower average accuracy and over-fitted data. In contrast, the prediction accuracy of the feature set combined with $\overline{\Delta d o t}$ and $\overline{\Delta D}$ was higher than that of features $\overline{\Delta D}$ alone, and the resulting tree is shown in Figure 4.

figure_4.png

Figure 4. Decision tree generated by $\{\overline{\Delta d o t}, \overline{\Delta D}\}$ feature set

Table 4. Number of nodes, layers, precision and 95% confidence intervals for the trees formed by each feature set

Table 5. Number of nodes, layers, precision for evaluating trees based on the target feature set

The time window limit may change depending on the task, but the expected target is fixed, which is much better than considering the entire target's motion trajectory.

Finally, using a single head-target relative parameter, such as $\overline{\Delta d o t}$, and a single handle-target relative parameter, such as $\overline{\Delta D}$, may be of no or little use in tasks where the number of targets is large and their locations vary.

Tables 5 shows that the feature set of combination $\overline{\Delta d o t}$, $\overline{\Delta D}$, and r1, r2 obtained the best prediction result, exceeding the average precision of the reference standard {r₁, r₂} by almost 7%, but the resulting tree is too large for this study. Table 5 also shows that feature set $\left\{\overline{\Delta D}, r_1, r_2\right\}$ produces higher accuracy than $\left\{\overline{\Delta d o t}, r_1, r_2\right\}$ and {r₁, r₂}. The high accuracy of the feature set of combination $\overline{\Delta D}$ and r1, r2 indicates that in the simple task of dynamic target prediction, the function of the size and distance of the sphere target can predict the chosen sphere well.

By integrating $\overline{\Delta D}$, it shows a significant increase in information gain, reflecting the correlation between handle and target poses. However, this only occurs after a certain task preparation time.

(6) Experimental findings

Experimental results show that integrating head-target feature set $\overline{\Delta d o t}$ and handled-target feature set $\overline{\Delta D}$ can effectively predict user intention in dynamic target selection task. These features were calculated in about 67% of the selection time, which was better than the reference standard prediction. In addition, the combined features of $\overline{\Delta d o t}$ and $\overline{\Delta D}$ obtained higher accuracy than the separate features $\overline{\Delta D}$.