The Measurement of Evacuation Effectiveness Regarding Dynamic Evacuation Routing System (DERS) in High-Rise Building Using Virtual Reality Simulation

The Measurement of Evacuation Effectiveness Regarding Dynamic Evacuation Routing System (DERS) in High-Rise Building Using Virtual Reality Simulation

Adithya SudiarnoDito Abrar Amanullah Reza Aulia Akbar

Industrial System and Engineering Department, Institut Teknologi Sepuluh Nopember, Surabaya 60111, Indonesia

Corresponding Author Email: 
adithya.sudiarno@gmail.com
Page: 
115-122
|
DOI: 
https://doi.org/10.18280/ijsse.120114
Received: 
14 January 2022
|
Revised: 
3 February 2022
|
Accepted: 
10 February 2022
|
Available online: 
28 February 2022
| Citation

© 2022 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).

OPEN ACCESS

Abstract: 

Nowadays, the existing evacuation method for emergencies in a building still relies on exit signs and evacuation maps, which usually take longer for the victim to read thoroughly in the case of panic. Therefore, Dynamic Evacuation Routing System (DERS) is developed as a guiding evacuation system that comes in lines, and the lights are programmable to show the safest and shortest path to escape. We developed DERS with Virtual Reality (VR) as a training system tool. Three variables will be tested, namely DERS implementation, type of building, and the starting floor level. This quantification combined three methods: Presence Questionnaire (PQ), System Usability Scale (SUS), and usability matrix. It was suggested that DERS implementation was able to improve all results collectively with different building and starting floor levels as other significant factors affecting the results and the quality of the simulation were good for immersion level (3.90 out of 5.00) and decent for usability level (71.72 out of 100). Therefore, DERS implementation could improve evacuation effectiveness in the case of an emergency, and VR simulation has been successfully utilized as a medium for evaluating new evacuation systems using DERS.

Keywords: 

emergency, evacuation, immersion, usability, virtual reality

1. Introduction

The problem of building fire disasters can be caused by various factors, such as human error, short circuit, and gas leakage. According to Brushlinsky et al. [1], there are 86.4 million fire incidents have caused 1 million fire deaths in the past to decades. Fire hazard in building is defined as the potential accident to property and threaten human life in a building [2]. Fires can occur in developed and developing countries with varying degrees of severity. Developing countries such as Pakistan and India suffered the highest number of fires with 100,000-600,000 special per year and 10,000-25,000 of the number fire casualties [3]. Since fire accidents frequently occur in buildings, the focus of the research should be directed to the fire accidents that happen in buildings.

In the state of panic in a disaster circumstance, the knowledge of escape route and evacuation procedures are really important to optimize the survival chance of the disaster victims. The type of knowledge that is most preferable in the case of a disaster is tacit knowledge. Tacit knowledge is acquired from personal experience and developed into a habit [4]. This knowledge is not given through a textual matter but through directly experiencing the environment. However, in the case of evacuation, sometimes safety induction is not effective enough to give tacit knowledge to the potential victim. Moreover, the existing evacuation method in the building only relies on exit signs and evacuation maps which usually takes time for the victim to understand or read thoroughly in the case of panic. Therefore, a more effective way to evacuate is very demanded in this condition, especially in a high-rise or highly complex building.

According to one of the producers of this technology, INOTEC, Dynamic Evacuation Routing System (DERS) is an addition to conventional emergency and safety lighting connected to an automatically triggered power supply. In the event of a power cut, the latter ensures that certain areas are still lit and that emergency exit luminaries and direction-giving signs are front or backlit as appropriate [5]. Ideally, DERS is used to automatically light up the safest and shortest escape route in a smoke-induced room or a room with low luminaries. One of the common examples of DERS is implemented in commercial airlines. However, using DERS to give environment-based information to the victim is a relatively new concept and technology, especially in Indonesia, and the acceptance from society is still unknown.

Since it is a new technology, the effectiveness of the implementation is still questionable. Moreover, Indonesian people sometimes have difficulty adapting to new technology quickly. Therefore, simulation is needed to understand how Indonesian people implement DERS, especially high-rise buildings. This simulation can be made through a virtual environment or a Virtual Reality (VR) that represents the real condition of the simulated building. VR is the technology to create a simulated environment that allows users to interact with 3D worlds by simulating as many senses as possible, such as vision, hearing, touch, even smell but limited to the availability of content and computing power [6]. The virtual environment can be created by using VR software such as Unity, and the software can be run through a Head-Mounted Device (HMD). In terms of benefits, VR can be used to give people tacit knowledge about the evacuation process in a fire disaster. Furthermore, the VR simulation is used to give training to children in the case of danger. The previous research divides the simulation into three main components: behavioral test, in-situ training or training inside VR, and in-situ assessment or improvement within VR. This method can give a good result in terms of the method's usability [7]. However, other research mainly discusses the same thing, such as developing a simulation or a behavioral measurement [8, 9]. Moreover, the improvements are mainly used for achieving better simulation. However, VR simulation for prototyping is still rare, and this research will try the possibility of VR-based product prototyping.

Based on the various cases of disaster explained before, this research will measure the compatibility of DERS implementation in Institut Teknologi Sepuluh Nopember (ITS) building, especially in Research Center and Rectorate Building. The places are picked because of the importance of the building. Practically, Research Center and Rectorate building are located in ITS, and most of the important people (Rector, Vice-Rector, Dean, and Lecturer) in ITS reside in those buildings. The activities in those buildings include internal meetings, external meetings, office work, and others. Therefore, the high-security level demands are much higher than the other buildings in ITS. This research aims to understand people’s behavior when interacting with DERS in a panic situation, especially high-rise buildings with different heights. The measurement will be taken implemented in VR simulation technology to represent the real condition and the performance in the virtual environment. It will be measured as the approach to know the effectiveness of DERS implementation. This research will be beneficial for ITS to know the impact of investing in this technology in the pursuit of achieving a more reliable disaster response system. Moreover, the benefit of this research might give a glimpse of insight into Indonesian people’s behavior when interacting with DERS and thus, enlarge the new perspective on the variety of evacuation methods that may be implemented in Indonesia.

2. Materials and Methods

2.1 Virtual Reality (VR) simulation

Virtual Reality (VR) is the technology to create a simulated environment that allows users to be able to interact with 3D worlds by simulating as many senses as possible, such as vision, hearing, touch, even smell but limited to the availability of content and computing power [6]. The virtual environment can be created by using VR software such as Unity, and the software can be run through a Head-Mounted Device (HMD). In this research, the type of HMD used is Samsung Gear VR correlated with the Samsung Galaxy S8 smartphone. In terms of evaluation, virtual reality may become an excellent alternative to real-life testing. The simulation result will be more trusted if virtual reality immersion gives a better representation of the real or actual world [10].

2.2 Presence Questionnaire (PQ)

In the case of VR simulation, the method of measuring human presence inside VR still has not been explored thoroughly at that time. Thus, the Presence Questionnaire is developed to capture the unspoken words that may represent how well the VR simulation does, especially for the immersion or the degree of realness compared to the real-world situation. In the ideal form, PQ consists of 19 questions that will be clustered to quantify four presence factors, namely involvement, sensory fidelity, adaptation/immersion, and interface quality [11].

According to Witmer, these factors have their focuses. Involvement is defined as a psychological state experienced as being involved or interacting in the environment. Sensory fidelity measures how the VR simulation gives accurate and consistent stimuli to the users. Immersion is defined as a state of being in the situation or included in the environment. Lastly, interface quality is the measure of usability of the VR simulation and the consistency of the interface. Those factors are interconnected to each other. Involvement is influenced by adaptability/immersion and sensory fidelity. Immersion is influenced by involvement and sensory fidelity. Sensory fidelity is influenced by involvement and immersion. Interface quality is influenced by the level of immersion [11].

2.3 System Usability Scale (SUS)

System Usability Scale (SUS) is a “quick and dirty” way to quantify a system or product usability. John Brooke initially introduced this method in 1986 and then SUS was developed further by Jeff Sauro in 2011 [12]. This method has become the industry standard for over 600 publications. SUS gives an effortless scoring mechanism to quantify whether a product is excellent and comfortable to use or not [13]. The SUS consists of ten-item questionnaire parameters with five response options (Likert-scale) for respondents who strongly disagree (1) and strongly agree (2). This questionnaire can be used to evaluate mobile devices, hardware, application, and software usability.

The System Usability Scale (SUS) has a score range 0-100 and can be calculated by following these steps: 1) For each odd-numbered question (1,3,5,7,9), the equation to calculate the SUS score is (X-1), 2) For each even-numbered question (2,4,6,8,10), the equation to calculate the SUS score is (5-X), 3) All of the odd-numbered score and even-numbered score are added to get the total score and multiply by 2.5. Therefore, the SUS questionnaire scores 0 (low usability) - 100 (high usability), and the average acceptable usability score is 68. If the scores are below 68, the object needs to be resolved and needs significant improvements [12]. The SUS assessment is classified into 5 (five) ranks (A, B, C, D, F) using the SUS Scoring Matrix. Details of the SUS Scoring Matrix and assessment are shown in Table 1.

In terms of the scoring method, Sauro and Lewis (2011) describes that the end value of the result is based on a 0-100 scale. In the calculation, the Likert scale of 1 to 5 is converted into 0 to 4. First, for odd items, the value must be subtracted by one. Second, for even-numbered items, the value must be subtracted from 5. Lastly, all values are summed and then multiplied by 2.5. The score of 68 acquired through this method is the middle point. On the other hand, the “A” score is gained if the score reaches a minimum of 80.3 scores [14].

Table 1. SUS scoring matrix and assessment

SUS Score

Grade

Adjective Ratings

Acceptability

80.3 - 100

A

Best Imaginable

Acceptable

74 – 80.2

B

Excellent

68 – 73

C

Good

51 – 67

D

Poor

Not Acceptable

0 – 50

F

Worst Imaginable

Table 2. Research scenarios

No

Usability and Immersion Level

DERS Implementation

Building

Starting

Floor level

Detailed

Location

Number of Participants

1

Combined Questionnaire

DERS Implemented

Research Center

2

Location A

12

2

3

Location B

3

5

Location C

4

Rectorate Building

1

Location D

12

5

2

Location E

6

3

Location F

7

Without DERS

Research Center

2

Location A

12

8

3

Location B

9

5

Location C

10

Rectorate Building

1

Location D

12

11

2

Location E

12

3

Location F

Total of Participants

48

Table 3. Questionnaire for assessment tool

No

Combined Questions

No

Combined Questions

No

Combined Questions

No

Combined Questions

1

How much were you able to control events?

8

How much did the auditory aspects of the environment involve you?

15

How easily did you adjust to the control devices used to interact with the virtual environment?

22

I think that I would need the support of a technical person to use this system.

2

How responsive was the environment to actions that you initiated (or performed)?

9

How well could you identify sounds?

16

How much delay did you experience between your actions and expected outcomes?

23

I found that the various functions in this system were well integrated.

3

How naturally did your interactions with the environment seem?

10

How well could you localize sounds?

17

How much did the visual display quality interfere or distract you from performing assigned tasks or required activities?

24

I thought there was too much inconsistency in this system.

4

How much did the visual aspects of the environment involve you?

11

How proficient in moving and interacting with the virtual environment did you feel at the end of the experience?

18

How much did the control devices interfere with the performance of assigned tasks or with other activities?

25

I would imagine that most people would learn to use this system quickly.

5

How natural was the mechanism which controlled movement through the environment?

12

How well could you concentrate on the assigned tasks or required activities rather than on the mechanisms used to perform those tasks or activities?

19

I think that I would like to use this system frequently

26

I found the system very cumbersome to use.

6

How much did your experiences in the virtual environment seem consistent with your real-world experiences?

13

How completely were your senses engaged in this experience?

20

I found the system unnecessarily complex.

27

I felt very confident using the system.

7

How compelling was your sense of moving around inside the virtual environment?

14

Were there moments during the virtual environment experience when you felt completely focused on the task or environment?

21

I thought the system was easy to use

28

I needed to learn many things before going with this system.

2.4 Usability matrix

Usability is a quality attribute that assesses how easy user interfaces are to use. Usability focuses on how easy and comfortable the object is to be used. In terms of usability, there are five quality components: learnability, efficiency, memorability, errors, and satisfaction. Learnability is the degree of ease in understanding the product or system mechanism. Efficiency is the measure of the speed of learning regarding the object. Memorability is the degree of ease in remembering the product or system mechanism. Error is the measure of how many errors happen during the testing. Lastly, satisfaction is the user's degree of comfort or pleasant. Those components are assessed through a series of questions [15].

2.5 Dynamic Evacuation Routing System (DERS)

According to one of the producers of this technology, INOTEC, Dynamic Evacuation Routing System (DERS) is an addition to conventional emergency and safety lighting connected to an automatically triggered power supply. In the event of a power cut, it ensures that certain areas are still lit and that emergency exit luminaries and direction-giving signs are front or backlit as appropriate [5]. Other than giving light, it also gives a sense of direction to the evacuees while in the state of panic. It also may give immediate and accurate information regarding the route used to evacuate quickly and ensure safety.

There are three main advantages of using DERS: flexibility, adaptability, and understandability. Flexibility covers the changeability of the sign used for guidance to the evacuees. Adaptability covers the speed to adapt to changes in the environment. Finally, understandability covers the ease of knowledge transfer even in a panic situation. Those advantages will significantly improve survivors' decision-making in a dangerous situation since they do not have to read and understand a complicated evacuation plan. Therefore, the survivor may make safer steps without risking making errors in the evacuation process.

2.6 Study design

Three independent variables will be tested, namely DERS implementation, type of building, and the starting floor level. This study uses the Research Center Building and Rectorate Building as a sample building in this research case study with two different characteristics. First, in accordance with the independent variables, there were three dependent variables as the response of the experiment; completion time, reaction time, and the number of errors. Based on those variables, there were 12 scenarios carried out by 48 participants between and within the experiment, as shown in Table 2. Moreover, in accordance with the fact that simulation was used in this experiment, then the degree of immersion and usability of the simulation were also needed to be quantified to strengthen the simulation result. This quantification combined three methods: Presence Questionnaire (PQ), System Usability Scale (SUS), and usability matrix. Those quantification methods will be presented through an assessment tool in a questionnaire containing 28 questions, as shown in Table 3.

Through several literature reviews, some hypotheses were related to this experiment.

  • Hypothesis 1. There was a significant improvement when a building used DERS as their evacuation method
  • Hypothesis 2. There was a significant difference when the evacuation process was conducted in different types of building
  • Hypothesis 3. There was a significant difference when the evacuation process was conducted in different building floor-levels
  • Hypothesis 4. The developed simulation had a good level of immersion with a PQ score above 3.50
  • Hypothesis 5. The developed simulation had a good level of immersion with the SUS score above 74 and the value of the Usability Matrix dimension of at least 74% for each dimension.

According to hypotheses one up to 3, the simulation was tested through MANOVA Test to find the significance. Moreover, for hypotheses 4 and 5, the simulation was tested before using the assessment tool, which utilized PQ, SUS, and Usability Matrix methods.

3. Results

3.1 Participant’s simulation result

During the data gathering process, the participants had used the simulation, and the results were recorded and analyzed. The simulation interface is shown in Figure 1. The yellow arrow in the interface was the DERS visualization. The arrow did not guide participants only to the exits but to all possible routes, and the participants needed to find the correct route to the assembly point.

Figure 1. Simulation interface (DERS visualization)

In terms of the effect of DERS implementation in MANOVA, it turned out that the DERS implementation had a significant effect on all dependent variables at once. The MANOVA result for all methods indicated the number of 0.000, which meant that the DERS implementation significantly affected all dependent variables collectively. In terms of the effect of different buildings in MANOVA, it turned out that the different buildings were significantly affecting the evacuation effectiveness (completion time, reaction time, and the number of errors) collectively with the minimum P-value of 0.000. In terms of the effect of different starting floor levels in MANOVA, it turned out that the different starting floor levels were significantly affecting the evacuation effectiveness (completion time, reaction time, and the number of errors) collectively with the minimum P-value of 0.002.

According to Table 4, a p-value provides a significant indicator if the p-value is < 0.05. This p-value is not the result of the sum of the seven sources of variation (independent). First, it was known that the combination of DERS implementation and different buildings gave a P-value of 0.088, which was more than 0.05. Therefore, the combination was not significant. Then, it was also known that the combination of DERS implementation and different starting floor levels gave a P-value of 0.334, which was more than 0.05. Therefore, the combination was not significant. Other than those combinations, the P-value was less than 0.05. Therefore, the remaining combination, the combination of different building and starting floor levels and all of the independent variables, significantly affected all of the dependent variables.

Table 4. MANOVA result

No

Source of Variation

P-value

Significance

1

DERS

0.000

Significant

2

Building

0.000

Significant

3

Start

0.002

Significant

4

DERS*Building

0.088

Insignificant

5

DERS*Start

0.334

Insignificant

6

Building*Start

0.000

Significant

7

DERS*Building*Start

0.048

Significant

*If p<0.05, the variation is significant

Figure 2. Factorial plot result (main effect and interaction)

3.2 Simulation’s factorial plot result

Other than the MANOVA result given before, there were also simulation results in the form of factorial plots. These factorial plots gave a more comprehensive view and detailed facts about the simulation effect. These factorial plots were made according to the amount of independent and dependent variables present in the experiment. The factorial plots for the main effect and the interaction are given in Figure 2. It was known that the usage of DERS reduced the completion time needed by the participants slightly.

Moreover, the Research Center building had a higher completion value than the Rectorate Building. Then, the starting floor turned to have a random pattern where the highest completion time was the highest floor, and the lowest was on the middle floor. Furthermore, the DERS and building combination contradicted results for implementing DERS in a different building. For the DERS and start a combination, the DERS usage improved all the starting floor levels. Then, the effect of different buildings in the combination of building and starting floor level gave a significant difference.

It was known that the usage of DERS reduced the reaction time needed by the participants significantly. Moreover, the Research Center building had a higher value in reaction time rather than the Rectorate Building. Then, the starting floor turned to have a random pattern where the highest reaction time was the highest floor, and the lowest was on the middle floor. After that, the DERS implementation in a different building improved reaction time in all combinations. For the DERS and start a combination, the DERS usage improved all the starting floor levels. Then the effect of different buildings in the combination of building and starting floor level gave a significant difference.

It was known that the usage of DERS reduced the number of errors performed by the participants slightly. Moreover, the Research Center building had a higher value in the number of errors rather than the Rectorate Building. Then, the starting floor turned to have a random pattern where the highest number of errors was on the highest floor, and the lowest was on the middle floor. The DERS implementation in a different building improved the number of errors in all combinations. For the DERS and start a combination, the DERS usage improved the lowest and middle starting floor level. Then, the effect of different buildings in the combination of building and starting floor level gave a significant difference.

3.3 Simulation assessment result

For the immersion level, the overall PQ results gave the number of 3.90 out of 5, which already met the standard used in the PQ method, which was 3.50. However, since the PQ method had four presence factors that explained the result that appeared in the overall results, to analyze those presence factors. The presence factors were involvement, sensory fidelity, immersion, and interface quality. The involvement received a score of 4.01 out of 5, which meant that the participants could easily be able to involve themself in the simulation. For the sensory fidelity, the score was 3.41 out of 5, which meant that the participants could decently sense the reality inside the simulation. The immersion received a score of 4.04 out of 5, which meant that the simulation gave the user the realness and gave a mostly accurate representation of the real world. Lastly, in terms of interface quality, the score obtained was 4.15 out of 5, which meant that the simulation was mainly lag or delay-free. The detailed result is shown in Table 5.

In terms of usability, the SUS result gave the number 71.72, which was not enough to satisfy the standard to get the B-rank in the SUS scoring matrix. This result suggested that the simulation was typical or average. This implied that the usability aspect of the simulation must be improved. The usability matrix has five quality components: learnability, efficiency, memorability, error, and satisfaction. In order to calculate the percentage, each question would be fitted to the quality component of the matrix, and then the percentage would be calculated by dividing the number of people choosing the scale of 4 to 5 of the respected quality component’s questions with the total of people included in this experiment times 100%. Therefore, it was known that 74% of the participants agreed that this evacuation simulation was easy to learn and understand. Then, 83% of participants agreed that the participants could efficiently use the evacuation simulation. It means that the DERS evacuation training method based on Virtual Reality (VR) is more time and cost-efficient than real evacuation simulations. Moreover, 61% of participants said that this simulation was easy to be memorized. Therefore, this technology makes it easier for users to understand and remember the actual evacuation procedure. Then, 72% of the participants agreed that the simulation was error-free. These results provide information that users can use the tool according to its function. Lastly, 65% of the participants were satisfied with how the simulation was designed and felt.

Table 5. Presence questionnaire result

Question No

Presence Factor

Question Score

Factor Score

PQ Score

1

Involvement

4.19

4.01

3.90

2

4.15

3

3.92

4

4.31

5

3.71

6

4.02

7

3.81

8

Sensory

Fidelity

3.48

3.41

9

3.92

10

2.83

11

Immersion/ Adaptation

4.10

4.04

12

4.13

13

3.38

14

4.08

15

4.52

16

Interface Quality

4.17

4.15

17

4.04

18

4.23

4. Discussion

4.1 The effect of DERS implementation regarding participant’s evacuation effectiveness

The DERS improved all of the dependent variables significantly in the main effect of the factorial plot. However, some interactions gave an insignificant result in the MANOVA result. This result suggested that the factor of DERS Implementation could alter the result of the evacuation success.

After MANOVA and ANOVA, the result of the main effect and interaction plot showed how each combination of independent variables, especially DERS implementation, affected each dependent variable. Regarding DERS implementation, the main plot of all dependent variables implied that DERS implementation improved the evacuation effectiveness. The result also supported this statement, which suggested that the number of participants’ failures when using DERS was lower than the number of failures without using DERS. However, in the interaction plot, the combination of DERS and buildings and the combination with starting floor levels gave slightly different pattern variations. For the completion time as the dependent variable, the combination of DERS and buildings suggested that the implementation of DERS in the Rectorate building was not suitable to reduce the completion time, as shown with the declining trend in Figure 2. However, the remaining dependent variables suggested that implementing DERS in the Rectorate building would reduce the reaction time and the number of errors. This result might cause insignificancy in the MANOVA test result since the results between dependent variables were not consistent.

Nevertheless, this result might happen because participants already perceived the way out in the Rectorate building. However, it was somehow interfered with by the lines provided by DERS, and it somehow slowed down their decision-making process. That statement was also in-line with the fact that the reaction time and the number of errors were improved while only the completion time had a worse result. Furthermore, another fact also highlighted that for the number of errors as the dependent variable, the combination of DERS and starting floor level suggested that the highest simulated floor level gave worse results when DERS was implemented. However, this result was inconsistent with other dependent variables, suggesting that the DERS implementation improved, specifically in completion and reaction time. This also became the reason for the insignificancy in the MANOVA result. This might happen because, for higher floor levels, the current type of DERS tested in this research was not suitable since the complexity was high. Somehow, the provided lines led people to pick wrong routes even if they could complete the simulation quickly with faster reaction time.

4.2 The effect of different building regarding participant’s evacuation effectiveness

For different buildings, the main effect of the MANOVA Result suggested a significant impact of this independent variable on all dependent variables. However, the interaction was not always significant. This result suggested that the factor of the different buildings could alter the result of the evacuation success. After MANOVA and ANOVA, the result of the main effect and interaction plot showed how each combination of independent variables, especially different buildings, affected each dependent variable. The main effect plot of the building suggested that the Rectorate building was much easier to handle than the Research Center building. However, the interaction plot for the combination of building and starting floor level with the number of errors as the dependent variable suggested that the participants had a slight tendency to make more mistakes when placed in the Rectorate building for the highest floor level rather than the Research Center building. This result might be the reason for the slight significance in the MANOVA result. The significant difference in evacuation effectiveness data regarding the effect of the different buildings lay in the uniqueness of the building layout and the familiarity with the place. It turned out that following the complexity or uniqueness of the layout, the Rectorate building was way more superficial than the Research Center building, as explained several times in this analysis. In the Rectorate building, the layout of the first floor until the third floor was pretty similar, square-shaped. However, the Research Center building had a square layout but a different pathway for the first floor until the third floor and mainly a squared fixed pathway for the fourth until the eleventh floor. Moreover, the familiarity of the place also becomes why people tend to escape better in the Rectorate building because students have ever gone to the place. On the contrary, the Research Center was a new building where most students have not even known about the building.

4.3 The effect of starting floor level regarding participant’s evacuation effectiveness

For different starting floor levels, the main effect of the MANOVA Result suggested a significant impact of this independent variable on all dependent variables. However, the interaction was not always significant. This result suggested that the factor of the different buildings was able to alter the result of the evacuation success.

After ANOVA and MANOVA, the result of the main effect and interaction plot showed how each combination of independent variables, especially different buildings, affected each dependent variable. It turned out that a higher floor level did not always cause people to dwell longer inside the building. This evidence was backed up by the fact that the middle floor level made people dwell less than the lowest floor level. Then, building and starting floor level interaction also gave an exciting pattern. For the Rectorate building, the completion time and the number of errors gave linearly typical results where the fastest cleared floor was the lowest floor, and the most extended cleared floor was the highest. This might happen because the layout of each floor in the Rectorate building was consistent, where each floor was square-shaped. The completion time and error factors were the distance and the increasing number of possible routes on each floor. However, the highest sequences were the highest, lowest, and middle floors for the reaction time data. This might happen because of the participants' learning curve, or the building layout might cause it in the middle floor where the participants could directly see the door which led to the room's exit.

Moreover, the completion time and error results suggested that the longest completion time and the highest error rate were achieved on the lowest floor for the Research Center building. On the other hand, the fastest completion time and the lowest error rate were achieved on the highest floor. Then, for the reaction time, the sequences were the highest floor, lowest floor, and then the middle floor, which might happen because, on the highest floor, the participant needs to pick between 2 follow-up points. For the lowest floor, it was longer because of the participant’s learning curve. This pattern might imply that building heights were not the problem, but the problem was the complexity of the building layout on those floor levels. For example, the Research Center on the highest floor or a fifth-floor level was very close to the emergency exit. Many escaping options on the middle floor or the third-floor level usually caused the evacuees to escape easily but sometimes made them think before choosing, which made it longer than the highest floor. However, it had a vast area for the lowest floor or the second-floor level, which confused the evacuees and struggled to find the exits.

4.4 Immersion and usability level of the VR simulation

In terms of appraising the quality of VR simulation, the factor of immersion and usability were becoming very important. Those parameters became vital because they determined whether the simulation was real enough and easy enough for the user to operate. In this research, three methods were used to quantify the quality of the simulation. First, one method was used to appraise the simulation's immersion level: the PQ method. Then, there were two methods used to appraise the usability of the simulation.

For the immersion level, the overall PQ results gave the number of 3.90 out of 5, which already met the standard used in the PQ method, which was 3.50. However, since the PQ method had four presence factors, analyze those presence factors. The presence factors were involvement, sensory fidelity, immersion, and interface quality. The involvement received a score of 4.01 out of 5, which meant that the participants could easily be able to involve themself in the simulation. For the sensory fidelity, the score was 3.41 out of 5, which meant that the participants could decently sense the reality inside the simulation. The immersion received a score of 4.04 out of 5, which meant that the simulation gave the user the realness and gave a mostly accurate representation of the real world. Lastly, in terms of interface quality, the score obtained was 4.15 out of 5, which meant that the simulation was mainly lag or delay-free.

In terms of usability, the SUS result gave the number 71.72, which was not enough to satisfy the standard to get the B-rank in the SUS scoring matrix. This result suggested that the simulation was typical or average. This implied that the usability aspect of the simulation must be improved. Furthermore, the detail of the analysis of usability was given in the usability matrix method result. Five quality components must be considered; learnability, efficiency, memorability, error, and satisfaction. It was known that 74% of the participants agreed that this simulation was learnable. Then, 83% of participants agreed that the participants could efficiently use the simulation.

Moreover, 61% of participants said that this simulation was easy to be memorized. Then, 72% of the participants agreed that the simulation was error-free. Lastly, 65% of the participants were satisfied with how the simulation was designed and felt. This number said there was a lack of memorability, error, and satisfaction of the simulation because the percentage of people agreeing was less than 74%. Some improvements might increase memorability by using a more dynamic routing that directly gives the direction to the exit, and the interface must be easier to control. For satisfaction, visual and audio quality must be improved. For the visual, it must engage people to interact with the environment, and for the audio, that source of sounds must be clear to give participants information about where the fire breaks out. Then, it was also suggested that the moving mechanism had to be more natural to mimic how people walk.

In terms of suggestions from participants, there were several downsides to this simulation that became the basis for further improvements. The flaws or improvement points were located within the visual quality, level of interaction, audio engineering, orientation consistency, DERS type, and a moving mechanism. First, the blurry image became a problem in visual quality. The minuscule amount of interaction possibilities for the participants within the VR environment is also becoming a problem. Then, the audio should be engineered to have more realistic sounds. After that, some image disorientations were left within the simulation that needed to be fixed. Then, the type of DERS simulated was better to provide lighting, leading to only the exit and not all directions. Lastly, the moving mechanism must be more natural by mimicking a human’s walking method.

4.5 Compatibility of DERS implementation

Firstly, in order to determine whether DERS Implementation was compatible, it was needed to determine whether the simulation was valid or not. From the analysis, it was clear that the simulation is immersive and decently usable. This result gave the foundation that the experiment was accurate and valid enough.

The DERS improved all of the dependent variables even though they were insignificant. This consistent result made the MANOVA method give a significant result. So, the implementation of DERS was suitable for people, and it improved the evacuation effectiveness parameters as a whole but not significantly. Then, the effect of the building implemented turned out to be necessary. It turned out that a simple layout building was slightly better if not using DERS for the completion time, and the improvement for other dependent variables was decent. From the data, it was suggested to implement DERS in a more complex building rather than the simple one. Through all of those discoveries, it was known that in making or engineering buildings, it was essential to consider the layout where the emergency exit must be near to the essential places in that building, such as offices, and make a clear hint about the place of the emergency exit with redundant lighting, signs, and decorations.

5. Conclusions

The DERS implementation gave a significant result in the MANOVA (0.000) because it gave a consistent result on each dependent variable. Then the main plot suggested that the DERS implementation improved the evacuation effectiveness. The different building and starting floor levels gave significant results in the MANOVA (0.000 & 0.002) because they gave consistent results on each dependent variable. Then, the main plot suggested that the most accessible building to be escaped was the Rectorate building, and the height did not affect the effectiveness linearly. However, the building layout of each floor level was the main contributor to the increasing evacuation effectiveness. The immersion level of the simulation represented by the PQ method gave the number of 3.90, which could meet the standard. However, the SUS (71.72) and usability result (min. 61% for Memorability) were still lacking, and further improvement was needed to be executed to improve the usability of the simulation such as the visual quality, level of interaction, audio engineering, orientation consistency, DERS type, and a moving mechanism.

Acknowledgment

This research was conducted based on research contract number 1261/PKS/ ITS/2019 funded by ITS.

  References

[1] Brushlinsky, N.N., Ahrens, M., Sokolov, S.V., Wagner, P. (2017). World Fire Statistics. CTIF - International Association of Fire and Rescue Services. https://www.ctif.org/sites/default/files/ctif_report22_world_fire_statistics_2017.pdf, accessed on March 3, 2022.

[2] Kodur, V., Kumar, P., Rafi, M.M. (2020). Fire hazard in buildings: Review, assessment and strategies for improving safety. PSU Research Review, 4(1): 1-23. http://dx.doi.org/10.1108/PRR-12-2018-0033

[3] Brushlinsky, N.N., Ahrens, N.N., Sokolov, M., Wagner, P. (2016). World Fire Statistics. CTIF - International Association of Fire and Rescue Services. http://www.ctif.org/sites/default/files/ctif_report21_world_fire_statistics_2016.pdf, accessed on March 3, 2022.

[4] Mohajan, H. (2016). Sharing of tacit knowledge in organizations: A review. American Journal of Computer Science and Engineering, 3(2): 6-19. https://mpra.ub.uni-muenchen.de/id/eprint/82958

[5] INOTEC. (2017). D.E.R: Dynamic Escape Routing. Germany: INOTEC. https://www.inotec-licht.de/fileadmin/user_upload/Bilder/Aktuelles/Presse/2001-BDB-diretk-dynamische_flu.pdf, accessed on March 3, 2022.

[6] Elmqaddem, N. (2019). Augmented reality and virtual reality in education. Myth or reality? International Journal of Emerging Technologies in Learning, 14(3): 234-241. https://doi.org/10.3991/ijet.v14i03.9289

[7] Çakiroğlu, Ü., Gökoğlu, S. (2019). Development of fire safety behavioral skills via virtual reality. Computers & Education, 133: 56-68. https://doi.org/10.1016/j.compedu.2019.01.014

[8] Fabroyir, H., Teng, W.C. (2018). Navigation in virtual environments using head-mounted displays: Allocentric vs. egocentric behaviors. Computers in Human Behavior, 80: 331-343. https://doi.org/10.1016/j.chb.2017.11.033

[9] Cha, M., Han, S., Lee, J., Choi, B. (2012). A virtual reality based fire training simulator integrated with fire dynamics data. Fire Safety Journal, 50: 12-24. https://doi.org/10.1016/j.firesaf.2012.01.004

[10] Passig, D., Tzuriel, D., Eshel-Kedmi, G. (2016). Improving children's cognitive modifiability by dynamic assessment in 3D immersive virtual reality environments. Computers & Education, 95: 296-308. https://doi.org/10.1016/j.compedu.2016.01.009

[11] Witmer, B.G., Jerome, C.J., Singer, M.J. (2005). The factor structure of the presence questionnaire. Presence: Teleoperators & Virtual Environments, 14(3): 298-312. https://doi.org/10.1162/105474605323384654

[12] Brooke, J. (2013). SUS: A retrospective. Journal of Usability Studies, 8(2): 29-40.

[13] Sauro, J. (2011). A Practical Guide to the System Usability Scale: Background. Benchmarks & best practices. North Charleston: SC: Create Space Independent Publishing Platform. 

[14] Sauro, J., Lewis, J.R. (2011). When designing usability questionnaires, does it hurt to be positive? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2215-2224. https://doi.org/10.1145/1978942.1979266

[15] Nielsen, J., Budiu, R. (2013). Mobile Usability. MITP-Verlags GmbH & Co. KG.