Chaima Bouali | Olivier Habert | Abderrahim Tahiri
OPEN ACCESS
This work describes a ‘detection of abnormal activities and health-related changes’ system for an elderly person at her/his home. The analysis is based on the data collected by a domotic box of the market. The box was initially designed to continuously recognize the owner’s daily activities in order to anticipate anomalies and consequently prevent health complications and enhance the rate of disease prevention. The box uses non-intrusive home automation sensors to detect the activity level of the occupants. It is equipped also with other technologies, including humidity sensors, bed and chair sensors to name a few. In order to build a system capable of intercepting warning signs for early intervention, we adopt a Hidden Markov Model based approach that we will initialize beforehand with the activity sequences of the user within a given period. The outcomes of the model paves the way for deducting the final judgement and reporting a relevant context-aware alert to healthcare service experts. Other statistical processes might complete this behavioural analysis later on to enhance the alerts accuracy.
activity monitoring, sensors, data processing, smart homes, daily activities sequence, elderly, independent living
The purpose of a smart home for assisted living is to enhance particularly the quality of life of a resident in his/her daily functioning. The user either might be an old person with specific cognitive impairment or, broadly, with age related problems in functional abilities.
Smart homes are no longer needed just for comfort and luxury life but beyond that, these technologies respond nowadays to various vital human needs in various ranges. Healthcare, security and energy management are the fields that smart residences focus on [1]. Not only are these technologies useful in different areas but also for different age ranges. In other terms, smart homes can be adapted for the daily lives of elderly people too. Assisting them, helping them in daily tasks and tracking their activities in order to prevent injuries and harmful behaviours are the major uses of a smart home for assisted living.
Accurate recognition processes improve the efficiency of the smart home applications. One of the challenges resides in building a system that learns, understands and automatically establishes the resident’s ADL (Activities of Daily living). These characteristics provide an extreme reliability of the identification system. Among the issues, we also have the identification of an action in real time. Many works have been dedicated to this aspect [2] in order to deal with ADL instantly and reveal unwanted behaviours for example [3].
The company NOVIATEK has deployed a set of sensors inside their clients’ homes and has installed a box in their place representing features extracted from the collected sensor data related to the daily functioning of the occupant. The sensors are located on the furniture, floors, walls and doors in an unobtrusive way. The box permits the provision of a rich ground truth. From the data, we can distinguish the different daily activities performed by the client (Sleeping, toileting, eating, etc.). Even though this paper is not intended to dive deep into the complex approaches of an accurate Activity Recognition, we need to precisely determine that if the activity detection algorithms are wrongly performed, the whole detecting anomalies process might fail.
Humans in general, elderly people in particular, trust and accept technologies that will fit with their needs and will serve them appropriately in the different areas of their daily living. The main use of the box is to deal with abnormal behaviours of the user and generate context-aware alerts regarding alarming behaviours. These alerts might either be generated on a daily basis or regarding observations made on a given period. However, the accuracy of the alerts must be as high as possible. It opens the way to allow tracking different activities and predicting safe and unsafe behaviours and furthermore, to supervise changes in daily activities and permit medical monitoring.
A key aspect of assistive technologies consists in the availability of a system capable of automatically detecting future behaviours or abnormalities, significantly reaching a higher rate of correct predictions.
The aim of this work is to provide valuable information for evaluating the functional abilities recognized by the box. The system built should examine the relevant factors related to the occupant’s daily performance and verbally alert the caregiver or an expert of the possible future inconsistency.
Using the long-term context history, the system would learn the behavioural pattern of the person. The elderly has specific health issues and mobility limitations, which means that we should implement personalised and user-centric monitoring tools.
Many projects [4, 5] are aimed at developing systems of health deterioration or unnatural process of ageing detection. However, little work is focused on developing an intelligent change detection model centered on the client’s needs. In general, these needs are closely related to the specific cognitive impairment of each person, such as memory problems, difficulty walking, poor eyesight and to the disease in question that may be causing them. In addition, these solutions were based mostly on laboratory samples and data which means that they are being tested in a non-realistic context. In our paper, we demonstrate the effectiveness of our system by using several real-world datasets.
The main objective of the system presented in this review is twofold: firstly, it analyses the daily activity sequences considered in our seniors homes, as well as the appearance of the most relevant activities on a daily basis. Secondly, it takes into consideration the timing of the performance of the indoor activities in case of alarming displacement. The sequence of activities changes according to the individual user and the duration of each activity. Based on this reflection, we have chosen the Hidden Markov Model (HMM) which is a performant probabilistic model for dealing with such uncertainties. Using the HMM, it is possible to predict future states from current observations as well as the sequence of states from an observed sequence.
1.1 Motivations
The primary goal of our research is to build an accurate system, permitting an early diagnosis of abnormal behaviours that can deliver adaptive services with regard to a knowledge base of the user’s lifestyle. A system that detects incident cognitive decline in seniors living habits by deploying an unobtrusive assessment to help healthcare professionals in long-term diagnosis strategies. The detection process must respect the intimacy of the resident and the data has to be stored conforming to the general regulation of personal data. The achievement of these desired purposes is fuelled by the following motivations:
1.2 State-of-the-art
The perpetual increase in the number of elderly people with a decrease in independence, that is related either to age or to some handicap, is one of the motivations of smart home conception. Many works have highlighted the use of remote monitoring in order to deal with the growing needs of providing health services for these persons [5]. This section presents an overview of the different proposed solutions for e-Health applications. In this article, the main objective is to have a system that provides a steady, non-invasive, personalised and highly accurate service.
‘Health monitoring and the detection of daily abnormalities’ is one of the flagship applications of smart homes. In this context, Wang et al. [6, 7] focus basically on a sensor based conception; environmental and on-board sensors. Wang et al. [6] implements a platform that allows the supervision of the daily behaviours of the resident; with a prior knowledge of the chronic disease that affects the user.
For its part, the project of [7] is intended specifically for people suffering from dementia. In this case for example, the aim is to provide, in due course, adapted services that assist in enhancing the memory.
In the ref. [8], the targeted public is broad. The conception of the system is based on the Hidden Markov Models (HMM).
In this work, we find our research problematic; the detection of alarming situations in the daily living of an elderly person. For the establishment of the project of [8], laboratory data has been exploited.
2.1 Data
The context domain we will firstly focus on is the user activity. The purpose of the model is to learn the behavioural patterns of the elderly person using the historical data collected by the smart box to detect a consequent change that might occur in an observed sequence. The database provided by the smart box company is composed of several tables. Each table contains a variety of attributes including the activities. We solely take into account the activities that reflect a considerable change in the user’s routine.
The domotic box gives us the identified activities on a daily basis. We have chosen this sampling of 24 hours to match with the natural rhythm of life of a person in his/her home. A longer duration (a month, a year) might be possible in order to identify more issues related to health deterioration (in a pathological order). In order to maximize the efficiency of a long-term context-aware health-decline detection system, there should be a broad-focus on the essential needs of the elderly. A fully functional prediction system should appeal to all the essential needs of the elderly in order to improve and enhance the reliability of the alerts. The system can compute a daily score that gives information on the coherence of the input activity sequence, regarding the given settings of the system. After the analysis of the sequence, the returned information has to be detailed enough to be able to detect either automatically or manually the causes of each abnormality. With the computed score, we should get, as well, a classification of each activity of the sequence using the labels “Normal”, “Abnormal” …
The domotic box generates a database that includes much more information related to the time, the duration and the place of the activities… To this day, this article considers just the analysis of the activities (Figure 1).
Figure 1. List of activities
2.2 Markov model
Markov model [9] is a stochastic model composed of states, transitions and emissions. It is used to study the dynamic of stochastic processes and to model organized data. Thus, it is possible to model data at a given time regarding the previous data. The Markov model assumes that the next state is only influenced by the current state. The main applications of Markov model are statistical learning, prediction or recognition of similarity. Figure 2 illustrates a model of three states {E1, E2, E3} of which the transitions between the states have been established by using the collected data.
Figure 2. Markov Model
For each state, the sum of the transition probabilities to other states including to the state itself has to be 1. This model permits to get the probability of a states’ sequence from an initial state. For example, the probability of the sequence E2, E2, E3 will be the probability of the sequence E2, E3 knowing E2: P(E2)= P(E2).P(E2,E2)= P(E2).P(E2)=0.4*0.35=0.14
2.3 Hidden Markov Model
A Hidden Markov Model (HMM) is a statistical model where the states are hidden (Figure 3). Each state can emit an observable output. Considering the previous model, each state emits k observable outputs {S1,S2,…,Sk}, each one has a probability P(Ei) with 1≤i≤3 and 1<j<k.
The Markov model permits, after initialization of the parameters, to compute the probabilities of the new states’ sequences but is still incapable of quantifying every abnormality detected in each sequence in order to be qualified on a normality scale. An HMM with set parameters evaluates an observations’ sequence. The model permits to find a states’ sequence that maximizes an observation sequence as well. Finally, it is possible to train an HMM with the maximization of the observations’ sequence probability chosen in an accurate way for the application needed.
All these issues (evaluation, decoding and learning) can be solved using known algorithms, like the algorithms Forward, Viterbi, Forward-Backward (Baum-Welch) that we will present later on.
Figure 3. Hidden Markov Model
2.4 Application for the detection of abnormalities in activities sequences
Figure 4. Model of detection of the activities abnormality
The HMM can be used to resolve our problem of daily activities’ sequences qualification for the dependent person. In fact, these activities’ sequences {A1, A2, …, Ak} constitute the observations’ sequences of the HMM (Figure 4).
The hidden states of the model are the labels qualifying the normality of the activities sequence. We chose at first three labels: “Normal”, “Abnormal” and “Critical”. As a starting point, we have the model with set parameters; it is possible then to compute the probability of a states’ sequence. Using the Viterbi algorithm, we can find the states’ sequence that matches the most with an observations’ (activities) sequence. This succession of labels allow an abnormality analysis deeper than a simple calculation of the likelihood.
2.5 Implementation of the HMM
There are three major problems for Hidden Markov Models:
The first and the second problem can be solved by using the dynamic programming algorithms known as the Viterbi algorithm and the Forward-Backward algorithm. The last one can be solved by an iterative Expectation-Maximization (EM) algorithm, known as the Baum-Welch algorithm.
(1) Viterbi
This algorithm [10] helps to find the best states’ sequence for a given observations’ sequence and a given model. Viterbi consists in computing the probability of generation and resolving the problem of the evaluation following all the possible paths to choose among them, the one with the highest probability.
(2) Algorithm Forward-Backward
The Forward-Backward algorithm [11] starts by a progressive computation of probabilities, a “forward” calculation that gives the possibility to get the ‘n’ first observations in a given sequence finishing with each possible state of the Markov model. It allows us to filter the probabilities that progress over time. Afterward, the algorithm computes the probability of getting other observations posterior to an initial state. These two probability groups can be combined to get, at every time the probability of the hidden state knowing the whole sequence of the observed data. The Forward-Backward algorithm can be used then to find the most probable states for a Markov model at any instant.
(3) Baum-Welch
Baum-Welch [12] is an Expectation Maximization (learning) algorithm. Given the observations’ sequences O and the initial model λ, Baum-Welch re-estimates the parameters of the model in order to increase the likelihood of the generation of O. Baum-Welch uses the algorithms Forward and Backward to decrease the complexity of the calculation.
In our case, we use the Baum-Welch algorithm to estimate the parameters of our HMM; emission probabilities, transition matrix and the initial distribution. We then get our model λ using the activities’ sequences specific to the user. After this, Viterbi’s algorithm permits the identification of the sequences that might be generated by our model. This part is concerned with the abnormality detection in a sequence using λ. Baum-Welch re-estimates afterward the parameters of the model for the learning part for the analysis in the later stages.
3.1 Used data
The processing of the data coming from the monitoring of users in a real situation of dependence will be the added value to our work. The box is installed at the homes of different people with a decreasing autonomy. This diversity offers us a wide range of possibilities to approve the system, either in the learning stage or in the analysis of the sequences.
Naturally, it is necessary to ensure the consent of individuals in the use of their personal data.
Figure 5 shows an example of an activities’ sequence, specific to a user, generated through MATLAB. We are at a trial stage of the algorithms using the available MATLAB’s tools. We’re testing the different steps by submitting to the system some learning sequences corresponding to the lifestyle of a particular case. We then inject some disturbed sequences to observe the evaluation result given by the system.
Figure 5. An activities’ sequence of a day
3.2 User profile
Dominique is 66. She is a widow of three years, therefore, she has been living independently for this period. During this time, Dominique has had a pretty stable lifestyle. Usually, she goes straight to the toilet after waking up in the morning. After this, she washes and gets dressed in her bedroom. Which is followed by breakfast and another trip to the toilet. Mid-morning, she leaves her home and takes a walk for an hour. When she returns, she occasionally goes to the toilet before having lunch. After eating, she gets some rest in her bedroom. Around 4 PM, normally she has a snack. In the evening, she has dinner and goes to sleep around 10:30 after getting ready for bed.
Following the presentation of the specifics of Dominique’s routine, we can present a sequence of activities that perfectly describes her usual day using the labels from our database.
PETIT_WC – GRANDE_HYGIENE – SHORT_SLEEP (Dressing up)– PETIT_DEJEUNER – LONG_WC – LAVAGE_MAIN (Getting ready to go out) – DEPART_AUTO – RETOUR_AUTO – PETIT_WC – DEJEUNER – SHORT_SLEEP – COLLATION_JOUR – PETIT_WC – DINER – LAVAGE_MAIN (Getting ready for bed) – SHORT_SLEEP – LONG_SLEEP.
Given Dominique’s typical day, a sequence remains normal even if, for example, she does not eat breakfast until she gets changed or does not leave her place during the day. On the other hand, if she does not eat all day or spends the whole day in bed for example, this can be alarming [13-17].
3.3 Evaluation
In order to evaluate our system for processing activities’ sequences and detecting anomalies, we design our model according to the following steps:
Figure 6. The evaluation of sequences by the system
This first stage of the project consisted in choosing, appropriating and validating the tools that seemed to us the most relevant for evaluating in Situ the sequences of activities given on a daily basis by the box. The algorithms have been adapted to the format of the data collected by the box. An attempt to inject normal sequences was carried out to initialize the model. An evaluation test was also carried out for situations considered normal and abnormal. We must now move on to the study of several concrete cases from field data. This requires planning a training period over several weeks and then starting the evaluation of the activity sequences. An adaptation to the profiles of the users involved (pathologies, layout of the apartment, etc.) will be necessary each time. The model could then be enriched by considerations linked to the location of people or by taking more specifically into account the temporality of the activities
The authors would like to thank the Noviatek society for its commitment and involvement in the project. The information about the organization of the database tables and the field data provided by our partners were very useful for the progress of our work.
[1] Augusto, J.C., McCullagh, P., McClelland, V., Walkden, J.A. (2007). Enhanced healthcare provision through assisted decision-making in a smart home environment. In 2nd Workshop on Artificial Inteligence Techniques for Ambient Inteligence, 27-32.
[2] Bouchard, K. (2014). Unsupervised spatial data mining for human activity recognition based on objects movement and emergent behaviors (Doctoral dissertation, Université du Québec à Chicoutimi).
[3] Zolfaghari, S., Keyvanpour, M.R. (2016). SARF: Smart activity recognition framework in Ambient Assisted Living. In 2016 Federated Conference on Computer Science and Information Systems (FedCSIS), 1435-1443.
[4] Bourobou, S.T.M., Yoo, Y. (2015). User activity recognition in smart homes using pattern clustering applied to temporal ANN algorithm. Sensors, 15(5): 11953-11971. https://doi.org/10.3390/s150511953
[5] Fahad, L.G., Khan, A., Rajarajan, M. (2015). Activity recognition in smart homes with self verification of assignments. Neurocomputing, 149: 1286-1298. https://doi.org/10.1016/j.neucom.2014.08.069
[6] Wang, L., Gu, T., Tao, X., Lu, J. (2012). A hierarchical approach to real-time activity recognition in body sensor networks. Pervasive and Mobile Computing, 8(1): 115-130. https://doi.org/10.1016/j.pmcj.2010.12.001
[7] Fergani, L., Fergani, B., Fleury, A. (2015). Improving human activity recognition in smart homes. International Journal of E-Health and Medical Communications (IJEHMC), 6(3): 19-37. https://doi.org/10.4018/IJEHMC.2015070102
[8] Majumder, S., Aghayi, E., Noferesti, M., Memarzadeh-Tehran, H., Mondal, T., Pang, Z., Deen, M.J. (2017). Smart homes for elderly healthcare—Recent advances and research challenges. Sensors, 17(11): 2496. https://doi.org/10.3390/s17112496
[9] Pulkkinen, T., Son, Y.S., Lee, J., Lee, Y.H., Sallinen, M., Park, J.H. (2013). Progressive monitoring and treatment planning of diabetes mellitus in smart home environment. In 2013 IEEE International Conference on Consumer Electronics (ICCE), 206-207. https://doi.org/10.1109/ICCE.2013.6486860
[10] Karakostas, A., Lazarou, I., Meditskos, G., Stavropoulos, T.G., Kompatsiaris, I., Tsolaki, M. (2015). Supporting cognitive skills of people suffering from dementia through a sensor-based system. In 2015 IEEE 15th International Conference on Advanced Learning Technologies, 460-461. https://doi.org/10.1109/ICALT.2015.129
[11] Forkan, A.R.M., Khalil, I., Tari, Z., Foufou, S., Bouras, A. (2015). A context-aware approach for long-term behavioural change detection and abnormality prediction in ambient assisted living. Pattern Recognition, 48(3): 628-641. https://doi.org/10.1016/j.patcog.2014.07.007
[12] Rabiner, L.R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2): 257-286. https://doi.org/10.1109/5.18626
[13] Forney, G.D. (1973). The viterbi algorithm. Proceedings of the IEEE, 61(3): 268-278. https://doi.org/10.1109/PROC.1973.9030
[14] David, S., Ferrer, M.A., Travieso, C.M., Alonso, J.B., y Comunicaciones, D.D.S. (2004). gpdsHMM: A hidden Markov model toolbox in the MATLAB environment. CSIMTA, Complex Systems Intelligence and Modern Technological Applications, 476-479.
[15] Welch, L.R. (2003). Hidden Markov models and the Baum-Welch algorithm. IEEE Information Theory Society Newsletter, 53(4): 10-13.
[16] Botros, M., Heskes, T., de Vries, I.A. (2017). Supervised Learning in Human Activity Recognition Based on Multimodal Body Sensing (Doctoral dissertation, Bachelor’s Thesis, Radboud University, Nijmegen, The Netherlands).
[17] Wong, J.K.W., Leung, J., Skitmore, M., Buys, L. (2017). Technical requirements of age-friendly smart home technologies in high-rise residential buildings: A system intelligence analytical approach. Automation in Construction, 73: 12-19. https://doi.org/10.1016/j.autcon.2016.10.007
