Ambient intelligence: Placement of Kinect sensors in the home of older adults with visual disabilities

Abstract

BACKGROUND:

Although a number of research studies on sensor technology for smart home environments have been conducted, there is still lack of consideration of human factors in implementing sensor technology in the home of older adults with visual disabilities.

OBJECTIVE:

This paper aims to advance knowledge of how sensor technology (e.g., Microsoft Kinect) should be implemented in the home of those with visual disabilities.

METHODS:

A convenience sample of 20 older adults with visual disabilities allowed us to observe their home environments and interview about the activities of daily living, which were analyzed via the inductive content analysis.

RESULTS:

Sensor technology should be integrated in the living environments of those with visual disabilities by considering various contexts, including people, tasks, tools, and environments (i.e., level-1 categories), which were further broken down into 22 level-2 categories and 28 level-3 categories. Each sub-category included adequate guidelines, which were also sorted by sensor location, sensor type, and data analysis.

CONCLUSIONS:

The guidelines will be helpful for researchers and professionals in implementing sensor technology in the home of older adults with visual disabilities.

Keywords

Older adults visual disability sensor home human factors

1. Introduction

Although a great amount of efforts have been made to manage healthcare costs, quality, and access in the United States and some improvements have been observed there are still some gaps to address. For example, healthcare costs keep rising as the US healthcare spending increased 4.6 percept (to reach $3.6 trillion) in 2018, i.e., a faster growth rate than the rate of 4.2 percent in 2017 [1]. A reactive healthcare approach was typically taken, e.g. healthcare professionals rely on patents to contact them when noticeable symptoms are found by patients; patients are passive recipients of interventions; and clinical visits are treatment-focused as opposed to patient-centered (i.e., holistic and root-cause care) [2]. In order to better manage the healthcare system, a proactive healthcare approach has recently gotten a lot more attention, e.g. patients are active partners in managing health conditions on a daily basis; and chronic conditions are prevented with promotion and disease prevention strategies that patients are allowed to navigate and control [2, 3]. Thus, it is important to monitor the daily living activities and assess how much his/her own activities are deviated from the norm.

There are a variety of technologies available (e.g., motion tracking sensors, networks, less invasive computing, and artificial intelligence) contributing to detecting accurately and collecting adequately different activities of daily living (e.g., gait characteristics), which is referred to as ambient intelligence that incorporates intelligence to our everyday lives and makes it sensitive and responsive to the presence of an individual [4]. For instance, Kinect is a Microsoft’s motion sensor add-on for the Xbox 360 gaming console. Engineers and scientists often use the Kinect sensor in monitoring and analyzing various human behaviors in natural settings. The Kinect sensor is equipped with a set of microphones, motion sensors, a color camera and a depth camera that emits a grid of infrared light [5]. The Kinect sensor calculates the distance between objects via time-of-flight analysis of reflected light beams. The Kinect sensor can detect an object in the distance of 0.5–4.5 meters and an angular field of view of 70 ${}^{\circ}$ horizontally and 60 ${}^{\circ}$ vertically. The software development kit (SDK) of the Kinect sensor can detect 25 anatomical landmarks (Kinect joints) of an individual and identify up to six individuals at once. The Kinect sensor can accurately detect audio input from $+$ and –50 degrees in front of the sensor. The Kinect’s microphone array can be pointed at 5 ${}^{\circ}$ increments within the 180 ${}^{\circ}$ range such that it can precisely recognize the incoming direction of sounds without removing other ambient noise. The Microsoft company has recently released a new Azure Kinect that is an upgraded version of Kinect v2 [6]. For example, Azure Kinect has a full 6-axis inertial measurement unit (IMU) while Kinect v2 provides 1-axis. The Kinect sensor technology has widely been used in different fields such as computer vision [7], 3D mapping [8], robotics [9], health [10], and human tracking [11, 12, 13].

A smart home concept is a good example of using such ambient intelligence technologies to facilitate daily living activities [14], leading to promotion of an individual’s quality of life through; for instance, a mobile emergency response system [15], a fall detection system [16], and a recommender system for promoting a healthy lifestyle [17]. Sensor technologies are also anticipated to contribute to collecting and assessing clinical data for dementia [18], abnormal sleep disorder [19] heart rate problems [20], and an early sign (or onset) of Alzheimer’s disease [21].

However, studies on ambient intelligence technologies are often introduced by targeting general populations over those with special needs (e.g., older adults with visual disabilities). For example, many studies on smart home environments tried to identify ideal locations to place sensors in the home [22, 23, 24, 25] by relying on mathematical models such as a Monte Carlo algorithm, a hill climbing algorithm, and a genetic algorithm computational modeling [22]. While a few research reports discussed the importance of integrating sensors in a human behavior monitoring system for vulnerable populations [26, 27, 28], they merely focused on older adults who do not have disabilities; for example, assistive robot hands to help older adults’ daily living activities [29], a stand-up robotic chair [30], and a fall detection alarm [31]. Technology designs suitable for sighted users, would not guarantee that they are also suitable to users with visual disabilities (e.g., visual impairments and blindness) because the two user groups are likely to have different performance capabilities, limitations, and preferences [32, 33, 34].

There is lack of consideration of the user-centered approach; that is, a systematic analysis of the end users’ living and working contexts, users’ preferences, tools users use, tasks users carry out on a daily basis, users’ capabilities and limitations that affect the system designs developments, and implementations [35, 36]. The technology that engineers develop will eventually be used by end users. Without in-depth consideration of the end users’ needs and concerns, any technology is likely to be abandoned by users, leading to becoming useless technology despite technical advancements. The user-centered approach should be integrated in the ambient intelligence and smart home by rigorous scientific methods – not simply relying on a computational modeling or a human common sense only. This paper aims to advance knowledge of how sensor technology (e.g., Microsoft Kinect) should be implemented in the home of people with disabilities, particularly older adults with visual disabilities.

2. Methods

A descriptive research design (involving interviews and observations in the field) was used to describe systematically and accurately the characteristics of various contexts of research participants without experimental manipulation or control of variables [37]. To ensure the visual acuity, each participant’s visual acuity was measured with a Snellen eye chart [38]. Approval for this study was obtained from the Institutional Review Board.

2.1 Participants

A convenience sampling method helped to recruit 20 older adults with visual disabilities who live in various towns across North Carolina in the United States. Individuals who met the following eligibility criteria were invited: (1) English speaking, (2) 65 years old or older, (3) community-dwelling, and (4) visual acuity levels worse than 20/70 [39]. Table 1 shows the participants’ characteristics.

Table 1
Characteristics of the participants

Participants	$n=$ 20
Visual acuity
Between 20/70 and 20/200	2
Between 20/200 and 20/400	11
Between 20/400 and 20/1200	1
Less than 20/1200, but has light perception	1
No light perception at all	5
Duration of visual impairments (years)	28.35 $\pm$ 23.04
Age (years)	72.85 $\pm$ 7.96
Gender
Male	4
Female	16
Race/ethnicity
Black or African American	12
White	8
Marital status
Married	6
Not married	4
Widow/widower	4
Divorced	6
Education
High school or equivalent	7
Bachelors	7
Masters	5
Doctorate	1
Occupation
Full time	1
Unemployed	6
Retired	13
Household income
$<$ $25,999	8
$26,000–$51,999	7
$52,000–$74,999	2
$\geqslant$ $75,000	2
Declined to say	1

2.2 Procedure

A researcher visited each participant’s home and conducted the interview and observation to obtain a deep understanding of their daily living contexts. It lasted approximately 60 minutes. The interview was carried out with semi-structured questions, e.g. “How do you walk in the home and outside?”, “Do you use any tools/aids for walking (e.g. a white cane, a service animal, personal help)?” “Are there any barriers or facilitators to your activities of daily living?”, “What can you tell us about environmental factors (e.g., sunlight, noise, location) and its impact on your daily living activities?”, “What is your daily routine?”, and “How do you think about having a sensor technology that keeps track of your activities of daily living in the home?” We also observed their living environments such as housing types, rooms, walking in the home, household goods, and interaction with household goods. The participants’ comments were audio-recorded to capture all details and transcribed for content analysis, and also the observation was recorded making notes (including quick sketches, see Fig. 1).

Figure 1.

Quick sketch map of a participant’s home, which was drawn from observation to show the main features of the area.

2.3 Data analysis

The inductive content analysis [40] with QSR International’s NVivo 11 software [41] helped to understand the interview and observation data via open coding, axial coding, and selective coding In the open coding phase, the raw data were sorted into several groups to interpret them. Detailed word-by-word and line-by-line analysis was accomplished by assigning an appropriate label to each sentence (or idea and concept), and they were then regrouped as needed. The axial coding contributed to regrouping and linking themes into each other in a rational manner. The last step was a selective coding that helped to select a primary theme and then relate it to other themes appropriately. Another coder was invited to assess the inter-rater reliability using Cohen’s kappa statistic. There was strong agreement among the raters as the inter-rater reliability was found to be $\kappa=$ 0.80 (95% CI: 0.596 to 1.004), $p<$ 0.001.

3. Results

The observation and interview data analysis produced in-depth insights into user-centered implementations of sensor technology (e.g., Kinect) in the home where people with visual disabilities live. As shown in Table 2, it is recommended that sensor technology be integrated in the living environments of those with visual disabilities by considering various contexts, including the following four categories: people, environment, tasks, and tools (i.e., level-1 categories). The level-1 categories were further broken down into 22 level-2 categories and 28 level-3 categories For example, the category of Tool has sub-categories such as white cane and sensor (distance from the sensor to the target, parallel connection

Table 2
Determinants for implementation of Kinect sensor technology in the home of older adults with visual disabilities

Categories			Contexts	Determinants of implementing sensors	Data- analysis	Sensor- location	Sensor- type
Tool	White cane	Us	Participants (i.e., older adults with visual disabilities) mostly used a white cane outside but would still carry it from a room to an entrance door when going outside. A Kinect-based monitoring system can be interfered with a walking aid in detecting and analyzing the human gait.	Avoid having the Kinect sensors aim at the entrance door when a study is set to monitor gait characteristics of an individual who walks without a white cane; Incorporate algorithms that can distinguish the gait data between people with and without the white cane	x	x
		Not us	Participants with moderate visual impairments relied on their residual vision or just relied on family members’ help while walking, which would result in different gait characteristics compared to others with severe visual impairments (or total blindness) and those living alone.	Analyze an individual’s gait data adaptively by referring to a variety of different conditions associated with visual acuity and personal assistant	x
	Sensor	Distance to the target	A Kinect sensor can cover the distance of 1.2 to 3.5 meters and an angular field of view of 57 ${}^{\circ}$ horizontally and 43 ${}^{\circ}$ vertically.	Understand the technical limitations of a Kinect sensor and place the sensor in the home by considering an individual’s living contexts and walking routes
		Parallel connection of sensors	If it is a large room, it may be too far for two sensors (placed on two opposite walls) to detect an individual walking in the middle of the room due to its technical limitations.	Avoid placing two sensors in parallel (facing each other) if the target room is too large		x
		Series connection of sensors	The Kinect sensor’s infrared laser is emitted in a fan-shape. When a sensor is placed to be adjacent to another sensor (i.e., series connection), a blind spot will be generated.	Avoid the placement of series connection that could generate a blind spot unless additional sensors would be available to cover all the blind spots; yet, that is inefficient implementation of sensor technology		x
Peopl	Emotio		Participants shared their experience with emotional challenges (e.g., lonely, depressed) followed by changes in his/her gait characteristics such as slower gait speed and shorter stride length.	Consider the relations between emotional challenges and walking characteristics, which should be reflected in analyzing the gait data and also consider a sensitive sensor technology that can detect subtle changes in their body moments	x		x
	Cognitio	Fall history & fear of falling	Participants (including those with a fall history) expressed fear of falling, which would result in abnormal gait characteristics (e.g., too much cautious or fearful walking, leading to lower gait velocity, higher stride length variability, and higher stride time variability).	Consider an individual’s fall history and fear of falling when analyzing the gait data as compared to their peers with no fall history and/or less fear of falling
		Cognitive maps	Participants who lived in the current home longer enough to form a clear cognitive map of their home felt comfortable walking in the home.	Consider the individual differences in developing a cognitive map of a physical area and relevant effects on his/her walking characteristics

Table 2, continued
Categories			Contexts	Determinants of implementing sensors	Data- analysis	Sensor- location	Sensor- type
	Privacy		With regard to the concept of smart home technology (e.g., sensors) and privacy, participants wanted to have a full control of the sensor technology; for example, by switching on and off whenever they want (i.e., user empowerment).	Provide an adequate means of controlling the monitoring system (e.g., on/off switch) (or avoid installing sensors in private places, e.g. bedrooms and bathrooms if feasible)
	Preference	Technology adoption	In relation to the implementation of the Kinect sensor technology in the home, participants were divided into early and late adopters. While the early adopters wanted to try the Kinect sensor to monitor their daily living activities, the late adopters were unsure about it yet	Refer to valid usability and accessibility standards (e.g., Web Content Accessibility Guidelines, WCAG) in designing, developing, and implementing technology to ensure ease of use, secure, safe, and accessible to users with disabilities			x
		Favorite place in the home	Participants tended to spend most of their time in a particular room or location (e.g., a particular sofa/table/chair) while staying at home during a day.	Identify the location where a target user tends to spend most of his/her time during a day and install the sensor to capture fully his/her daily living activities, i.e., rich data collection		x
		Monitoring the house	As participants struggle with the visual disability, they would like to have a sensor technology system that helps to take care of health but also manage the home.	Consider the opportunity to monitor a target user’s daily living activities but also home environments (e.g., notifications for when a door is left open/unlocked, which should notify a user via alternative formats – sound and haptic/tactile)			x
	Visitor/cohabitant		Participants had regular/occasional visitors (e.g., friends, social workers, family) and/or lived with sighted cohabitants (e.g., other family members and friends in a relationship)	Include algorithms in a system to distinguish the target individual from others in the house (e.g., gait biometrics)	x
Environment	Wall	Bump into wall	Participants typically used their hands and feet to scan their surroundings to detect obstacles or orientation-marks as they were less likely to use a white cane in the home. Yet, they often bumped into walls, which would increase the risk of falling	Avoid installing the Kinect sensors on/near the walls that those with visual disabilities use as their orientation-marks while walking; Consider it that the gait data may show abnormal gait patterns due to bumping into walls although they have no health problems, which should be reflected accordingly in data analysis	x	x
		Walls in small homes	Participants living in a small studio apartment showed that the walking routes and patterns tended to be short, non-linear, and/or discontinuous (e.g., frequently stop and go). Their small studio apartment does not have a divider/wall to separate a bedroom from a living area.	Analyze the gait data by differing walking conditions such as walking in a wide-open area (e.g., a wide living room, a straight hallway) versus a small area with many household goods; Set the Kinect sensors to aim at the target zone that does not include a bed area	x	x
	Home size		Participants lived in various residence types and sizes – ranging from a studio apartment to a house with multi-stories, which will affect one’s activities of daily living.	Determine the number of sensors by comprehensively considering the target user’s living environments. Yet, a larger number of sensors do not always result in great data accuracy as the sophisticated algorithms with a smaller number of sensors could also lead to a great performance.		x

Table 2, continued

Categories

Contexts

Determinants of implementing sensors

Data- analysis

Sensor- location

Sensor- type

Location

Participants often found that his/her family members rearranged household goods (e.g., furniture) such that they often hit the household goods that were located in their routinized walking paths. His/her walking area should be free from clutter.

Avoid placing the Kinect sensor in the area that interferes with his/her walking paths

Too much

stuff in the

home

A house/room was often crammed full of household goods, and the household goods could block the view of sensors. As a user-centered design approach, the household goods should not be removed or relocated intentionally for the sensor system.

Find a less-crowded place in the home to install the Kinect sensor such that the sensor could have a clear view

Interfered with

pets & service animals

Pets (including a service dog) walked on the floors as well as climbed furniture (e.g., chair, desk, sofa).

Install the Kinect sensors in a secure place to avoid any interference with dogs/cats chewing or blocking the view of sensors; Include algorithms to distinguish the gait data between humans and dogs/cats

Noisy place in the house

Participants experienced that their walking routes were affected by noisy sound, e.g., a participant lived with his cousin who made a loud noise in his room by playing musical instruments such that the participant occasionally changed his walking paths away from the noisy room.

Install the Kinect sensors by comprehensively considering his/her living environments (e.g., noisy place) affecting their daily living activities such as walking paths

External environment

External environment factors such as weather could affect the activities of daily living, e.g., participants spent most of their time on the porch in the summer as compared to indoors

Install the Kinect sensors by considering outdoor environments (i.e., individual preferences)

Task

Activities at home

While staying at home, participants kept themselves busy with various activities of daily living (e.g. watching TV, conversation on the phone, reading books, and other tasks) that would occur in different places, at different times, and under different circumstances.

Install the Kinect sensors to be aligned with his/her daily activities’ routines, locations and timelines

Exercise

In the home

Participants did occasionally or regularly exercise at home by using a variety of equipment (e.g., a treadmill, a stationary bike). If they do exercise within the detection zone of the Kinect sensors, it may appear to be that he/she is walking abnormally in the home.

Avoid setting Kinect sensors to focus on the exercise equipment (e.g., treadmill) unless the sensors aim to monitor exercise activities. Alternative option to consider will be to set up a separate Kinect sensor aiming solely at the exercise equipment if it is necessary to monitor exercise activities.

Table 2, continued
Categories			Contexts	Determinants of implementing sensors	Data- analysis	Sensor- location	Sensor- type
		Outsid	Although participants were retired, they were still active and regularly participated in social events and physical exercise programs at YMCA and senior centers.	Find ideal timeframes for monitoring by considering his/her daily living activities outside; or enable a user to control the monitoring system through an on/off switch	x		x
	Cooking		Participants tended to avoid cooking due to their vision loss.	Avoid installing Kinect sensors in the kitchen for those who rarely or never use the kitchen		x
	Walking	Walking with visual impairment	Participantscounted their steps and constantly looked down by using his/her parallel vision while walking. The gait characteristics of an individual may be different from that of others who have different visual acuity levels.	Consider individual differences in gait characteristics,which should be reflected in analyzing his/her gait data
		Parallel paths	Participants established his/her walking paths that they frequently used while walking in the house.	Identify places where the Kinect sensors can cover as many walking paths as possible, leading to rich data collection
		Contact (crossed) points of path	The walking paths were often crossed at some points in the house; for example, the living room is a place where an individual frequently transitions between rooms.	Identify places where many walking paths are crossed, which would be good locations for the Kinect sensors to capture the gait data
		Favorite paths	Although participants were aware that there were multiple walking paths leading to a particular place/room in the house, they tended to use his/her favorite path(s) (e.g., a shorter path, a more convenient path, a safer path, and so on).	Identify his/her favorite path(s) and accordingly install the Kinect sensors to monitor their gait data
		Gait direction	There is ample evidence in literature to suggest that a Kinect sensor can accurately monitor gait characteristics when the sensor is set to aim at the lengthways of his/her gait direction as compared to crossways.	Set the Kinect sensor to be aligned with the gait direction (i.e., lengthways)

of sensors, and series connection of sensor). The category of People has sub-categories including emotion, cognition (fear of falling and cognitive maps), privacy, preferences (technology adoption, favorite place in the home, monitoring the house), and visitors/cohabitants. The category of Environment has sub-categories such as walls (bump into walls and walls in small homes), home size, stairs, home adjustment, doors (door alignment, door swing direction, doorway without a door, and entrance door), hallway (narrow hallway and short hallway), kitchen, household goods (tables, chairs, location, and too much stuff in the home), interference with pets and service animals, noisy place in the house, and external environment. The category of Task is further broken down into the following sub-categories: activities in the home, exercise (in the home and outside), cooking, walking (walking with visual impairments, parallel paths, contact points of paths, favorite paths, and walking direction) Each subcategory was given adequate guidelines, which were also sorted by sensor location, sensor type, and data analysis. It should be noted that the design guidelines were developed by targeting older adults (age 65+) who have visual impairments or blindness, a video-based sensor technology (e.g., Microsoft Kinect), and gait data collection and analysis in the home.

4. Discussion

As compared to general populations, less attention has been paid to older adults with visual disabilities with regard to implementations of smart sensors in the home (e.g., ambient intelligence and smart home). This study has conducted contextual inquiries (i.e., observations and interviews in participants’ home) to advance knowledge of the target user’s living environments, needs and concerns about placing sensors in his/her home to keep track of daily living activities. This study suggested a set of guidelines that would be useful for researchers and professionals in determining on how to implement sensor technology (e.g., sensor type, data collection and analysis) for older adults with visual disabilities in the home. The guidelines were constructed by comprehensively reviewing various factors such as tools, people, environment, and tasks.

As previous studies in literature have been found to support the results of this study (i.e., the guidelines), we discussed further the guidelines by referring to previous studies. For example, participants in this study (i.e., older adults with visual disabilities) perceived that they often felt lonely while staying at home. Besides a decrease in lower extremity muscle mass and muscle strength, such human emotion as a feeling of loneliness can cause an individual to change his/her gait characteristics [42, 43, 44], such as slower gait speed, shorter stride length and lower cadence, which should be reflected when analyzing the gait data. Yet, the changes in gait characteristics – induced by emotion – may occur in certain contexts only (e.g., a particular location or time) and they may go back to normal in different contexts [45, 46]. The gait data of older adults with visual disabilities should be analyzed adaptively by considering such various conditions. As lonely people may have a limited range of movement [47], a sensitive sensor technology (e.g., micro-scale motion sensing technologies [48, 49]) may be used to detect and distinguish the subtle changes in the body movements of older adults with visual disabilities.

Participants mentioned that they typically spend most of their time in the home instead of going outside due to retirement or vision loss. It is well documented that people with visual disabilities tend to encounter outdoor mobility restrictions [50] in that 35% of adults with visual disabilities are likely to decrease the frequency of socializing after vision loss and 47% depend on other people [51]. A Kinect sensor may not aim at the entrance door as they are less likely to use the entrance door compared to other locations in the home.

Participants showed different techniques to get around safely by preventing from tripping and falling even when they were in the home. For example, they tended to count their steps or constantly look down by using his/her parallel vision while walking. However, individual differences within the group of people with visual disabilities should be accordingly reflected in the gait data analysis. For instance, an empirical gait study [52] found that the walking characteristics (e.g., gait speed, stride length, time of stance phase, and time of swing phase) would vary, depending on different visual status (e.g., late blind and congenitally blind) Participants used their hands and feet to scan their surroundings for obstacles or orientation marks; yet, it was observed that participants often bumped into walls although it is their home. Various installation kits (e.g., poles, tripods, or stands) are typically used to install the Kinect sensors on/near the walls, but the sensors should not be placed on/near the walls that older adults with visual disabilities are likely to bump into or use as their orientation marks. The gait data should also be carefully analyzed by considering it that the gait data may show abnormal gait patterns due to bumping into walls although they have no health problems.

When participants walked on stairs, especially with too short tread depth for adults or with a little darker or no lighting, they tended to experience a fear of falling and thus change their gait to prevent from falls It is well documented that walking on stairs is among the most challenging and hazardous activities of daily living for older adults, leading to a fear of falling and changes in walking patterns [53, 54]. According to the Stair Behavior Model [55], an individual walking on stairs is likely to perform a series of information process tasks, i.e. initial conceptual scan for sensory input, detection of hazards, choice of route, visual perception of step location, and continuous monitoring scans. Any interruption of these processes puts the individual at increased risk of falling. As people with visual disabilities are significantly affected by the lack of visual input, he/she would result in the information processing interruption such that they are likely to increase the risk of falling while walking on stairs and thus, change their walking characteristics to prevent from falling. The walking changes should be considered in analyzing the gait data if the sensors are needed to be located near/on the stairs.

Participants had a set of walking paths to frequently use while walking in the home, which is likely to be routinized. The Kinect sensors could be installed in a place where the sensors can cover as many walking paths as possible, leading to rich data collection. The walking paths are often crossed at some points in the house (e.g., a living room), which would also be good locations for the Kinect sensors to aim. Even though there were multiple walking paths leading to a certain place/room in the house, participants tended to use his/her favorite paths (e.g., a shorter path, a more convenient path, a safer path, and so on). It is recommended to identify his/her favorite path(s) and accordingly install the Kinect sensors to monitor the gait. According to an empirical study with sighted participants [56], a Kinect sensor can accurately monitor gait characteristics when the sensor is set to aim at the lengthways of walking direction as compared to crossways. It would, thus, be ideal for the Kinect sensor to be aligned with the walking direction (i.e., lengthways) of people with visual disabilities.

Participants stated that after experiencing a fall, they tended to pay much more attention to their walking and surroundings, ultimately affecting their gait patterns. There is ample evidence to suggest that a history of falls can cause fear of falling, which is likely to result in abnormal gait characteristics (e.g., too much cautious or fearful walking, leading to lower gait velocity, higher stride length variability, and higher stride time variability) [57, 58, 59, 60]. People with visual disabilities are already overwhelmed with their vision loss and likely to adjust their gait patterns to prevent from bumping into objects and people. The fear of falling caused by the fall history would cause them to change their walking characteristics furthermore, which should carefully be reflected in analyzing the gait data of people with a history of a fall as compared to people with no history of a fall, especially among people with visual disabilities.

Participants took advantage of a home modification [61] (i.e. alteration made to a home to meet the needs of people with disabilities) such that they can live independently and safely. While the home modification can positively affect the activities of daily living [62], it could ironically interfere with the Kinect sensors. Participants in this study installed handrails on the walls to prevent from falls, which may change the gait characteristics as they walk closely with the walls The gait data should be analyzed by considering the two different conditions, i.e. walking with and without handrails. Furthermore, the home modification may include extra overhead lighting and a larger mirror [62] which could make a strong glare that makes it hard for the Kinect sensor to identify an individual or keep track of an individual’s movements [63, 64]. The Kinect sensor should avoid aiming at the walls equipped with mirrors or highly reflective objects.

A group of participants were willing to adopt and install the Kinect sensor in their home to monitor various activities of daily living to improve the quality of life; on the other hand, another group of participants were unsure about it in that they were hesitant to even replace their conventional technology applications (e.g., a flip phone) with a new one (e.g., a smartphone). In literature, users with visual disabilities are anticipated to adopt a new technology if it is ease of use, secure, safe, and designed for people with special needs [65, 66, 67], which should be referred in designing, developing, and implementing the Kinect sensor technology in the home for those with visual disabilities.

The sensor placement could be affected by not only humans but also animals in the home. Participants lived with a service dog (and/or pets) in the home. Those animals could climb furniture (e.g., chair, desk, sofa) so that the Kinect sensors should carefully be placed in the home to avoid any interference with those animals chewing or blocking the sensors. The Kinect technology is also recommended to be equipped with algorithms that can distinguish the behavioral data of humans from that of animals. A research team by Munsell et al. [68] introduced a person identification algorithm by using human full-body motion and anthropometric biometrics, acquired from Kinect sensors. It can help to distinguish a person from others but also two people who have similar full body motion characteristics. Additional research is needed to explore the quantitative relationship between gait parameters measured by the Kinect system in home environments and those measured by the conventional, clinical assessment in a controlled setting. The research team by Stone et al. [69] found that the conventional measures for gait parameters tended to show significantly greater variation from month to month than the Kinect-based measures. The variation would be induced by intraindividual variation between sessions of measurements [70, 71]; for example, changes in activities of daily living (e.g., a very slow gait speed) may be unrelated to a “true” change in one’s physical function but merely reflect normal variation or noise data in the measurement [69]. When a chair is located between a faller and the Kinect sensor, the Kinect sensor could consider the chair as part of the faller, leading to miscalculating a person’s fall risk. Other behavioral examples (leading the Kinect sensor to inaccurate evaluation for falls) are the situations where a person performs certain yoga poses on the floor; kids drop to the floor while playing; and pets jump down from furniture. A sophisticated tracking and analysis algorithm is recommended to avoid triggering a false alarm [72]. A personalized data analytics approach may contribute to adequately determining the optimal threshold for an individual at risk by considering his/her own contexts. For example, a research team by Skubic et al. [73] developed a fall alert system for older adults at home, which was equipped with a personalized algorithm that was developed via retrospective analysis and collaboration with clinicians. They kept monitoring an older adult and reviewed his/her emergency room visits, hospitalizations, and fall histories, which were compared with home sensor data. Potential algorithms were iteratively tested, and adequate thresholds were then determined for the individual.

Biometrics can help to make the Kinect system applicable to multiple people in that a single Kinect sensor can monitor multiple users and analyze each individual based on his or her anatomical and/or behavioral traits (i.e., biometric algorithms) even when they enter the detection zone at the same time [74]. A range of Kinect-based interventions offer various benefits, such as relevant feedback in real time to a person performing a physical exercise [75]; classification of Parkinson’s disease stages based on on’s own gait characteristics [76]; real-time fall alerts via an acoustic fall detection system, FADE [77] or infrared sensors [78]; in-home self-assessment of fall risk [79]; and real-time monitoring of facial expressions of emotions to inform a caregiver online [80].

5. Conclusion

Although a variety of research studies on sensor technology for smart home environments have been conducted, there is still lack of consideration of user-centered approach in implementing sensor technology in the home, especially for older adults with visual disabilities. This study conducted observations and interviews to advance knowledge of human factors-based implementations of sensor technology (e.g., Kinect) in the home of people with visual disabilities. It is recommended that sensor technology be integrated in the living environments by considering various contexts of those with visual disabilities, such as people, environment, tasks, and tools, each of which was further broken down into several sub-categories. Each sub-category included adequate guidelines, which were also sorted by sensor location, sensor type, and data analysis. The guidelines will be helpful for researchers and professionals in designing, developing, and installing sensors in the home to be usable, safe, and accessible to older adults with visual disabilities.

Footnotes

Acknowledgments

This material is based upon work supported by the National Science Foundation under Grant No. 1831969.

Conflict of interest

The author has no conflicts of interest to report.

References

Hartman

, et al., National Health Care Spending In 2018: Growth Driven By Accelerations In Medicare And Private Insurance Spending. Health Affairs. 2019; 39(1): 101377/hlthaff. 2019.01451.

Health

Care

. Preventing and Managing Chronic Disease: Ontario’s Framework. 2007 [cited 2020 January 10]; Available from: http//www.health.gov.on.ca/en/pro/programs/cdpm/pdf/framework_full.pdf.

National Clinical Guideline Centre for Acute Chronic Conditions, Patient Experience in Adult NHS Services: Improving the Experience of Care for People Using Adult NHS Services: Patient Experience in Generic Terms. 2012: National Clinical Guideline Centre at The Royal College of Physicians.

Debes

, et al., Monitoring activities of daily living in smart homes: Understanding human behavior. IEEE Signal Processing Magazine. 2016; 33(2): 81-94.

Microsoft KINECT for Windows Human Interface Guidelines (v2.0). 2020.

Microsoft. Azure Kinect DK Document. 2020 [cited 2020 October 1]; Available from: https//docs.microsoft.com/en-us/azure/kinect-dk/.

Eric

Jang

. Kinect depth sensor for computer vision applications in autonomous vehicles. In 2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN). 2017.

Khoshelham

Elberink

. Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors. 2012; 12(2): 1437-1454.

Afthoni

Rizal

Susanto

. Proportional derivative control based robot arm system using Microsoft Kinect. In 2013 International Conference on Robotics, Biomimetics, Intelligent Computational Systems. 2013. IEEE.

10.

Tahavori

Alnowami

Wells

. Marker-less respiratory motion modeling using the Microsoft Kinect for Windows. In Medical Imaging 2014: Image-Guided Procedures, Robotic Interventions, and Modeling. 2014. International Society for Optics and Photonics.

11.

Southwell

Fang

, Human object recognition using colour and depth information from an RGB-D Kinect sensor. International Journal of Advanced Robotic Systems. 2013; 10(3): 171.

12.

Amon

Fuhrmann

Graf

. Evaluation of the spatial resolution accuracy of the face tracking system for kinect for windows v1 and v2. In Proceedings of the 6th Congress of the Alps Adria Acoustics Association. 2014. Australian alps.

13.

Wei

Jia

Chen

. Real-time facial expression recognition for affective computing based on Kinect. In 2016 IEEE 11th Conference on Industrial Electronics and Applications (ICIEA). 2016. IEEE.

14.

García Hernando

de la Cruz

. The elderly’s independent living in smart homes: A characterization of activities and sensing infrastructure survey to facilitate services development. Sensors. 2015; 15(5): 11312-11362.

15.

Lloret

, et al., A smart communication architecture for ambient assisted living. IEEE Commun. Mag, 2015; 53(1): 26-33.

16.

Chen

Brown

. Hidden Markov Model-Based Fall Detection With Motion Sensor Orientation Calibration: A Case for Real-Life Home Monitoring. IEEE Journal of Biomedical and Health Informatics. 2018; 22(6): 1847-1853.

17.

Ali

, et al. KARE: A hybrid reasoning approach for promoting active lifestyle. In Proceedings of the 9th International Conference on Ubiquitous Information Management and Communication. 2015. ACM.

18.

Moore

, et al., Detection of the onset of agitation in patients with dementia: real-time monitoring and the application of big-data solutions. International Journal of Space-Based and Situated Computing. 2013; 3(3): 136-154.

19.

Ravichandran

, et al. Making Sense of Sleep Sensors: How Sleep Sensing Technologies Support and Undermine Sleep Health. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2017. ACM.

20.

Elliot

Hamlin

Lizamore

. Validity and reliability of the Hexoskin® wearable biometric vest during maximal aerobic power testing in elite cyclists. Journal of Strength and Conditioning Research. 2017.

21.

Lyons

, et al., Pervasive computing technologies to continuously assess Alzheimer’s disease progression and intervention efficacy. Frontiers in Aging Neuroscience. 2015; 7: 102.

22.

Thomas

Crandall

Cook

. A Genetic Algorithm approach to motion sensor placement in smart environments. Journal of Reliable Intelligent Environments. 2016; 2(1): 3-16.

23.

Dawadi

Cook

Schmitter-Edgecombe

. Smart home-based longitudinal functional assessment. In Proceedings of the 2014 ACM international joint conference on pervasive and ubiquitous computing: Adjunct publication. 2014. ACM.

24.

Synnott

Nugent

Jeffers

. Simulation of smart home activity datasets. Sensors. 2015; 15(6): 14162-14179.

25.

Cook

, et al., CASAS: A smart home in a box. Computer. 2013; 46(7): 62-69.

26.

Wolbring

Leopatra

. Sensors: views of staff of a disability service organization. Journal of Personalized Medicine. 2013; 3(1): 23-39.

27.

Shi

. Recent Advances of Sensors for Assistive Technologies. Journal of Computer and Communications. 2015; 3(5): 80.

28.

Totter

Bonaldi

Majoe

. A human-centered approach to the design and evaluation of wearable sensors-Framework and case study. In Pervasive Computing and Applications (ICPCA), 2011 6th International Conference on. 2011. IEEE.

29.

Chen

Tee

Chew

C-M

. Assistive grasping in teleoperation using infra-red proximity sensors. In RO-MAN, 2013; IEEE. 2013. IEEE.

30.

, et al. A survey on assistive chair and related integrated sensing techniques. In Manipulation, Manufacturing and Measurement on the Nanoscale (3M-NANO), 2013 International Conference on. 2013. IEEE.

31.

Cagnoni

, et al. Sensor fusion-oriented fall detection for assistive technologies applications. In ISDA’09. Ninth International Conference on Intelligent Systems Design and Applications. 2009. IEEE.

32.

Pigeon

Marin-Lamellet

. Evaluation of the attentional capacities and working memory of early and late blind persons. Acta Psychol (Amst). 2015; 155: 1-7.

33.

Dormal

, et al., Early but not late blindness leads to enhanced arithmetic and working memory abilities. Cortex. 2016; 83: 212-221.

34.

McDonald

Rodrigues

. Sighted and visually impaired students’ perspectives of illustrations, diagrams and drawings in school science. Wellcome Open Research. 2016; 1.

35.

Hendrick

Kleiner

. Macroergonomics: An introduction to work system design (HFES issues in human factors and ergonomics book series volume 2). 2001.

36.

Kleiner

, et al., Sociotechnical attributes of safe and unsafe work systems. Ergonomics. 2015; 58(4): 635-649.

37.

Dulock

. Research design: Descriptive research. Journal of Pediatric Oncology Nursing. 1993; 10(4): 154-157.

38.

Falkenstein

, et al., Comparison of Visual Acuity in Macular Degeneration Patients Measured with Snellen and Early Treatment Diabetic Retinopathy Study Charts. Opthalmology. 2007; 115(2): 319-323.

39.

World Health Organization, Change the definition of blindness. 2008.

40.

Strauss

Corbin

. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. 1990, Newbury Park, CA Sage.

41.

QSR International

Pty Ltd.

,NVivo qualitative data analysis software, 2015.

42.

Naugle

, et al., Emotional state affects gait initiation in individuals with Parkinson’s disease. Cognitive, Affective, & Behavioral Neuroscience. 2012; 12(1): 207-219.

43.

Randhavane

, et al., Identifying emotions from walking using affective and deep features. arXiv preprint arXiv1906.11884, 2019.

44.

Ray

, et al., Clinical Assessment of Functional Movement in Adults with Visual Impairments. Journal of Visual Impairment & Blindness. 2007; 101(2): 108-113.

45.

Kuppens

Verduyn

. Looking at emotion regulation through the window of emotion dynamics. Psychological Inquiry. 2015; 26(1): 72-79.

46.

Liu

, et al., Intrinsic emotional fluctuation in daily negative affect across adulthood. The Journals of Gerontology: Series B. 2016; 73(1): 100-112.

47.

Perissinotto

Stijacic Cenzer

Covinsky

. Loneliness in older persons: a predictor of functional decline and death. Arch Intern Med. 2012; 172(14): 1078-83.

48.

Yonetani

Kitani

Sato

. Recognizing micro-actions and reactions from paired egocentric videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.

49.

Zeng

Zhao

. Sensing movement: Microsensors for body motion measurement. Sensors. 2011; 11(1): 638-660.

50.

Lamoureux

Hassell

Keeffe

, The determinants of participation in activities of daily living in people with impaired vision. American Journal of Ophthalmology. 2004; 137(2): 265-270.

51.

Boerner

Wang

S-W

Cimarolli

. The impact of functional loss: nature and implications of life changes. Journal of Loss and Trauma. 2006; 11(4): 265-287.

52.

Nakamura

. Quantitative analysis of gait in the visually impaired. Disabil Rehabil. 1997; 19(5): 194-7.

53.

Startzell

, et al., Stair negotiation in older people: a review. Journal of the American Geriatrics Society. 2000; 48(5): 567-580.

54.

Reed-Jones

, et al., Vision and falls: a multidisciplinary review of the contributions of visual impairment to falls among older adults. Maturitas. 2013; 75(1): 22-28.

55.

Templer

, The staircase: studies of hazards, falls, and safer design. 1995: MIT press.

56.

Stone

Skubic

. Evaluation of an inexpensive depth camera for passive in-home fall risk assessment. In Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2011 5th International Conference on. 2011. Ieee.

57.

Brach

, et al., Too much or too little step width variability is associated with a fall history in older persons who walk at or near normal gait speed. Journal of Neuroengineering and Rehabilitation. 2005; 2(1): 21.

58.

Ayoubi

, et al., The influence of fear of falling on gait variability: results from a large elderly population-based cross-sectional study. Journal of Neuroengineering and Rehabilitation. 2014; 11(1): 128.

59.

Wang

, et al., Differences between gait on stairs and flat surfaces in relation to fall risk and future falls. IEEE Journal of Biomedical and Health Informatics. 2017; 21(6): 1479-1486.

60.

Reelick

, et al., The influence of fear of falling on gait and balance in older people. Age and Ageing. 2009; 38(4): 435-440.

61.

Swenor

, et al., Evaluation of the Home Environment Assessment for the Visually Impaired (HEAVI): an instrument designed to quantify fall-related hazards in the visually impaired. BMC Geriatrics. 2016; 16(1): 214.

62.

Cho

, et al., Accessible home environments for people with functional limitations: a systematic review. International Journal of Environmental Research and Public Health. 2016; 13(8): 826.

63.

Microsoft. Troubleshoot body tracking. 2018 [cited 2018 November 1]; Available from: https//support.xbox.com/en-BZ/xbox-360/kinect/body-tracking-troubleshoot.

64.

Lachat

, et al., Assessment and calibration of a RGB-D camera (Kinect v2 Sensor) towards a potential use for close-range 3D modeling. Remote Sensing. 2015; 7(10): 13070-13097.

65.

Korwatanasakul

. Factors influencing technology adoption of people with visual impairment: case study of financial transactions through an automatic teller machine (ATM). Kasetsart Journal of Social Sciences, 2018.

66.

Loiacono

Djamasbi

Kiryazov

. Factors that affect visually impaired users’ acceptance of audio and music websites. International Journal of Human-Computer Studies. 2013; 71(3): 321-334.

67.

Lee

H-N

Lim

S-H

Kim

J-H

. UMONS: Ubiquitous monitoring system in smart space. IEEE Transactions on Consumer Electronics. 2009; 55(3): 1056-1064.

68.

Munsell

, et al., Person identification using full-body motion and anthropometric biometrics from kinect videos. In European Conference on Computer Vision. 2012. Springer.

69.

Stone

Skubic

. Unobtrusive, Continuous, In-Home Gait Measurement Using the Microsoft Kinect. IEEE Transactions on Biomedical Engineering. 2013; 60(10): 2925-2932.

70.

Bohannon

Schaubert

. Long-term reliability of the timed up-and-go test among community-dwelling elders. Journal of Physical Therapy Science. 2005; 17(2): 93-96.

71.

Nordin

Rosendahl

Lundin-Olsson

, Timed “Up & Go” test: reliability in older people dependent in activities of daily living – focus on cognitive state. Physical Therapy. 2006; 86(5): 646-655.

72.

Stone

Skubic

, Fall detection in homes of older adults using the Microsoft Kinect. IEEE Journal of Biomedical and Health Informatics. 2015; 19(1): 290-301.

73.

Skubic

Guevara

Rantz

. Automated health alerts using in-home sensor data for embedded health assessment. IEEE Journal of Translational Engineering in Health and Medicine. 2015; 3: 1-11.

74.

Gavrilova

, et al., Kinect sensor gesture and activity recognition: new applications for consumer cognitive systems. IEEE Consumer Electronics Magazine. 2017; 7(1): 88-94.

75.

Chiang

A-T

, et al., Kinect-based in-home exercise system for lymphatic health and lymphedema intervention. IEEE Journal of Translational Engineering in Health and Medicine. 2018; 6: 1-13.

76.

Dranca

, et al., Using Kinect to classify Parkinson’s disease stages related to severity of gait impairment. BMC Bioinformatics. 2018; 19(1): 471.

77.

, et al. Improvement of acoustic fall detection using Kinect depth sensing. In 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 2013. IEEE.

78.

Mastorakis

Makris

. Fall detection system using Kinect’s infrared sensor. Journal of Real-Time Image Processing. 2014; 9(4): 635-646.

79.

Ejupi

, et al., Kinect-based five-times-sit-to-stand test for clinical and in-home assessment of fall risk in older people. Gerontology. 2016; 62(1): 118-124.

80.

Khanal

, et al. Using emotion recognition in intelligent interface design for elderly care. In World Conference on Information Systems and Technologies. 2018. Springer.

Ambient intelligence: Placement of Kinect sensors in the home of older adults with visual disabilities

Abstract

BACKGROUND:

OBJECTIVE:

METHODS:

RESULTS:

CONCLUSIONS:

Keywords

1. Introduction

2. Methods

2.1 Participants

Table 1 Characteristics of the participants

3. Results

Table 2 Determinants for implementation of Kinect sensor technology in the home of older adults with visual disabilities

5. Conclusion

Footnotes

Acknowledgments

Conflict of interest

References

Table 1
Characteristics of the participants

Table 2
Determinants for implementation of Kinect sensor technology in the home of older adults with visual disabilities