Extending the Passive-Sensing Toolbox: Using Smart-Home Technology in Psychological Science

Abstract

New smart-home devices provide the opportunity to advance psychological science and theory through novel research opportunities in home environments. These technologies extend the in vivo research and intervention capabilities afforded by other assessment techniques such as Ecological Momentary Assessment methods as well as mobile and wearable devices. Smart-home devices contain a multitude of sensors capable of continuously and unobtrusively collecting multimodal data within home contexts. These devices have some complementary strengths and limitations compared with other assessment methods. This article (a) briefly reviews data collection methods in home environments, (b) discusses the unique advantages of smart-home devices, (c) describes the extant smart-home literature, (d) explores how these devices may advance evaluation and refinement of psychological theories, (e) describes examples of psychological processes that are potential targets for smart-home assessment and intervention, (f) considers methodological challenges and barriers, (g) discusses ethical considerations, and (h) concludes with a discussion of future directions for research and the merging of passive-sensing technologies with active self-report methods. This article aims to highlight the potential utility of smart-home devices within psychological research to evaluate psychological theories related to behavior within the home context.

Keywords

digital sensors passive sensing behavioral observation smart home

Humans spend considerable time in their home environments, which are arguably the location of many of the most meaningful, profound, and distressing aspects of humans’ lived experience. For many people, the home is the location for their closest and most intimate relationships. For example, it is where they raise children, eat together, sleep, mate, argue, relax, and recover. However, aside from an older literature on direct human observation in the home (with all the attendant inefficiencies one might imagine), the ability of psychological scientists and clinicians to directly observe behavior and intervene in the home has been limited by methodological and logistical barriers. However, this may soon change. The widespread commercial availability of Internet-connected sensors in the home creates significant opportunities (and some dangers) that psychological scientists should consider carefully. Our review is theoretical and prospective with the goal of elucidating how smart-home technology can be leveraged to further psychological science and theory.

It is estimated that by 2020, 50 billion devices will be connected to the Internet (Anderson & Rainie, 2012), and by 2021, 60.7% of households in the United States will contain smart-home technologies (Statista, 2017). This rapid expansion and adoption of these consumer technologies suggests that these devices will soon become ubiquitous (especially in high-income countries), as have smartphones, wearable devices (e.g., wrist wearables that track actigraphy and/or heart rate), wearable cameras (e.g., GoPro cameras), and augmented reality devices (e.g., Google Glass), which have recently been leveraged for psychological research and interventions (Harari et al., 2016; Hernandez & Picard, 2014; Insel, 2017; Mehl, 2017; Miller, 2012).

Research in the computer and medical sciences has already begun to use smart-home technology as a means of collecting and validating data on human behavior, but this has largely been limited to assessing device functioning and cognitive health in elderly populations. Through this process, important information on the opportunities, challenges, and technological limitations of using this technology has being revealed. The primary aim of this review is to call attention to smart-home technology and elucidate the ways in which it can work in tandem with current active and passive ecological assessment methods to provide continuous observational multimodal data on individual and relational functioning, specifically within home environments.

Current Methods for Indexing Psychological and Behavioral Health in the Home

Traditional assessment of affective and behavioral functioning in individuals and families relies almost entirely on questionnaires, self-report interviews, or laboratory-based measurements. Although each of these approaches has important strengths, they are also (like all methods) subject to limitations. Some of these include reporter bias, lack of ecological validity, Hawthorne effects, memory limitations, and incomplete assessment of affect and behavior (Baumeister, Vohs, & Funder, 2007; Furr, 2009; Gosling, John, Craik, & Robins, 1998; Harari et al., 2016; McCambridge, Witton, & Elbourne, 2014; Miller, 2012; Reis & Gosling, 2010). Moreover, many methods of assessment have not fundamentally changed in research for more than 40 years. More recently, the Center on Everyday Lives of Families at the University of California, Los Angeles, has used live observers with hand-held computers to record the location and activity of subjects in videos during 10-min intervals both during the work week and on weekends (Graesch, Mittmann, Bradbury, & Repetti, 2006; Ochs, Shohet, Campos, & Beck, 2010; Saxbe, Repetti, & Graesch, 2011; Wang, Repetti, & Sears, 2013), although the necessity to use human observers makes such techniques expensive and time consuming.

Recent technological developments have become permissive of active intensive-repeated-measure (IRM) designs, which collect multiple time points of data using methods such as daily diaries (Repetti, Reynolds, & Sears, 2015), and ecological momentary assessment (EMA; Shiffman, Stone, & Hufford, 2008); these methods have been used to provide greater temporal resolution of data collection within an individual’s daily life—including their home setting (for review, see Repetti et al., 2015). Indeed, these newer methods do address some of the methodological limitations mentioned above. In addition, the recent incorporation of passive electronic recordings (Mehl, 2017), as well as smartphone and wearable devices (Harari et al., 2016), into psychological research has allowed for truly passive sensing (i.e., requiring no effort on the part of the participant) of behaviors during individuals’ daily activities. Smart-home technology has the capacity to further extend the toolbox of active IRM and passive-sensing techniques, and it has some complementary strengths and limitations compared with these methods.

What Does Smart-Home Technology Add to Current Methods?

As we outline below, a number of key issues within psychological science and intervention science can be addressed by the development of assessment methods that can be deployed within the home and that allow for the measurement of observed behavior in real time, continuously, unobtrusively, and eventually in a scalable manner that has the potential to reduce the burden on researchers and participants. The achievement of such methods may advance psychological science by allowing for more objective and multimodal measurement of observed affective and behavioral patterns in home contexts. These methods can supplement the existing strengths of the current approaches mentioned above to better understand behavior in the home context and provide novel methods for evaluating and refining psychological theories.

Current commercially available smart-home devices include a multitude of sensors capable of collecting video, audio (i.e., ambient environmental sounds and conversations), movement of participants within and between rooms in the house, temperature, physiology, and ambient light data, as well as data on interpersonal interactions and participants’ interactions with home utilities such as televisions or refrigerators. Table 1 describes examples of some current smart-home device sensors, the data they produce, and the psychological and behavioral processes that may be indexed through those methods.

Table 1.

Overview of Smart-Home Data: Indexing Psychological and Behavior Processes

Type of data	Function	Psychological and behavioral constructs captured
Type of data	Function	Affect	Instrumental behavior and device use	Physical activity and locomotion	Social/ interpersonal	Environmental
Video	Social interactions/behavior	X	X	X	X	X
Microphone sensors	Acoustic voice; speech content	X	X	X	X	X
Ambient light sensors	Lighting gradients		X	X	X	X
Ambient temperature sensors	Room temperature gradients					X
Motion sensors	Movement		X	X
Proximity sensors	Movement		X	X
RFID tags	Object interactions		X	X
Chemical sensors	Toxin and drug exposure					X
Wi-Fi data usage	Internet searches	X	X	X	X

Note: RFID = radio-frequency identification.

Locating smart-home technology within the psychological assessment “space”

The use of smart-home devices for psychological research reveals both common and distinct features compared with other assessment techniques. To highlight and clarify these unique and overlapping capabilities of smart-home methods in relation to other techniques, we have used a framework drawn from Mehl (2017). In his description of the electronically activated recorder (EAR; a wearable research tool that collects recordings of ambient sounds), he identified a number of key dimensions on which psychological assessment techniques may vary, including (a) ecological versus nonecological (e.g., laboratory) setting, (b) active versus passive measurement (i.e., whether the method requires an active response from participants to collect data), and (c) first-person perspective versus third-person or objective perspective (i.e., whether the assessment method is based primarily on the participant’s point of view). Figure 1 presents a three dimensional “assessment space” that is defined by these continua, and we have approximated the placement of various assessment methods within this space to identify their common and unique features.

Fig. 1.

Three dimensional “assessment space.” The x-axis represents a dimension of active versus passive measurement (i.e., whether the method requires an active response from participants to collect data), the y-axis depicts the dichotomous dimension of first-person versus third-person or objective perspective (i.e., whether the assessment method is based primarily on the participant’s point of view), and the z-axis represents the dichotomous dimension of ecological versus nonecological (e.g., laboratory) setting methods. This figure provides a conceptual model showing the ways in which smart-home technology differs from other recent passive-sensing and active-response methods as well as examples of other routinely used assessment methods. Note that points in the lower left of the figure have been jittered for clarity. fMRI = functional magnetic resonance imaging; EMA = ecological momentary assessment; EAR = electronically activated recorder.

The most commonly used assessment method in psychological science, self-report questionnaires, defines one corner of the cube (i.e., active, first person, usually nonecological) and, consequently, so does a large proportion of psychological research. These methods are optimized for assessing participants’ subjective experiences and judgments, but are also limited in a number of ways—as noted above. IRM methods, such as EMA (Shiffman et al., 2008), and passive methods, such as those using EAR devices (see Mehl, 2017) or smartphone and wearable devices (Harari et al., 2016), are all typically associated with assessment in ecological settings; however, EMA is an active technique, whereas electronic recorders and mobile and wearable computing methods are typically passive. The low participant burden associated with these passive techniques enables more intensively repeated and sustained assessment, and therefore they can provide higher temporal resolution (e.g., minutes, seconds), and potentially longer assessment periods, than are possible with EMA methods (with temporal resolution usually in the range of hours to days). Smart-home methods are also passive and ecological, and thus many of their strengths and limitations overlap with those of mobile and wearable techniques. The main features that distinguish smart-home techniques from other passive-sensing methods include their location (i.e., in the home) and the fact that they are particularly suited to third-person assessment. This latter point, however, is not straightforward. Although sensors worn on a participant’s body, such as smartphones or wearables (e.g., wearable cameras), provide objective data, they often do so from the individual’s perspective (e.g., what that individual can see or hear), whereas smart-home methods are more suited to observing the person from an external viewpoint (e.g., observing the individual’s behavior from fixed cameras). However, it should be noted that some sensors on smartphones and wearables do provide data on behavior from a third-person perspective, just not direct observation of that behavior (e.g., measures of geographic movement or physical activity).

Compared with other passive-sensing methods, a number of more pragmatic and less conceptual features of smart-home assessment may have implications for their feasibility of use in specific clinical and research settings. For example, compared with smartphone and wearable devices, smart-home devices

are usually connected to continuous power sources, meaning they do not need to cease data collection to be charged, which prevents data loss due to battery-life limitations and/or instances when participants forget to reengage with their devices;

may have the capacity to collect and transfer larger and continuous data files (as a result of stable and potentially wired—Ethernet based—Internet connections);

allow for the recording of participants’ environmental contexts (e.g., the interpersonal and physical characteristics of specific environments within the home), rather than solely collecting information on what participants are doing;

allow the manipulation of home environments, such as changing ambient lighting, sound, and temperature gradients that may allow for novel psychotherapeutic and health interventions; and

have an ability to index participants’ interaction with various Internet of Things devices in the home (e.g., refrigerator and bathroom use or medication management) that may be relevant to behavioral and psychological health.

Smart-home methods are best suited for assessing the (a) continuous collection and (b) passive collection of (c) longitudinal data and (d) multimodal data on (e) objective, third-person observable individual and relational interactions, appliance usage, and events within (f) an ecologically valid context, the home. It is important to note that smart-home technology is not the panacea for the methodological limitations of other methods. Smart-home technology has its own limitations, such as being limited to behavioral observation in the home, and some devices (e.g., cameras, sound recorders) being fixed in position, making them unable to “follow” participants as they move around their homes. Moreover, the home is considered a distinctly private space, and some participants may find smart-home assessment techniques to be particularly intrusive (see below regarding ethical issues). However, these various assessment techniques (e.g., EMA, mobile and wearable computing, self-report, smart home) can complement one another by being used in combination in order to extend the methodological toolbox.

Combining existing EMA self-report methods with smart-home technology will allow for these methods to capitalize on their respective strengths and allow for the assessment of both first-person subjective experience and judgments and third-person observation of behavior and affect. This will be important for research questions that require both an understanding of a person’s experience of their environment and an objective understanding of what that current environment entails. For example, the psychological constructs of loneliness and social isolation are two overlapping yet distinct constructs that cannot be fully evaluated without the merging of self-report (e.g., EMA) and smart-home methods. Within a home setting, a participant might report being subjectively lonely even when surrounded by people (e.g., family members, roommate, friends), whereas another participant might report low levels of loneliness when alone. Using either EMA or smart-home technologies alone would miss key information that might be important to providing a complete picture of both the subjective and the environmental aspects of loneliness and social isolation. In contrast, combining wearable, smartphone, and EMA methods in daily environments may help tease apart these constructs in real-world environments outside the home.

Applications of smart-home methods in psychological science

Smart-home technologies have multiple applications: research, assessment, and intervention. In terms of their relevance for intervention, they can be used both as an intervention tool (e.g., to provide behavioral feedback and suggestions for behavior change via “nudges”; see Hertwig & Grüne-Yanoff, 2017) and as a method of evaluating the effectiveness of interventions designed to change home-based behaviors—for example, family therapy and treatment of home-based behaviors related to mental-health disorders (e.g., obsessive compulsive disorder or sleep problems). The use of smart-home technologies, especially in conjunction with EMA, smartphone, and wearable technology, may also present an opportunity for psychological science to move toward precision psychology—improving psychological and behavioral health by providing nuanced assessment of individual differences in behavioral patterns.

In the clinical realm, reducing the public burden associated with mental-health disorders by improving treatment is a pressing need (Kessler et al., 2003; Mojtabai & Jorm, 2015; Reavley & Jorm, 2014), especially among young people (Mojtabai, Olfson, & Han, 2016), minorities (Ronzio, Guagliardo, & Persaud, 2006), and people in rural environments who have reduced access to mental-health services (Wang et al., 2005). Smart-home technologies may facilitate a new wave of evidence-based psychological intervention that can be delivered from a distance and may be able to be tailored to the individual rather than to the disorder. Clinical psychological science would benefit from such research to improve treatment outcomes, as well as to facilitate more timely interventions. For example, “just-in-time” interventions highlight how the timing of an intervention can be as important as the specific type of intervention used (Nahum-Shani, Hekler, & Spruijt-Metz, 2015; Nahum-Shani et al., 2018), and smart-home technology may be able to be used to trigger certain interventions at the time of greatest need or effectiveness as discussed in greater depth below, but first we briefly review existing smart-home research.

Extant Smart-Home Research

In the past few years, there have been an increasing number of research studies conducted in the computational and medical sciences using smart-home technologies. These studies have provided a proof of concept that these devices can be self-installed by participants with a high success rate (Hu et al., 2016) and used to capture data on cognitive health (Dawadi, Cook, Parsey, Schmitter-Edgecombe, & Schneider, 2011; Dawadi, Cook, & Schmitter-Edgecombe, 2013) and behavior change (Dawadi, Cook, & Schmitter-Edgecombe, 2014).

Feasibility of installation

Researchers have tested the feasibility of having participants (age 50 or over) self-install a smart-home kit (Hu et al., 2016). Self-installation components consisted of server boxes, relays, temperature sensors, magnetic door sensors, motion sensors, power cords, Ethernet cables, batteries, and adhesive strips. Overall, results indicated a high rate of installation success for most devices (failure rate < 12%), except for door sensors (failure rate = 46.15%), indicating that smart-home kits are intuitive for nontechnical seniors to install. Success rates are likely to be higher with younger generations more familiar with modern technology.

Cognitive assessment and longitudinal data collection

Some research has begun to use smart-home technologies in conjunction with machine learning algorithms to longitudinally assess cognitive health. For example, smart-home technology has been able to distinguish between healthy older adults and those suffering from dementia based on how well participants performed activities in their home environments (Dawadi et al., 2011; Dawadi, Cook, & Schmitter-Edgecombe, 2013). These researchers have also demonstrated feasibility of longitudinal research using these methods by conducting a 2-year study to determine changes in participants’ cognitive and physical health behaviors (Dawadi, Cook, & Schmitter-Edgecombe, 2014). These methods allowed researchers to capture physical mobility patterns, sleep behaviors, and bed-to-toilet transitions, as well as activities such as cooking, eating, relaxing, personal hygiene, and leaving the home. Overall, cognitive health was significantly associated with changes in sleep and activity. These findings provide a proof of concept that this technology can be used to further psychological research by identifying and predicting aspects of both cognitive functioning and physical health.

Behavioral interventions

Similar technologies, though not smart-home based, have been used for behavioral interventions in elderly populations. For example, some studies have focused on prompting seniors with dementia to increase hand-washing compliance (Labelle & Mihailidis, 2006; Mihailidis, Barbenel, & Fernie, 2004; Mihailidis, Boger, Craig, & Hoey, 2008). Findings indicated that behavioral “nudging” with audio and/or video prompts could significantly increase the completion of hand-washing steps without a caregiver present (by 25%; Mihailidis et al., 2004) and significantly reduce the need for human caregiver instruction (by 78%; Mihailidis et al., 2008). Indeed, as the robust literature on behavioral nudging and therapeutic feedback have shown, these interventions can be personalized (Hertwig & Grüne-Yanoff, 2017; Kaptein, Markopoulos, De Ruyter, & Aarts, 2015) and can be an important psychological component for behavior change (for reviews, see DiClemente, Marinilli, Singh, & Bellino, 2001; Marchiori, Adriaanse, & De Ridder, 2017). Although identifying when participants need to wash their hands is probably less nuanced than recognizing other potential targets for behavior change (e.g., emotional shifts associated with interpersonal conflict or mental-health states), these findings do provide some preliminary support for the notion that smart-home technologies can be used not only to identify behaviors but also to potentially facilitate behavior change.

Instrumental behavior, device use, and physical-activity recognition

Researchers have also used smart-home devices to recognize participants’ activity and instrumental behavior. One way to assess participants’ interaction with objects is to use radio-frequency identification (RFID) tags, which can be attached to household items. These small and unobtrusive sensors are battery free and inexpensive. An RFID reader connected to other devices, such as a smartphone, can then track participants’ interaction with objects to index activities of daily living (Philipose et al., 2004). Other tools allow for the recognition of independent versus relational behavior in smart-home environments by using motion sensors and probabilistic models (Singla, Cook, & Schmitter-Edgecombe, 2010). Such methods can be used to assess when participants are independently performing activities in their home environment as opposed to interacting with other people. Current technological advances may allow for the consolidation of smart-home technologies, making some sensors, such as RFIDs, obsolete in the future. For example, Amazon has recently opened a store, Amazon Go, that has no cashier (https://www.amazon.com/b?node=16008589011). Customers simply enter the store, pick up the items they want, and then leave. Amazon uses technology similar to that used in self-driving cars (machine deep learning, computer vision, sensor fusion) to identify a customer and the products they purchased. In the near future, such technologies could be adapted to identify human behaviors, such as medication adherence and food choice, within the home.

Although smart-home research has been increasing in the computer and medical sciences, its potential has yet to be leveraged for furthering psychological theory and science. Below we discuss ways in which this technology can advance theory and then offer two salient examples of relevant psychological and behavioral constructs that are potential targets for smart-home research, including their potential role in assessment and intervention.

Advancing Psychological Theory

Smart-home technologies may be able to facilitate the development of psychological science and theory more broadly by extending the passive-sensing toolbox through the unique integration of types of data that have not previously been available, especially within the home context. Specifically, smart-home devices allow for the (a) continuous collection and (b) passive collection of (c) intensively longitudinal and (d) multimodal data from a (e) third-person observational perspective that (f) preserves ecological validity by taking place outside of the laboratory and within participants’ own homes. Whereas each of these methods alone has been used to study various theories, smart-home measures combine these specific features, as can be gleaned from Figure 1.

For example, one of the most notable capabilities associated with data collected from smart-home sensors is that they bring the laboratory into the home. Specifically, as noted above, they provide passive, third-person (objective) assessment in the home context that is intensively longitudinal and can be collected with minimal burden on participants. Greater availability of these data will likely help adjudicate a range of psychological theories, especially those that address the temporal dynamics of psychological phenomena. Indeed, researchers across subdisciplines, from developmental and clinical psychology to cognitive neuroscience, have often been severely limited by their inability to study the temporal dynamics of observed behavior over extended periods of time, especially in naturalistic settings. Current longitudinal research designs typically involve significant time gaps between waves of data collection, and these gaps are often insufficiently fine grained to allow examination of the dynamics of those behaviors (e.g., interpersonal relationships, eating behavior, emotional dynamics) that unfold over shorter timescales (e.g., minutes, days, weeks). Recent work, as mentioned above, using EMA (Shiffman et al., 2008) or diary methods (Repetti et al., 2015) is providing some greater insight into these issues, but even EMA techniques are typically not able to completely capture the full temporal dynamics of behaviors over shorter time scales (minutes, seconds). Here we present some examples of psychological theories that may be advanced by the collection of smart-home data.

Interpersonal theories

Given the significance of home environments for close interpersonal relationships, smart-home assessment may be especially relevant to theories of relational processes. For example, the theory of social buffering can be further evaluated and extended by studies incorporating smart-home technologies. Social buffering posits that supportive social relationships can buffer (or moderate) the impact of stress on psychological and biological functioning (Hostinar, Sullivan, & Gunnar, 2014; Nelson et al., 2017; Pietromonaco & Collins, 2017). This body of research has been largely dependent on discrete laboratory paradigms that observe individual and family functioning or use self-reported measures of perceived relational functioning. A study using a smart-home approach would allow for a novel investigation that might extend the theory of social buffering in ways that are difficult (if not impossible) to achieve with other techniques.

First, smart-home observation allows for the objective description of third-person observation of behaviors within a key ecological environment (i.e., the home), thereby extending the evaluation of this theory to observed behaviors in the home (as opposed to subjective self-report of social behavior in the home or objective observation of social buffering in the laboratory). This may ultimately bolster the ultimate validity and translatability of the findings into what is clearly a key ecological environment for these processes.

Second, it is possible that the temporal dynamics of social-buffering effects may be observable at smaller timescales, such as when an act of social support results in an immediate reduction of behavioral or physiological indices of stress (e.g., eating behavior, sleep, angry or dysphoric vocal affect). Smart-home devices would allow for continuous recording of observed behavior to advance the understanding of when and how social buffering takes place. This would provide a unique perspective distinct from that available from continuous first-person recordings with wearables or less temporally detailed EMA of subjective report on social experiences.

Third, the longer time periods over which data can be collected with smart-home methods could help to determine the time lag between social support and buffering effects and how long they are sustained. The ability to compare buffering effects over different timescales may help to adjudicate key mechanisms (e.g., whether the effects are mediated by immediate autonomic physiology or longer-term changes in social and health behaviors) and, ultimately, to guide interventions.

Finally, the passive collection of these data in home environments will prevent participants from having to pause their normal activities to provide a response or enter a novel laboratory environment, therefore potentially allowing for a more natural unfolding of social buffering in a real-world context.

Emotion dynamics

Data that allows for intensive temporal sequencing of emotional expressions and behaviors is essential to fully assess and refine theories of emotional dynamics, such as emotional inertia and critical slowdowns. Emotional inertia is a phenomenon in which the autocorrelation of an individual’s emotions over time predicts psychological maladjustment, rumination, and depressive severity (Koval & Kuppens, 2012; Koval, Kuppens, Allen, & Sheeber, 2012; Kuppens, Allen, & Sheeber, 2010; Kuppens et al., 2012); critical slowdowns, which are partly characterized by greater variance in emotion, higher temporal autocorrelation of emotion, and greater correlation of emotions of similar and dissimilar valence, have also been shown to predict transitions toward depression (van de Leemput et al., 2014). These phenomena have thus far largely been studied in discrete laboratory settings. Using smart-home devices would allow for the passive collection of observed third-person behavior and affect as they unfold within the home setting, which may increase the ecological validity of laboratory studies and translate these findings into real-world settings. In addition, the temporally detailed and longitudinal collection of data may elucidate both micro-level (moment to moment) and macro-level (week to week) inertia of behavior and affect, which have yet to be investigated. These advantages would address a number of significant questions regarding these phenomena, including their persistence across contexts (interpersonal and environmental), the time frames over which they are observable, and the relationship between the temporal dynamics of different measures of emotion. Furthermore, this method could be used in conjunction with self-reported methods to see how both observed and experienced inertia of affect and behaviors relate to one another across time.

Risk for suicide

Finally, the application of smart-home technology can be used for furthering our understanding of psychopathological processes that typically occur in the home. One salient example that we discuss in detail here is the psychological and behavioral processes associated with acute risk for mental-health crises, particularly suicide attempts. There are a number of long-standing theories of suicidal behavior, such as Beck’s hopelessness theory and Joiner’s interpersonal theory (Beck, Kovacs, & Weissman, 1975; Joiner, 2007). In particular, because of methodological limitations, these processes have largely been studied as distal risk factors for suicide attempts, whereas some of the affective and social processes described in these theories (i.e., hopelessness, thwarted belongingness/perceived burdensomeness) may in fact be more potent as short-term risk factors. For example, the onset of suicidal ideation and selection of suicide methods often occur years before an attempt, whereas the proximal steps to suicide often occur within a week, and mostly within hours, before the attempt (Millner et al., 2017), which suggests that different risk factors may operate at different time scales. The significance of smart-home and other IRM methods is that they offer, in some cases for the first time, the opportunity to examine these issues in a nonretrospective way.

Indeed, the majority of suicide attempts occur in the home (Kellermann et al., 1992), which emphasizes the particular suitability of home-based monitoring technology for addressing this issue. Recent research using timeline follow-back interview approaches and EMA methods have demonstrated alterations in first-person affect before a suicide attempt, which makes clear that there is typically a rapid increase in emotions related to dissatisfaction, hostility, loneliness, fear, hopelessness, and burdensomeness in the hours and minutes before a suicide attempt (Bagge, Littlefield, & Glenn, 2017; Kleiman et al., 2017). Using smart-home technology to passively collect intensively longitudinal data on behavioral processes associated with depression and suicide risk may better elucidate the behavioral patterns associated with these changes in psychological states (e.g., hopelessness, interpersonal distress), especially how these may precipitate and sustain suicidal risk. Data on observed third-person behavior collected in real time, rather than retrospective timeline follow-back interviews, may allow for a more accurate categorization of behavioral and affective dynamics that emerge along both discrete and longitudinal timescales leading up to a suicide attempt. Finally, using smart-home technologies in tandem with EMA methods would allow for the collection of observed facial, speech, and behavioral data as well as experiential data on affective states; such data could both allow for a greater understanding of short-term risk processes and have great clinical significance by detecting people who are likely to attempt suicide and facilitating timely intervention at moments of greatest risk.

In summary, naturalistic observation via smart-home technology provides a set of tools that, while not suited for all research questions, can make a unique contribution to psychological research and theory by providing what was described above: the (a) continuous collection and (b) passive collection of (c) intensively longitudinal and (d) multimodal data from a (e) third-person observational perspective that (f) preserves ecological validity by taking place outside of the laboratory and within participants’ own homes. Some of these methods may initially be labor intensive because they require human coding of behaviors until automated coding methods have been developed and validated, they primarily use stationary sensors, and they are limited to collection of data within the home environment. However, by providing intensive longitudinal data within the home environment that can capture concurrent multimodal processes (face, voice, physiology, behavior, device usage, home environment), from a third-person perspective, on both individual and relational functioning, in a temporally precise and ecologically valid manner that is independent from, yet complementary to, self-report, EMA, smartphone, and wearable devices, smart-home methods may plausibly allow us to address unique questions relevant to diverse psychological theories. Below we further explicate the potential contributions of smart-home methods to psychological-intervention science and behavior change.

Examples of Psychological and Behavioral Targets for Interventions Using Smart-Home Technology

Interpersonal relationships

In addition to potential research applications, smart-home devices can also be used for novel assessment and intervention techniques relevant to a wide range of home-based behaviors that can be targets for behavior change. As discussed previously, researchers have already used such devices to distinguish between healthy older adults and those suffering from dementia (Dawadi et al., 2011; Dawadi, Cook, & Schmitter-Edgecombe, 2013). Assessments of this kind could be extended to other populations, which could allow for diagnosis from a distance and a better understanding of the dynamics that are associated with deterioration or recovery of mental health and interpersonal functioning. Take for example a couple who attends psychotherapy to reduce patterns of interpersonal conflict. At the start of each session, the therapist, rather than relying solely on the couple to retrospectively report on their functioning in the past week, could pull up summary descriptive charts of relational quality (i.e., frequency and duration of conflictual interactions as detected by vocal tone and facial expression) as well as a host of other metrics (e.g., sleep habits) that may be influencing a couple’s mood.

Another relevant example for the realm of intervention science is the number of home-based clinical-research interventions that use video feedback to improve parenting skills, with the ultimate goal of improving family relationships and preventing mental health and other disorders in offspring. These typically require a researcher or clinician to enter a participant’s home to record naturalistic relational interactions between parent-child dyads. One such example is the Filming Interactions to Nurture Development (Fisher, Frenkel, Noll, Berry, & Yockelson, 2016) intervention. This intervention consists of a video-coaching program for caregivers of children, during which therapists record parent-child relational interactions within the home to positively reinforce existent supportive social interactions between caregivers and their children. Such interventions have the advantage of providing opportunities to practice skills in naturalistic settings, as opposed to bringing families into the laboratory or clinic, and have greatly propelled our understanding of relational and intervention science. However, these techniques are able to evaluate only specific interactions and for relatively short periods of time because of the burdens of microsocial coding. The assessment of in-home behavior using smart-home technologies would initially require an equal or greater burden of microsocial coding with a less frequent sampling rate to establish a “ground truth” to optimize machine-learning algorithms; however, after this is accomplished, smart-home devices could be combined with automated behavioral coding, which would allow for assessment of, and intervention in, a much larger range of in-home behaviors.

There are also a number of remote therapeutic interventions that could be administered with such technology. One potential is a therapeutic nudge, in which clients would be prompted to change an aspect of their behavior to increase adaptive functioning, such as modifying eating and sleep patterns (Marteau, Ogilvie, Roland, Suhrcke, & Kelly, 2011; Nahum-Shani et al., 2015; Roberto & Kawachi, 2014; Tavernier & Adam, 2017). For example, behavior and language captured using home-monitoring cameras could be analyzed in real time; future algorithms could determine and code for conflictual interpersonal behavior (e.g., deviation from acoustic norms and occurrences of aggressive behavioral posturing). If such deviations from normal behavioral and acoustic functioning are positively identified, then voice-activated speakers or text messaging could initiate therapeutic “nudges” (e.g., reminders to use interpersonal skills that have been taught during face-to-face therapy interventions) to move individuals out of conflict and toward resolution.

Sleep

Sleep is another example of a salient set of home-based behaviors that may be tracked and potentially modified by smart-home technology. Indeed, sleep disturbance is a common symptom of a variety of mental-health disorders, such as depressive, stress, and anxiety disorders (APA, 2013), demonstrating its relevance to mental health. The current standard for sleep measurement is polysomnography, which requires research participants or patients to spend a night in a sleep clinic. Diary methods have also been used and have provided valuable data on how relational functioning can influence sleep (Repetti et al., 2015), such as how one person’s health problems can affect the sleep behaviors of their spouse (Martire, Keefe, Schulz, Stephens, & Mogle, 2013). Recently, with the advent of consumer wearable devices, there has been a push toward using wearable actigraphy sensors to capture behavioral sleep data (Marino et al., 2013). Although smart-home technologies cannot yet capture all the physiological data that can be acquired through polysomnography, new methods involving radio-frequency signals sent via Wi-Fi can now be used to index respiration, heart rate (Adib, Mao, Kabelac, Katabi, & Miller, 2015; Zhao, Adib, & Katabi, 2016), and sleep stages (Zhao, Yue, Katabi, Jaakkola, & Bianchi, 2017) even in the absence of wearable sensors. In addition, a number of devices can be used to both study sleep habits and assess sleep hygiene (e.g., through sensor-equipped beds and pillows) and provide sleep interventions.

A range of key sleep variables, including sleep-onset latency (i.e., time from getting in bed to falling asleep), duration, and quality, are strongly influenced by environmental variables such as light-dark cycles and ambient temperature: Cooler temperatures are more conducive to sleep onset (Okamoto-Mizuno & Mizuno, 2012). Smart-home technologies may provide a means to study how each of these variables alters sleep patterns as well as to intervene at the contextually appropriate time by modifying each of these aspects of the home environment using multifactorial designs or microrandomized trials (Klasnja et al., 2015) to support healthy sleep practices through intervention.

Smart-home devices also have the potential to index and intervene in other behavioral patterns that may prove useful in improving healthy sleep. For example, evening routines around television and other screen use could be monitored to determine how the use of home devices influences sleep patterns. Monitoring device use could provide precise information on the timing and location of the devices that are being used and could examine how falling asleep on your couch while watching television and then waking up later in the evening influences sleep stages. It would also be possible to examine when, on nighttime awakening, participants simply stay in bed until they fall back to sleep, check devices, or move to certain rooms of the house (e.g., the kitchen or bathroom) before falling back to sleep and to evaluate how these behaviors affect overall sleep hygiene.

Moreover, these technologies also have potential applications for intervention for people suffering from sleep disorders. Nudging interventions, as described above, from voice-activated smart speakers, could prompt individuals that it is time to get ready for bed to reduce sleep variability. Indeed, similar nudging methods using text messages in the smartphone literature have been found to improve objective sleep duration in adolescents (Tavernier & Adam, 2017). Time for bed could be personalized by capturing individually indexed sleep onset and awakening times and combining this data with weekly calendars to optimize these variables. Simultaneously, automated thermostats could slowly lower bedroom and home temperatures on the basis of average activity-pattern data detected from motion sensors and video cameras. Furthermore, as an individual’s average bedtime approaches, light-emitting devices (e.g., smartphones, televisions, and tablets) could automatically reduce blue-light exposure (e.g., using the f.lux and Twilight apps or Apple’s iPhone Night Shift mode) and dim lights to create environmental cues for sleep. In each of these ways, new approaches to intervention can be tested to better understand sleep-hygiene practices to identify which smart-home behavior-change methods are most conducive to providing optimal sleep environments in order to improve sleep health.

Methodological Challenges and Research Barriers for Smart-Home Research

A number of exciting opportunities are afforded by the adoption of smart-home technologies in psychological research, as described above; however, a number of current technological limitations are noteworthy for identifying which questions and theories the psychological sciences are best suited to address using these methods. A number of current logistical and technological challenges to measuring naturalistic behavior in the home affect the feasibility and fidelity of using smart-home technology. First, background, ambient environmental acoustics (e.g., television, radio, music, devices) have the potential to obscure speech signals from participants, causing loss of usable data. Second, resolution on many current fixed cameras designed for home observation (e.g., fish-eye type lenses) are not fine enough to allow for the measurement of microlevel behavior, such as facial-feature detection that captures enough facial landmarks to accurately deduce facial expression for automated analysis (De la Torre et al., 2015). It should be noted that technological advances in the automated detection of human facial features will likely change this in the future by providing cameras that can automatically zoom in on the features of greatest interest. Furthermore, camera angle, positioning, and lighting gradients may each differentially affect the quality of video and therefore the type of signal that can be extracted from collected data. For example, data can be lost if a participant simply moves out of visible camera range or enters a dark room, although less is likely to be lost than with current conventional data-collection techniques.

Finally, it is the hope that future smart-home data collection and signal processing will become automated using machine-learning techniques to capitalize on large data sets and reduce researcher expense and time burden related to coding behavioral observations, but extensive research will be needed to ensure measurement accuracy, construct validity, and reliability of recorded signals to ensure replicability and reproducibility of findings. For example, the collection of other types of data along with smart-home data—such as manual coding of behavior by trained observers, EMA, and diary methods—will be important in order to establish a nomological network of variables (i.e., “the interlocking system of laws which constitute a theory”; Cronbach & Meehl, 1955, p. 11) that can help to establish the construct validity of smart-home data that are passively collected and coded via automated methods. Furthermore, these more traditional data sets can serve as “ground truth” labels on which to train machine-learning algorithms using data from smart-home devices, which can then automate the analysis of these signals in future research and application.

There are also other challenges associated with the use of smart-home methods. The use of multiple sensors capable of continuous data collection means that very large data sets will be collected, resulting in a number of challenges that are common to big data and data mining, such as being able to sift through vast amounts of data to identify the signal in the noise. As mentioned below, research using passive-sensing methods will likely require interdisciplinary data integration that builds collaborations across diverse specialties. Furthermore, these commercially available devices, similar to smartphones and wearable devices, rely on proprietary algorithms that are at times undisclosed for commercial reasons. Consequently, before commercially available smart-home technologies can be used for scientific research and intervention, validation studies must be conducted to verify the reliability and validity of each device with respect to standard research methodologies for assessment (where such exist). This validation work should take care to report firmware versions because this is still not documented in almost all studies validating wearables (Nelson & Allen, 2018). Without known firmware versions, reliable comparison studies will not be possible because frequent software updates may change underlying algorithms, making these devices more or less accurate over time.

Ethical Considerations for Smart-Home Research and Intervention

Ethical and legal considerations for the collection of passive-sensing data through emerging technologies is a rapidly expanding area of ongoing discussion because of the novel and highly identifiable nature of the data (for examples, see Carter, Liddle, Hall, & Chenery, 2015; Kelly et al., 2013; Nebeker et al., 2016; Pisani et al., 2016; Torous & Nebeker, 2017; Torous & Roberts, 2017b). It should be noted that although smart-home technologies may seem invasive to some today, similar observations were made of smartphones continually tracking location or advertising companies scanning personal emails, and these have now become technological norms that are commonly accepted (albeit begrudgingly). In fact, research indicates that once smart-home devices, particularly smart speakers, are incorporated into daily living, 65% of individuals would not want to go back to living without the devices and 45% report that they expect to purchase additional devices (National Public Radio & Edison Research, 2017).

Although there will be a need for existing institutional review boards (IRB)—originally designed at a time when research took place in one location with the collection of small data sets—to adapt to these new technologies and methodologies (Bloss et al., 2016), more immediate changes can be adopted by researchers and clinicians today. From a researcher’s perspective, a number of considerations related to transparency, confidentiality, security, trust, and privacy will be required for the ethical collection of passive sensing data. For example, some research bodies, such as the Substance Abuse and Mental Health Services Administration (2015), have outlined a number of considerations for technology-assisted care that can likely be adapted for data collected via smart-home technologies. These include special attention to issues related to confidentiality (protecting participants’ data), security (preventing participants’ data from unauthorized access), and privacy (allowing participants to control access to their data). In addition, there are a number of legal challenges that may have different outcomes depending on state laws regarding data collection (e.g., recording telephone conversations with one- or two-party consent). Furthermore, because of the highly identifiable nature of the data collected with these methods, researchers will need to be especially vigilant in following ethical principles outlined in The Belmont Report, including respect for persons, beneficence, and justice (National Institutes of Health, 1979, pp. 3–4).

Crowdsourcing digital ethics

Regarding the ethical collection of passive-sensing data, there is one promising avenue that could be used by researchers, IRBs, and industry partners: the Connected and Open Research Ethics (CORE) initiative that was recently designed and launched to develop ethical practices in digital research (Torous & Nebeker, 2017). The CORE initiative was launched in 2016 and provides a research space for members to share knowledge, resources, questions, and answers regarding the use of passive-sensing technology for research purposes (The Regents of the University of California, 2016). This platform currently contains members from over 10 countries with expertise spanning bioethics, privacy, technology, and research ethics. In other words, CORE crowdsources ethics from a transdisciplinary team to provide resources for both researchers and IRBs to help guide research design and to provide an ethical framework that encourages sound research practices.

Ethics for clinical application of digital technologies

Researchers have also begun to discuss the various ways in which to incorporate digital technologies into the lives of clients, patients, and participants in a way that provides safety and privacy (American Psychiatric Association, 2017) and also delivers these novel interventions in nonintrusive ways (Pisani et al., 2016). For example, the American Psychiatric Association has created an “App Evaluation Model” that provides mental-health providers with the following five steps to consider when incorporating digital technologies, specifically app usage, with clients: (a) background information, (b) risk/privacy and security, (c) evidence, (d) ease of use, and (e) interoperability (American Psychiatric Association, 2017). Likewise, other researchers provide a decision-tree model that allows providers to incorporate ethical safeguards into clinical practice (Torous & Roberts, 2017a).

Protecting confidentiality of participants and third parties

Similar to data collected via other passive-sensing technologies, data from smart-home technologies will be highly identifiable and will therefore require a greater degree of protection for participants and third parties (Nebeker et al., 2016; Torous & Nebeker, 2017). Important methods for achieving greater confidentiality include data encryption and automated identity recognition via facial or voice data. In one solution, continuous data collection would be saved to an encrypted cloud service protected according to the provisions of the Health Insurance Portability and Accountability Act (HIPAA). Video and voice data would then be scanned by automated facial and speech recognition software to match collected data with participants’ facial and audio samples (Hansen & Hasan, 2015; Jain & Li, 2011; Turk & Pentland, 1991). Data containing video or audio material that was not from a target participant, or that included the addition of a third party with a target participant, would be automatically deleted before any humans viewed the data. An additional layer of protection for third parties in couple studies could be achieved by programing smart-home devices to record only when consented participants’ Bluetooth-enabled wearable devices are within a specified diameter of one another, which would allow wearables to “speak” to one another and then trigger recording of devices. These techniques would not only reduce the degree of third-party data collected but also cut down the size of data files, which would likely be a concern for smart-home studies.

Future Directions and Conclusions

As noted above, incorporating smart-home technology into psychological research will first have to focus on device validity and reliability to ensure that commercially available devices are indeed measuring what they are intended to measure. Furthermore, feasibility research is needed for smart-home studies, which may run into problems similar to those facing wearables research (e.g., participant compliance and cost). This early research will have to detail needed sample sizes, percentage of participants accepting these devices in their homes, levels of study attrition, and how device adherence varies by number of participants in the home (i.e., it may be harder to recruit entire households than to recruit individuals living alone). It is important to highlight that moving forward with this type of research will likely require interdisciplinary data integration, necessitating collaboration between experts in behavioral science, affective computing, mobile sensing, mental health, signal processing, artificial intelligence, biomedical engineering, data mining, computer networks, machine learning, bioethics, and technology development, along with industry partners. In addition, combining validated smart-home technologies with passive smartphone and wearable technologies as well as valuable self-report EMA methods may also allow researchers and clinicians to continuously track both first-person subjective affect and third-person observed affective behavior both within the home and in participants’ wider environment.

Finally, there is always the potential that these passive-sensing technologies will not deliver what they promise (e.g., potential lack of use and adoption, no behavior improvement). Other potentially promising areas of research, including the translation of neuroscientific findings into new treatments ( “Back to basics,” 2013) or using genetics to improve treatment outcomes (Joyner, Paneth, & Ioannidis, 2016), have recently been subjected to critical evaluations that have raised the question of when a direction of research should be abandoned or significantly redirected. Likewise, smart-home technologies will need to be subjected to regular critical evaluations of their usefulness and incremental effectiveness to ensure that we do not persist with invasive and expensive procedures that do not have clear utility for science or application.

Research has yet to find a way to more fully measure behavior in participants’ homes in a manner that is scalable, unobtrusive, continuous, longitudinal, and ecologically valid. Home environments tend to be emotionally salient locations where people conduct important interpersonal relationships, such as interacting with their romantic partners, raising children, and hosting friends. Homes are also the site of many self-care behaviors that are critical to mental and physical health. Commercially available smart-home technologies provide researchers with a novel means to observe psychological and behavioral functioning continuously and longitudinally with rich time-series data while reducing the burden for both researchers and participants. Overall, the use of smart-home devices in psychology promises to further psychological theory and science by producing both novel research findings regarding individual and relational health and novel strategies for prevention and intervention.

Footnotes

Action Editor

Brad J. Bushman served as action editor and Robert J. Sternberg as editor-in-chief for this article.

ORCID iD

Nicholas B. Allen

Declaration of Conflicting Interests

The author(s) declared that there were no conflicts of interest with respect to the authorship or the publication of this article.

References

Adib

Mao

Kabelac

Katabi

Miller

R. C.

(2015). Smart homes that monitor breathing and heart rate. In Begole

Kim

Inkpen

Woo

(Eds.), CHI ’15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 837–846). doi:10.1145/2702123.2702200

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.

American Psychiatric Association. (2017). Mental health app evaluation model. Retrieved from https://www.psychiatry.org/psychiatrists/practice/mental-health-apps/app-evaluation-model

Anderson

Rainie

(2012). The future of smart systems. Retrieved from http://www.pewinternet.org/2012/06/29/the-future-of-smart-systems/#fn-163-1

Back to basics [Editorial]. (2013). Nature Neuroscience, 16, Article 983. doi:10.1038/nn.3483

Bagge

C. L.

Littlefield

A. K.

Glenn

C. R.

(2017). Trajectories of affective response as warning signs for suicide attempts: An examination of the 48 hours prior to a recent suicide attempt. Clinical Psychological Science, 5, 259–271.

Baumeister

R. F.

Vohs

K. D.

Funder

D. C.

(2007). Psychology as the science of self-reports and finger movements: Whatever happened to actual behavior? Perspectives on Psychological Science, 2, 396–403.

Beck

A. T.

Kovacs

Weissman

(1975). Hopelessness and suicidal behavior: An overview. Journal of the American Medical Association, 234, 1146–1149.

Bloss

Nebeker

Bietz

Bae

Bigby

Devereaux

. . . Klemmer

(2016). Reimagining human research protections for 21st century science. Journal of Medical Internet Research, 18(12), Article e29. doi:10.2196/jmir.6634

10.

Carter

Liddle

Hall

Chenery

(2015). Mobile phones in research and treatment: Ethical guidelines and future directions. JMIR mHealth and uHealth, 3(4), Article e95. doi:10.2196/mhealth.4538

11.

Cronbach

L. J.

Meehl

P. E.

(1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.

12.

Dawadi

Cook

Parsey

Schmitter-Edgecombe

Schneider

(2011). An approach to cognitive assessment in smart home. In Chawla

Ghani

Krishnapuram

Kumar

Madigan

Younger

. . . J.

(Eds.), KDD ’11: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining: Workshop on Data Mining for Medicine and HealthCare (pp. 56–59). doi:10.1145/2023582.2023592

13.

Dawadi

Cook

D. J.

Schmitter-Edgecombe

(2014). Smart home-based longitudinal functional assessment. In Brush

A. J.

Friday

(Eds.), UbiComp ’14 Adjunct: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 1217–1224). doi:10.1145/2638728.2638813

14.

Dawadi

P. N.

Cook

D. J.

Schmitter-Edgecombe

(2013). Automated cognitive health assessment using smart home monitoring of complex tasks. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 43, 1302–1313. doi:10.1109/TSMC.2013.2252338

15.

De la Torre

Chu

Xiong

Vicente

Ding

Cohn

(2015). IntraFace. In Bowyer

Leonardis

Cohn

(Eds.), 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (Vol. 1). doi:10.1109/FG.2015.7163082

16.

DiClemente

C. C.

Marinilli

A. S.

Singh

Bellino

L. E.

(2001). The role of feedback in the process of health behavior change. American Journal of Health Behavior, 25, 217–227.

17.

Fisher

P. A.

Frenkel

T. I.

Noll

L. K.

Berry

Yockelson

(2016). Promoting healthy child development via a two-generation translational neuroscience framework: The filming interactions to nurture development video coaching program. Child Development Perspectives, 10, 251–256.

18.

Furr

R. M.

(2009). Personality psychology as a truly behavioural science. European Journal of Personality, 23, 369–401.

19.

Gosling

S. D.

John

O. P.

Craik

K. H.

Robins

R. W.

(1998). Do people know how they behave? Self-reported act frequencies compared with on-line codings by observers. Journal of Personality and Social Psychology, 74, 1337–1349.

20.

Graesch

A. P.

Mittmann

Bradbury

Repetti

(2006). Video ethnography and ethnoarchaeological tracking. In Pitt-Catsouphes

Kossek

E. Ernst

Sweet

(Eds.), Work and family handbook: Multi-disciplinary perspectives and approaches (pp. 387–409). Mahwah, NJ: Erlbaum.

21.

Hansen

J. H. L.

Hasan

(2015). Speaker recognition by machines and humans: A tutorial review. IEEE Signal Processing Magazine, 32(6), 74–99. doi:10.1109/MSP.2015.2462851

22.

Harari

G. M.

Lane

N. D.

Wang

Crosier

B. S.

Campbell

A. T.

Gosling

S. D.

(2016). Using smartphones to collect behavioral data in psychological science: Opportunities, practical considerations, and challenges. Perspectives on Psychological Science, 11, 838–854.

23.

Hernandez

Picard

R. W.

(2014, October). SenseGlass: Using google glass to sense daily emotions. In Benko

Dontcheva

Wigdor

(Eds.), Uist’14: Adjunct Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology. (pp. 77–78). doi:10.1145/2658779.2658784

24.

Hertwig

Grüne-Yanoff

(2017). Nudging and boosting: Steering or empowering good decisions. Perspectives on Psychological Science, 12, 973–986.

25.

Hostinar

C. E.

Sullivan

R. M.

Gunnar

M. R.

(2014). Psychobiological mechanisms underlying the social buffering of the hypothalamic–pituitary–adrenocortical axis: A review of animal models and human studies across development. Psychological Bulletin, 140, 256–282.

26.

Tilke

Adams

Crandall

A. S.

Cook

D. J.

Schmitter-Edgecombe

(2016). Smart home in a box: Usability study for a large scale self-installation of smart home technologies. Journal of Reliable Intelligent Environments, 2, 93–106.

27.

Insel

T. R.

(2017). Digital phenotyping: Technology for a new science of behavior. Journal of the American Medical Association, 318, 1215–1216.

28.

Jain

A. K.

S. Z.

(2011). Handbook of face recognition. New York, NY: Springer.

29.

Joiner

(2007). Why people die by suicide. Cambridge, MA: Harvard University Press.

30.

Joyner

M. J.

Paneth

Ioannidis

J. P.

(2016). What happens when underperforming big ideas in research become entrenched? Journal of the American Medical Association, 316, 1355–1356.

31.

Kaptein

Markopoulos

De Ruyter

Aarts

(2015). Personalizing persuasive technologies: Explicit and implicit personalization using persuasion profiles. International Journal of Human-Computer Studies, 77, 38–51.

32.

Kellermann

A. L.

Rivara

F. P.

Somes

Reay

D. T.

Francisco

Banton

J. G.

. . . Hackman

B. B.

(1992). Suicide in the home in relation to gun ownership. New England Journal of Medicine, 327, 467–472.

33.

Kelly

Marshall

S. J.

Badland

Kerr

Oliver

Doherty

A. R.

Foster

(2013). An ethical framework for automated, wearable cameras in health behavior research. American Journal of Preventive Medicine, 44, 314–319.

34.

Kessler

R. C.

Berglund

Demler

Jin

Koretz

Merikangas

K. R.

. . . Wang

P. S.

(2003). The epidemiology of major depressive disorder: Results from the National Comorbidity Survey Replication (NCS-R). Journal of the American Medical Association, 289, 3095–3105.

35.

Klasnja

Hekler

E. B.

Shiffman

Boruvka

Almirall

Tewari

Murphy

S. A.

(2015). Microrandomized trials: An experimental design for developing just-in-time adaptive interventions. Health Psychology, 34, 1220–1228.

36.

Kleiman

E. M.

Turner

B. J.

Fedor

Beale

E. E.

Huffman

J. C.

Nock

M. K.

(2017). Examination of real-time fluctuations in suicidal ideation and its risk factors: Results from two ecological momentary assessment studies. Journal of Abnormal Psychology, 126, 726–738.

37.

Koval

Kuppens

(2012). Changing emotion dynamics: Individual differences in the effect of anticipatory social stress on emotional inertia. Emotion, 12, 256–267.

38.

Koval

Kuppens

Allen

N. B.

Sheeber

(2012). Getting stuck in depression: The roles of rumination and emotional inertia. Cognition & Emotion, 26, 1412–1427.

39.

Kuppens

Allen

N. B.

Sheeber

L. B.

(2010). Emotional inertia and psychological maladjustment. Psychological Science, 21, 984–991.

40.

Kuppens

Sheeber

L. B.

Yap

M. B.

Whittle

Simmons

J. G.

Allen

N. B.

(2012). Emotional inertia prospectively predicts the onset of depressive disorder in adolescence. Emotion, 12, 283–289.

41.

Labelle

K. L.

Mihailidis

(2006). The use of automated prompting to facilitate handwashing in persons with dementia. American Journal of Occupational Therapy, 60, 442–450.

42.

Marchiori

D. R.

Adriaanse

M. A.

De Ridder

D. T.

(2017). Unresolved questions in nudging research: Putting the psychology back in nudging. Social & Personality Psychology Compass, 11, 1–13.

43.

Marino

Rueschman

M. N.

Winkelman

J. W.

Ellenbogen

J. M.

Solet

J. M.

. . . Buxton

O. M.

(2013). Measuring sleep: Accuracy, sensitivity, and specificity of wrist actigraphy compared to polysomnography. Sleep, 36, 1747–1755.

44.

Marteau

T. M.

Ogilvie

Roland

Suhrche

Kelly

M. P.

(2011) Judging nudging: Can nudging improve population health. BMJ, 342, Article d228. doi:10.1136/bmj.d228

45.

Martire

L. M.

Keefe

F. J.

Schulz

Stephens

M. A. P.

Mogle

J. A.

(2013). The impact of daily arthritis pain on spouse sleep. Pain, 154, 1725–1731. doi:10.1016/j.pain.2013.05.020

46.

McCambridge

Witton

Elbourne

D. R.

(2014). Systematic review of the Hawthorne effect: New concepts are needed to study research participation effects. Journal of Clinical Epidemiology, 67, 267–277.

47.

Mehl

M. R.

(2017). The Electronically Activated Recorder (EAR): A method for the naturalistic observation of daily social behavior. Current Directions in Psychological Science, 26, 184–190.

48.

Mihailidis

Barbenel

J. C.

Fernie

(2004). The efficacy of an intelligent cognitive orthosis to facilitate handwashing by persons with moderate to severe dementia. Neuropsychological Rehabilitation, 14, 135–171.

49.

Mihailidis

Boger

J. N.

Craig

Hoey

(2008). The COACH prompting system to assist older adults with dementia through handwashing: An efficacy study. BMC Geriatrics, 8, Article 28. doi:10.1186/1471-2318-8-28

50.

Miller

(2012). The smartphone psychology manifesto. Perspectives on Psychological Science, 7, 221–237.

51.

Millner

A. J.

Lee

M. D.

Nock

M. K.

(2017). Describing and measuring the pathway to suicide attempts: A preliminary study. Suicide and Life-Threatening Behavior, 47, 353–336.

52.

Mojtabai

Jorm

A. F.

(2015). Trends in psychological distress, depressive episodes and mental health treatment-seeking in the United States: 2001–2012. Journal of Affective Disorders, 174, 556–561.

53.

Mojtabai

Olfson

Han

(2016). National trends in the prevalence and treatment of depression in adolescents and young adults. Pediatrics, 138(6), Article e20161878. doi:10.1542/peds.2016-1878

54.

Nahum-Shani

Hekler

E. B.

Spruijt-Metz

(2015). Building health behavior models to guide the development of just-in-time adaptive interventions: A pragmatic framework. Health Psychology, 34 (Suppl.), 1209–1219. doi:10.1037/hea0000306

55.

Nahum-Shani

Smith

S. N.

Spring

B. J.

Collins

L. M.

Witkiewitz

Tewari

Murphy

S. A.

(2018). Just-in-time adaptive interventions (JITAIs) in mobile health: Key components and design principles for ongoing health behavior support. Annals of Behavioral Medicine, 52(6):446–462. doi:10.1007/s12160-016-9830-8

56.

National Institutes of Health. (1979). The Belmont report. Retrieved from https://www.hhs.gov/ohrp/sites/default/files/the-belmont-report-508c_FINAL.pdf.

57.

National Public Radio & Edison Research. (2017). The smart audio report. Retrieved from http://nationalpublicmedia.com/smart-audio-report/

58.

Nebeker

Lagare

Takemoto

Lewars

Crist

Bloss

C. S.

Kerr

(2016). Engaging research participants to inform the ethical conduct of mobile imaging, pervasive sensing, and location tracking research. Translational Behavioral Medicine, 6, 577–586.

59.

Nelson

B. W.

Allen

N. B.

(2018). Accuracy of wearables during a continuous and ecologically valid 24-hour period that approximates actual consumer device use conditions within an individual. JMIR Preprints. doi:10.2196/preprints.10828

60.

Nelson

B. W.

Byrne

M. L.

Simmons

J. G.

Whittle

Schwartz

O. S.

Reynolds

E. C.

. . . Allen

N. B.

(2017). Adolescent sympathetic activity and salivary C-reactive protein: The effects of parental behavior. Health Psychology, 36, 955–965. doi: 10.1037/hea0000516

61.

Ochs

Shohet

Campos

Beck

(2010). Coming together at dinner: A study of working families. In. B. Schneider (Ed.), Workplace flexibility: Realigning 20th-century jobs for a 21st-century workforce (pp. 57–70). Ithica, NY: ILR Press.

62.

Okamoto-Mizuno

Mizuno

(2012). Effects of thermal environment on sleep and circadian rhythm. Journal of Physiological Anthropology, 31, Article 14. doi:10.1186/1880-6805-31-14

63.

Philipose

Fishkin

K. P.

Perkowitz

Patterson

D. J.

Fox

Kautz

Hahnel

(2004). Inferring activities from interactions with objects. IEEE Pervasive Computing, 3(4), 50–57. doi:10.1109/MPRV.2004.7

64.

Pietromonaco

P. R.

Collins

N. L.

(2017). Interpersonal mechanisms linking close relationships to health. American Psychologist, 72, 531–542.

65.

Pisani

A. R.

Wyman

P. A.

Mohr

D. C.

Perrino

Gallo

Villamar

. . . Brown

C. H.

(2016). Human subjects protection and technology in prevention science: Selected opportunities and challenges. Prevention Science, 17, 765–778.

66.

Reavley

N. J.

Jorm

A. F.

(2014). Mental health reform: Increased resources but limited gains. Medical Journal of Australia, 201, 375–376.

67.

The Regents of the University of California. (2016). Connected and Open Research Ethics (CORE) Platform. Retrieved from http://thecore-platform.ucsd.edu/

68.

Reis

H. T.

Gosling

S. D.

(2010). Social psychological methods outside the laboratory. In Fiske

S. T.

Gilbert

D. T.

Lindzey

(Eds.), Handbook of social psychology (pp. 82–114). Hoboken, NJ: John Wiley & Sons Inc. doi:10.1002/9780470561119.socpsy001003

69.

Repetti

R. L.

Reynolds

B. M.

Sears

M. S.

(2015). Families under the microscope: Repeated sampling of perceptions, experiences, biology, and behavior. Journal of Marriage and Family, 77, 126–146.

70.

Roberto

C. A.

Kawachi

(2014). Use of psychology and behavioral economics to promote healthy eating. American Journal of Preventive Medicine, 47, 832–837.

71.

Ronzio

C. R.

Guagliardo

M. F.

Persaud

(2006). Disparity in location of urban mental service providers. American Journal of Orthopsychiatry, 76, 37–43.

72.

Saxbe

D. E.

Repetti

R. L.

Graesch

A. P.

(2011). Time spent in housework and leisure: Links with parents’ physiological recovery from work. Journal of Family Psychology, 25, 271–281.

73.

Shiffman

Stone

A. A.

Hufford

M. R.

(2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 1–32.

74.

Singla

Cook

D. J.

Schmitter-Edgecombe

(2010). Recognizing independent and joint activities among multiple residents in smart environments. Journal of Ambient Intelligence and Humanized Computing, 1, 57–63.

75.

Statista. (2017). Smart home. Retrieved from https://www.statista.com/outlook/279/109/smarthome/united-states#market-smarthomes

76.

Substance Abuse and Mental Health Services Administration. (2015). A treatment improvement protocol: Using technology-based therapeutic tools in behavioral health sciences. Treatment improvement protocol (TIP) series 60. HHS Publication No. (SMA) 15-4924. Retrieved from http://www.medicaid.gov/medicaid-chip-program-information/by-topics/benefits/mental-health-services.html

77.

Tavernier

Adam

E. K.

(2017). Text message intervention improves objective sleep hours among adolescents: The moderating role of race-ethnicity. Sleep Health, 3, 62–67.

78.

Torous

Nebeker

(2017). Navigating ethics in the digital age: Introducing Connected and Open Research Ethics (CORE), a tool for researchers and institutional review boards. Journal of Medical Internet Research, 19(2), Article e38. doi:10.2196/jmir.6793

79.

Torous

Roberts

L. W.

(2017a). The ethical use of mobile health technology in clinical psychiatry. The Journal of Nervous and Mental Disease, 205, 4–8.

80.

Torous

Roberts

L. W.

(2017b). Needed innovation in digital health and smartphone applications for mental health: Transparency and trust. JAMA Psychiatry, 74, 437–438.

81.

Turk

M. A.

Pentland

A. P.

(1991). Face recognition using eigenfaces. Journal of Cognitive Neuroscience, 3, 72–86.

82.

van de Leemput

I. A.

Wichers

Cramer

A. O.

Borsboom

Tuerlinckx

Kuppens

. . . Derom

(2014). Critical slowing down as early warning for the onset and termination of depression. Proceedings of the National Academy of Sciences, USA, 111, 87–92.

83.

Wang

P. S.

Lane

Olfson

Pincus

H. A.

Wells

K. B.

Kessler

R. C.

(2005). Twelve-month use of mental health services in the United States: Results from the National Comorbidity Survey Replication. Archives of General Psychiatry, 62, 629–640.

84.

Wang

S. W.

Repetti

Sears

(2013). Using direct observational methods to study the real lives of families: Advantages, complexities, and conceptual and practical considerations. In Grzywacz

J. G.

Demerouti

(Eds.), New frontiers in work and family research (pp. 191–210). East Sussex, England: Psychology Press.

85.

Zhao

Adib

Katabi

(2016, October). Emotion recognition using wireless signals. In Gruteser

Chen

(eds.), MobiCom ’16: Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking (pp. 95–108). doi:10.1145/2973750.2973762

86.

Zhao

Yue

Katabi

Jaakkola

T. S.

Bianchi

M. T.

(2017). Learning sleep stages from radio signals: A conditional adversarial architecture. Proceedings of the 34th International Conference on Machine Learning, PMLR, 70, 4100–4109.