Extending the Passive-Sensing Toolbox: Using Smart-Home Technology in Psychological Science

Locating smart-home technology within the psychological assessment “space”

The use of smart-home devices for psychological research reveals both common and distinct features compared with other assessment techniques. To highlight and clarify these unique and overlapping capabilities of smart-home methods in relation to other techniques, we have used a framework drawn from Mehl (2017). In his description of the electronically activated recorder (EAR; a wearable research tool that collects recordings of ambient sounds), he identified a number of key dimensions on which psychological assessment techniques may vary, including (a) ecological versus nonecological (e.g., laboratory) setting, (b) active versus passive measurement (i.e., whether the method requires an active response from participants to collect data), and (c) first-person perspective versus third-person or objective perspective (i.e., whether the assessment method is based primarily on the participant’s point of view). Figure 1 presents a three dimensional “assessment space” that is defined by these continua, and we have approximated the placement of various assessment methods within this space to identify their common and unique features.

OPEN IN VIEWER

Fig. 1.

Three dimensional “assessment space.” The x-axis represents a dimension of active versus passive measurement (i.e., whether the method requires an active response from participants to collect data), the y-axis depicts the dichotomous dimension of first-person versus third-person or objective perspective (i.e., whether the assessment method is based primarily on the participant’s point of view), and the z-axis represents the dichotomous dimension of ecological versus nonecological (e.g., laboratory) setting methods. This figure provides a conceptual model showing the ways in which smart-home technology differs from other recent passive-sensing and active-response methods as well as examples of other routinely used assessment methods. Note that points in the lower left of the figure have been jittered for clarity. fMRI = functional magnetic resonance imaging; EMA = ecological momentary assessment; EAR = electronically activated recorder.

The most commonly used assessment method in psychological science, self-report questionnaires, defines one corner of the cube (i.e., active, first person, usually nonecological) and, consequently, so does a large proportion of psychological research. These methods are optimized for assessing participants’ subjective experiences and judgments, but are also limited in a number of ways—as noted above. IRM methods, such as EMA (Shiffman et al., 2008), and passive methods, such as those using EAR devices (see Mehl, 2017) or smartphone and wearable devices (Harari et al., 2016), are all typically associated with assessment in ecological settings; however, EMA is an active technique, whereas electronic recorders and mobile and wearable computing methods are typically passive. The low participant burden associated with these passive techniques enables more intensively repeated and sustained assessment, and therefore they can provide higher temporal resolution (e.g., minutes, seconds), and potentially longer assessment periods, than are possible with EMA methods (with temporal resolution usually in the range of hours to days). Smart-home methods are also passive and ecological, and thus many of their strengths and limitations overlap with those of mobile and wearable techniques. The main features that distinguish smart-home techniques from other passive-sensing methods include their location (i.e., in the home) and the fact that they are particularly suited to third-person assessment. This latter point, however, is not straightforward. Although sensors worn on a participant’s body, such as smartphones or wearables (e.g., wearable cameras), provide objective data, they often do so from the individual’s perspective (e.g., what that individual can see or hear), whereas smart-home methods are more suited to observing the person from an external viewpoint (e.g., observing the individual’s behavior from fixed cameras). However, it should be noted that some sensors on smartphones and wearables do provide data on behavior from a third-person perspective, just not direct observation of that behavior (e.g., measures of geographic movement or physical activity).

Compared with other passive-sensing methods, a number of more pragmatic and less conceptual features of smart-home assessment may have implications for their feasibility of use in specific clinical and research settings. For example, compared with smartphone and wearable devices, smart-home devices

Smart-home methods are best suited for assessing the (a) continuous collection and (b) passive collection of (c) longitudinal data and (d) multimodal data on (e) objective, third-person observable individual and relational interactions, appliance usage, and events within (f) an ecologically valid context, the home. It is important to note that smart-home technology is not the panacea for the methodological limitations of other methods. Smart-home technology has its own limitations, such as being limited to behavioral observation in the home, and some devices (e.g., cameras, sound recorders) being fixed in position, making them unable to “follow” participants as they move around their homes. Moreover, the home is considered a distinctly private space, and some participants may find smart-home assessment techniques to be particularly intrusive (see below regarding ethical issues). However, these various assessment techniques (e.g., EMA, mobile and wearable computing, self-report, smart home) can complement one another by being used in combination in order to extend the methodological toolbox.

Combining existing EMA self-report methods with smart-home technology will allow for these methods to capitalize on their respective strengths and allow for the assessment of both first-person subjective experience and judgments and third-person observation of behavior and affect. This will be important for research questions that require both an understanding of a person’s experience of their environment and an objective understanding of what that current environment entails. For example, the psychological constructs of loneliness and social isolation are two overlapping yet distinct constructs that cannot be fully evaluated without the merging of self-report (e.g., EMA) and smart-home methods. Within a home setting, a participant might report being subjectively lonely even when surrounded by people (e.g., family members, roommate, friends), whereas another participant might report low levels of loneliness when alone. Using either EMA or smart-home technologies alone would miss key information that might be important to providing a complete picture of both the subjective and the environmental aspects of loneliness and social isolation. In contrast, combining wearable, smartphone, and EMA methods in daily environments may help tease apart these constructs in real-world environments outside the home.

Applications of smart-home methods in psychological science

Smart-home technologies have multiple applications: research, assessment, and intervention. In terms of their relevance for intervention, they can be used both as an intervention tool (e.g., to provide behavioral feedback and suggestions for behavior change via “nudges”; see Hertwig & Grüne-Yanoff, 2017) and as a method of evaluating the effectiveness of interventions designed to change home-based behaviors—for example, family therapy and treatment of home-based behaviors related to mental-health disorders (e.g., obsessive compulsive disorder or sleep problems). The use of smart-home technologies, especially in conjunction with EMA, smartphone, and wearable technology, may also present an opportunity for psychological science to move toward precision psychology—improving psychological and behavioral health by providing nuanced assessment of individual differences in behavioral patterns.

In the clinical realm, reducing the public burden associated with mental-health disorders by improving treatment is a pressing need (Kessler et al., 2003; Mojtabai & Jorm, 2015; Reavley & Jorm, 2014), especially among young people (Mojtabai, Olfson, & Han, 2016), minorities (Ronzio, Guagliardo, & Persaud, 2006), and people in rural environments who have reduced access to mental-health services (Wang et al., 2005). Smart-home technologies may facilitate a new wave of evidence-based psychological intervention that can be delivered from a distance and may be able to be tailored to the individual rather than to the disorder. Clinical psychological science would benefit from such research to improve treatment outcomes, as well as to facilitate more timely interventions. For example, “just-in-time” interventions highlight how the timing of an intervention can be as important as the specific type of intervention used (Nahum-Shani, Hekler, & Spruijt-Metz, 2015; Nahum-Shani et al., 2018), and smart-home technology may be able to be used to trigger certain interventions at the time of greatest need or effectiveness as discussed in greater depth below, but first we briefly review existing smart-home research.

Feasibility of installation

Researchers have tested the feasibility of having participants (age 50 or over) self-install a smart-home kit (Hu et al., 2016). Self-installation components consisted of server boxes, relays, temperature sensors, magnetic door sensors, motion sensors, power cords, Ethernet cables, batteries, and adhesive strips. Overall, results indicated a high rate of installation success for most devices (failure rate < 12%), except for door sensors (failure rate = 46.15%), indicating that smart-home kits are intuitive for nontechnical seniors to install. Success rates are likely to be higher with younger generations more familiar with modern technology.

Cognitive assessment and longitudinal data collection

Some research has begun to use smart-home technologies in conjunction with machine learning algorithms to longitudinally assess cognitive health. For example, smart-home technology has been able to distinguish between healthy older adults and those suffering from dementia based on how well participants performed activities in their home environments (Dawadi et al., 2011; Dawadi, Cook, & Schmitter-Edgecombe, 2013). These researchers have also demonstrated feasibility of longitudinal research using these methods by conducting a 2-year study to determine changes in participants’ cognitive and physical health behaviors (Dawadi, Cook, & Schmitter-Edgecombe, 2014). These methods allowed researchers to capture physical mobility patterns, sleep behaviors, and bed-to-toilet transitions, as well as activities such as cooking, eating, relaxing, personal hygiene, and leaving the home. Overall, cognitive health was significantly associated with changes in sleep and activity. These findings provide a proof of concept that this technology can be used to further psychological research by identifying and predicting aspects of both cognitive functioning and physical health.

Behavioral interventions

Similar technologies, though not smart-home based, have been used for behavioral interventions in elderly populations. For example, some studies have focused on prompting seniors with dementia to increase hand-washing compliance (Labelle & Mihailidis, 2006; Mihailidis, Barbenel, & Fernie, 2004; Mihailidis, Boger, Craig, & Hoey, 2008). Findings indicated that behavioral “nudging” with audio and/or video prompts could significantly increase the completion of hand-washing steps without a caregiver present (by 25%; Mihailidis et al., 2004) and significantly reduce the need for human caregiver instruction (by 78%; Mihailidis et al., 2008). Indeed, as the robust literature on behavioral nudging and therapeutic feedback have shown, these interventions can be personalized (Hertwig & Grüne-Yanoff, 2017; Kaptein, Markopoulos, De Ruyter, & Aarts, 2015) and can be an important psychological component for behavior change (for reviews, see DiClemente, Marinilli, Singh, & Bellino, 2001; Marchiori, Adriaanse, & De Ridder, 2017). Although identifying when participants need to wash their hands is probably less nuanced than recognizing other potential targets for behavior change (e.g., emotional shifts associated with interpersonal conflict or mental-health states), these findings do provide some preliminary support for the notion that smart-home technologies can be used not only to identify behaviors but also to potentially facilitate behavior change.

Instrumental behavior, device use, and physical-activity recognition

Researchers have also used smart-home devices to recognize participants’ activity and instrumental behavior. One way to assess participants’ interaction with objects is to use radio-frequency identification (RFID) tags, which can be attached to household items. These small and unobtrusive sensors are battery free and inexpensive. An RFID reader connected to other devices, such as a smartphone, can then track participants’ interaction with objects to index activities of daily living (Philipose et al., 2004). Other tools allow for the recognition of independent versus relational behavior in smart-home environments by using motion sensors and probabilistic models (Singla, Cook, & Schmitter-Edgecombe, 2010). Such methods can be used to assess when participants are independently performing activities in their home environment as opposed to interacting with other people. Current technological advances may allow for the consolidation of smart-home technologies, making some sensors, such as RFIDs, obsolete in the future. For example, Amazon has recently opened a store, Amazon Go, that has no cashier (https://www.amazon.com/b?node=16008589011). Customers simply enter the store, pick up the items they want, and then leave. Amazon uses technology similar to that used in self-driving cars (machine deep learning, computer vision, sensor fusion) to identify a customer and the products they purchased. In the near future, such technologies could be adapted to identify human behaviors, such as medication adherence and food choice, within the home.

Although smart-home research has been increasing in the computer and medical sciences, its potential has yet to be leveraged for furthering psychological theory and science. Below we discuss ways in which this technology can advance theory and then offer two salient examples of relevant psychological and behavioral constructs that are potential targets for smart-home research, including their potential role in assessment and intervention.

Interpersonal theories

Given the significance of home environments for close interpersonal relationships, smart-home assessment may be especially relevant to theories of relational processes. For example, the theory of social buffering can be further evaluated and extended by studies incorporating smart-home technologies. Social buffering posits that supportive social relationships can buffer (or moderate) the impact of stress on psychological and biological functioning (Hostinar, Sullivan, & Gunnar, 2014; Nelson et al., 2017; Pietromonaco & Collins, 2017). This body of research has been largely dependent on discrete laboratory paradigms that observe individual and family functioning or use self-reported measures of perceived relational functioning. A study using a smart-home approach would allow for a novel investigation that might extend the theory of social buffering in ways that are difficult (if not impossible) to achieve with other techniques.

First, smart-home observation allows for the objective description of third-person observation of behaviors within a key ecological environment (i.e., the home), thereby extending the evaluation of this theory to observed behaviors in the home (as opposed to subjective self-report of social behavior in the home or objective observation of social buffering in the laboratory). This may ultimately bolster the ultimate validity and translatability of the findings into what is clearly a key ecological environment for these processes.

Second, it is possible that the temporal dynamics of social-buffering effects may be observable at smaller timescales, such as when an act of social support results in an immediate reduction of behavioral or physiological indices of stress (e.g., eating behavior, sleep, angry or dysphoric vocal affect). Smart-home devices would allow for continuous recording of observed behavior to advance the understanding of when and how social buffering takes place. This would provide a unique perspective distinct from that available from continuous first-person recordings with wearables or less temporally detailed EMA of subjective report on social experiences.

Third, the longer time periods over which data can be collected with smart-home methods could help to determine the time lag between social support and buffering effects and how long they are sustained. The ability to compare buffering effects over different timescales may help to adjudicate key mechanisms (e.g., whether the effects are mediated by immediate autonomic physiology or longer-term changes in social and health behaviors) and, ultimately, to guide interventions.

Finally, the passive collection of these data in home environments will prevent participants from having to pause their normal activities to provide a response or enter a novel laboratory environment, therefore potentially allowing for a more natural unfolding of social buffering in a real-world context.

Emotion dynamics

Data that allows for intensive temporal sequencing of emotional expressions and behaviors is essential to fully assess and refine theories of emotional dynamics, such as emotional inertia and critical slowdowns. Emotional inertia is a phenomenon in which the autocorrelation of an individual’s emotions over time predicts psychological maladjustment, rumination, and depressive severity (Koval & Kuppens, 2012; Koval, Kuppens, Allen, & Sheeber, 2012; Kuppens, Allen, & Sheeber, 2010; Kuppens et al., 2012); critical slowdowns, which are partly characterized by greater variance in emotion, higher temporal autocorrelation of emotion, and greater correlation of emotions of similar and dissimilar valence, have also been shown to predict transitions toward depression (van de Leemput et al., 2014). These phenomena have thus far largely been studied in discrete laboratory settings. Using smart-home devices would allow for the passive collection of observed third-person behavior and affect as they unfold within the home setting, which may increase the ecological validity of laboratory studies and translate these findings into real-world settings. In addition, the temporally detailed and longitudinal collection of data may elucidate both micro-level (moment to moment) and macro-level (week to week) inertia of behavior and affect, which have yet to be investigated. These advantages would address a number of significant questions regarding these phenomena, including their persistence across contexts (interpersonal and environmental), the time frames over which they are observable, and the relationship between the temporal dynamics of different measures of emotion. Furthermore, this method could be used in conjunction with self-reported methods to see how both observed and experienced inertia of affect and behaviors relate to one another across time.

Risk for suicide

Finally, the application of smart-home technology can be used for furthering our understanding of psychopathological processes that typically occur in the home. One salient example that we discuss in detail here is the psychological and behavioral processes associated with acute risk for mental-health crises, particularly suicide attempts. There are a number of long-standing theories of suicidal behavior, such as Beck’s hopelessness theory and Joiner’s interpersonal theory (Beck, Kovacs, & Weissman, 1975; Joiner, 2007). In particular, because of methodological limitations, these processes have largely been studied as distal risk factors for suicide attempts, whereas some of the affective and social processes described in these theories (i.e., hopelessness, thwarted belongingness/perceived burdensomeness) may in fact be more potent as short-term risk factors. For example, the onset of suicidal ideation and selection of suicide methods often occur years before an attempt, whereas the proximal steps to suicide often occur within a week, and mostly within hours, before the attempt (Millner et al., 2017), which suggests that different risk factors may operate at different time scales. The significance of smart-home and other IRM methods is that they offer, in some cases for the first time, the opportunity to examine these issues in a nonretrospective way.

Indeed, the majority of suicide attempts occur in the home (Kellermann et al., 1992), which emphasizes the particular suitability of home-based monitoring technology for addressing this issue. Recent research using timeline follow-back interview approaches and EMA methods have demonstrated alterations in first-person affect before a suicide attempt, which makes clear that there is typically a rapid increase in emotions related to dissatisfaction, hostility, loneliness, fear, hopelessness, and burdensomeness in the hours and minutes before a suicide attempt (Bagge, Littlefield, & Glenn, 2017; Kleiman et al., 2017). Using smart-home technology to passively collect intensively longitudinal data on behavioral processes associated with depression and suicide risk may better elucidate the behavioral patterns associated with these changes in psychological states (e.g., hopelessness, interpersonal distress), especially how these may precipitate and sustain suicidal risk. Data on observed third-person behavior collected in real time, rather than retrospective timeline follow-back interviews, may allow for a more accurate categorization of behavioral and affective dynamics that emerge along both discrete and longitudinal timescales leading up to a suicide attempt. Finally, using smart-home technologies in tandem with EMA methods would allow for the collection of observed facial, speech, and behavioral data as well as experiential data on affective states; such data could both allow for a greater understanding of short-term risk processes and have great clinical significance by detecting people who are likely to attempt suicide and facilitating timely intervention at moments of greatest risk.

In summary, naturalistic observation via smart-home technology provides a set of tools that, while not suited for all research questions, can make a unique contribution to psychological research and theory by providing what was described above: the (a) continuous collection and (b) passive collection of (c) intensively longitudinal and (d) multimodal data from a (e) third-person observational perspective that (f) preserves ecological validity by taking place outside of the laboratory and within participants’ own homes. Some of these methods may initially be labor intensive because they require human coding of behaviors until automated coding methods have been developed and validated, they primarily use stationary sensors, and they are limited to collection of data within the home environment. However, by providing intensive longitudinal data within the home environment that can capture concurrent multimodal processes (face, voice, physiology, behavior, device usage, home environment), from a third-person perspective, on both individual and relational functioning, in a temporally precise and ecologically valid manner that is independent from, yet complementary to, self-report, EMA, smartphone, and wearable devices, smart-home methods may plausibly allow us to address unique questions relevant to diverse psychological theories. Below we further explicate the potential contributions of smart-home methods to psychological-intervention science and behavior change.

Interpersonal relationships

In addition to potential research applications, smart-home devices can also be used for novel assessment and intervention techniques relevant to a wide range of home-based behaviors that can be targets for behavior change. As discussed previously, researchers have already used such devices to distinguish between healthy older adults and those suffering from dementia (Dawadi et al., 2011; Dawadi, Cook, & Schmitter-Edgecombe, 2013). Assessments of this kind could be extended to other populations, which could allow for diagnosis from a distance and a better understanding of the dynamics that are associated with deterioration or recovery of mental health and interpersonal functioning. Take for example a couple who attends psychotherapy to reduce patterns of interpersonal conflict. At the start of each session, the therapist, rather than relying solely on the couple to retrospectively report on their functioning in the past week, could pull up summary descriptive charts of relational quality (i.e., frequency and duration of conflictual interactions as detected by vocal tone and facial expression) as well as a host of other metrics (e.g., sleep habits) that may be influencing a couple’s mood.

Another relevant example for the realm of intervention science is the number of home-based clinical-research interventions that use video feedback to improve parenting skills, with the ultimate goal of improving family relationships and preventing mental health and other disorders in offspring. These typically require a researcher or clinician to enter a participant’s home to record naturalistic relational interactions between parent-child dyads. One such example is the Filming Interactions to Nurture Development (Fisher, Frenkel, Noll, Berry, & Yockelson, 2016) intervention. This intervention consists of a video-coaching program for caregivers of children, during which therapists record parent-child relational interactions within the home to positively reinforce existent supportive social interactions between caregivers and their children. Such interventions have the advantage of providing opportunities to practice skills in naturalistic settings, as opposed to bringing families into the laboratory or clinic, and have greatly propelled our understanding of relational and intervention science. However, these techniques are able to evaluate only specific interactions and for relatively short periods of time because of the burdens of microsocial coding. The assessment of in-home behavior using smart-home technologies would initially require an equal or greater burden of microsocial coding with a less frequent sampling rate to establish a “ground truth” to optimize machine-learning algorithms; however, after this is accomplished, smart-home devices could be combined with automated behavioral coding, which would allow for assessment of, and intervention in, a much larger range of in-home behaviors.

There are also a number of remote therapeutic interventions that could be administered with such technology. One potential is a therapeutic nudge, in which clients would be prompted to change an aspect of their behavior to increase adaptive functioning, such as modifying eating and sleep patterns (Marteau, Ogilvie, Roland, Suhrcke, & Kelly, 2011; Nahum-Shani et al., 2015; Roberto & Kawachi, 2014; Tavernier & Adam, 2017). For example, behavior and language captured using home-monitoring cameras could be analyzed in real time; future algorithms could determine and code for conflictual interpersonal behavior (e.g., deviation from acoustic norms and occurrences of aggressive behavioral posturing). If such deviations from normal behavioral and acoustic functioning are positively identified, then voice-activated speakers or text messaging could initiate therapeutic “nudges” (e.g., reminders to use interpersonal skills that have been taught during face-to-face therapy interventions) to move individuals out of conflict and toward resolution.

Sleep

Sleep is another example of a salient set of home-based behaviors that may be tracked and potentially modified by smart-home technology. Indeed, sleep disturbance is a common symptom of a variety of mental-health disorders, such as depressive, stress, and anxiety disorders (APA, 2013), demonstrating its relevance to mental health. The current standard for sleep measurement is polysomnography, which requires research participants or patients to spend a night in a sleep clinic. Diary methods have also been used and have provided valuable data on how relational functioning can influence sleep (Repetti et al., 2015), such as how one person’s health problems can affect the sleep behaviors of their spouse (Martire, Keefe, Schulz, Stephens, & Mogle, 2013). Recently, with the advent of consumer wearable devices, there has been a push toward using wearable actigraphy sensors to capture behavioral sleep data (Marino et al., 2013). Although smart-home technologies cannot yet capture all the physiological data that can be acquired through polysomnography, new methods involving radio-frequency signals sent via Wi-Fi can now be used to index respiration, heart rate (Adib, Mao, Kabelac, Katabi, & Miller, 2015; Zhao, Adib, & Katabi, 2016), and sleep stages (Zhao, Yue, Katabi, Jaakkola, & Bianchi, 2017) even in the absence of wearable sensors. In addition, a number of devices can be used to both study sleep habits and assess sleep hygiene (e.g., through sensor-equipped beds and pillows) and provide sleep interventions.

A range of key sleep variables, including sleep-onset latency (i.e., time from getting in bed to falling asleep), duration, and quality, are strongly influenced by environmental variables such as light-dark cycles and ambient temperature: Cooler temperatures are more conducive to sleep onset (Okamoto-Mizuno & Mizuno, 2012). Smart-home technologies may provide a means to study how each of these variables alters sleep patterns as well as to intervene at the contextually appropriate time by modifying each of these aspects of the home environment using multifactorial designs or microrandomized trials (Klasnja et al., 2015) to support healthy sleep practices through intervention.

Smart-home devices also have the potential to index and intervene in other behavioral patterns that may prove useful in improving healthy sleep. For example, evening routines around television and other screen use could be monitored to determine how the use of home devices influences sleep patterns. Monitoring device use could provide precise information on the timing and location of the devices that are being used and could examine how falling asleep on your couch while watching television and then waking up later in the evening influences sleep stages. It would also be possible to examine when, on nighttime awakening, participants simply stay in bed until they fall back to sleep, check devices, or move to certain rooms of the house (e.g., the kitchen or bathroom) before falling back to sleep and to evaluate how these behaviors affect overall sleep hygiene.

Moreover, these technologies also have potential applications for intervention for people suffering from sleep disorders. Nudging interventions, as described above, from voice-activated smart speakers, could prompt individuals that it is time to get ready for bed to reduce sleep variability. Indeed, similar nudging methods using text messages in the smartphone literature have been found to improve objective sleep duration in adolescents (Tavernier & Adam, 2017). Time for bed could be personalized by capturing individually indexed sleep onset and awakening times and combining this data with weekly calendars to optimize these variables. Simultaneously, automated thermostats could slowly lower bedroom and home temperatures on the basis of average activity-pattern data detected from motion sensors and video cameras. Furthermore, as an individual’s average bedtime approaches, light-emitting devices (e.g., smartphones, televisions, and tablets) could automatically reduce blue-light exposure (e.g., using the f.lux and Twilight apps or Apple’s iPhone Night Shift mode) and dim lights to create environmental cues for sleep. In each of these ways, new approaches to intervention can be tested to better understand sleep-hygiene practices to identify which smart-home behavior-change methods are most conducive to providing optimal sleep environments in order to improve sleep health.

Crowdsourcing digital ethics

Regarding the ethical collection of passive-sensing data, there is one promising avenue that could be used by researchers, IRBs, and industry partners: the Connected and Open Research Ethics (CORE) initiative that was recently designed and launched to develop ethical practices in digital research (Torous & Nebeker, 2017). The CORE initiative was launched in 2016 and provides a research space for members to share knowledge, resources, questions, and answers regarding the use of passive-sensing technology for research purposes (The Regents of the University of California, 2016). This platform currently contains members from over 10 countries with expertise spanning bioethics, privacy, technology, and research ethics. In other words, CORE crowdsources ethics from a transdisciplinary team to provide resources for both researchers and IRBs to help guide research design and to provide an ethical framework that encourages sound research practices.

Ethics for clinical application of digital technologies

Researchers have also begun to discuss the various ways in which to incorporate digital technologies into the lives of clients, patients, and participants in a way that provides safety and privacy (American Psychiatric Association, 2017) and also delivers these novel interventions in nonintrusive ways (Pisani et al., 2016). For example, the American Psychiatric Association has created an “App Evaluation Model” that provides mental-health providers with the following five steps to consider when incorporating digital technologies, specifically app usage, with clients: (a) background information, (b) risk/privacy and security, (c) evidence, (d) ease of use, and (e) interoperability (American Psychiatric Association, 2017). Likewise, other researchers provide a decision-tree model that allows providers to incorporate ethical safeguards into clinical practice (Torous & Roberts, 2017a).

Protecting confidentiality of participants and third parties

Similar to data collected via other passive-sensing technologies, data from smart-home technologies will be highly identifiable and will therefore require a greater degree of protection for participants and third parties (Nebeker et al., 2016; Torous & Nebeker, 2017). Important methods for achieving greater confidentiality include data encryption and automated identity recognition via facial or voice data. In one solution, continuous data collection would be saved to an encrypted cloud service protected according to the provisions of the Health Insurance Portability and Accountability Act (HIPAA). Video and voice data would then be scanned by automated facial and speech recognition software to match collected data with participants’ facial and audio samples (Hansen & Hasan, 2015; Jain & Li, 2011; Turk & Pentland, 1991). Data containing video or audio material that was not from a target participant, or that included the addition of a third party with a target participant, would be automatically deleted before any humans viewed the data. An additional layer of protection for third parties in couple studies could be achieved by programing smart-home devices to record only when consented participants’ Bluetooth-enabled wearable devices are within a specified diameter of one another, which would allow wearables to “speak” to one another and then trigger recording of devices. These techniques would not only reduce the degree of third-party data collected but also cut down the size of data files, which would likely be a concern for smart-home studies.