Psychology and Surveillance Capitalism: The Risk of Pushing Mental Health Apps During the COVID-19 Pandemic

The COVID-19 Pandemic, Disaster Capitalism, and the Mental Health App Push

According to the history of “disaster capitalism” outlined by Klein (2007), moments of economic crisis are often used by industry and powerful special interest groups to push through unpopular legislation that deepens inequality and undermines civil rights. The COVID-19 pandemic is already being used by governments to forward neoliberal economic policies and roll back regulations and civil rights protections (Adams, 2020). There have been increasing reports of distress in response to the pandemic in the public at large and clinicians suddenly have found that they are not able to meet in person with their clients. Both of these conditions have created the type of ideological environment that Klein (2017) argues is conducive to the machinations of disaster capitalism.

As social distancing restrictions began to impede in-person mental health treatments across the United States, leading medical journals such as The New England Journal of Medicine and The Lancet ran commentaries calling for the rapid expansion of remote service options beyond video sessions. In these commentaries, there was a clear emphasis on the urgent need to remove legal barriers that would prevent the widespread use of novel telehealth initiatives (e.g., Keesara et al., 2020; Liu et al., 2020). While some experts sought to normalize the experience of emotional distress during a crisis (Pfefferbaum & North, 2020), others raised the alarm arguing that “urgent action is required to transform health care delivery and to scale up our systems by unleashing the power of digital technologies” (Keesara et al., 2020, p. 1). In fact, when China moved mental health treatment to remote care during the initial outbreak, public policy experts recommended the use of AI screening programs and surveillance tools as a means to develop innovative telehealth platforms that could improve the effectiveness of clinical and emergency interventions (Liu et al., 2020). Other countries have also moved to ease regulations on telepsychology practices. For example, in the United States, the Department of Health and Human Services Office for Civil Rights changed its policies to reduce privacy protections. In March 2020, the Department of Health and Human Services Office for Civil Rights announced that it would waive penalties for Health Insurance Portability and Accountability Act violations against health care providers that used insecure communication technologies, such as FaceTime or Skype, during the COVID-19 pandemic (Centers for Medicare & Medicaid Service, 2020).

The push to expand telepsychology services and ease privacy restrictions goes beyond video visits and includes the dissemination of mental health apps designed to augment or replace traditional mental health services. These apps have proliferated over the last few years because they are easy to use, accessible, portable, and can reach people who do not seek help because of financial and other reasons. Past research on mental health apps has raised significant concerns about the lack of privacy protections and the potential for overdiagnosis. In fact, a review of 61 mental health apps found that the language used by these tools often promoted a flawed and pathologizing message that emotional responses to real life stressors are the result of abnormal neurophysiology (Parker et al., 2018).

Researchers have also found that the majority of these apps are not transparent about how they collect data and how that data will be used. A recent review of mental health apps found that 81% of apps sent data to Facebook or Google for use in data analytics or marketing and 92% sent that data to other third parties. The authors concluded, “users [of these mental health apps] are denied an informed choice about whether such sharing is acceptable to them [emphasis added]” (Huckvale et al., 2019). Despite these privacy concerns, policy makers have rushed to promote the use of mental health apps during the pandemic. In one example, the State of New York recently collaborated with Headspace to provide its mental health app to New Yorkers for free in response to the pandemic (Headspace, 2020).

From Biomarkers to Digital Phenotyping: Using Human–Computer Interactions to Predict Mood

Every app is shunting your data to 3^rd parties. The two goliaths [in this marketplace] are Facebook and Google. This means that when you download an app—it doesn’t stop there—[your anonymized behavioral surplus data] is going to 3^rd parties, primarily Facebook and Google. (Zuboff, 2019d)

Mental health apps vary in scope. Some apps, for example, Headspace, provide tips on meditation and improving sleep but do not offer diagnostic or therapeutic services such as access to a therapist. Other mental health apps go well beyond offering mindfulness and CBT-based interventions and use complex machine learning algorithms and AI to attempt to predict and diagnose mental distress based on users’ digital fingerprints. Mindstrong, a Silicon Valley-based start-up and self-described as a health care and tech company hybrid, is the current leader in the use of this technology. The former Chief Technology Officer of the antipiracy software company, MarkMonitor Inc., Paul Dagum, founded the company in 2014. Thomas Insel, former director of the National Institute of Mental Health, and Richard Klausner, the founder and director of Juno Therapeutics (a pharmaceutical company), are cofounders. In the world of digital health, Mindstrong is well funded, backed by tens of millions from venture capitalist firms, including Bezos Expeditions. In May of 2018, the company developed a smartphone app that “can tell you’re depressed before you know it” (Metz, 2018), by using a “paradigm shifting” technology.

What if we can detect symptoms getting worse? What if we can predict it? We’re measuring human-computer interactions, and using machine learning to develop breakthrough technology. This technology is integrated into our care platform to help guide targeted proactive care. (Mindstrong, 2020a)

The paradigm shifting technology to which Mindstrong refers is digital phenotyping. Unlike other mental health apps, Mindstrong is less interested in mining content such as health data and explicit personal information. Rather, the app’s digital phenotyping is focused on how users interact with their smartphones. Scrolling, clicking, tapping, and other touch screen behaviors are analyzed with machine learning to predict their cognition and mood (Mindstrong, 2020c). This human–computer interaction model—analyzing the way information is presented to the user and repeated measures of a user’s response time—is described as “provid[ing] passive, ecological, in-the-moment insight into a user’s cognitive and emotional state” (Dagum, 2019, p. 96). The ecological, real-time information (obtained by tracking and analyzing how digital devices are being used) is theorized to be an approximation for mental processing speed. Empirical support for this theory is based on a small (N = 27) study that correlated neurocognitive scale results with predictions derived from a kernel principal components analysis of over 1,000 human–computer interactions (Dagum, 2018, 2019). The aggregated results of these interactions (i.e., the patterns and timings of user activity on touch screen devices) are referred to as digital “biomarkers.” These digital biomarkers are then keystroke captured with an application running passively in the background of the device.

It should be noted that kernel principal components analysis is common in biomarker research as it enables detection and analysis of massive amounts of data that are related to the variable (often in nonlinear ways) under investigation (Schölkopf et al., 1998; Shiokawa et al., 2018). Yet the central question remains: What is the evidence to support use of the use of digital biomarkers and phenotyping to predict mental health symptoms and mood disturbances? There is one small (N = 23) prospective cohort study registered on clinicaltrials.gov whose aim is to validate Mindstrong’s app (ClinicalTrials.gov, 2017). Although the study was completed in May 2019, there are no study results posted and no peer-reviewed papers have been published in the scientific literature. Moreover, cognitive tests have not been proven to be accurate predictors of depression (Scult et al., 2017) and there could be many reasons why a person is typing more slowly (e.g., they have gloves on). However, in 2019, Dagum published a book chapter in which he reports on a subset of 10 of the 23 participants to support Mindstrong’s technology as a potential “continuous ecological surrogate for laboratory-based assessments for mood disorders and of clinical severity” (Dagum, 2019, p. 101).

When viewed through the lens of disaster capitalism, it is not surprising that, despite the limited empirical data to support its product, Mindstrong recently modified its website and encouraged viewers to download the app on a page titled, “Supporting you through COVID-19” (Mindstrong, 2020b). There has been a proliferation of news stories (e.g., Bender & Pannett, 2020; Brooks, 2020; Daley, 2020) reporting that mental health conditions, particularly depression and anxiety disorders, are developing or worsening as a result of the pandemic. Thus, on the one hand, using a mental health app to track one’s mood and mental health appears to be a critical innovation during a pandemic and a time of social distancing. In fact, there have been reports that use of mental health apps is on the rise since the pandemic began (Heilweil, 2020), including one that specifically targets children and adolescents (Staines, 2020). On the other hand, we must also consider the limited data available to support the use of digital phenotyping for the prediction of mental health conditions, the conflicted nature of the empirical evidence (i.e., it appears that one of the cofounders of the company is the only one to date that has assessed the efficacy of the company’s product) and the large financial incentive to promote this app. In the following section, we use the framework of surveillance capitalism to identify a less obvious, but equally concerning, iatrogenic effect of this technology—the use of users’ data to both predict and shape behavior.

Mindstrong and Surveillance Capitalism: Writing the Music to Make Them Dance

It is no longer enough to automate information flows about us; the goal now is to automate us. (Zuboff as quoted in Naughton, 2019).

Shoshanna Zuboff (2019c) coined the term surveillance capitalism to describe a new form of capitalism—a behavioral futures marketplace. The term surveillance capitalism, points to the connection between digital tools, collecting, and monitoring of data from large swaths of the population, and the promotion of consumer-oriented behavior that further the interests of neoliberal capitalism. Significant debates exist regarding how to provide genuine informed consent when personal information shared online is used to develop targeted advertisements (Smit et al., 2014). What Zuboff exposes, however, goes well beyond such debates. She reveals that the most significant data collected and utilized by digital technologies is not the content of what is shared online, but behavioral data about how one navigates the online environment. These data are then used to produce advanced predictions of human behavior. In an age of surveillance capitalism, it is not just behavioral data, such as posting on Facebook that you are going to dinner, that is collected and sold, it is the “behavioral surplus data,” concerning “whether you say, ‘I’ll meet you later’ or whether you say, ‘I’ll meet you at 6:45,’” that is valued (Zuboff, 2019b).

Indeed, within this “information panopticon,” Zuboff points out that health care apps play a significant role in harvesting this type of meta-data and feeding it into the prediction markets and third parties, an architecture of power she christens “Big Other” (Zuboff, 2015). This meta-level behavioral surplus data is analyzed and utilized or sold for its predictive value:

You download a diabetes app, it takes your phone, it takes your microphone, it takes your camera, it takes your contacts. Maybe it helps you manage your diabetes a little bit, but it’s also just a part of this whole supply-chain dynamic for behavioral surplus flows. The stuff that they’re taking from you has nothing to do with the diabetes functionality for which you downloaded the app. Absolutely nothing. It’s simply siphoning off data to third parties for other revenue streams that are part of these surveillance capitalists’ ecosystems. (Zuboff, 2019a)

In light of the fact that the public is looking for (and being encouraged to access) mental health advice and services during the COVID-19 pandemic, mental health apps like Mindstrong’s, have great appeal. Although users of Mindstrong’s app may not be thinking about this, the boundary between predicting mood and shaping or controlling behavior is, of course, tenuous. Zuboff quotes one data scientist stating, “We can engineer the context around a particular behavior and force change that way. . . . We are learning how to write the music, and then we let the music make them dance” (Zuboff, 2019b, para. 33). As Hacking’s “looping effect of human kinds” (Hacking, 1995) demonstrates, attempts to classify patterns of behavior simultaneously create possibilities for subjectivity. The looping effect describes a recursive process where the development of new classifications leads those classified to identify with the classification and to increasingly behave in ways that conform with the expectations of the classification (Brinkmann, 2005). As the fields of psychiatry and psychology develop and propose new constructs for classifying people, these classifications inevitably reflect the philosophical assumptions of those proposing the framework, particularly those concerning agency and personhood (Kirmayer & Gómez-Carrillo, 2019). These classifications are circulated in societal discourses and reinforced by the machinations of institutions, which serves to maintain the perceived legitimacy of the initial assumptions, limiting possibilities for paradigmatic change. For example, when exposed to biomedical explanations for depression, people are more likely to attribute causality to stable and internal processes, rather than as the result of social and environmental conditions (Lebowitz & Appelbaum, 2019; Zimmermann & Papa, 2019). The psy-disciplines, as institutions that often posit and legitimate these classifications, participate in the production of forms of subjectivity that satisfy the needs of current ideological systems, such as neoliberal capitalism, and undermine efforts to reform prevailing structures (Cosgrove & Karter, 2018).

Within this framing, mental health apps such as Mindstrong can be understood to operate at a powerful intersection of digital surveillance technologies in the service of markets and the cultural legitimacy granted to the psy-disciplines. Casting behavioral surplus data in the seemingly objective and scientific language of mental disorders may thus serve to legitimate and naturalize psychiatric classification systems, while simultaneously shaping the behavior of the person using the app toward the demands of capitalism. Additionally, casting behavioral data in this way runs counter to a humanistic focus on dignity, meaning making, and the sociopolitical determinants of well-being, as the needs of the market and the expertise of psy-disciplines reign supreme.

Analyzing the promotion of a mental health app during this pandemic, especially when seen through the double lens of disaster and surveillance capitalism and from a humanistic perspective, raises complicated ethical questions and issues. Even a cursory review of Mindstrong’s disclosure reveals that Zuboff’s warning about the siphoning off of behavioral surplus data to for-profit entities should be strongly heeded, as the following statements are made:

We may use your Personal Information to create data that is de-identified in accordance with HIPAA, other applicable laws and our Organization Agreements with applicable Organizations. This de-identified information is not Personal Information, because it cannot be used to identify you, and may be used by us for any lawful purpose. . . . We may share de-identified information and other De-identified Non-Personal Information in all legally permissible ways. (Mindstrong, 2019, para. 10-11, Italics added)

The Ethical Entanglements of Promoting Mental Health Apps in a Pandemic: Lessons From Humanistic Psychology

Humanistic psychologists have historically been at the forefront of challenging conceptual models that reduce human experience to quantifiable measures and metrics (Aanstoos, 2015; Elkins, 2016; Taylor, 1999; Wertz, 1998); the “third-force” of humanistic psychology arose in opposition to behaviorist interventions meant to produce populations “beyond freedom and dignity” (DeCarvalho, 1990). Humanistic psychologists remain at the forefront of scholarly opposition and advocacy efforts to reform diagnostic practices based on a limited view of emotional distress (Kamens et al., 2017; Kinderman et al., 2020). Although the advent of digital phenotyping presents categorizations of behavior patterns of markedly increased complexity, these phenotypes remain abstractions based on cognitions and behaviors captured within a particular context (e.g., through the use of smartphones). As such, there remains the “hard problem” (Chalmers, 2007) that stands between any attempt to map diagnoses or phenotypes onto neurobiological processes (see, e.g., Karter, 2019). There is also an ethical imperative to avoid measurement-based symptom reduction models that fail to bear witness to the distressed person’s unique life circumstances. Thus, claims that digital phenotyping can predict neuronal function and mood should be interrogated by philosophical and humanistic psychologists alike.

Most important, the algorithmic model on which mental health apps are based takes “major depressive disorder” to be the object and neglects the unique experiences of the person in the midst of a COVID-19 pandemic. Computational analysis substitutes for an empathetic attunement to the lifeworld of the individual, for moods can now be “detected.” However, as Heidegger argued, human beings are not rational creatures with neutral mood states who may sometimes experience mood episodes (e.g., a “major depressive mood” episode). He invoked the concept of Befindlichkeit in order to demonstrate the impossibility of “having” a mood (Stimmung); to be human is to always be situated in and attuned to the world (Heidegger, 1927/1962). Indeed, the focus on the prediction of moods is antithetical to how humanistic psychologists understand personhood. This is because one is always “found” in a mood, for emotions engage (or impede engagement) with the (intersubjective) world; Dasein is always immersed in a particular comportment toward the concerns of its everyday life and activities. (Heidegger, 1962; see also Cosgrove et al., 2019). Thus, measuring human–computer interactions as a proxy for understanding a person and predicting their moods reduces the subject to a collection of data points identified by computational systems, and fails to appreciate the fact that people are always “absorbed” in the world at hand—the world is the very fabric of Dasein’s existence. The person using the app thus becomes reified through mathematical modeling and machine learning—a modern day example of Whitehead’s notion of “misplaced concreteness” and James’ “vicious abstractionism” (James, 1895; Whitehead, 1926). In this way, digital tracking and computational analysis, as a replacement for contemplative clinical practice, undermines possibilities for agency and autonomy in people struggling with emotional distress.

Moreover, it is not only the client’s agency and autonomy that are reduced through the use of digital phenotyping, but the clinician’s humanity is similarly reduced, for all that is needed when the app predicts a shift toward a negative mood, is a robotic technician. The intuitive talent of the caring professional who strives for a deeply felt attunement to the other (Churchill, 2014), and the critical importance of contemplative practice, have no place in this brave new algorithmic world. Mindstrong, for example, offers 20-minute text therapy sessions with “credible therapists” based on techniques such as CBT and emotion regulation. Yet no details about what constitutes a credible therapist are provided. As humanistic psychologists have long decried, technical training cannot substitute for attunement to the “affairs of consciousness” (Churchill, 2013) or to a depressed or anxious person’s effortful existing. Insofar as these “credible therapists” lack conceptual and structural competence (Karter & Kamens, 2019), they may very well facilitate unintended human rights violations (e.g., forced treatment). For example, Orwellian though it may sound, it is all too easy to imagine that when the techno-therapist receives an alert that the person’s digital behavior correlates with suicidality, first responders will be called in to hospitalize them.

Conclusion

By conflating well-being with narrowly defined technological fixes, mental health apps cannot address the psychosocial and sociopolitical determinants of mental health and the context in which people experience emotional distress. Clearly, this technology runs counter to humanistic psychology’s appreciation for an individual’s unique lifeworld and concomitantly, the importance of contemplative clinical practice. Also, as can be seen in this brief case study, there are limited data to support the use of digital phenotyping to predict negative mood states and symptom worsening. The double lens of disaster and surveillance capitalism shows that something more nefarious is happening than the promotion of a diagnostic tool that is not yet validated. Mental health apps that use digital phenotyping and other surveillance technologies position people as unwitting profit-makers; they take individuals at their most vulnerable and make them part of a hidden supply chain for the marketplace.