Abstract
The production and promotion of “AI” technology involves dehumanization on many fronts. I explore these processes of dehumanization and the role that cognitive science can play by bringing a richer picture of human cognition to the discourse.
The ways in which so-called “artificial intelligence,” 1 or “AI,” is described in the research literature, the popular press, and blogs or other advertizing copy from tech companies involve dehumanization on many fronts. Fortunately, cognitive scientists are well positioned to resist this trend on the basis of our research practice and expertise. Figure 1 summarizes both the kinds of dehumanization and the possiblities of resistance. In this article, I detail how the practice of AI enacts dehumanization and the ways in which cognitive scientists can push back. To set the stage, I describe my path to this topic and provide a working definition of dehumanization.
Six ways in which AI practice enacts dehumanization (top) and six ways cognitive scientists can resist (bottom).
Researcher’s Path
My path to this topic runs through two articles I coauthored. The first is Bender and Koller (2020), which was written in reaction to widespread claims that language models actually understand language. Language models are systems trained to output plausible sequences of words or letters based on a context of sequences of words or letters. Since about 2019, language models for English (e.g., GPT-3; Brown et al., 2020) model the distribution of word forms closely enough to produce seemingly coherent text. A linguistic perspective, however, shows that this coherence is in the eye of the beholder: Language models have no communicative intent nor understanding of what the word sequences mean. Language is a system of signs (i.e., pairings of form and meaning or signifier and signifed; de Saussure, 1959), but language models are trained only on form and have no access to the meaning side of it. What is not in the training data cannot be “learned” by the model.
The second article is Bender et al. (2021), in which we observed the trend, already very evident in 2020, toward ever larger language models and asked: What could go wrong here? We surveyed the literature on risks associated with language models and mitigation strategies. We considered environmental costs through the lens of environmental racism; financial costs and their impact on research participation; how the training data sets come to be filled with hegemonic viewpoints and worse, without documentation or accountability for their content; and finally how synthetic text-generating machines can reproduce the systems of oppression from their data sets while also misleading humans who cannot help but make sense of the text that we encounter. We coined the term stochastic parrots to evoke the way in which large language models, running as text-synthesis machines, “haphazardly [stitch] together sequences of linguistic forms . . . according to probabilistic information about how they combine, but without any reference to meaning” (p. 617).
I now often get asked: “How do I know that you’re not just a stochastic parrot?” I have decided I am not going to have conversations with people who will not posit my humanity as a basic axiom of the conversation. Acknowledging my privilege (living as a white cis person in the United States) to not have noticed before, that phrasing led me to think about dehumanization across AI.
Dehumanization: Working Definition
From the broad literature on dehumanization, I draw on two quotes as touch points in making a working definition. The first quote is from Kronfeldner (2021): “Dehumanization happens when people are depicted, regarded, or treated as not human or less human. . . . I start with such a thin notion since not much agreement exists beyond it in the scholarship on dehumanization” (p. xvii). The second quote is from Weheliye (2014):
If racialization is understood not as a biological or cultural descriptor but as a conglomerate of sociopolitical relations that discipline humanity into full humans, not-quite-humans, and nonhumans, then blackness designates a changing system of unequal power structures that apportion and delimit which humans can lay claim to full human status and which humans cannot. (p. 3)
Kronfeldner’s words are from the preface of an edited volume on dehumanization and reflect the difficulty of operationalizing a definition. Weheliye’s remarks, not directly a definition of dehumanization, are informative for their clarity about the experience of racialization.
From those starting points, I come to a three-part working definition: dehumanization is (a) the cognitive state of failing to perceive another human as fully human, (b) an act that expresses that cognitive state or otherwise entails the assertion that another human is not fully human, or (c) the experience of being subjected to an act that expresses a lack of perception of one’s humanity and/or denies human experience or human rights, or combinations thereof.
I use the phrase “fully human” to mean entitled to all rights recognized as human rights, equally in possession of internal life and point of view, and welcomed as one’s full self. This working definition acknowledges the cognitive process of the person doing the dehumanization, the acts that express it, and the experience of being the target of it.
Dehumanization in the Research, Development, and Sales of “AI”
This section outlines six ways in which the research, development, and sales of “AI” contribute to dehumanization.
The computational metaphor
Baria and Cross (2021) analyzed the computational metaphor in neuroscience: THE BRAIN IS A COMPUTER. 2 They noted it is a bidirectional metaphor, for which the other half, THE COMPUTER IS A BRAIN, is used pervasively by technologists. About the metaphor as a whole, Baria and Cross (2021) wrote that it “afford[s] the human mind less complexity than is owed, and the computer more wisdom than is due” (p. 2). In short, this metaphor builds up computers at the expense of how we understand humans. Furthermore, Baria and Cross identified in the computational metaphor a hierarchy of human value defined in terms of ideologies around intelligence, in which “rationality” is valued above “emotionality,” affording more power to those who display more machine-like qualities. Ultimately, “in its fake-ness as a human intelligence, AI paradoxically succeeds in being a more trustworthy form of intelligence, by being the epitome of rational thought” (p. 6).
Especially pernicious is the appropriation of the experiences of disabled people to assert the humanity of AI. This rhetorical turn was elaborated by Agüera y Arcas (2021), who asserted that large language models are like deaf-blind people. Under the heading “modality chauvinism,” he called on the writings of Daniel Kish, who is blind, and Helen Keller, who was deaf-blind, to argue that no one sensory system is required for humans to develop concepts, even sensory concepts. But his purpose in doing so was to argue that large language models might therefore also be developing concepts. He could not show that large language models are like people with internal lives and relationships and full personhood, and so he ended up dehumanizing blind and deaf-blind people by saying that they are like something that is patently not human, specifically because of their disability.
Digital physiognomy
Researchers pursuing digital physiognomy claim to predict such things as criminality, sexual orientation, employability, political leanings, and psychopathy on the basis of photos, videos, voice samples, and so on (see Stark & Hutson, 2022). Thus the long discredited pseudoscience of physiognomy has come back, using computers for a veneer of objectivity (Agüera y Arcas et al., 2017). Classifying people by gender or race on the basis of how they look is equally problematic: It flattens human identities and experiences into fabricated categories that are falsely imagined to be intrinsic, immutable, and externally observable. Not only are such classifications fundamentally not possible (the information simply is not in the input signal), they are also harmful, because attempting them entails the objectification of the people being subjected to such systems, the rigidification of categories, and the misattribution of characteristics and identities.
Ground lies
Next consider the way training data sets for “AI” are mythologized as being representative (Paullada et al., 2021; Raji et al., 2021; Scheuerman et al., 2021). This viewpoint holds that data collected without care is “naturally occurring” and therefore a true representation—despite biasing decisions of where to collect data from, how to collect it, how to filter it, what labels to apply, who should apply the labels, how the labels are verified, and more. If we do not actively work to curate the data sets that we want, then we will be collecting data sets that are representative of dehumanizing ideologies such as white supremacy and calling lies “ground truth” (Raji, 2020). 3
Irrelationality
Humans are thoroughly relational in our experience of ourselves, our lives, and our world. As Birhane (2021, p. 5) and others have argued, our understanding of our world is inherently bound up in our culture, history, and lived experience, all of which are primarily built with and through our relationships to other people. Kyselo (2014, p. 8) made the case that even our very selves emerge in interaction. The “knowing” that we program into “AI” is, in contrast, irrelational, that is, ostensibly abstracted from the web of relations within which we have all of these experiences. Birhane (2021, p. 3) observed and elucidated the deep commitment to rationalism and the supposed potential of a “God’s eye view” built into these systems—in which the “God’s eye view” aligns with the perspective of those with power in society.
Machines are not designed to apply what Scott (1998) termed metis: the way people working with rules creatively navigate through them on the basis of the facts at hand. Computers can work with hard, hand-coded rules or statistical processing (either so-called deep learning or more traditional stastical methods) based on historical data but never in relation to the full situation at hand and thus never with wisdom (Weizenbaum, 1976). This irrelationality ends up devaluing humanity while also leaving no space for it. We must recognize that attempting to make decision-making more fair by replacing computers with humans is wishful thinking: Pushing off difficult decisions to supposedly impartial machines that encode hegemonic values and lack flexibility is an exercise in shirking accountability while again devaluing the human web of relationships (Alkhatib, 2021; Roberts, 2021).
Ghost work
Human effort is everywhere in these systems: labeling data, design and evaluation, and as a backstop for when the computer fails on some input. Tech firms ship those tasks off to microworkers on crowdsourcing platforms (Gray & Suri, 2019; Roberts, 2021), hiding often grueling labor (e.g., content moderation) and the humanity of the microworkers behind the illusion of “AI.” Furthermore, crowdwork platforms encapsulate microworkers in worker IDs, so requestors are encouraged to treat workers as interchangeable software components accessible through an application-programming interface (Gray & Suri, 2019).
Reinforcement of the white racial frame
The (Anglophone) discourse around “AI” reinforces the white racial frame: the implicit and assumed understanding of racial categories that supports systemic racism and white supremacy, shaping decision-making wherever it is not actively resisted. Cave and Dihal (2020) documented how (within Anglo-Western culture) “AI” is racialized as white: Depictions of robots and actual robots are frequently made with white exteriors, and even robots without physicality (voice assistants or text-based chatbots) adopt white-coded speaking styles (Marino, 2014). 4 Cave and Dihal (2020) hypothesized that this shows the influence of the white racial frame: The traits that are associated with “AI”—intelligence, professionalism, power—are those that the white racial frame ascribes to white people. Further and damningly, they hypothesize that white people overrepresented in the AI workforce are designing a set of servants who would let them avoid interacting with people who are not white. This is problematic on its face but also in more subtle ways: The whiteness of “AI” is dehumanizing because the white racial frame itself is dehumanizing to anybody who is not ascribed whiteness—recall Weheliye’s (2014) description of Blackness within the white racial frame.
What Can Cognitive Scientists Do About This?
My purpose is not just to surface these many facets of dehumanization in “AI” but also to talk about what we can do about it. Here are six suggestions.
Problematize simplified tasks
Machine learning (ML) research is driven by tasks that are defined either through informal descriptions of what the algorithm is supposed to do (e.g., transcription of spoken Kinyarwanda from audio recordings) or through data sets pairing inputs with expected outputs (Schlangen, 2021). Tasks are supposed to represent vague capabilities that are hypothesized to underlie the possibility of doing the task. However, many ML tasks lack construct validity (see Raji et al., 2021): We do not know that an algorithm’s score on test data means that it has the capability that a human would have to do the task ostensibly represented by the examples.
How does such misinterpretation of the results of ML tasks come about? I think it follows from the idea that computer science (CS) should provide general solutions. To achieve this, many researchers do not look at specific data to avoid creating systems overly tailored to that data. But this also prevents understanding the shape of the problem. The division of labor between those who construct the data sets and those who build the algorithms sets the conditions for wildly excessive claims about system capabilities. The “marketing” parts of data-set articles get repeated as serious, well-founded descriptions of the capabilities the tasks supposedly represent. For example, Wang et al. (2019) framed their SuperGLUE benchmark as a “rigorous test of language understanding” (p. 2) composed of tasks that “test a system’s ability to understand and reason about texts in English” (p. 4). This led to a 2021 article titled “Microsoft DeBERTa Surpasses Human Performance on the SuperGLUE Benchmark” that claimed “to get the right answer, the model needs to understand the causal relationship between the premise and [two] plausible options” (He, Liu, & Chen, 2021, para. 1). The media then reported it as “Microsoft’s AI Model Has Outperformed Humans in Natural Language Understanding.” 5
Cognitive scientists are the domain experts in the capabilities that these tasks supposedly test for. We must bring our expertise to bear in contextualizing how the tasks relate to the capabilities we study. Bender and Koller (2020) responded to the claim that language models “understand” (supported by scores on benchmarks such as SuperGLUE) by laying out a linguist’s perspective on meaning and understanding. Raji et al. (2021) similarly problematized claims of “generality” in ML. Anyone who researches things that humans do with our cognition can probably find something for which the folks doing ML are making spurious claims—and then be in a position to say “That’s not how that works!”
Critically analyze claims of “AI” capabilities
Problematizing simplified tasks hones skills for critically examining claims of “AI” capabilities. Faced with hype-driven headlines such as “Can A.I.-Driven Voice Analysis Help Identify Mental Disorders?” 6 and “Algorithm Predicts Crime in US Cities Before It Happens,” 7 one can ask:
How is this task defined?
What is the input and what is the output?
Does the input provide sufficient information to produce accurate output?
Where did the training data set come from and how was it validated?
Can this technology be used for surveillance, harassment, or otherwise denying people their rights?
Exploring these questions publicly, even just raising them, helps deflate AI hype.
The hype is especially dangerous in cases in which it would be beneficial to have something that can do X with only Y input, but we have not established that it is possible. For example, it would be useful to give people accurate actionable mental-health diagnoses on the basis of their voice (assuming we could prevent the negative surveillance use cases). This is a danger zone for ML tech solutionism because we can always construct ML systems that look like they are doing the job, taking Y and giving X. But if we cannot validate system outputs they are useless. Until and unless we have comprehensive and robustly enforced regulation of this kind of misuse of “AI” technologies, it is up to us to expose it for what it is.
Another angle for critical analysis is looking for the people in the “AI” system. Lanier and Weyl (2020) argued that “AI” is an ideology, not a technology, reminding us that “the AI way of thinking can distract from the responsibility of humans” (para. 4). Here, we are concerned with the people who designed the system and decided how to use it, whom we should ask: Why is this a safe thing to do, why did you frame the task this way, and whose interests does it serve?
Decenter whiteness/English
A third act of resistance is the decentering of identities or characteristics that get accorded the status of “default” or “unmarked” (e.g., whiteness, speaking English). If we do not name English when we are working on it (Bender, 2019), or white people, WEIRD (Western, educated, industrialized, rich, and democratic) societies, or middle- and upper-class people when we we are working on them, then we misinterpret results about those groups as general and results about other specific groups as parochial. This both weakens our research and contributes to maintaining (in the U.S. context at least) the white racial frame. Likewise, we should insist on success criteria that do not leave the concerns of people of color as an afterthought (Birhane, 2021; Raji, 2020). A system is not accurate if it is not accurate for everybody: If it is failing people of color it is failing.
We should also question the entire metaphor of “AI” for how it aligns intelligence with whiteness and for how it devalues cognitive capabilities that are outside of those prized by rationality. Here, I believe that the field of psychology has some work to do, taking accountability for how “intelligence” is discussed and how it relates to the white racial frame. Doing so will be a powerful force for resisting dehumanization from “AI.”
Engage in public scholarship
“AI” has captured the public imagination for decades. Tech firms selling “AI” are shaping the regulatory landscape, asserting claims to data in the digital world (Zuboff, 2019) and selling surveillance technology and other deeply problematic applications of “AI.” Sensible regulation requires an informed public and informed policymakers—which in turn requires public scholarship.
One way to do this is through social media. As a first step, cultivate a set of accounts to follow and learn from, especially people who experience different forms of oppression. Then build a network of people who are speaking out about similar things: Spaces to offer mutual support are key to doing this work without burning out. Likewise, connecting with traditional media can be both valuable and time-consuming. Prepare for this by doing institutional media training (if available), learning to vet journalists before engaging with them, and practicing speaking within your expertise. For both social and traditional media, be prepared to say the same things over and over to educate new audiences.
Public scholarship also includes engagement with policymakers and policy advocacy. When I have been invited to talk with policymakers, I have taken the opportunity to advocate for policy goals on the basis of my understanding of how technology such as large language models works and how human language-processing capability leaves us vulnerable to being misled by the technology. These goals include transparency (of the fact of the use of automation, of the data used for training), accountability (held by people for system output), application of existing regulations (not asssuming that new technology means that previous protections of rights become moot), and sufficient funding of social-science and humanities research to be able to understand the impact of technology on society.
Finally, in addition to engaging in public scholarship, we must also hold space for others who are doing so. We can support public scholarship whenever we review tenure packets, allocate grant funding, or otherwise wield power.
Advocate for broader distribution of research funds
I believe that CS, and especially AI/ML, is overfunded, creating a power imbalance between CS and the domain areas it should be partnering with. A stark example is the nominally interdisciplinary U.S. National Science Foundation Program on Fairness in Artificial Intelligence in Collaboration with Amazon, 8 in which proposals required principal investigators from CS departments. When questions are about fairness of technology, the core scholarship area required to answer those questions is not necessarily CS. With more equitably distributed research funding, computer scientists would have to enter into interdisciplinary collaborations with others as equals.
Large language models are especially problematic here. With systems that can generate what looks like legal contracts, medical advice, scientific papers, and so on, we might think we are about to have systems that can actually do those things. Without significant funding of non-CS research in the content areas implicated and the way technology in those content areas would affect society, we risk finding ourselves in a situation in which the computer scientists proclaim “Solved it!” on the strength of their stochastic parrots and regulators and other decision makers believe them.
Envision alternative prohuman research paths
AI research is currently throwing resources at made-up problems such as automating morality judgments, including not just financial or research-time resources but also things such as carbon budget and other natural resources (Strubell et al., 2019). We should be identifying practical problems that could benefit from computational solutions, not ML solutions that could benefit from problems. One step is to engage in the ongoing discussion of ethical considerations in ML conferences. Cognitive scientists are particularly well positioned to focus on the people involved in this kind of research, so we have a role to play in helping to shape that conversation. The more work in AI and ML is held accountable for its impacts in the world, the more we can shift the research focus to humanistic concerns.
Conclusion
I have reviewed the ways in which “AI” research, development and sales involve dehumanization, from the computational metaphor, to digital physiognomy, to our “ground lies,” to “AI’s” insistence on irrelationality, to the way we hide ghost work, and finally the way in which “AI” reinforces the white racial frame. I have also explored the ways in which cognitive scientists are well positioned to resist this dehumanization. We have many roles to play: We can problematize simplified tasks; critically analyze claims of “AI” capabilities; work to decenter whiteness, WEIRD societies, and English; engage in and support public scholarship; advocate for the broader distribution of research funds; and envision alternative prohuman development paths.
Recommended Reading
Raji, D. (2020). (See References). Succinctly and powerfully summarizes the dangers posed to Black people by data sets that encode systemic racism.
Broussard, M. (2019). (See References). Exposes the phenomenon of techochauvanism or the belief that technology can solve all problems and traces how attempted “solutions” cause more harm.
Alkhatib, A. (2021). (See References). Captures the tragedy and absurdity that follow when we structure our world around algorithmic rather than human systems.
Baria, A. T., & Cross, K. (2021). (See References). Explores the origins of the computational metaphor in neuroscience, its uptake in computer science, and its effects.
Footnotes
Acknowledgements
This article has benefited greatly from discussions with Leon Derczynski, whom I also thank for his encouragement. Likewise, I am grateful to Andy Perfors and the other organizers of the COGSCI 2022 conference for their invitation to present the keynote talk that is the basis of this article. I am also indebted to Deb Raji, Amandalynne Paullada, Alex Hanna, Remi Denton, Timnit Gebru, and Margaret Mitchell for the wide-ranging discussions (largely over group chat) that helped shape these ideas. Finally, I thank the reviewers and editor for their input.
Transparency
Action Editor: Robert L. Goldstone
Editor: Robert L. Goldstone
Declaration of Conflicting Interests
The author(s) declared that there were no conflicts of interest with respect to the authorship or the publication of this article.
