Interview with N. Katherine Hayles

LA & VP: A major contribution of your 2017 book Unthought is the concept of the ‘cognitive nonconscious’. Where cognition for you (after Shannon) is ‘a process that interprets information within contexts that connect it with meaning’ (p. 22), the faculty of nonconscious cognition is distributed across biological and technical cognizers. Could you elaborate for us some of the implications of paying careful attention to the cognitive nonconscious?

NKH: Allow me to offer an important correction. My definition of cognition is not inspired by Shannon's information theory; indeed, in some ways I offer it as a correction to Shannon, as it explicitly includes references to ‘interpretation’, ‘context’ and ‘meaning’, all of which are absent from Shannon's probabilistic theory, which is based only on the relative frequencies of the message elements (Shannon and Weaver, 1971 [1948]). Rather, my definition of cognition is indebted to a remark by Edward Fredkin, a theoretical physicist who has suggested that in some sense, reality itself may be computational in nature (Fredkin, 2018). Without necessarily accepting this implication, I found his idea of connecting information to meaning through context to be extremely insightful.

It was a key that enabled me to rethink cognition in terms that broadened it well beyond ‘thinking’, with its long tradition of anthropocentric interpretation as something only humans can do. A second key move is to break the link between cognition and consciousness; cognition as I define it is a much more capacious activity that far exceeds the bounds of human conscious thought. For humans and other conscious organisms, cognition extends beyond the brain into the body and environment, and for nonconscious organisms, it extends throughout the entire biological realm of all lifeforms, including plants. Once cognition is seen not to require consciousness, it also extends to computational media in all its forms, including networked and programmable machines. Obviously, this view of cognition has a low threshold for something to count as cognitive, but it can also scale upward in complexity to the most sophisticated human and computational cognitive achievements. Cognition in this view exists as a spectrum rather than as a single point; it also is defined as a process rather than an entity, so it is inherently dynamic and transformative.

LA & VP: In your book you suggest that the human/non-human binary, characterizing so much of the contemporary debate in humanities and the social sciences, serves to reinstall the privileging of a liberal humanist subject. You offer in its place a rather fascinating distinction between what you call ‘cognizers’ and ‘noncognizers’ so that cognizers extend across humans and other biological life forms as well as technical systems, while noncognizers embrace material processes and inanimate objects. What is at stake in this distinction between cognizers and noncognizers? How does it transform what might count as agency?

NKH: The crucial features distinguishing cognizers from noncognizers are interpretation and choice (or selection). The two are entwined, because without choice, there can be no interpretation, which requires at least two available options. Agency, as I understand the term, denotes the ability to act. Without question, material forces have agency, from the water that wears away a rock surface to the landslide that rips off a mountainside. What material forces cannot do, however, is interpret and choose. Their actions rather are the resultant of all the forces acting upon them and can be understood in these terms (with criticality phenomena, the situation is more complex because they are sensitive to very small perturbations. Nevertheless, their actions can still be adequately understood through simulations and other numerical methods).

All lifeforms, by contrast, possess the signal characteristics associated with cognition, namely flexibility, adaptability, and evolvability. Their actions can never be entirely constrained within a rigid stimulus-response model that denies them the capacity for creative interpretation and choice. The issue for me is thus not agency, which exists in noncognizers and cognizers alike, but rather the distinction between agents (material forces possessing the ability to act) and actors (cognizers who can interpret and make choices).

There is a longer argument that I cannot detail here about computational media and the role of choice or selection in their operations (Hayles, 2019). Suffice it to say that computational media are built in layers that proceed from the minimally cognitive (the basic selection between five volts or none, one or zero) up to increasingly sophisticated decision trees (subroutines nested inside routines, routines inside libraries, etc.) that can deal with highly ambiguous or conflicting information and arrive at interpretations about it. In these cases, it is necessary to understand ‘meaning’ in terms similar to pragmatists such as John Dewey, who argued that meaning derives from the consequences of actions (Dewey, 2000: 128). All computational media employ an explicit or implicit ‘if/else’ logical structure that connects choices or selections with consequences, and hence (in the pragmatist sense) with meaning.

LA & VP: The cognitive assemblage you map for the reader in Unthought appears as something rather different from the Deleuze and Guattari inspired assemblages of connections, desires, affects and resonances. What does a cognitive assemblage approach do? How might a cognitive assemblage shift the locus of responsibility for decision so that, for example, decision-making capacities could extend from neural network algorithms to RFID chips, or from city bankers and regulators to the probability weightings of a high frequency trading algorithm?

NKH: My use of ‘assemblage’ shares some common ground with Deleuze and Guattari (Deleuze and Guattari, 1987; DeLanda, 2016), and also with Bruno Latour's actor network theory (Latour, 2007). It also has some distinctive differences in its emphasis on cognition, information, and interpretation. Whereas Deleuze and Guattari write against the subject, the sign, and the organism (and hence refuse the presumed boundaries of the same), my ‘cognitive assemblages’ are entirely consistent with the idea of pre-existing entities such as humans and computers. Rather, the emphasis falls on the ways in which these entities symbiotically interpenetrate each other's actions, so that cognition and agency are understood as always already distributed throughout the assemblage. Cognitive assemblages also differ from actor network theory in that I make a strong distinction between cognizers and noncognizers, a difference that Latour seeks to obliterate in placing material forces and cognitive actors on the same plane.

Given my framework, it is clear that both humans and computational media have agency and thus are ethical actors, in the sense of being able to perform actions that have ethical consequences. There is nevertheless a profound difference in the ethical responsibility of human and nonhuman actors, because humans are the ones designing, implementing, and overseeing complex cognitive systems such as high-frequency trading algorithms, neural network architectures, and (at a minimally cognitive level) RFID chips. In most ethical theories of which I am aware, one must have free will in order to be an ethical actor. This framework is simply inadequate to deal with our present situation, because ‘free will’ is deeply associated with consciousness and hence does not apply to nonconscious cognizers such as nematode worms, plants or computational media. We desperately need ethical theories that expand upon the idea that responsibility may be differentially assigned within a cognitive assemblage on the basis of who (or what) has control over its design parameters, socio-economic implementations, and likely consequences. ‘Control’ in this sense should not be understood as connoting ‘free will’ (an ill-defined concept in any case) but rather the places within a cognitive assemblage where effective interventions can be made by ethically responsible actors. I have no special expertise as an ethicist, but it is clear to me that this is a lacuna that needs to be addressed and reconceptualized within ethical theories.

LA & VP: What does it mean to be human in the context of a cognitive assemblage? In the book you detail the example of Brandon Bryant, the drone sensor pilot for the US Air Force who suffered from PTSD. What are the relations between human emotions and cognitive assemblages such as the UAV? To what extent is the design of a socio-technical system such as the drone also fundamentally a redesigning of the meaning of being human?

NKH: To be human in a cognitive assemblage means to participate in the deep symbiotic relation between biological and technical cognizers. This may be done with or without conscious awareness that such is the case; for example, most people in developed countries do not think much about their participation in the electric grid, which is completely dependent upon computational controllers, connectors and transmitters, until something goes wrong and a blackout disrupts our normal routines. Then it becomes apparent how much of contemporary life is utterly dependent on our computational symbionts. In the case of a drone pilot, the system is so entirely interpenetrated with technical cognition that it would be almost impossible not to be consciously aware of this involvement.

There are some features of biological cognition not generally present in technical media, for example, human emotion. As I try to make clear in Unthought, I consider emotion to be very much a part of cognition. Emotion, and affect more generally, taps into some very potent meaning-events, since it has evolved as a means to cope with changing and unpredictable environments in ways that promote an organism's survival and reproduction. Lacking these evolutionary developments, technical media rely instead on design and purpose; these, in turn, are constructed by humans. Insofar as humans bear ethical responsibility for the systems they design and in which they participate, their affective responses do and should have bearing on system design and implementation, which in turn implies an ethical responsibility to understand the full implications of how the cognitive assemblage as a whole works. This is especially important for humanities disciplines, which for too long and too often have regarded intimate knowledge of technological systems as unnecessary or beyond their concern, even (or especially) when they want to criticize them. Effective interventions (as opposed to preaching) require a reasonable amount of knowledge and dialogue with those designing and building complex technical systems.

LA & VP: A rather extensive theme in contemporary social and legal debates on ‘autonomous’ systems technologies and deep learning algorithms is that one should advocate for ‘more transparent’, less ‘biased’ models where there is a ‘human in the loop’ as the guarantor of ethics. The persuasive arguments you make in Unthought seem to us to run entirely counter to the idea that a cognitive assemblage could or should be transparent, or indeed that a human could oversee the workings of the assemblage. You write, for example, that, in a complex assemblage of ‘human and technical cognizers’, ‘the choice is not between human decision versus technical implementation’ (p. 136). What do you think of the many contemporary calls for algorithmic accountability, ethical design of systems, or human oversight? Do you envisage a different kind of ethico-political response better attuned to the extent of collaboration among human and technical cognizers?

NKH: I find this question puzzling, since at many places in Unthought I argue precisely for the necessity of a ‘human in the loop’, for example when I consider autonomous weapons or the functioning of high-frequency trading algorithms. No doubt biases of many kinds do get incorporated into technical systems; one thinks, for example, of criteria for a mortgage or a loan, which often have sources of bias much subtler than simple redlining. It may be a red herring, however, to focus too much on ‘transparency’; most complex systems require detailed technical knowledge to understand completely, and it is not clear to me what ‘transparency’ would mean in this regard. Having a human in the loop, by the way, is no guarantee against bias, since most humans have conscious or unconscious biases; indeed, in some ways the demands for more ‘unbiased’ and more ‘transparent’ systems have vectors that point in opposite directions.

When I suggested that human choice and technical implementation is not a binary opposition, I meant to imply that human choices are very much involved in technical implementations at many levels; far from being a binary, such situations should more accurately be considered as interpenetrations. That said, I am very much in favour of human oversight and ethical interventions into the design, implementation, and functioning of technical systems, as I exemplify through my discussion of batch auctions and slow trading in the concluding section of the chapter on high-frequency trading algorithms. Such interventions, to be effective, may have a high entry cost in terms of learning how complex technical systems actually work. AIDS activism provides a compelling model for how to make effective intervention. To change medical practices, activists needed to learn the concepts and vocabulary at issue, and the fact that they were willing to do so is a large reason why they were able to convince medical professionals to make constructive changes in how medical tests and experiments were configured.

In my view, it may be naïve to expect complex systems to be ‘transparent’, since by definition their architectures have many interacting components with massive feedback loops connecting diverse parts. A more achievable goal is to require transparency for those small parts of a complex system that directly interface with consumers. For example, transparency in one's electric bill is a good thing and can easily be achieved, whereas ‘transparency’ in how the entire electrical grid operates is much more difficult and would require considerable technical knowledge of the system's many networks, subsystems and connecting parts, as well as the dynamic interactions between them.

I take this to be Latour's point when he takes his fellow sociologists to task for evoking ‘the social’ without requiring a detailed technical knowledge of how a given complex system operates (Latour, 2007: 85). Ethical judgments, he argues, require not easy mystifications but the kind of technical analysis that he typically undertakes. Accused by his fellow sociologists of lacking an emphasis on politics, he defends his practice as more deeply political than those who eschew close technical analysis and simply call for reform instead.

I find his argument compelling that there are deep connections between oversight, ethical responsibility, and technical knowledge. Such an argument also implies that, given the limits of a finite lifetime, it is important to pick one's battles and spend one's resources accordingly.

LA & VP: As with some of your previous works, such as My Mother Was a Computer (2005) and How We Think (2012), in Unthought you offer close readings of a novel (Colson Whitehead's The Intuitionist, 1999) as a way of engaging anew with computational theory on the incomputable. Reflecting on the character of Lila Mae and her observations on the catastrophic accident as that which cannot be predicted, you propose that ‘the power of the black box does not lie in concealing a knowable answer, but rather in its symbolization of the limits of knowledge, both Empirical and Intuitionist’ (p. 189). Why is this symbolization of the limits of what can be known empirically or intuitively so important to your project? Does this threshold of the knowable offer a potential critical response to the damaging effects of algorithms used in financial trading or in criminal justice?

NKH: The power of the limits to knowability discussed by Luciana Parisi (2013, 2019) and also the limits of computability, a related topic explored by M. Beatrice Fazi in Contingent Computation (2018), lies in their absolute nature. For example, Gödel's Theorem proves that, for a formal system strong enough to do arithmetic, at least one statement within the system cannot be proven to be true or false. Wherever such limits appear, they tend to be related to a system's recursivity, its ability to turn back on itself and re-enter the system from a different perspective. Niklas Luhmann used this insight as a central characteristic of his systems theory (Luhmann, 1996); I find it interesting that recursivity (in the form of massive re-entry) has also been theorized by Edelman and Tononi as an essential condition for the emergence of consciousness (Edelman and Tononi, 2000). These findings suggest to me that complexity requires recursive interactions, and at the same time recursive architectures limit how much we can know about the system, a result relevant to the ‘hidden layer’ in neural net and deep learning algorithms.

I think the central issue here is the difference between linear causality and circular or recursive causality. For example, there is no problem in analysing the dynamics of a chemical system in which reagents A and B interact to form C and D. But when C and D also interact to form A and B, the system becomes recursive, and analysis becomes much more difficult because all the parts are interacting simultaneously in ways that affect how the parts interact. Resisting formal closures, recursive systems are associated with limits on how precisely the system's workings can be understood. Without downplaying the significance of these results in a theoretical sense, I question how useful they are in systems that are not formal mathematical structures but rather complex social structures involving many different kinds of interactions, including human emotions and biases as well as computational media, whose design parameters may not always be fully known. One would have to know much more about how the system in question works to draw a connection between theoretical limits to knowledge and the system's recursive complexity.

My point about Colson Whitehead's novel is exactly this: in a system as complex and multifaceted as entrenched racism, only something as potent as an absolute theoretical limit could be powerful enough to demolish it completely and send the city along a different trajectory. Of course, as a novelist, Whitehead only needs to gesture toward this possibility rather than analyse it in a systematic way. It remains in his novel a tantalizingly utopian gesture toward an uncertain future, a lifeline thrown out of desperation and hope in equal measure, betting that a small measure of grace is left open by the system's inability to achieve complete closure.

LA & VP: In Unthought you are interested, almost in a methodological sense, in whether novels enact dynamics that are not already present in mathematics or computer science. Do you think that the novel, perhaps paradoxically, is becoming more and not less significant as medium in a time when deep reading and concentrated attention seems to be undermined by algorithmic hyper attention? Do you see the contemporary novel as up to the task of opening critically onto the errors and contingencies that have, as Luciana Parisi argues, become the very terrain of the machine learning algorithm?

NKH: Every human society constructs narratives, and it seems fair to conclude that narratives are hardwired into the ways in which humans understand the world and their places in it. Of course, many genres rely upon narratives, not only novels: computer games; qualitative histories; biographies and autobiographies; films; children's stories; Biblical parables; and so on. The issue for me is what novels can do that other narrative (and non-narrative) forms cannot. It is surely no accident that context, interpretation and meaning feature centrally in my provisional list of what novels can do; these features imply that novels are devices that employ cognitive devices to stimulate and render more complex human cognitions.

It may well be the case, as you suggest, that digital media are leading us as a population toward hyper attention and away from deep attention (Hayles, 2011), a hypothesis that may cause us to wonder if novels can compete effectively for audience share in our attention-challenged era. Before we write the novel off, however, we should acknowledge that it is the most protean of forms, having adapted through its centuries-old traditions to a wide variety of different cultural and reading practices. Already there are Twitter fictions and fictions for small screens such as cell phones (whether these can be considered ‘novels’ is another issue). For me it is not a question of whether the novel is ‘up to the task’ of ‘opening onto the errors and contingencies’ of algorithms, for I can think of at least a dozen contemporary works that do precisely this (an outstanding example is Pynchon's Gravity's Rainbow). The question should perhaps rather ask whether readers are up to the task of perusing and understanding such complex and difficult novels. I certainly hope so, and as a teacher of such novels, I am doing my small part to make sure this will be so.

LA & VP: In the final chapter of Unthought you reflect back on the cybernetics that animated your 1999 book How We Became Posthuman. You suggest that the cybernetic paradigm was both prophetic and misguided: prophetic in its modes of communication between humans, non-human life forms, and machines, and yet misguided in its belief that feedback mechanisms would afford a means of control. Can you explain why it is that you consider the idea of control to be ‘increasingly obsolete, if not outright dangerous’ (p. 202)? Given this obsolescence, and given your argument for ‘inflection points’ in place of control, how do you envisage the impulses or political will to send the cognitive assemblage in different or unforeseen directions?

NKH: I would say that control is not so much obsolete as applicable to only a small fraction of systems employing linear causality. As soon as complexity and recursivity enter the picture, control becomes more elusive and more unpredictable. To illustrate, I can reference a short fable by Stanislaw Lem, disguised as a Nobel acceptance speech, entitled ‘The New Cosmogony’ (Lem, 1983: 197–214). The speech, made by a theoretical physicist accepting the prize, postulates that there exists an incredibly advanced civilization, much older than ours, that has achieved the ability to alter the laws of the cosmos. The problem is that they are also part of the cosmos, so when they alter the laws, they also alter their own functioning in unknowable and unpredictable ways – a neat example of recursive re-entry within a complex system. The very conditions that enable them to exert control also limit how that control can be exercised.

In Unthought I argue that a more fruitful approach than straightforward control, and one better suited to complex systems, is to look for ‘inflection points’, places in time and space where a complex system enters a meta-stable state and therefore becomes more sensitive to small perturbations such as a group of individuals might exercise. These do not in themselves guarantee that the interventions will be positive; they could well be negative, for example, causing the system to crash rather than simply move in another direction. But these are the places to look if one wants to make constructive changes.

Needless to say, the more deeply one understands the system's dynamics, the more likely it is that the intervention will be effective. Many smart people are involved at present in precisely these kinds of activities, for example investment bankers with the goal of making money. I say it is high time for humanists and progressive thinkers to be involved as well – people who may have other goals in mind, such as sustainability, environmental stability, legal systems able to reliably deliver justice, and so forth. These for me are the stakes of understanding the implications of living and acting within cognitive assemblages and the cognitive planetary ecologies which they mutually co-constitute.