Abstract
In 1976, two researchers declared a revolution in cryptography: With the invention of public key encryption, cryptography could be used not only to share secret messages, but to secure and authenticate communications networks, and, eventually, to enable radically new kinds of social relationships facilitated by networked communication technology. This article explores a series of transformations in the meaning of cryptography in the 1960s and 1970s that led to the declaration of a revolution. Drawing on archival materials, the article considers how public key cryptography was the product of an emerging consensus among cryptographers of the importance of privacy in the wake of abuses of surveillance powers by government agencies. Shaped by a changing technological and political environment, it situates cryptography at the center of a focused effort to assert control over information in an era of sociopolitical upheaval, concluding that the invention of public key encryption both marked a change in the imaginary surrounding cryptography and offered a technical solution that foreclosed other approaches to addressing the problem of surveillance.
How do certain technologies become the fulcrums of our imagined solutions to social problems? Cryptography – and particularly public key encryption – holds special significance in the minds of advocates who see it as a critical intervention to protect our privacy, mathematicians fascinated by its intellectual challenges, and law enforcement agencies who contend that it presents an existential threat to the surveillance of alleged criminals. In recent years, encryption has sometimes been positioned as a teleological goal in itself, a solution for problems of surveillance and privacy that are also social and political. This rhetorical move often serves to obscure underlying power differentials along lines of race, gender, class and ability that influence who is most affected by the harms of surveillance (Gürses et al., 2016).
Public key cryptography is a kind of cryptographic system that uses two sets of keys: A public key, which can be used by the sender of a message to encrypt a message, and a private key, which can be used by the receiver to decrypt it. The significance of this system is that it enables anyone to send an encrypted message to anyone else, without requiring that the two parties already have some method to establish a shared code system.
Despite its many centuries of history, cryptography remains a relatively understudied area in Science and Technology Studies (STS). Though there is ample popular literature on the subject, including several well-researched histories (Fagone, 2017; Kahn, 1967; Mundy, 2017; Singh, 1999), academic scholarship of cryptography tends to be concentrated in technical fields of study. Only over the last decade have a number of new studies begun to contribute to our critical understanding of encryption (Blanchette, 2012; Dupont and Cattapan, 2017; Hellegren, 2017; Rider, 2018) and of cryptocurrency (Brunton, 2019; Dupont, 2018; Lustig, 2019).
Though emergent, this literature connects to a larger body of STS research concerned with the management of information under security regimes (Vogel et al., 2016). Scholars have examined the relationship between information and power through the study of state secrecy and classification regimes (Birchall, 2011; Dennis, 1999; Masco, 2010), explored the sociomaterial dimensions of practices and technologies of surveillance (Ball et al., 2012; Browne, 2015; Lyon, 2007; Suchman et al., 2017), and situated security as a foundationally sociotechnical and infrastructural construct (Ellis, 2020; Ferreira et al., 2014; Goerzen et al., 2019).
Building on this scholarship, my analysis centers on parsing contestations over the regulatory, economic and cultural factors influencing the usage and adoption of encryption, alongside evolution in the materiality of cryptographic systems. The politics of encryption is mutable and constantly evolving. In the eyes of the cryptographers I studied, encryption became a means of situating power and control over information in the hands of those with the ability to encrypt it. 1
As cryptographer Phil Rogaway (2015: 3) writes: ‘That cryptographic work is deeply tied to politics is a claim so obvious that only a cryptographer could fail to see it’. How the application of encryption embodies specific forms of power and authority matters as much as the discursive meaning of its hidden text – and STS scholarship offers a useful set of conceptual and analytical resources for interrogating how and why these are embodied in particular applications of encryption. In particular, I draw connections between concepts associated with the social construction of technology (SCOT) and of the sociotechnical imaginary. As a framework, SCOT is particularly focused on making sense of the evolving social meaning of technology, suggesting that the process of technological design evolves in tandem with the social circumstances of development, and that the design of an artifact reaches a relatively stable state not because it objectively ‘works’, but because the relevant social groups accept that it works for them (Bijker et al., 1987). This means, therefore, that the ‘successful’ version of any technological artifact is not the only version possible. Through this history of public key cryptography, I explore why a particular set of views on the proper role of cryptography – as a technologically determined means of asserting control over information – stabilized as the dominant meaning.
Though SCOT is useful for making sense of processes of technological development at the level of micro-interaction, it has been criticized for a neglect of larger structural influences (Klein and Kleinman, 2002). In the case of cryptography, systemic concerns around government surveillance played an important role in the process of stabilization. While there were many proposals around how best to address the problem of surveillance, government actors influenced the adoption of technical approaches in two respects: by opening up cryptographic research outside the auspices of state intelligence apparatuses (though this opening proved to be tenuous and fraught), and by encouraging the development of cryptographic standards to be used for computer security.
I find Jasanoff and Kim’s (2009, 2015) development of the sociotechnical imaginary to be particularly helpful in accounting for these dimensions. In earlier work, I described how ‘cryptographic imaginaries’ – conceptualizations about what encryption is, what it does and what it should do – become embodied in technological architecture and social practice (West, 2018). This framing builds, among other things, on the conceptual entry points offered by ‘sociotechnical imaginaries’ for theorizing how collectively imagined forms of social life and social order become embodied in the design of technological projects.
Drawing on these conceptual frames, the case of cryptography is uniquely interesting in several respects. Though cryptographic imaginaries implicate nation-states and visions of sovereignty, over the long course of its history cryptography has been invoked across a number of other domains; it has ancient associations with religious authority and the occult, and more modern associations with civil libertarian values around free expression and association (West, 2018). Across these different realms, sociotechnical imaginaries around cryptography act as a focal point for making sense of the role of information in society – to whom and under what conditions information should be revealed or obscured.
Synthesizing these analytical frames, I delve into the history of public key encryption to argue that the community of academic, industry and state cryptographers in the 1960s and 70s together positioned cryptography as a medium for working through debates over how power and authority would be reconfigured within and alongside emerging database and networked communications technologies. As I will show, this focus on cryptography as a means for negotiating power and authority is the product of both an emerging consensus among cryptographers that produced a stabilized vision for cryptography and changing conditions at the level of the nation-state that made this stabilized meaning possible.
Academic, industry and state actors all converged on the notion that technical solutions were the appropriate means to addressing the new challenges to privacy and security that emerged in the 1960s and 70s. This convergence occurred coincided with an opening for cryptographic research after a period of restrictive controls by military and state intelligence agencies during World Wars I, II and the Cold War. The centrality of cryptography to challenges posed by networked communications systems became particularly potent because of, rather than in spite of, the differences between these actors. We can see in this history the production of a new imaginary that situated power and control over information in the hands of those who could encrypt it.
Methods
This article is part of a larger project exploring the series of transformations in the imaginary around cryptography from the 1960s to the present day. I conducted archival research across a number of different sites, including the Martin Hellman papers at Stanford Library, Paul Armer papers at the Smithsonian Museum of National History, IBM Research, Computer History Museum and oral history collections at the Charles Babbage Institute. I selected these sites in an effort to trace the origins of public key cryptography through the work of the cryptographers who developed it. I began with the Hellman papers and turned to additional archival sites to investigate the references I found more deeply or to explore contrasting perspectives. At each archive, I also conducted general searches, often with the assistance of librarians, for all materials relating to cryptography and encryption. In subsequent visits, I added to these searches the names of companies that were active in this space as well as prominent individuals who were engaged in the study of cryptography, generating further sources of material to study.
In my pursuit of answers to the questions that drove this project, I noticed gaps in the archives. For one, those who spoke were primarily men with high levels of technical expertise and education, even though women and people of color were actively involved in cryptologic enterprises during World War II (see Fagone, 2017; Mundy, 2017. This remains a notable gap in this point in history, and one that I hope to address in future research. I also realized early on that some of what I sought remained under government or commercial classification. I found it necessary to triangulate documents across multiple archives in order to gain a fuller picture of materials that were redacted or that were inaccessible. In analyzing these texts, I found articles published in popular and trade presses from the 1960s and 1970s to be invaluable in providing additional context. Most of the articles referenced here are clippings that I found in the archives themselves, and thus shaped the thinking of those directly engaged in this work.
What is public key cryptography?
Cryptography is the art and science of writing codes. 2 Public key cryptography is a specific kind of cryptographic system that uses two sets of keys: A public key, which can be used by the sender of a message to encrypt a message, and a private key, which can be used by the receiver to decrypt it. The significance of this system is that it enables anyone to send an encrypted message to anyone else, without requiring that the two parties already have some method to establish a shared code system. This means that private conversations could be conducted in public view, on the basis of mutual trust – not between the communicating parties, but in the cryptographic system.
The architecture of public key encryption was a radical innovation in many respects. It embodied a shift away from a one-way, transmission model 3 and toward a two-way, communicative model – a change with broad social and organizational implications. Over most of their history, cryptographic systems have required carefully guarded methods for controlling the distribution of decryption keys so that they are not lost or uncovered by third parties. In order to make cryptography functionally useful there must be a channel established to selectively share decryption keys, and to update them if they ever change. This means that the parties who wish to communicate in secret must establish an additional secret channel to share their method. This is commonly known as the key sharing problem.
A number of different solutions to the key sharing problem were developed prior to the invention of public key cryptography. For instance, one highly secure method, if used perfectly, is to use one-time pads, a deck of encryption keys that are each used and disposed of after a message is sent. But using one-time pads presents organizational challenges: Over time, there have been many examples where the transmitters of messages re-used keys for the sake of convenience, not knowing that a third-party adversary had cracked the code. Solutions like this seemed to work best in organizational structures that are hierarchical, that enable oversight and control of the management of keys and operate with a high degree of trust between parties.
This is, in part, why cryptography has historically proven such a good fit for organizations like military and intelligence agencies, as their organizational structures are well-suited to the management and distribution of keys. But globalization introduced strains on these systems: The firms rapidly adopting commercial cryptographic systems had to find safe channels for communicating keys that could reach much farther and incorporate more people into the system to work effectively. This greatly increased the risk of errors.
Public key cryptography proposed a new method drawing on basic principles of mathematics – one-way functions – in order to enable keys to be shared in public. Rather than distribute keys through extensive secret channels, public key cryptography splits the key into a public and private version (see Figure 2). The sender then obtains the public part of the receiver’s key and uses it to encrypt their message, transforming its contents through one-way mathematical functions difficult to crack, so that only the receiver’s private key can be used to decrypt it. Today, we use public key cryptography in much of our digital infrastructure, from authenticating web connections, to securing the transmission of financial data, to protecting the content of our messages. Public key cryptography is well integrated into our digital lives, often in ways that we may not realize.

Symmetric key crypto system: The same key is used to encrypt and decrypt the message.

Public key crypto system: A combination of public and private keys are used to perform and solve a one-way mathematical function to decrypt the message.
Cryptography during the Cold War: A brief history
The development of public key cryptography was shaped by a dialectic of openness and secrecy over the course of the 20th century, a dialectic that became increasingly intertwined with US state interests in geopolitical affairs. At the turn of the century, literature on cryptography was generally easily available to the public. It was a somewhat esoteric subject, but the publication of works like Edgar Allen Poe’s ‘The Gold Bug’ helped to popularize codebreaking among budding mathematicians and enthusiasts seeking pirate treasure and occult secrets (Rosenheim, 1997). In the 1910s, the world’s foremost codebreakers, or cryptanalysts, were employed by a privately run facility, Riverbank Laboratories, on a failed mission to seek out messages allegedly hidden in the works of William Shakespeare by Sir Francis Bacon (Fagone, 2017). 4 But by World Wars I and II, the US and UK governments consolidated expertise in cryptography and cryptanalysis within their military intelligence divisions: MI-8 (eventually known as the Black Chamber), the Signal Intelligence Service and Bletchley Park. Across these disparate and secretive sites, government officials began to treat cryptographic research as a closely guarded national security resource, critical to the war effort in World War II (Schneier, 1996).
This view of cryptography as a state secret carried through into the Cold War. In 1953, the nation’s top scientists convened to review the nation’s cryptologic capabilities (Brown, 1965). The committee determined cryptography would be one of the most important weapons in the Cold War and recommended the formation of an effort much like the Manhattan Project devoted to improving the country’s cryptographic capabilities (Bamford, 1982). Given this central focus on military cryptanalysis, cryptography largely fell off the radar of public research between the late 1930s and early 1960s. This meant that many young researchers were discouraged from working in the field by the likelihood that any of their published work would have to undergo classification (Hellman, 2004). Moreover, the newly formed National Security Agency used a loophole in the patent system to prevent work on cryptography from being commercialized by issuing secrecy orders for patents deemed to have implications for national security (Diffie and Landau, 2007). 5 What little work could be conducted outside of the auspices of military research began to quickly fall behind the state of the art (Schneier, 1996).
IBM and the Cryptography Research Group
This changed during the 1960s as a result of evolving commercial and government needs. As a growing number of businesses adopted the use of mainframe computers and time-sharing systems, IBM – then undergoing a period of rapid growth and consolidation in the computing industry – identified a need to develop cryptographic techniques for private sector use and founded a cryptography research group at the company’s research center in Yorktown Heights, NY. 6
In the shift to using time-sharing terminals, the number of computer users and points of vulnerability in the security of networked computing systems increased. In the early days, computer security had primarily meant guarding against threats from within the physical buildings housing mainframes, employing methods like locking the doors to computer rooms and hiring security guards. However, with the introduction of time-sharing it became necessary to consider new kinds of threats from outside (MacKenzie and Pottinger, 1997; Yost, 2015a).
Decades before the heyday of computer hacking, businesses began to express concerns about the safety of their computer networks. As TIME Magazine explained, In today’s world, the integrity of secret messages can be crucial not only to national security but to commercial and industrial operations as well. Yet as society becomes increasingly reliant on electronically relayed communications – and more sophisticated new gadgetry is developed to intercept them – it is becoming harder than ever to keep a transmitted secret. (TIME, 1978: 5)
These fears led to the emergence of computer security as a field of study in the early 1970s (Bishop, 1998).
Electronic banking
IBM’s nascent Cryptography Research Group focused on developing techniques designed for an industry in urgent need of data security: banking and finance. Large banking firms like Lloyd’s Bank, based in London, were undergoing a shift toward electronic banking, and contracted IBM to develop a system of 600 networked cash-issuing terminals – an early version of the Automated Teller Machine. The first such terminal had been built by competitor Barclays Bank in 1967 and worked by collecting ten-pound vouchers with an ‘unbreakable punch code’ that were issued to approved customers. A customer would sign the voucher and place it in the terminal’s drawer to be tested against the code the customer would enter (Murray, 2017). IBM, which already operated a computerized clearing system in Surrey for London’s banks, wanted to create a new version of the terminal system that replaced paper vouchers with an online real-time system (Batiz-Lazo et al., 2014).
But Lloyd’s executives expressed some reservations about IBM’s proposal: Could the lines transmitting signals between the network of cash-dispensing machines and the centralized computer monitoring and checking transactions be wiretapped by eavesdroppers? IBM believed cryptography would present a solution to this problem and set out to develop a method that would ensure that the machines could safely transmit requests for cash to and from banking institutions. As IBM Vice President and Chief Scientist Lewis Branscomb described it: ‘The goal in cryptography is to render information undecipherable. We don’t need a perfect encryption scheme. What we do need is an encryption so difficult to decipher that the very small chance of success isn’t worth the effort’ (Bode, 1978).
Horst Feistel, a German-born mathematician who had been working for the US Air Force and MITRE Corporation, headed the new research group charged with developing the cryptographic system. Feistel long held an interest in cryptography but had faced challenges finding work in the area due to political sensitivities around his German origins at the military research centers that employed him. When he joined IBM in 1968, he was finally given room to explore his interests fully, working with a team to invent a new encryption method, which they named ‘Lucifer’.
Recognizing a larger potential market, IBM filed for patents for the new commercial application 7 but were initially blocked by an NSA-issued secrecy order (Konheim, 2015). Once the order was lifted, Walter Tuchman, who managed data security products at IBM’s System Communications Division in Kingston, took up work on the algorithm, realizing that it needed considerable strengthening before it could withstand massive commercial use. After years of testing implementations in software and hardware, Lucifer was finally ready for the wider market. IBM launched three commercialized cryptographic products in 1977: the IBM 3845 and 3846, which were table-top and rack-mounted data encryption units compatible with all computer terminals, and the IBM Cryptographic Subsystem, which was designed for use with IBM computers and data processing networks (Bode, 1978; Morris, 1977).
Government regulation and the search for a standard
IBM’s growing interest in developing commercialized encryption systems for businesses coincided, conveniently, with a series of changes in government regulation of cryptography. Agencies across the US government were adopting new networked computing technologies, leading Congress to recognize a need for new standards. In 1965, Congress charged the National Bureau of Standards with developing security standards for the federal government’s use of automatic data processing services under the Brooks Act, which led it in 1968 to commission a series of studies on the government’s Federal Information Processing Standards (FIPS), including safety and security standards (Burns and Radack, 1977).
In early 1972, NBS held discussions with NSA, and received guidance that encryption should be incorporated in civilian applications. 8 In a Memorandum of Understanding between NSA and NBS dated September 18, 1972, the two organizations outlined ‘the provision of adequate security measures for the protection of data and control of data access in data processing systems is vital to the national interest. The federal government … [h]as a basic responsibility to assure the development and application of appropriate computer security measures’. Initially, the memo suggests, this was envisioned as a plan to transfer ‘techniques for the protection of data and the control of data access which have been developed by NSA’ (National Security Agency, 1972).
By 1972, the NBS concluded that a technical solution should be developed and issued a call a year later for an encryption method that would be published as a public standard and used to protect the storage and transmission of data government-wide (Burr, 2001). In a status report outlining NBS progress in acquiring an encryption algorithm for data protection, the agency outlined that it believed ‘based on expert guidance, that data encryption is the only acceptable means for protecting data during transmission between a computer and its terminals or other computers’. 9 Unsurprisingly given the dearth of research in cryptography at the time, IBM’s Cryptography Research Group won the bid.
Databases and privacy
Though his IBM work was overtly designed around data security, Horst Feistel had his own concerns. Feistel worried about databases. He worried that their growing sophistication – and the increased appetite for data to fill them – would lead to surveillance of the public. In a 1973 article about Lucifer in Scientific American, Feistel outlined an urgent need to protect individual privacy from the threats posed by computer systems, and particularly databases containing personal information. ‘Since many computers contain personal data and are accessible from distant terminals, they are viewed as an unexcelled means of assembling large amounts of information about an individual or group’, he wrote. He described the ‘dangerous ease’ with which dossiers could be compiled on citizens, noting that computers made it easier than before to gather together information from disparate locations to monitor citizens (Feistel, 1973).
Feistel’s suspicions of government may have begun with his experiences as a German émigré. Born in Berlin, he left Germany during the years of Hitler’s rise and immigrated to the United States in 1934. He was placed under house arrest in his home in Cambridge at the outset of the war, but was granted citizenship and a security clearance to work for the Air Force on secretive research projects in the years following. Given these early experiences, h would have been familiar with the abuses of power that can result from the excesses of surveillance.
He was not alone: A series of scandals spurred concerns among other technologists about government surveillance. In 1971, the Citizens’ Commission to Investigate the FBI exposed the COINTELPRO program, revealing that the agency had compiled thousands of dossiers on predominantly Black Americans, targeting a wide range of groups that included members of the Civil Rights Movement and Black Panther Party. 10 The Watergate investigations into wiretapping in the White House were also well underway by 1973, with the Senate’s televised hearings into the scandal starting only two weeks after the publication of Feistel’s article.
Paul Armer, then a fellow at the Center for Advanced Study in the Behavioral Sciences (and formerly the director of the Stanford Computing Center) wrote an article in the magazine Computers and People describing the state of the art in computing technology as ‘a most important sub-set of surveillance technology’. Armer warned that the advances in computing technology of the time – improvements in computing power, introduction of microprocessors in everyday devices like trucks and appliances, and networking systems like the ARPANET – would have profound effects on individual privacy. He was particularly concerned that the very kinds of electronic funds transfer systems Feistel was at work building were primed to become ‘the best surveillance system we could imagine with the constraint that it not be obtrusive’ (Armer, 1975). 11
While many in the tech industry acknowledged the problem of surveillance, there was not, at this point, any consensus around cryptography – or any kind of technical solution for that matter – as the appropriate approach to addressing it. When Congress held hearings over the use of databanks by the federal government in 1972, a collective called Computer People for Peace asserted that technical safeguards could not ensure the safety of the data in federal databanks. Drawing on their expertise as technologists, the members said in their testimony: Because we work in the field we know that there are no software or hardware constraints which can be incorporated into any system to make it foolproof. … In no way could we abdicate our responsibility to the public by making them believe that we could technically design data bank systems which could offer them the protections they so desperately need. The solution must rest with the people and the enforcement mechanisms they devise; it can not rest in the computer industry, although the industry must take responsibility for its actions. (Computer People for Peace [CPP], 1972, emphasis added)
In hindsight, these fears were warranted. In 1977, a Congressional investigation meant to inform an update to the country’s wiretap law revealed the National Security Agency had developed substantial dragnet eavesdropping capabilities. For years, the investigation revealed, the NSA had been collecting telegram traffic in bulk, storing it in computers and using lists of key words to search and retrieve messages deemed suspicious – the kinds of capabilities Feistel warned were possible with ‘dangerous ease’ with new database computing capabilities. The NSA was also caught illegally monitoring the phone conversations of US citizens (Volkman, 1977). An article in Science noted that NSA capacities were helped by three technologies: The capability to send multiple telegraph messages on a single stream, sorting them at the receiving end, the growth in computer storage capacity and the ability to retrieve with precision selected information from databases (Shapley, 1977).
As the debates over databases made clear, by the mid-1970s there was an urgent need for protections against the kinds of incursions on privacy made possible by database computing systems – but it was unclear what solutions were most needed. Where Armer and the activists of CPP situated solutions to the database problem in policy, Feistel, IBM and the National Bureau of Standards turned to technology. Feistel in particular believed cryptography could present a potential solution to these incursions on privacy. It would limit abuse of such systems by ensuring that only those who are meant to have access could gain access to the information, making it difficult for intruders to take advantage of computerized databases for nefarious purposes.
Moreover, Feistel asserted that cryptography would make it possible to protect against attempts to alter the information in a system, thus hindering computer-assisted fraud. Feistel was concerned with the potential for deception through database systems. As he described in a classified report to IBM, ‘machine communications systems, in contrast to systems which can enlist the subtle filtering capabilities of the human brain are very sensitive to interference and deception. Without special protection computers are easily fooled and this can become an intolerable burden to a data bank operation if this remains unnoticed’ (Feistel, 1970). He concluded that cryptography could be used not only to ensure confidentiality in communication, but to provide privacy and authentication for communities of databank users. His invention of Lucifer was an important, but ultimately only partial, step toward addressing the problems he outlined.
Opposition
Around the same time, Stanford Associate Professor Martin Hellman and his graduate student, Whitfield Diffie, began working together to research ideas around cryptography. Hellman’s interest was first sparked after he completed his PhD and began working with Feistel and the Cryptography Research Group at IBM. By 1971, Hellman moved on to Stanford’s Electrical Engineering department, and, despite the discouragement of his department, had begun his own research into cryptography. He was joined in 1974 by Diffie, at the suggestion of a former IBM colleague, who was likewise looking for others with a passion for cryptographic research. Hellman encouraged Diffie to sign up as a doctoral student at Stanford, and they were joined shortly afterward by Ralph Merkle, another researcher working independently on cryptography at the University of California Berkeley (Yost, 2015b).
When IBM won the bid to develop the NBS system, it immediately encountered scrutiny from Hellman, Diffie and others. The company used a variant of Lucifer termed the Alternative Encryption Technique, later renamed the Data Encryption Standard (DES) (Mollin, 2006). To produce the new standard, IBM reduced the size of its encryption key from 64 key bits to 56 bits. Among the most vigorous objectors, Hellman and Diffie outlined a series of concerns about the proposed standard, arguing that the reduced key size made it easier than necessary to conduct a ‘brute force attack’, through which a computer would try all possible keys until it arrived at the correct one. Through a series of calculations, they found that it would take an actor with a $20 million system slightly less than one day to break the encryption. Though this would be economically infeasible for any commercial entities, it would make any systems protected with the standard vulnerable to a nation-state actor.
Moreover, they argued, using all 64 key bits would exponentially increase the system’s security, raising the cost to over $5 billion for a one-day search time – a requirement of computing power that next to no adversaries of the US would be able to marshal (Hellman, 1975). 12 Based on his calculations, Hellman felt that it was likely there were other motives at play in the reduction of the key size. In a letter to cryptography expert David Kahn, he confided that he believed NSA, which NBS consulted in its choice of a standard, was responsible for the choice of a less-than-optimal standard, prioritizing its need to be able to solve intercepted communications from other nations over the security of US commercial systems (Hellman, 1976). 13 His reasoning was motivated by concerns about state security: He worried that the weakening of a commercial standard would open up the communications of US businesses to spying by other nations. Since the US had far more computing systems than the Soviet Union, Hellman later recalled, they feared it would have the most to lose from insecure encryption – thus, they felt was in the country’s national security interest to strengthen the standard (Hellman, 2004).
Though there is no definitive proof, the notion that there was some level of intervention by NSA in limiting the strength of DES is persistent among cryptographers. The IBM documents I analyzed, many of which were given ‘classified’ status within the company, at a minimum confirm the existence of longstanding backchannel communications between the company and the NSA in its development. Though there was a compelling government interest in U.S. commercial enterprises protecting their communications abroad, these interests may not have overcome intelligence agencies’ demands to limit widespread adoption of encryption or the promulgation of technologies that made this possible.
In their opposition, Hellman, Kahn and Diffie drew connections to the database debates that situated cryptography as a solution to concerns about government surveillance and invasions of privacy and leveraged these concerns to stage a public campaign opposing the DES standard. In an op-ed in the New York Times about the DES standard David Kahn noted, ‘Like people, computers talking to one another can be wiretapped…this has led to demand for a common cipher – a system that would both permit intercommunication among computers and safeguard the privacy of data transmissions’. He went on to consider the broader effects of computer networking for surveillance: Why should the National Security Agency be so passionately interested in the 56-bit key that it asked to attend a meeting that Hellman set up on the question and flew a man across the country for it? The N.S.A. expert declined to say. But one obvious reason is that, with a solvable cipher, N.S.A. would be able to read the increasing volumes of data that are flowing into the United States time-sharing and other computer networks from abroad. (Kahn, 1976)
In a letter to US Secretary of Commerce Elliot Richardson, Stanford Professor Martin Hellman agreed that DES ‘may pose a threat to individual privacy’, as it made it possible for NSA to ‘misuse its ability to delve, almost at will and undetected, into the supposedly private files of other agencies’ (Hellman, 1976). Hellman, Diffie and Kahn were successful in rallying institutions including Alcoa, Bell Telephone Labs, Sperry Univac and MIT in opposition to the standard. But despite their efforts, the National Bureau of Standards adopted DES in 1976 (Hellman, 1976).
This led to a rapid spread in the use of cryptography among commercial firms in the years following. Only two years later, the Wall Street Journal wrote of hundreds of large firms that were now encrypting their data: Exxon, Shell and US Steel used cryptography to protect computerized personnel files. Ford encrypted administrative memos it sent between its headquarters and global plants to prevent their interception. Oil companies began using ciphers to protect the geological and drilling information stored in their computers. Construction companies used cryptography to encode bids sent to countries where the competitors were government sponsored, and others encrypted messages about executive travel to nations known for terrorism. The Department of Agriculture even used encryption to secure the information used to make monthly forecasts of US crop production (Shaffer, 1978).
The leaders in adopting encryption technologies, however, were banks and financial institutions. They were uniquely motivated; as Richard Shaffer put it in a Wall Street Journal article, ‘computers have largely replaced checks and letters as the means for moving large amounts of money. The machines are connected in globe-spanning webs of telephone lines, and tapping the lines could enable someone to steal huge sums’. Shaffer cited the New York-based Greenwich Savings Bank, an early innovator in the use of debit cards, as one exemplar: The bank followed the lead of IBM’s client Lloyd’s and encrypted the passwords its customers punched into the Bank’s new debit card terminals. The SWIFT computer network, used to secure international financial transactions, encrypted the messages to and from the 500 international banks it linked. Citibank also began to encode all traffic on some of its private wires, such as those linking New York City and London (Shaffer, 1978). Notably lagging were credit monitoring companies: Shaffer cites Equifax and TRW Credit Data as among those who felt their files were safe enough without encrypting, and even expressed fear that governments would force them to impose the technique.
But these were exceptions to the rule: By the end of the 1970s, cryptography was commonly used by companies to secure their increasingly networked communications. The primary reason for using cryptography, at this time, was for data security: Keeping communications and transactions confidential and preventing manipulation of databases by the emerging computer fraud industry. Though adoption of cryptography rapidly grew outside the previously limited scope of state intelligence agencies, its underlying cultural meaning was largely unchanged: Cryptography was designed for secrecy and security of information. This would soon shift, the product of changing technological and political conditions. However, as the debates around DES suggest, an important set of articulations had already occurred among government, commercial and academic actors as they converged on cryptography as a technical solution to these emerging problems.
New directions: Inventing public key cryptography
The same year, Diffie and Hellman made a remarkable discovery that would solidify the relationship between cryptography and privacy, dramatically changing the social conditions around data encryption. Like Feistel, Diffie and Hellman were motivated both by their politics and belief, at the time, in a technical solution to the privacy problem. They also had an additional motivation: They recognized a strong commercial need for encryption. As Hellman later described it, ‘The fact that IBM was spending a huge amount of money on cryptography told me there were commercial applications for it’ (Hellman, 2004).
But outside IBM and NSA, there were still few researchers working on problems relating to cryptography, in part due to fears that new discoveries would immediately be classified and kept from public view (Hellman, 2004). IBM even began discouraging further research into cryptography after the development of DES, instead turning its focus to secure operating systems (Yost, 2015a).
Despite this, Hellman and Diffie worked together on what they hoped at the time would be the foundation for an entire new theory of cryptography. The product of their work, public key encryption, is a reflection of their goal to revolutionize the field. In an article titled ‘New Directions in Cryptography’, the pair outlined how both technological and economic change opened up new possibilities for incorporating cryptographic devices in commercial applications. ‘We stand today on the brink of a revolution in cryptography’, they announced, arguing that cryptography would be transformed from an ‘ancient art into a science’ (Diffie and Hellman, 1976).
Public key encryption and social transformation
If one-way systems lend well to hierarchical organizational structures, the introduction of public key cryptographic systems opened up new kinds of social relationships that could be fostered through cryptography. As one writer in Science put it, ‘This proposal may be arriving just in time to overcome the massive logistical problems that exchanging codes will pose if computerization of communications continues as expected’ (Kolata, 1977a: 747). Initiating encrypted communications was no longer a one-way affair that required cipher systems to be shared clandestinely – in fact, it no longer required that the communicating parties even know one another in order to encrypt their communications. Private conversations could be conducted in public view, on the basis of mutual trust in the cryptographic system. This materially challenged the idea of cryptography as being solely and fundamentally about secrecy; it could instead be about privacy, security, and communication.
The cultural importance of public key cryptography was in many ways bolstered by a concerted effort by the US government to keep it secret. As scholars of nuclear technology have observed, this was a frequently used management technique by the military science establishment, which sought to compartmentalize the production of knowledge (Birchall, 2011; Dennis, 1999). However, as Masco (2010) notes, secrecy can become the basis for creating new kinds of power – in this case, by imbuing work on public key cryptography with a kind of cultural cache.
Other members of the academic community also played important roles in public key cryptography’s invention. Ralph Merkle developed an early version of public key distribution in a paper published shortly after Diffie and Hellman’s (Merkle, 1978). 14 Hellman also cites John Gill, an associate professor at Stanford and one of the first black graduates of Georgia Tech, as the person who came up with the idea of using factoring or modular exponentiation/discrete logarithms for the one-way function in the paper. Ron Rivest, Adi Shamir and Leonard Adleman, all at MIT, played a crucial role by developing a practicable implementation of Diffie and Hellman’s system. They were quick to patent their ideas, and eventually built a successful business, RSA, off of the scheme, demonstrating its commercial viability.
The nature of the invention of public key cryptography as a concept that emerged out of an academic community destabilizes the common notion that it was a ‘great man’s invention’ or embodied a singular big-bang moment for the field. Instead, it reinforces that this technical development, despite the cultural cache it later developed as an instrumental solution to the problems of privacy and surveillance, is the product of work within a small, but growing community of researchers to a widely known set of social and technical problems. As Jean-Francois Blanchette writes, ‘What perhaps secured “New Directions” such a lofty place in the pantheon of cryptography was its authors’ ability to situate their discoveries in the context of a broad historical progression’ (Blanchette, 2012: 40).
Hellman wrote years later: …We were almost being channeled in that direction, and what I say when I give this talk on the evolution of public key cryptography, I say, ‘Well, initially your reaction may be how did they come up with something so earth shattering, so ground breaking, so different from what was before?’ But after I describe all the threads that were leading us there consciously or subconsciously, I hope your reaction will be, ‘Why did it take them so long?’ (Hellman, in Yost 2004: 24)
Why was it that public key cryptography emerged in the mid-1970s, and not at another time? The key-solving problem had proved a persistent challenge in cryptography over its centuries-long history. In the face of an urgent need for privacy protections, why did it take so long for a solution to emerge, and even longer to come to fruition?
Competition
One possibility is that public key encryption’s development was actively hindered by competing parties in both the public and private sector. For one, there was already a competing standard in DES, with the weight of both IBM and the US government behind it. DES adapted cryptography as it is conventionally understood for use in computers, itself a novel approach. Given IBM’s centrality to computing at the time, it is unsurprising that DES had considerable weight. Standards are powerful forces within technical processes (DeNardis, 2009), and can carry a kind of political agency by inscribing the ideas and viewpoints of their creators. Contests over standards can thus become a site for ‘politics by other means’ (Abbate, 1999: 179).
Moreover, the inventors faced competition of another sort, though they could not have known it at the time. They were beaten to the punch by other cryptographers who had already developed a similar cryptographic system a few years earlier – but at a moment in time and in a context in which the potential uses of public key encryption were difficult to envision. This reinforces the premise that the environment in which a technology is developed matters greatly for its downstream use.
James H. Ellis, a researcher at the British signals intelligence agency Government Communications Headquarters, conceived of a system of ‘non-secret encryption’ as early as 1969, but couldn’t solve how to implement it. When he brought it to his supervisors, he was told that it was ‘garbage’ (Espiner, 2010). Ellis moved on, but his colleague, Clifford Cocks, picked it back up a few years later at the impetus of a fellow mathematician. Cocks successfully developed an implementation of Ellis’ ideas essentially similar to what became known as RSA, a commercialized form of public key cryptography. Cocks judged it to be most important for military use, and again kept the discovery secret. Years later, he recalled he could not foresee its implications until the invention of the World Wide Web in 1989 (Espiner, 2010).
The invention of ‘non-secret encryption’ reveals important aspects of the secretive environment around cryptographic research engendered by the intelligence agency. Both Cocks and Ellis later claimed that they were unable to resolve important issues because they could not publish their ideas. As soon as the research on public key cryptography was out in the open, many of these challenges were solved relatively quickly. This suggests that despite the intelligence agencies’ efforts to build a monopoly on cryptographic innovation by making investments in the best and brightest mathematical minds the nations could offer, the effect of compartmentalization and a closed working environment ultimately hindered the academic creativity of their work.
Academic freedom
Another reason that public key cryptography was initially slow to take off was the intimidation of academics pursuing cryptographic research. Hellman and his students Steve Pohlig and Ralph Merkle planned to present their work on public key encryption at the International Symposium on Information Theory in October 1977, but received pushback in the form of a letter written by a man named JA Meyer to the IEEE, saying: I have noticed in the past months that various IEEE Groups have been publishing and exporting technical articles on encryption and cryptology – a technical field which is covered by Federal Regulations, viz: ITAR (International Traffic in Arms Regulations, 22 CFR 121-128)… These modern weapons technologies, uncontrollably disseminated, could have more than academic effect. (Meyer, 1977)
The IEEE replied to the letter stating that it had determined its publications were exempt, but it nevertheless engendered concern among members of the group. IEEE’s Director of Technical Activities, Nirendra Dwivedi, urged the scientists to clear their papers with their companies or submit them to the State Department for vetting prior to publication.
In the meantime, Gina Kolata of Science Magazine took up the case, uncovering that Meyer worked for the NSA. After contacting the agency’s public affairs office, Kolata received an official statement asserting that Meyer wrote the letter as a private citizen and not in his capacity as an NSA employee. Kolata interviewed the State Department Office of Munitions Control and found that nearly every interpretation made in the Meyer letter was inaccurate: Publications made available to the public were exempted from ITAR. However, if publications were submitted to the Office, as Dwivedi suggested they do, the NSA would gain control over the work, because the Office would refer the papers to the NSA for rulings. As Kolata put it, ‘Meyer was proposing, in effect, a censorship system by the NSA over the research of the Information Theory group’ (Kolata, 1977b).
In response, Hellman turned to Stanford University Counsel John Schwartz for his determination. In a letter to Schwartz, he made a case for the value of public domain crypto research: We were motivated to work in this area by the growing commercial need for data security and encryption, and the almost total lack of unclassified knowledge which could be applied to these needs. While there is a body of classified knowledge, it is unavailable for commercial user… Because of this motivation we have made all of our research in this area freely available through publication in reputable technical journals. (Hellman, 1977)
Hellman described the ‘understandable’ desire for the NSA to maintain its monopoly on cryptographic knowledge, but outlined several reasons why the conditions that made such a monopoly tenable during World War II had changed: First, there is a commercial need today that did not exist in the 1940′s. The growing use of automated information processing equipment poses a real economic and privacy threat. Although it is a remote possibility, the danger of initially inadvertent police state type surveillance through computerization must be considered. From that point of view, inadequate commercial cryptography (which our publications are trying to avoid) poses an internal national security threat. A second difference between the situation in the 1940s and today is the relative ease of building cheap, highly secure cryptographic devices. … Complex cryptographic devices which are impossible to break would have been impractically expensive and unreliable in the 1940s, and any edge that a nation possessed in cryptographic knowledge gave it a tremendous advantage. Today, these ‘complex’ devices can be built on a highly reliable, single integrated circuit which sells for $10 in large quantities. ... The third difference with the 1940s is that we are not currently involved in a hot war where lives depend on cryptanalytic intelligence. (Hellman, 1977)
Schwartz concluded that Hellman and his students could legally publish and present their work, but added, ‘If you are prosecuted, Stanford will defend you. But if you’re found guilty, we can’t pay your fine and we can’t go to jail for you’ (Corrigan-Gibbs, 2014). They concluded that if ITAR was construed broadly enough to cover their work, it was unconstitutional, but were wary of the fact that the only way to determine that would be in a court case (Hellman, 2004).
Schwartz recommended against Merkle and Pohlig presenting, even though it was the norm for student authors to lead presentations. Hellman had the benefit of the tenure system to support him if he were to get into any trouble. Hellman left the decision up to the students, who at first said they wanted to go ahead. However, after speaking with their families, they changed their minds and said they’d let Hellman present on their behalf (Hellman, n.d.).
Ultimately, the papers were well-received at the symposium, and Hellman and his students emerged without any further harassment. Hellman later said that he received evidence that the highest echelons of NSA were extremely troubled by the publication, even though they had suggested Meyer had acted alone (Hellman, n.d.). As Stanford Magazine described it decades later in an article commemorating the work: ‘That a group of nongovernmental researchers could publicly discuss cutting-edge cryptographic algorithms signaled the end of the US government’s domestic control of information on cryptography’ (Corrigan-Gibbs, 2014). Nevertheless, the episode marked an important shift in the social conditions around cryptography, bringing the field of study back into the open. This enabled future work that focused on developing applications for public key cryptography, though debates on the legality of cryptographic research and use persisted through the 1990s and into the present day.
A final, crucial component influencing the adoption of public key encryption was that of technological change. Though the increased use of computers by the late 1970s meant there was a larger market for computer security, the invention of a new method wasn’t enough. Public key encryption needed an application, and computers were not yet powerful enough to be of practical use for implementing public key encryption software requiring large amounts of memory. It would take the widespread adoption of communications networks and increased computer power to bring public key cryptography into full fruition.
Conclusion
Between the mid-1960s and early 1970s, cryptographers working in government, the private sector and in academia developed a shared imaginary for encryption as a viable technical solution to the problems of commercial data security and government surveillance. This positioning was facilitated by a series of changes in the sociotechnical imaginary surrounding the use of encryption that fostered associations between cryptography and privacy, as well as by the practical ability of researchers to engage in work on building cryptographic systems as the field emerged out of the cloisters of national intelligence agencies and into the public domain. Ultimately, these changes offered up a new interpretation of control over information that situated power in the hands of those with the capacity to encrypt it, and a new form of cryptography – public key cryptography – that materially embodied these ideas.
As this account illustrates, public key cryptography was not a singular invention. Its significance can be best understood as the product of an emerging consensus among computer scientists and the technology industry of the importance of privacy and security in the wake of abuses of surveillance powers by government agencies. This was combined with an attempt to open up a new field of potential uses for cryptography suited to a changing technological, political and commercial context. Diffie and Hellmann were, paradoxically, both a decade too early and a decade too late in their pronouncement. The ‘revolution’ they declared in their paper began in earnest a decade before, when government and commercial cryptographers began to consider the implications of cryptography for the era of networked communications. But it would take a decade longer, with advancements in computing power and the spread of timesharing systems, for it to come to full fruition.
The invention of public key cryptography was, however, a key inflection point for a series of changes in the social conditions around encryption, as a constellation of different actors turned to cryptography as they confronted the need to address the growing problems of computer security and surveillance. Lucifer was a precursor, demonstrating that cryptography could be implemented in computerized systems and backed by the weight of industry players like IBM and government institutions like NBS. But public key cryptography ultimately became the solution most compatible to the needs of networked computing systems, by facilitating the adoption of secure cryptographic systems that could be set up in the open. It is at the material level, in the design of public key cryptographic systems, that the evolving ideas around the cryptographic imaginary took root.
Ironically, cryptography then became itself the focus of efforts to control access to information, because in the wake of the publication of Diffie and Hellman’s paper the NSA sought to reassert state secrecy prohibitions on publications about encryption. Despite these efforts, the study of cryptography re-emerged in the public domain during the 1960s and 1970s, first through the work of Feistel and IBM’s Cryptography Research Group, which demonstrated the commercial viability of cryptography in the tech sector, and then through academic work at Stanford and MIT. This battle over the right to conduct cryptographic research undergirded a deeper debate over the use of cryptography as a means of controlling access to information – if it could be used by average citizens, they too would have access to protect their information from the prying eyes of the government. Though similar debates arose in the 1990s, over the Clipper Chip, and in the 2010s, following the Snowden revelations, it is during this early period of the 1970s that the notion of cryptography as a technical solution for protecting privacy first emerged, and carries through to the present day.
Footnotes
Acknowledgements
I’d like to thank Lucy Suchman and Sergio Sismondo for their invaluable support in editing this piece, and to the external reviewers for their generative comments on the manuscript. I presented early versions of this work at the International Communication Association, Microsoft Research and AI Now Institute and received invaluable feedback. Thanks also to Mike Ananny, Finn Brunton, Manuel Castells, Christina Dunbar-Hester, MC Forelle, Elizabeth Kaziunas, Chris Kelty, Lee McGuigan, Britt Paris, and Joy Rankin for their thoughts on this work.
Funding
The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article was produced under fellowship support from the Annenberg School of Communication and Journalism and the AI Now Institute.
