Metahuman systems = humans + machines that learn

Abstract

Metahuman systems are new, emergent, sociotechnical systems where machines that learn join human learning and create original systemic capabilities. Metahuman systems will change many facets of the way we think about organizations and work. They will push information systems research in new directions that may involve a revision of the field’s research goals, methods and theorizing. Information systems researchers can look beyond the capabilities and constraints of human learning toward hybrid human/machine learning systems that exhibit major differences in scale, scope and speed. We review how these changes influence organization design and goals. We identify four organizational level generic functions critical to organize metahuman systems properly: delegating, monitoring, cultivating, and reflecting. We show how each function raises new research questions for the field. We conclude by noting that improved understanding of metahuman systems will primarily come from learning-by-doing as information systems scholars try out new forms of hybrid learning in multiple settings to generate novel, generalizable, impactful designs. Such trials will result in improved understanding of metahuman systems. This need for large-scale experimentation will push many scholars out from their comfort zone, because it calls for the revitalization of action research programs that informed the first wave of socio-technical research at the dawn of automating work systems.

Keywords

Machine learning learning theory technology work groups job design organizational forms monitoring embodiment autonomy

Introduction

Machines that learn interact adaptively with their environment while increasing their capabilities through experience. Such machines are now being embedded in apps, news feeds, video streaming services, and email filters. Machines that learn do so in a manner similar to human trial-and-error learning. But because of digitally enabled transfer capability, they differ in learning speed, scope and scale. We use the term machines that learn here, however, as a distinct category from the common noun phrase in computing fields called machine learning. Machine learning is a specialized kind of computational process (Domingos, 2012), whereas machines that learn abstract from the details of the computation and focus on the process features, outcomes and emergent features of systems like their scale, scope, speed and architecture. This paper addresses machines that learn as parts of wider systems where both humans and machines learn jointly (Davenport and Kirby, 2016). These are called metahuman systems. The capabilities of machines to learn can change in such systems at a different rate than in how humans learn. Our focus is on the socio-technical consequences of introducing machines that learn to work settings and systems as organizations start deploying metahuman systems.

We use the Greek meta as a prefix for such systems because it fits with the nature of these systems etymologically, colloquially, and prescriptively. Etymologically meta means “after” and “beyond,” as in post-human. Metahuman systems invite us to explore things that go beyond what machines or humans alone possess. Such superhuman aspects of metahuman systems show now often up in popular books (Bostrom, 2017). They can also be oppressive (Zuboff, 2015, 2019). There is also a colloquial meaning of meta in denoting “about” and “above” as in metastudies that analyze and synthesize findings of other studies. In our case, meta used colloquially relates to expanding and synthesizing ways in which humans and machines are connected in work systems and that study of metahuman systems encompass all systems using machines that learn in the context of human work systems. Our prescriptive use of meta is focused on “beyond” in the sense of models of models and related metamodels that humans use to reason about assessments made by machines come to fore in the examination of metahuman systems. Before humans delegate to a machine a task as they would to a colleague, for example, when doctors who use artificial intelligence (AI) to diagnose cancer and delegate that task to a machine they want to understand how the AI machine has been trained to calibrate their delegation. In this regard, they either understand the process or engage in constant monitoring of it (Cai et al., 2019). In such settings, higher metalevels affect always design of systems at work. For example, a fleet of autonomous vehicles operates in an ecology of designers, passengers, controllers and regulators who need to work together and react to some situations that the (meta)system creates. They must always consider the next generation and understand also the design for design. Metadesign, if done well, allows to design better systems, and possibly paves way to systems that design themselves.

When machines learn, their capabilities change in unexpected ways and at a speed and scale different than we have encountered before. Currently, information systems (IS) research does not reason pragmatically and diligently about this. Some IS research presupposes that machines do not change or change very slowly, as seen in on use (Venkatesh et al., 2003) and affordance (Leonardi, 2011) related theorizing. Some advanced sociotechnical research presupposes symmetry between machine and human (Latour, 2005). The truth lies likely in between: machines capability changes, sometimes rapidly, even if not rivaling the capabilities of humans because machine and human learning processes are embodied differently (Lawrence, 2017).

Organizational bureaucracies where metahuman systems now operate serve important purposes (Du Gay, 2005). This is particularly true with respect to delegation and training of people that comes with the design of metahuman systems (Barnard, 1968 [1938]; Simon, 1997 [1947]). Metahuman systems are changing and will change these dramatically in future. For example, knowledge production in Wikipedia is already now shared between humans and bots which both approve and manage processes (Zheng et al., 2019). The proposed IS research agenda involves multiple levels of meta: first understanding differences in cognitive architecture and how this affects the design of organizational metaprocesses for delegation and monitoring (operational processes), cultivation (capability-building processes), and reflection (systems improvement processes).

It is still early days of metahuman systems, but these systems already now receive growing attention under different monikers in multiple discourses. Simon (1997 [1947]: 234ff) anticipated metahuman systems that build with their own initiative knowledge beyond the developments of the organization’s internal research departments. Now such systems are used in automated drug discovery (Schneider, 2018) and other fields. The concept is creeping in management discourse when Uber drivers are managed by algorithms that delegate driver tasks irrespective of human resistance (Lee et al., 2015; Möhlmann and Henfridsson, 2019). The notion appears when doctors now use adaptive robotic surgery systems to delegate surgery tasks to machines, but find at the same time that they must protect patients from swinging robotic arms (Sergeeva et al., 2018). Metahuman systems are showing up in multiple work settings: industrial organization, market making, election results, transportation and urban mobility, education, and scientific research, just to name a few. Machines that learn have been overhyped similar to the hype of 1930s Hugo Gernsback futuristic fanzines. They can produce negative, unanticipated effects (Marcus and Davis, 2019; Zuboff, 2015, 2019). Innocuous rectangles on traffic signs can fool autonomous vehicles leveraging the way machines are trained to cause accidents (Heaven, 2019). Given a certain brittleness of machines that learn, metahuman systems are susceptible to new types of attacks. IS research agenda has from early days blended social and technical (Hirschheim, 1985; Kling, 1980). IS researchers should now consider metahuman systems as unique kinds of socio-technical systems in their inquiries to remain relevant (Sarker et al., 2019).

An ongoing experiment in metahuman type learning provides an illustrative and powerful example of the immediate salience of the issues the field faces. The Never Ending Language Learner (NELL) learns substantially on its own and through humans about language and its uses. It reads web pages on its own to build a repository of contemporary beliefs, and then uses humans to explore the exact meaning of these beliefs (Pedro and Hruschka, 2012). By 2015, it had amassed over 80 million beliefs, with high confidence in over 2 million (Mitchell et al., 2018). When NELL fails to recognize a belief, it tweets human followers, parses the replies, and integrates the results to improve and expand its understanding (Carlson et al., 2010; Pedro and Hruschka, 2012). In learning terms NELL is a mixed-initiative, autonomous learning system (Horvitz, 1999; Parasuraman et al., 2000), and illustrates a growing trend in which machines use humans to further learning (Ekbia and Nardi, 2014). Any technology like NELL will go as part into metahuman systems. Such systems require novel research to address challenges like those identified by Zuboff (2015, 2019) regarding hidden or latent monetizing of consumer’s ‘behavioral surplus, by Aleksander (2017) that robotic systems far from human intelligence demonstrate character, capabilities and motivations different from humans while lacking transparency required for organizational use. Metahuman systems will blur the boundaries between the socio (the human) and the technical (the machine) in unanticipated ways which calls for novel inquiry.

The IS field grew out from the need for experts who can skillfully work at and bridge the boundary of the social and the technical (Su et al., 2017). Metahuman systems create a new kind of need for experts who can work at a rapidly shifting learning boundary in organizations. Advances in machine learning technologies will change learning assumptions on which the foundations of modern management were built. Herbert Simon hinted at this several decades ago when he noted the difference in speed between human learning and learning by machines (Simon, 1983: 26–27). Even if it takes a while to get machines to learn specific tasks or content, once learned, the machine can disseminate the learned capability quickly (Kallinikos et al., 2013; Lyytinen et al., 2016). Metahuman systems will bring higher speed of machines that learn and alter the scale and scope in which metahuman systems learn. A myriad of consequences is likely to follow—many of them not intended. This calls for a revision of the IS research agenda: how do we think of sociotechnical systems where learning forms one of the main dimensions of their behavior?

The concept of metahuman system

Metahuman systems are a hybrid of humans and machines that learn, complementing and amplifying capabilities that potentially make such systems better at learning than either humans or machines separately. What makes the story challenging and interesting is that machines and humans have different cognitive architectures and so far organizational design has operated under the assumption that all learning components in organizations have similar (though somewhat varying in level and skill) of cognitive architectures and ergo learning speed, scope and embodiment. The emergence of different cognitive architectures explains differences in learning speed which affects the scale and scope of learning at the system level. This calls for a joint common definition of learning that covers learning as an outcome irrespective of cognitive architecture. In this essay, we define learning as a process of increasing capabilities in a configuration of agents: human, machine, or mixed.

Past IS research assumes that humans learn and machines do not (Venkatesh et al., 2003). At the same time, humans have used machines to improve the overall performance of a system given machine characteristics of accuracy, speed or nearly fault-free behavior. In consequence, research on learning in organizational contexts has focused on human cognition and related learning (Leonardi, 2012). That machines can learn is sort of implied by actor network theory (Latour, 2005), critical realism (Mingers et al., 2013), and DeLanda’s (2019) theory of assemblages. However, they do not recognize the emerging learning capability of hybrids where both machines and humans learn. As the discussion of cognitive architectures below will show, it does not ultimately matter whether humans and machines learn through similar or dissimilar processes. Nevertheless, much of the research, including ours, in the context of metahuman systems proceeds on assuming the presence of mechanisms for learning in both humans and machines (Lake et al., 2015). The dynamics that surface when these types of learning agents with different architectures and skills interact are still unknown. Using a machine in a cognitive loop will, for example, causes humans also to anthropomorphize the machine expert (Cai et al., 2019), or only selectively follow machine advice when it reinforces human’s own biases (Green and Chen, 2019).

A thorough review of learning literature is beyond the scope of this essay. Learning spans many disciplines with many denotations and connotations. We constrain our discussion to learning relevant to metahuman systems. We differentiate between trial-and-error learning, when skills might be learned for the first time by humans and machines, and diffusion-based learning when skill are transferred (Argall et al., 2009). Given this distinction learning can be first simplified to the observation that any agent’s learning involves some sort of trial and error process (Thorndike, 1932) with intermittent rewards and punishments regarding goals (Sutton and Barto, 2018). An agent acts in an environment, and based on environmental reaction, decides what to do next. As it moves through this process it acquires new capabilities that make it better fit operating in that environment. The increased capabilities that accrue from learning can be measured by testing what an agent can do after a learning event in that setting. Testing might involve tasks to infer cognitively what agents can do when they are actively functioning as a learning system. Machines that learn receive reward or punishment based on the environment’s response to the machine’s action (Sutton and Barto, 2018). Even with their current limitations, this simple learning process allows machines to move into domains hitherto reserved to human learning. There are trade-offs: reinforcement learning in machines requires massive amounts of data, while humans learn through similar process using a small number of instances (Lake et al., 2017).

Trial-and-error learning encompasses also metalevel: learning to learn. Trial and error single loop learning builds specific capabilities to do something as well as generic capabilities to assimilate such specific capabilities. Double loop learning builds an ability to achieve higher learning capability which covers also changing goals and assumptions of learning (Argyris, 2004). Trial and error learning is often costly and slow for both humans and machines so specific solutions have been developed to circumvent it which enable diffusion-based learning. In such learning mode, humans and animals learn through imitation based learning (i.e. through observation or social participation). By watching others do something successfully or engaging in such activity with others, allows one to infer and repeat expected patterns of behavior (Frith and Frith, 2012). Multiple other mechanisms to diffuse knowledge might work in humans (and in many animals), with the fastest conjectured mechanism a kind of mirroring at the neurological level (Rizzolatti and Fogassi, 2014). Machines cannot do any of the above yet to diffuse knowledge. But this advantage of humans is met by a machine advantage in that they possess very fast mechanism of direct copying of learned capability accomplished by transferring lines of code, data, or matrix weights. Analogous tasks can therefore be transferred in machines by a process akin to human transfer learning (Mestre, 2006), wherein certain layers of a matrix are copied and recombined (Tan et al., 2018).

Four aspects influence generally how metahuman systems will learn either through trial-and-error learning or diffusion-based learning

Trial and error learning is context-dependent and requires language and cultural knowledge. Moral judgments that separate right from wrong fall here. Humans tend to be better at this than machines (Marcus and Davis, 2019; Smith, 2019). Sometimes humans can do it and machines cannot. Even when machines can do it, humans tend to do this more robustly. This might change over time, but for now humans have the advantage. Even basic language skills are elusive for machines: a prize has been offered for a machine agent that can sustain a 20-min conversation with a human, but by 2018 the median conversation lasted less than 2 min (Ram et al., 2018).

Trial and error learning takes time and requires significant attention. Performance time in humans might be less than in machines, but machines can operate relentlessly, 24 hours a day in dedicated mode. Elapsed time can be shorter for machines. Recent deep learning variants have made machines to progress in learning within realms where simulations can be used to produce infinite amounts of data.

Diffusion-based learning from one agent to another is fast if the agents are machines. Learning by knowledge transfer in machines speed up learning and response in metahuman systems. In contrast learning between humans and machines is slow due to significant differences in diffusion learning skills and mechanisms. This can only be improved by significant improvements in human–computer interfaces.

In diffusion based learning the use of sensory learning that extends human senses with those of machines can empower human learning (Sinz et al., 2019). Machine augmentation in vision alone includes an arsenal of microscopes, telescopes, night-vision equipment, and radar and such sensory capabilities have been a component of metahuman systems for some time. There are concerns about atrophication resulting from such advances for humans including the wide use of GPS tools reducing human unassisted navigation skills (Robbins, 2010) or how calculators have lowered arithmetic skills in humans.

A unique aspect of human learning is that it always happens in a cultural context (Engeström et al., 2016). Distributed cognition posits that any human knowledge, the consequence of learning, is embedded in the material organization of humans and machines in a cultural setting (Clark, 2017). Knowledge can reside in a human team, it can be embedded in their tools, or it can be inscribed in the physical environment through maps, signs, switches, and other kinds of interfaces. No one yet knows how metahuman systems distribute (or should distribute) learning objectives and capabilities as part of larger organizational or institutional processes. IS research involving metahuman systems requires deepening understanding of learning in ways that goes beyond notions of anthropocentric learning. Scholars should not project onto machines the capabilities and processes of humans, and vice versa. But speed, scope of learning matters, and differences in degree in any of them can and will become differences in kind at the metahuman system level.

Cognitive architecture

Embodiment in human learning suggests that humans’ cognitive processing related to learning depends on the architecture of human bodies (Lawrence, 2017). Inbound and outbound information used in cognitive processing associated with learning in humans is communicated through sensory experience embodied in the human body (this includes also spoken or written language). The human brain may have 100 million times the processing power of a desktop computer, but the computer can exchange information with another computer 100 million times as fast as humans. Metahuman system designs can and should take advantage of these differences in cognitive architectures. A human takes perhaps 10,000 h of learning to become an expert due to the biological foundation of human learning (Ericsson et al., 1993). Machines operate at different rates and the underlying technologies are improving. Moreover, as noted, machines already transfer information very fast. This was observed early on by Herbert Simon (1983): “The first obvious fact about human learning is that it’s horribly slow . . .” (pp. 26–27). Simon was not sanguine that machines could match human learning, though it was worth trying, because machines could replicate acquired capability quickly with other machines. Few doubt the organizational, social and economic consequences, if machines rival or surpass human learning and significant cognitive tasks observed in organizational settings. Even if it takes years to get there (such as conducting tax planning or trading equities with even 95% effectiveness), the machine’s ultimate advantage is in the fast transfer and replication of such capability which would have revolutionary consequences of related organizational tasks (Kallinikos et al., 2013; Lyytinen et al., 2016). But there is no need for machines to rival or surpass human learning in all aspects of complex knowledge tasks. The tasks can be distributed in specific ways in metahuman systems and such metahuman systems are already faster than biological-only-human based systems learning organizations. Acceleration of the scope of transfer learning and diffusion based learning between machines will affect how IS, cognitive psychology, computer science, biology, and other fields approach learning in social settings (Kallinikos et al., 2013; Lake et al., 2017; Lawrence, 2017; Rosen, 1991).

There are many sensational, science fiction-like claims about machines that learn without limits (Vinge, 1993). Most predictions are simple extrapolations of exponential improvements in raw “machine power” like those applying Moore’s law where a given amount of money buys roughly twice as much computing capability every 2 years (e.g. Kurzweil (2000)). It is not clear that machines can rival human learning or design themselves to learn and replicate because of just their ever greater computing power. Much depends on the definition of intelligence (Aleksander, 2017); it also depends on its origin. Human intelligence reflects human needs: surviving and adapting as biological and social organisms in evolving, complex environments. Machines possess no such needs. In consequence, machine intelligence might not serve all human needs, because teleology and meaning are part of human intelligence but not that of machine (Ackoff and Emery, 2005). Not surprisingly machine survival in the current discourse is treated primarily in technical terms and posed as a problem of security or fault tolerance that are high priorities for human existence. Prior to machines that learn it was not necessary to care about this in a similar way. That may be changing.

The emergent field of metahuman systems

Machines that can think and their effects have excited and informed IS research for more than half a century. We posit machines that learn constitute a new threshold event that will impact the future social organization and all disciplines that deal with organizing (Marcus and Davis, 2019; Smith, 2019). For example, the NELL system shows that there is a new direction where metahuman systems are evolving which will apply and shape organizational practices involving machines in the years to come. Whether still in the lab or deployed, it is time to anticipate research challenges that come with metahuman systems. We will note some of them below by illustrating features of metahuman systems operating in the laboratories and reviewing uses of metahuman systems in select industries.

Metahuman systems in the lab

Machines that can rapidly edit and copy knowledge (Yoo et al., 2010) make it possible for machine trainers to quickly pass new skills to machine trainees (Buciluǎ et al., 2006; Hinton et al., 2015; Rusu et al., 2016). Parallel processing can in addition speed this up by a factor of 10 (Nair et al., 2015). Finally, such knowledge can be compressed. As a result, data sets based on new learning models aid significantly both trial-and-error learning and diffusion learning: training machines can correct trainee machines. Then machines can distill their knowledge into smaller and more compact neural networks. Because of this, trainee machines can quickly match or surpass trainer machines.

There are trade-offs with these advances. Machines will learn faster and better than humans in specific knowledge tasks that rely on pattern recognition with clear goals (e.g. recognizing tumor patterns in radiographs). But in settings with high costs of failure this machine advantage might not be reliable enough. For example, autonomous vehicles that malfunction can cause significant damage. While high-speed driving games have been built and simulated to advance machine learning (Johnson-Roberson et al., 2016), these simulations don’t train for a wide enough range of potential driving hazards (Santana and Hotz, 2016). New techniques are now emerging to address this weakness such as using empirical data extracted from cameras in vehicles to ground the simulations. More generally, new models of agent behavior and learning patterns will be necessary to build machines that learn for tasks where learning takes place in a context of many human and machine agents interacting, cooperating, and competing, as with driving (Lowe et al., 2017; Panait and Luke, 2005).

Human intervention is still expected to set up initial machine capabilities and embedding them into the physical infrastructure, and for their subsequent modification and maintenance. Most machines that learn are not able to learn on their own (Blue and Andoh-Baidoo, 2010; Sutherland, 2008). However, this is now changing. After a learning harness is set up, a neural network system Deep Q uses reinforcement learning algorithms to take action while following a learned policy (Sutton and Barto, 2018). It can also learn on its own and pass its knowledge to other machines. For example, Deep-Q has been made to play video games by controlling a joystick, and then discovering strategies for playing the game that require complex cognitive processing (Figure 1). To accomplish this Deep Q considers multiple appropriate computational methods and evaluates how they meet its needs to maximize the consequences of its actions expressed in game points. As of 2015, Deep Q had learned 49 types of games, with skill on 29 comparable to a professional human game expert (Mnih et al., 2015).

Figure 1.

Deep-Q. Pixel images analyzed by three convolutional neural networks feed two fully connected networks that in turn transmit actions to a joystick that affects game play and changes the video signal. Some training online; some by experience replay when not playing.

Alphabet’s DeepMind has a Go game player that learns from humans (Moyer, 2016; Wang et al., 2016). It can also play by itself to learn game strategies no human invented (Silver et al., 2018). It can start from scratch, continue learning, and work with other machines to build related game knowledge. Similar machines play now capture-the-flag games and discover strategies that humans have not used (Jaderberg et al., 2019). These successes, however, are all in closed worlds and artificial domains. These are machines that do not understand cultural settings and their strategies therefore can have effects that are anti-social (Aleksander, 2017). For example, autonomous cars using machines that learn can sometimes brake sporadically after accelerating and cause accidents. For machines, this is acceptable behavior to learn to drive but such behavior lies outside expected norms in any normal traffic situation.

Early models of the human mind in AI sought to approximate thought through the manipulation of symbols: good old fashioned AI (Lieto et al., 2018). Subsequent models have been probabilistic. They used Bayesian modeling techniques to try to infer backward to causes from phenomenon (Lake et al., 2017; Tran et al., 2017). Indeed, recent AI based workflow research treats humans as “computational nodes” in larger networks of probabilistic hybrid agents. Machines recruit humans onto teams based on predicted compatibility with other group members and their behaviors (Retelny et al., 2014). An important topic for IS research in the metahuman systems is machines selecting humans for joint tasks. This problem already shows up in ride sharing algorithms and decisions about hiring employees.

Because of advances in machine learning the current understanding in the design and use of metahuman systems is that humans pick problems (knowledge tasks) and then create a learning environment for machines to learn about the tasks. After the initial choice, the machines can teach themselves. But there are caveats with this scenario: when the design of the scope is initially done by humans some important knowledge tasks might not be amenable to this approach (Marcus and Davis, 2019). Moreover, everyday tasks are not like a video game; furthermore, a person’s options in most tasks greatly exceed the possibilities presented by game controls to a machine. More generally, there is a larger system surrounding machines that learn which consists of people in companies and universities who train machine creators and the related models of building the machine software (Mackenzie, 2017). Overall, laboratory examples and results in machine learning suggest that machines will continue to improve in learning without close human monitoring. Possibly they may learn to design parts of themselves (Dean, 2019) which will pose new challenges and implications for IS research.

Metahuman systems in operational use

Metahuman systems are already in use in a plethora of industries and settings covering: chip design (Bricaud, 2012), 3D printing metamodels (Kyriakou et al., 2017), travel and tourism (Orlikowski and Scott, 2015), among others. The machines in these systems incorporate various algorithms (Faraj et al., 2018), and are not constrained to the classical organizational containers of earlier sociotechnical systems theory (Winter et al., 2014). They often cross boundaries of market and infrastructure in emergent, poorly understood, complex, and distributed forms. We selected three illustrative examples of metahuman systems which currently shape organization of work and have produced many unanticipated outcomes: high-frequency trading (HFT, sometimes called algorithmic trading), elections, and autonomous vehicles.

HFT shows how speed and scale effects arise from metahuman systems (Lawrence, 2017; MacKenzie, 2019). Trading machines rely now on complex learning algorithms to trade equities quickly and at very high volumes. HFT systems use timely, accurate and extensive market information and draw on that information to learn continuously. Older HFT without learning capability has become over the last 20 years a standard and have brought significantly lower prices per trade, narrowed spreads, created higher price fluctuations, fragmented markets (Arnoldi, 2016; Kearns and Nevmyvaka, 2013; Mattli, 2019). Machines that learn are bringing new capabilities and problems to how financial market operate and how investors and investment banks operate (Hendershott and Riordan, 2013; Lenglet, 2011).

Metahuman system HFT can transact equity trades within a few milliseconds—about a million times faster than human traders. It can also adjust to environmental changes including its own actions or those of competing metahuman systems (Cartea et al., 2016). Humans cannot react in time to prevent frequent, small, short-duration anomalies (e.g. crashes and spikes) (Mattli, 2019). Johnson et al. (2013) found 18,520 anomalies of less than 1.5 seconds duration in a 5-year period. This is much faster rate than traditional market crashes produced by human exuberance, bias and/or herd behavior. Metahuman systems HFT can now produce unexpected incidents like the flash crash of 2010 (Kirilenko et al., 2017) when humans become secondary to the machine and make decisions that come too late. Comparatively slow humans cannot act fast enough when metahuman HFT runs amok (MacKenzie, 2018). The fact that the US Securities and Exchange Commission (SEC) has mandated circuit breakers to stop runaway HFT suggests the potential for unintended consequences of such machine based behavior (Kim and Yang, 2004).

Metahuman system HFT is also becoming common and pervasive. Many large investment banks (e.g. Blackrock) no longer use human traders in several asset classes or markets (Thomas, 2017). The SEC estimated in 2013 that metahuman system HFT was used for more than 50% of all US stock trades (SEC Staff, 2014) and it has since grown to over 60% (Economist, 2019). The SEC has mandated large-scale monitoring and data collection to capture forensic information, new market making rules, real-time monitoring systems (e.g. MIDAS), and consolidated audit trails (e.g. CATS) (O’Hara, 2015). Metahuman HFT can do a great deal of damage before humans even know what is going on, and blaming human owners for prohibited trading practices makes little sense, if the prohibited behavior was executed by machines that do not “know” better due to lack of understanding of cultural setting. Tight coupling that come with electronic trading can precipitate “normal accidents” in adjacent metahuman system HFT (Perrow, 2010). For example, gas stations whose metahuman systems watch neighboring stations in real time and alter prices accordingly have been accused of collusion and price-fixing, (Schlechner, 2017). These systems can now be so complicated that consumers might need their own metahuman systems to cope (e.g. an app to avoid neighborhood fuel price fixing). Alternative monitoring and forensic investigation might be required to hold rule breakers accountable.

Manipulations of voter behavior illustrates the importance of scale in how these systems learn. In the 2016, US presidential election, social media manipulated voter attitudes in critical swing states (Valentino et al., 2017). Adjusting the algorithms or adding humans to cure problems created by machines might not solve this problem. Similarly, highly assistive (nearly autonomous) vehicles are an example of metahuman systems already deployed that redefine the relationship between human and machine. They illustrate the importance of scope. Drivers are expected to react correctly to unanticipated situations (Shapiro, 2016). To address this contemporary vehicles carry hundreds of millions of lines of code and 25%–30% of the value of a new car can be in software enabled functions (Charette, 2009). Collision avoidance aids have helped bad drivers get better, but they still work through drivers (Gage et al., 2015). Nearly autonomous vehicles that send data to other cars, raise questions of who is in control. Insurance companies entice drivers with less expensive insurance for continuous monitoring with, despite privacy implications (Ohlsson et al., 2015). Liability shifts from drivers to software designers, while rapid progress in sensors, data communications, and related software functions are met by lags in complements such as roads, signals, laws, training, licensing, insurance, and repair (King and Lyytinen, 2005). The significance and cost of social convention to drive on one side of the road was illustrated by Sweden’s right-hand drive switch (Högertrafikomläggningen) of 3 September 1967. Metahuman systems will affect such long-standing conventions. For example, in the transportation ecosystem Uber’s computer-assisted dispatch uses data originating from customer mobile phones to assign drivers and facilitate pickups (Chen et al., 2015), changing the convention of communication between passenger and dispatcher to communication between phone and dispatch, both metahuman systems.

Metahuman systems: toward an IS research agenda

The evolution of technology and organizing toward socio-technical thinking and metahuman systems

Machines that learn are rapidly penetrating work systems. It does not matter whether they are as intelligent as humans: they still provide an opportunity for design increasingly powerful metahuman systems. Rapid machine based accumulation, distribution and dissemination of knowledge will alter organizing and organizations for ever. The period of organizing where only humans learned will end. This section addresses how an IS research agenda might recognize and respond to this inevitable trend and technological juggernaut.

The IS field’s earlier focus on management IS will prevail. The field still focuses on the prerogatives and challenges of management and organizing in the context of using novel technologies. The best way to understand the new challenges metahuman systems pose for the IS field is by reviewing the evolving role of technology in the management of organizations and organizing of work over a long period. This helps put the recent change into a broader historical perspective and understand its significance in a context. Different scholars will frame the evolution differently, but for our purposes, we will organize the grand arc of human organizing into four epochal changes which were punctuated by shifts in the mechanisms and technologies of organizational learning.

The first shift took place about 10 millennia ago from hunter-gatherer to agrarian production (Day and Walter, 1989). This introduced change in organization due to changes in human mobility (hunter-gatherers are nomadic, farmers are settled), specialization (job skills differentiated), and governance (given the surplus of agrarian production and related needs for coordination settlements could now grow from tribes into empires). The outcome was an emerging state that became a testbed for organization, along with new agriculture practice and knowledge (such as predicting seasons), rudimentary manufacturing (tools), and trade. Other than a few examples of machines (the plow, possibly the wheel) played role in this shift.

The second epochal change, the industrial revolution that began in Europe in the 15th century in Italy and then spread, made machines and associated production factors (e.g. electricity) central to everyday human life. The speed of production, transportation, communication, and so on, enabled by machines changed radically in degree which became changes in kind. Records kept at that time demonstrate the importance of speed on scale and scope to improve production (Chandler and Hikino, 2009). Production governance dependent on scale and scope became essential to modern management (Hughes, 1994). Through affecting scale and scope managers complemented the invisible hand of the marketplace with the visible hand of direction (Chandler, 1993; Smith, 1937). Metahuman systems in management made machines as central to organizing as new forms of governance was in the shift from hunter-gatherer to agriculture. Machines that learn are central to metahuman systems that learn fast, thereby affecting scale and scope that requires rethinking of work systems design, as well as how to build and evolve them.

The third epochal change was synthetic, bringing together insights from centuries of governance, agriculture, trade, and industrialization into the functional view of executives as documented in Chester Barnard’s (1968 [1938]) classic The Functions of the Executive. He built on insights from Henri Fayol’s study of the French state (Parker and Ritson, 2005), scientific management (Taylor, 1911) and the sociology of organization (e.g., Max Weber’s studies of bureaucracy and organizing values such as the Protestant Ethic) (Kilcullen, 1996). This functionalist work formed a foundation for distinct management functions and functional separation and their related knowledge bases as still reflected in terms like “the marketing function,” or “the information systems function.” Metahuman systems involving information technology are likely to be affected and shaped by the IS function, reinforcing the centrality of IS research in metahuman systems. It should also draw IS researchers into shaping the design of future work systems. The functional view is stull the foundation of the IS research agenda below that concentrates on four managerial functions related to metahuman systems.

The fourth epochal change elevates the machine to a closer and more intimate status with humans in work systems. The socio-technical systems movement of the mid 20th century punctuated this trend when the British Government nationalized coal mining at the same time that fast, new long wall coal mining machines appeared. These machines brought about work system re-design needs as mining work had to refocus from getting coal out of the seam to getting coal out of the mine. To inform the re-design research took place in real coal mines and not in laboratories. The action research involved management and unions and mixed knowledge creation with real improvements in coal mining processes (Trist, 1981). Technologies and social systems need to be designed jointly and cover mutual relationships between people, technology, tasks, and organization. This perspective later fundamentally shaped the IS research goals and framing (Sarker et al., 2019). In the early wave of socio-technical research machines did not learn while teams of humans learned to help achieve system-level goals of efficiency and work satisfaction (Jaderberg et al., 2019; Sarker et al., 2019). Poorly designed sociotechnical systems with inadequate concern with mutual relationships were shown to fail and produce unintended or unwanted outcomes (such as alienation or vulnerability).

As computing machines moved in the 1960s and 1970s beyond performing simple arithmetic, sorting, and matching carried out by earlier punch-card systems to increasingly complex real time transaction recording where machine exhibit more autonomous and complex agency the shift called also for better learning among system users. In consequence, sociotechnical thinking had to change again (Alvesson and Sandberg, 2011; Ekbia and Nardi, 2014; Kallinikos, 2011; Seidel et al., 2018, 2019; Sutherland, 2008). Several extensions to understand the relationships between machines and people were put forward. These included Latour’s concept of Actor-Network Theory (ANT) (Leonardi, 2012) where machines join humans as “actants” to “perform” networks (Latour, 2005). Now machines can act autonomously (Parasuraman et al., 2000) as parts of “networks of humans and machines,” and ANT provides a “. . .powerful tool to regain the sense of heterogeneity, but is a ‘. . .bad tool for differentiating associations’ (Latour, 1996: 380). Because of this ANT still does not fully open to the possibility of what happens when machines learn by themselves and from other machines in association with humans. But it brings machines closer to the human level in the work system analysis. The relationship between human and machine is more nuanced but human agency is primary. As a result, humans and machines ‘imbricate’ one another (Leonardi, 2011). For example, different capability levels in machines that learn will bring about significantly different allocations of tasks and related work sequences. As these systems learn and adapt based on human action, the uses, goals and ways of using machines will also change.

Generally, the sociotechnical tradition embraces mutually enforcing, sustainable relations between people, technology, tasks, and organization but also expects causal effects to emerge from features of technology—which are mediated by specific structures and features of task, people and organization. In such causal analysis, humans and machines enter during work system design into relationships that need to be rendered analytically separable for causal analysis but which still remain ontologically intertwined. Markus and Rowe’s (2018) recent review of sociotechnical thinking in IS research identifies different notions of causality when machines of varying kinds are integrated into work systems with specific effects. Agency in networks will involve machines which have causal powers that differ from those that are subjected to clean human control. Therefore, sociotechnical theory and how it conceives the causal effects between the machines and humans must change. Machines that learn will have new kinds of emergent system level effects on work systems. For example, the concern for human autonomy, integrity and value in design needs to be contrasted to F.W. Taylor’s (1911) scientific management, which modeled humans as if they were machines, components of work systems. Now, if machines behave increasingly more like humans or beyond what humans can do cognitively should the principles of the work system design also change?

One important element in metahuman system design is that human learning makes humans versatile. When viewed as machines, humans can be seen to act like truly general purpose machines applicable to many, even so far unconceived, tasks (Helpman, 1998). Computer and communications technologies which underlie the capabilities of machines that learn can also be characterized as general purpose technologies (David, 1990). Machine based learning expands the general purpose character of such technologies making built machines closer to the human. In this sense, metahuman systems will create general purpose systems with new types of system level scale and scope economies that will shape the future of organizing.

Both effects are important in understanding the evolution of management thought so far and how they will play out in the future. Machines draw closer to humans as interpreted in the socio-technical and their variant ANT traditions. The origins of managerial thought suggest that when machines get better the joint capabilities with humans in metahuman systems will greatly improve organization’s ability to carry out a wider range of tasks in new ways and introduce new tasks. IS research into metahuman systems can be understood by applying a functional view into how to organize metahuman systems even though the growth of metahuman systems in organizations will shift their management away from the classic functional view.

A functional frame for is research involving metahuman systems

Four functions—delegation, monitoring, cultivating and reflecting—capture an initial IS research agenda involving metahuman systems with the objective of designing and integrating metahuman systems into work systems. As with most functional views of socio-technical systems, the agenda seeks to achieve a level of control that keeps the systems operational and helps achieve organization level goals that derive from new scale and scope economies. This type of control captures essentially the idea of management as it is now thought of (Yates, 1988) (p. xiv). Controlling metahuman systems will in the coming decade and beyond become part of management practice and essential element of work systems design. This control might accommodate levels of machine autonomy to acquire capabilities, but follow also human goals as part of cooperative effort (Lawrence, 2017), or in setting goals (Aleksander, 2017). At the same time, machines will use humans for goal-setting the way Google’s search engine uses human input to discover and present content deemed useful when measured by click-through-rates. With these new capabilities come new trade-offs. Narrow goals will constrain system level learning, while broader goals are likely to produce multiple unintended consequences (e.g. content manipulation, search results biased by stereotype). Controls that work for humans in a metahuman system will this not work for machines, and vice versa.

IS research into new IT capability, changing properties of decision making, and organizational effectiveness, will remain relevant for IT-based design and implementation generally, but are likely insufficient in guiding how to think of control of metahuman systems in future organizations. In future, managers are still likely to operate a visible hand that shakes the invisible hand of the market, but many hand-shaking tasks will be carried out by metahuman systems where the hand and the shake are based on machine-based learning (Chandler, 1993; Smith, 1937). The functionalist view will remain relevant in this new setting. Simon’s (1997 [1947]) classic Administrative Behavior anticipated new kinds of digital technologies that would affect all the functions, but he never questioned their value and role in organizing. This has also remained a core assumption in IS research: reap system-level rewards by exploiting technology use via effective adjustments of the social across the functions. Humans will set goals, refine plans, explain decisions, train other humans, and build or buy technology. It is human resistance and failure to learn that causes implementation failure. But machines that learn will make possible metahuman system where machines that learn operate with higher speed and autonomy, and this will affect the scale and scope of organizational learning ways of implementing systems. In consequence, organizing needs to be re-thought to ensure that technology and people remain re-aligned.

Four functional areas are suggested as key levers to manage metahuman systems (Table 1). The first two, delegating (letting machines that learn and humans do their job), and monitoring (anticipatory watching for consequences) relate to metahuman system operations. Delegating assumes agency, hierarchy, authority, and decision rights and responsibilities that are part of any metahuman system. Monitoring involves attention costs, appropriate levels of psychological monitoring to balance indifference versus micromanagement with adequate subordinate learning (Simon, 1997 [1947]). Both functional processes defined management in the 20th century (Holmstrom and Milgrom, 1991). The next two, cultivating (mindful enhancement of learning by setting up the right conditions) and reflecting (learning at the metahuman systems level) are new functions that relate to the autonomy and speed that characterizes all elements of metahuman systems. Cultivating is about system enhancement that touches both machine and human training, related management and organization. Reflecting captures the idea of double-loop learning that characterizes metahuman systems and allows critically evaluate their learning processes and outcomes (Argyris, 2004; Schön, 1983).

Table 1.

Research agenda to metahuman systems by function.

Function	Research questions	Theory base	Potential empirical sites	Design/experiment
Delegating	What types of governance processes are needed to map responsibility to humans, and how best to provide this?	Forms and sources of governance relevant to metahuman systems (Zheng et al., 2019)	Medical devices (Therac-25) Ride sharing companies (Uber, Lyft) Autonomous vehicle trials (Tesla, Waymo)	Experimental engineering Exploratory field studies Online gaming experiments
Delegating	What skills need to evolve to design and test responsibility-based processes?	Sociotechnical system design Business process models (Sergeeva et al., 2018)	Process re-engineering Study of effects and non-intended side effects Programming task evolution Data scientist career growth	New curricula and relevant evaluation Study of design curricula and effects
Monitoring	How should machine-based decisions operating at high speed scale and scope be monitored? How to achieve this sustainably?	(Real time) accounting/auditing High reliability organizing (Rozario and Vasarhelyi, 2018)	High frequency trading (flash crashes) Electric grid monitoring (failure and successes) Robotic surgery (Intuitive Surgical)	Studies of system scaling and interactions Simulation experiments of emergent system behaviors, Trading floor analyses and experiments
Monitoring	How should humans interact with machines augmented with new functions? How to achieve this sustainably?	HCI theory, gency theory involving groups and technology, Studies of digital transformation and digital ontologies (Cai et al., 2019)	Design of machines (e.g. Synopsys software), games (Ubisoft, Activision), autonomous vehicles (Tesla, Waymo, Uber), etc.	Interface theories and experiments Studies of work system change and behaviors
Cultivating	How best to achieve the right selection of training data for learning? How to achieve this sustainably?	Statistical learning theory (Wexler et al., 2019)	Bots (Microsoft, Amazon) Newsfeeds(Facebook, Twitter) Face recognition (United Kingdom, China, Las Vegas)	Adversarial networks (generative and otherwise)
Cultivating	What criteria are useful for evaluating learning regarding metahuman systems? How to affect learning?	Cognitive science Learning science Education (Lake et al., 2017)	Organizational learning in manufacturing and service industries	Cognitive modeling Simulation of metahuman systems Behavioral economic experiments
Reflecting	What processes help evaluate metahuman system progress? How to create such processes?	Human-Computer Interaction IT governance models (Buchwald et al., 2014)	Diffusion studies of past disruptive technologies: elevators, traffic signals, office automation.	Longitudinal analyses of technological change and evolution Metadesign experiments
Reflecting	What metahuman system features can be anticipated? How best to accomplish this?	STS theory Cybernetics Technology history and policy (Kallinikos et al., 2013)	Large projects (Manhattan, SAGE, space exploration, the Internet, etc.) Standardization efforts.	Technology-focused field studies Public policy interventions Regulatory actions

The suggested framework is similar to many early IS research frameworks that helped frame pertinent topics for a nascent research field (Mason and Mitroff, 1973). The framework is just one possible sensitizing approach—not the only approach. No one can predict accurately what will happen with metahuman systems or know how to best organize them. Much depends on framing and theorizing whereby IS researchers can learn to what differences metahuman systems make in different settings and how the management can learn to adjust machine and human relationships in ways that improve how all socio-technical elements fit together for established system goals (Markus and Rowe, 2018).

Delegating

Delegating grants authority and resources to agents to accomplish tasks, freeing resources for other work. In metahuman systems, agents can be machines, humans or hybrid agents. Delegation is about discovering what will happen if we shift the agency in the system. It is almost impossible to know ex ante how to configure the overall system if multiple possibilities are available for delegation (Colombo and Delmastro, 2004; Vickers, 1985). Delegation in metahuman systems integrates machines that learn into tasks formerly restricted to human learning. For example, delegation of driving control requires new kind of autonomy in traffic control, road systems with sensors, and changes in insurance systems (King and Lyytinen, 2005). Delegation to automated traders invites illegal or unpredictable trading (Scopino, 2015). Delegation of editorial decisions to newsfeeds can lead to the manipulation of voter behavior (Lazer et al., 2018).

The result of this delegation is new kinds of human/machine hybrids. Currently, our understanding of delegation advances through learning-by-doing and mostly in the field given the limitations of laboratory experiments. Delegation comes with new specific agency problems, and different policies can be used to evade agent responsibility in the event of failure (Bartling and Fischbacher, 2012). Research is needed on how humans can best delegate to machines, the parameters of delegation, undesired behaviors to be avoided, and desired behaviors to be hoped for. A good start might be to look at research into interactive algorithms for ways human/machine interaction can be improved (Goldin and Wegner, 2006; Wegner, 1998). Fine-grained individual interactions might be collected into to larger chunks of capability that allow higher-level delegation similar to the way sociologists use grounded interpersonal actions as a way of building higher level abstractions related to identity and organization (Abbott, 2016).

Machine delegation to humans is a nascent and important topic as machine capabilities continue to improve. Social media companies rely now on learning-by-doing to watch humans accept and place advertising, then later delegate this role to machines thereby for example generating election fairness concerns (Berghel, 2017; Lazer et al., 2018). Machines often do not understand what humans find important. Systems like NELL help reveal what works for the collective as a whole (including machines and humans), and enable machines to supervise humans (Pedro and Hruschka, 2012; Retelny et al., 2014). There are trade-offs in new forms of delegation. Legal constraints may prevent some forms of delegation, demanding responsibility from humans (Solaiman, 2017). New work quality issues will emerge: human deskilling can become a significant source of work dissatisfaction (Trist, 1981). Machine substitution for humans in pure profit-seeking can risk reputational or emotional loss. Machines cannot understand feelings that guide human systems (Hubbard, 2010). Observational studies will help assess how and when to shift agency (List and Pettit, 2011), and how roles are already shifting (Mortensen and Haas, 2018).

Monitoring

Agency requires monitoring (Jensen and Meckling, 1976), but monitoring mechanisms timed for humans will be too slow for machines. For example, HFT proceeds so quickly that monitoring needs to be performed by second level metahuman systems. Such monitoring is also social, as when algorithms act on changes in liquidity (Hendershott and Riordan, 2013). These systems are already driving research into how monitor and regulate market activity (Lenglet, 2011; Siering et al., 2017; Treleaven and Batrinca, 2017). Likewise, election monitoring systems can be audited to figure out what truly happened (Masterson, 2019). IS researchers have raised concerns for metahuman system surveillance for hire (Clarke, 2019). Monitoring autonomous driving systems is currently a complex topic and requires interdisciplinary research (Koopman and Wagner, 2017). For example, the complexity of handovers and takeovers in semiautonomous vehicles already taxes human cognitive load, and itself needs monitoring (Sibi et al., 2016).

During monitoring continuous auditing might help flag anomalous outputs (Kiesow et al., 2016) or invoke rule-based triggers inside metahuman processes (Rozario and Vasarhelyi, 2018; Singh et al., 2013). In institutionally regulated settings such as finance or transportation, authorities can and need to be notified of fraudulent behavior or emergent changes in goals (Abbasi et al., 2012). There is need for observational studies to show which architectures work best under different circumstances for monitoring goals. Election meddling is often a consequence of myopic architectural choice (Berghel, 2017) and can only be countered by taking a broader view. The recently introduced SEC’s forensic and real time monitors may lead to a better understanding of HFT (SEC staff, 2014) and better mechanisms.

Overall, IS research is needed to better understand the consequences of using different strategies of monitoring. Three approaches in particular deserve attention. Disclosive archeology help show what to look for—this is where humans run experiments on machines to determine how they work (Ananny, 2016; Barocas et al., 2013). However, algorithms like Facebook’s news feed are changing constantly, so the utility of disclosive archeology is limited. Transparent design involves stakeholders and users, and shows them explicitly what is driving the system (Introna, 2007; Lyons et al., 2017). But few organizations are willing to commit to such extensive transparency. Moreover, even with a goal of transparency, creating machines that divulge and explain their behavior in way humans can interpret is difficult. Data disclosure by the machine as it operates can provide some means for monitoring (Lenglet, 2019), but fast machines require fast monitoring to keep up: this in turn calls for more systems! In principle, monitoring computers can better report to humans as now seen in smart watches, phones, and clickstream analyzers. However, few companies disclose enough data to enable such monitoring of their own server-based capabilities.

Cultivating

The cultivation metaphor comes from agriculture (trimming, correct spacing, fertilizing and removal of insect pests and weeds). It is a stage in mentoring (Chao, 1997; Humberd and Rouse, 2016). Cultivating systems or technology has been discussed in the IS context for some time (Bergqvist and Dahlberg, 1999; Hanseth, 2010; Henfridsson and Bygstad, 2013; Markus and Benjamin, 1996). Systems are prone to evolve organically, so cultivation focuses attention on metahuman systems learning and knowledge allocation to different system components (Martinez, 2014). HFT, for example, has a long history of research that has focused no cultivation. Algorithmic traders interact and change the way to think about machine training (MacKenzie, 2019). Autonomous driving systems, in contrast, focus on space rather than time. In this regard cars can be trained to anticipate running animals, but we do not have models for all animals in any landscape. Kangaroos jump differently than deer (O’Rourke, 2017), and so a car may need to be trained differently to match the fauna in the environment. Election systems are complicated by serious social as well as technical challenges. In particular, social media has political impact (Kreiss and Mcgregor, 2018), and currently machine learning systems can be trained to recognize political ads (Qi et al., 2016). But controversies over political ads and how far we can go point to the challenges of cultivation. For example, can the AI discriminate between antisocial political ads and eusocial political ads? Should those who run the AI systems censor ads? There are no easy answers to these questions (Kreiss and Mcgregor, 2019; The Washington Post, 2019).

Organizations might not be ready now to cultivate their metahuman systems as suggested by multiple failures, examples of inherent bias, and systems running amok (Kulshrestha et al., 2017; Lambrecht and Tucker, 2019). However, work on these areas has proceeded and is receiving more attention. For bias, one can visualize causes and adjust the system accordingly (Wexler et al., 2019). As organizations acquire this capability, metahuman systems will come to address social processes with multiple viewpoints. An immediate need is to cultivate systems that serve human-defined and human-monitored purposes (Salimans et al., 2017), but eventually, machines might cultivate humans. Electronic tutoring might be an early exemplar of this (Wenger, 2014).

Reflecting

Reflective practice helps human experts to build novel knowledge in any professional domain (Schön, 1983). Reflection is also important to aesthetics and ethics of metahuman system design (Bostrom and Ord, 2006). For example, reflection about HFT should anticipate large scale changes in the financial industry (Gomber et al., 2018). Metahuman systems have now been accepted for increased liquidity and more efficient investing. But they also precipitate new approaches to regulation (Arner et al., 2016). Likewise, election systems are interwoven into media and other complex systems and call for reflection. Questioning algorithms that control news feeds and control advertising is far from affecting change. There are also disciplinary differences: sociology focuses on collective action (Coleman, 2017), while IS researchers concentrate on false news in social media (Aral and Eckles, 2019; Vosoughi et al., 2018). Autonomous vehicles may also pose a category error, attributing greater cognitive ability to robots than they warrant (Aleksander, 2017). Perhaps the road to full vehicle autonomy is very long though some say that benefits such as safety can be achieved without full automation (Bailey and Erickson, 2019). The argument for full automation calls for changes in economic systems to handle labor displacement (Srnicek and Williams, 2015). Other analyses of labor displacement are more nuanced, considering which jobs in which orders might be displaced (Brynjolfsson and McAfee, 2014). From the IS research perspective, reflection here is about any case of metahuman systems can lead to a far-ranging analysis of societal impacts and values.

Asimov’s (1942) famous laws for robots proved circumventable, perhaps inevitably, as they can be viewed as an early attempt to introduce reflection into automation studies. Clearly machines can harm humans because of poor design, careless programming, or malicious action. Machines that reflect on the behavior of other machines to detect problems seem far-off (Goertzel, 2012). We arguably don’t know how to build machines that understand and conform to human values (Etzioni and Etzioni, 2016). Thus far, grounding in human values requires reflection by humans. Reinforcement and reward engineering might be starting points for future research on how to design machines that reflect human values (Dewey, 2014). But the question of whether ethics for metahuman systems can be learned top-down, bottom-up, or both has not even been answered (Yampolskiy, 2016).

IS research has applied the idea of reflective practice to understand how system design works (cf. Córdoba, 2007; Mathiassen, 1998; Redmiles and Nakakoji, 2004; Stroulia and Goel, 1995). Reviews, consulting meetings, and coaching are but a few of the mechanisms used for such reflection. Reflection, however, has remained a uniquely human endeavor, and at present, machines do not engage in reflection. On the other hand, metahuman systems already outperform some human experts in law, finance, and medicine. But, as noted above, they have not demonstrated an ability to invent mechanisms for their own governance or to understand what they have learned, and consequently, change their parameters for delegation, monitoring, or cultivation.

IS researchers can do case studies of reflective practice for metahuman systems founded on action research of the sociotechnical systems tradition. However, many IS researchers within professional schools have seen such action research as methodologically flawed due to the within-system effects of the researcher. Moreover, there is concern about such research being supported by those who might have a particular goal related to the outcome (e.g. owners, executives, and workers). Conflicts of interest, confounds, cognitive dissonance and confirmation bias threaten action research. Metahuman systems might require IS researchers to move into the unfamiliar and get out of their comfort zone. Two recent studies are signposts in this regard: Zuboff’s (2019) look at Google and Microsoft and Google’s look at its own AI systems as used by doctors (Cai et al., 2019).

Conclusion

IS increasingly incorporate machines that learn. Together with humans, these machines that learn combine to form metahuman systems which exhibit sociotechnical systems of new ilk. On the one hand, they embody classic machine advantages of calculating, recording and transferring information in simple intellective tasks. On the other hand, they come with higher level cognitive skills that affect the speed of organizational learning, and shape the scale and scope economies of organizations in new ways. Metahuman systems afford IS research in four areas of work system organization: delegating, monitoring, cultivating, and reflecting. Differences in the ways humans and machines learn will affect how metahuman systems are organized.

This paper is a call to arms for IS research in the era of metahuman systems. It asks IS researchers to address issues of human goals and values in settings where metahuman systems evolve or are applied. Achieving benefits and avoiding problems will require better understanding of systems level learning, and how learning emanating from metahuman systems affects large ecosystems in manufacturing, agriculture, transportation, finance, medicine, and other fields. Design science researchers can explore and test alternative designs for metahuman systems similar to the sociotechnical systems tradition that confronted disruptive technology and was successfully adopted by IS research. IS researchers should join with practitioners and scholars from administration, policy, computing, law, and other fields relevant to understand, design and implement metahuman systems.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This material is based upon work supported by the National Science Foundation under Grants 1909803, 1717473, 1745463, 1442840, and 1422066.

ORCID iD

Jeffrey V Nickerson

Author biographies

Kalle Lyytinen is distinguished University professor and Iris S. Wolstein professor of Management Design at Case Western Reserve University, and a distinguished visiting professor at Aalto University, Finland. He is among the top five IS scholars in terms of his h-index (90); he has the highest network centrality among IS scholars. He was the LEO Award recipient (2013), AIS fellow (2004). He has published over 400 refereed articles and edited or written over 30 books, or special issues. He mainly conducts research on digital innovation, design work, requirements in large systems, and digital infrastructures.

Jeffrey V. Nickerson is the associate dean of research and professor in the School of Business at Stevens Institute of technology. His current research interests include peer production, remixing, and collective creativity.

John Leslie King is W.W. Bishop professor in the University of Michigan School of Information. His research concerns the relationship between technical and social change.

References

Abbasi

Albrecht

Vance

, et al. (2012) Metafraud: A meta-learning framework for detecting financial fraud. MIS Quarterly 36(4): 1293–1327.

Abbott

(2016) Processual Sociology. Chicago, IL: University of Chicago Press.

Ackoff

Emery

(2005) On Purposeful Systems: An Interdisciplinary Analysis of Individual And Social Behavior As a System of Purposeful Events. Piscataway, NJ: Transaction Publishers.

Aleksander

(2017) Partners of humans: A realistic assessment of the role of robots in the foreseeable future. Journal of Information Technology 32(1): 1–9.

Alvesson

Sandberg

(2011) Generating research questions through problematization. Academy of Management Review 36(2): 247–271.

Ananny

(2016) Toward an ethics of algorithms: Convening, observation, probability, and timeliness. Science, Technology & Human Values 41(1): 93–117.

Aral

Eckles

(2019) Protecting elections from social media manipulation. Science 365(6456): 858–861.

Argall

Chernova

Veloso

, et al. (2009) A survey of robot learning from demonstration. Robotics and Autonomous Systems 57(5): 469–483.

Argyris

(2004) Double-loop learning and implementable validity. In: Tsoukas

Mylonopoulos

(eds) Organizations as Knowledge Systems: Knowledge, Learning and Dynamic Capabilities. London: Palgrave Macmillan, pp. 29–45.

10.

Arner

Barberis

Buckey

(2016) FinTech, RegTech, and the reconceptualization of financial regulation. Northwestern Journal of International Law & Business 37: 371.

11.

Arnoldi

(2016) Computer algorithms, market manipulation and the institutionalization of high frequency trading. Theory, Culture & Society 33(1): 29–52.

12.

Asimov

(1942) Runaround. Astounding Science Fiction 29(1): 94–103.

13.

Bailey

Erickson

(2019) Selling AI: The case of fully autonomous vehicles. Issues in Science and Technology 35(3): 57–61.

14.

Barnard

(1968 [1938]) The Functions of the Executive. Cambridge, MA: Harvard University Press.

15.

Barocas

Hood

Ziewitz

(2013) Governing Algorithms: A Provocation Piece. Rochester, NY: Social Science Research Network.

16.

Bartling

Fischbacher

(2012) Shifting the blame: On delegation and responsibility. The Review of Economic Studies 79(1): 67–87.

17.

Berghel

(2017) Oh, what a tangled Web: Russian hacking, fake news, and the 2016 US presidential election. Computer 50(9): 87–91.

18.

Bergqvist

Dahlberg

(1999) Scalability through cultivation. Scandinavian Journal of Information Systems 11(1): 1.

19.

Blue

Andoh-Baidoo

(2010) Directive decision devices: Extending the reach of automation into the finance domain. Expert Systems with Applications 37(1): 45–54.

20.

(2017) Superintelligence. Dunod.

21.

Bostrom

Ord

(2006) The reversal test: Eliminating status quo bias in applied ethics. Ethics 116(4): 656–679.

22.

Bricaud

(2012) Reuse Methodology Manual: For System-on-a-Chip Designs. Berlin: Springer Science+Business Media.

23.

Brynjolfsson

McAfee

(2014) The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies. New York: W. W. Norton.

24.

Buchwald

Urbach

Ahlemann

(2014) Business value through controlled IT: Toward an integrated model of IT governance success and its impact. Journal of Information Technology 29(2): 128–147.

25.

Buciluǎ

Caruana

Niculescu-Mizil

(2006) Model compression. Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. Available at: https://dl.acm.org/citation.cfm?id=1150464 (accessed 16 April 2020).

26.

Cai

Winter

Steiner

, et al. (2019) Hello AI: Uncovering the onboarding needs of medical practitioners for human-AI collaborative decision-making. Proceedings of the ACM on Human-computer Interaction 3: 104.

27.

Carlson

Betteridge

Kisiel

, et al. (2010) Toward an architecture for never-ending language learning. In: Twenty-Fourth AAAI conference on artificial intelligence. Available at: https://www.aaai.org/ocs/index.php/AAAI/AAAI10/paper/viewPaper/1879 (accessed 16 April 2020).

28.

Cartea

Jaimungal

Kinzebulatov

(2016) Algorithmic trading with learning. International Journal of Theoretical and Applied Finance 19(04): 1650028.

29.

Chandler

(1993) The Visible Hand. Cambridge, MA: Harvard University Press.

30.

Chandler

Hikino

(2009) Scale and Scope: The Dynamics of Industrial Capitalism. Cambridge, MA: Harvard University Press.

31.

Chao

(1997) Mentoring phases and outcomes. Journal of Vocational Behavior 51(1): 15–28.

32.

Charette

(2009) This car runs on code. IEEE Spectrum 46(3): 3.

33.

Chen

Mislove

Wilson

(2015) Peeking beneath the hood of Uber. In: Proceedings of the 2015 Internet measurement conference, Tokyo, Japan, 28–30 October, pp. 495–508. New York: ACM.

34.

Clark

(2017) Embodied, situated, and distributed cognition. In: Bechtel

Graham

(eds) A Companion to Cognitive Science. Oxford: Blackwell, pp. 506–517.

35.

Clarke

(2019) Risks inherent in the digital surveillance economy: A research agenda. Journal of Information Technology 34(1): 59–80.

36.

Coleman

(2017) The mathematics of collective action. Abingdon: Routledge.

37.

Colombo

Delmastro

(2004) Delegation of authority in business organizations: An empirical test. The Journal of Industrial Economics 52(1): 53–80.

38.

Córdoba

J-R

(2007) Developing inclusion and critical reflection in information systems planning. Organization 14(6): 909–927.

39.

Davenport

Kirby

(2016) Just how smart are smart machines? MIT Sloan Management Review 57(3): 21.

40.

David

(1990) The dynamo and the computer: An historical perspective on the modern productivity paradox. The American Economic Review 80(2): 355–361.

41.

Day

Walter

J-L

(1989) Economic growth in the very long run: On the multiple-phase interaction of population, technology, and social infrastructure. In: Barnett

Geweke

Shell

(eds) Economic Complexity: Chaos, Sunspots, Bubbles, and Nonlinearity. Cambridge: Cambridge University Press, pp. 253–289.

42.

Dean

(2019) The deep learning revolution and its implications for computer architecture and chip design. Available at: http://arxiv.org/abs/1911.05289 (accessed 16 April 2020).

43.

DeLanda

(2019) A New Philosophy of Society: Assemblage Theory and Social Complexity. London: Bloomsbury Publishing.

44.

Dewey

(2014) Reinforcement learning and the reward engineering principle. In: AAAI spring symposium series. Available at: https://www.aaai.org/ocs/index.php/SSS/SSS14/paper/viewPaper/7704 (accessed 16 April 2020).

45.

Domingos

(2012) A few useful things to know about machine learning. Communications of the ACM 55(10): 78–87.

46.

Du Gay

(2005) The Values of Bureaucracy. Oxford: Oxford University Press.

47.

Ekbia

Nardi

(2014) Heteromation and its (dis) contents: The invisible division of labor between humans and machines. First Monday 19(6). Available at: http://firstmonday.org/article/view/5331/4090 (accessed 16 April 2020).

48.

Engeström

Lompscher

Rückriem

(2016) Putting Activity Theory to Work: Contributions from Developmental Work Research. Berlin: Lehmanns Media.

49.

Ericsson

Krampe

Tesch-Römer

(1993) The role of deliberate practice in the acquisition of expert performance. Psychological Review 100(3): 363.

50.

Etzioni

(2016) AI assisted ethics. Ethics and Information Technology 18(2): 149–156.

51.

Faraj

Pachidi

Sayegh

(2018) Working and organizing in the age of the learning algorithm. Information and Organization 28(1): 62–70.

52.

Frith

(2012) Mechanisms of social cognition. Annual Review of Psychology 63: 287–313.

53.

Gage

Bishop

Morris

(2015) The increasing importance of vehicle-based risk assessment for the vehicle insurance industry. Minnesota Journal of Law, Science & Technology 16: 771.

54.

Goertzel

(2012) Creating Internet Intelligence: Wild Computing, Distributed Digital Consciousness, and the Emerging Global Brain. Berlin: Springer Science+Business Media.

55.

Goldin

Wegner

(2006) Principles of interactive computation. In: Goldin

Smolka

Wegner

(eds) Interactive Computation: The New Paradigm. Berlin; Heidelberg: Springer, pp. 25–37.

56.

Gomber

Kauffman

Parker

, et al. (2018) On the fintech revolution: Interpreting the forces of innovation, disruption, and transformation in financial services. Journal of Management 35(1): 220–265.

57.

Green

Chen

(2019) The principles and limits of algorithm-in-the-loop decision making. Proceedings of the ACM on Human-Computer Interaction 3: 50.

58.

Hanseth

(2010) From systems and tools to networks and infrastructures-from design to cultivation: Towards a design theory of information infrastructures. In: Holmström

Wiberg

Lund

(eds) Industrial Informatics Design, Use and Innovation: Perspectives and Services. Hershey, PA: IGI Global, pp. 122–156.

59.

Heaven

(2019) Why deep-learning AIs are so easy to fool. Nature 574(7777): 163–166.

60.

Helpman

(1998) General Purpose Technologies and Economic Growth. Cambridge, MA: MIT Press.

61.

Hendershott

Riordan

(2013) Algorithmic trading and the market for liquidity. Journal of Financial and Quantitative Analysis 48(4): 1001–1024.

62.

Henfridsson

Bygstad

(2013) The generative mechanisms of digital infrastructure evolution. MIS Quarterly 37(3): 907–931.

63.

Hinton

Vinyals

Dean

(2015) Distilling the knowledge in a neural network. Available at: http://arxiv.org/abs/1503.02531 (accessed 16 April 2020).

64.

Hirschheim

(1985) Information systems epistemology: An historical perspective. In: Mumford

Hirschheim

Fitzgerald

, et al. (eds) Research Methods in Information Systems. Amsterdam: North-Holland, pp. 13–35.

65.

Holmstrom

Milgrom

(1991) Multitask principal-agent analyses: Incentive contracts, asset ownership, and job design. Journal of Law, Economics, & Organization 7: 24–52.

66.

Horvitz

(1999) Principles of mixed-initiative user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’99), Pittsburgh, PA, 15–20 May, pp. 159–166. New York: ACM.

67.

Hubbard

(2010) Do androids dream: Personhood and intelligent artifacts. Temple Law Review 83: 405.

68.

Hughes

(1994) Technological momentum. In: Smith

Marx

(eds) Does Technology Drive History. Cambridge, MA: MIT Press, pp. 101–114.

69.

Humberd

Rouse

(2016) Seeing you in me and me in you: Personal identification in the phases of mentoring relationships. Academy of Management Review 41(3): 435–455.

70.

Introna

(2007) Maintaining the reversibility of foldings: Making the ethics (politics) of information technology visible. Ethics and Information Technology 9(1): 11–25.

71.

Jaderberg

Czarnecki

Dunning

, et al. (2019) Human-level performance in 3D multiplayer games with population-based reinforcement learning. Science 364(6443): 859–865.

72.

Jensen

Meckling

(1976) Theory of the firm: Managerial behavior, agency costs and ownership structure. Journal of Financial Economics 3(4): 305–360.

73.

Johnson

Zhao

Hunsader

, et al. (2013) Abrupt rise of new machine ecology beyond human response time. Scientific Reports 3: 2627.

74.

Johnson-Roberson

Barto

Mehta

, et al. (2016) Driving in the matrix: Can virtual worlds replace human-generated annotations for real world tasks? Available at: http://arxiv.org/abs/1610.01983 (accessed 16 April 2020).

75.

Kallinikos

(2011) Bureaucracy Under Siege: On information, collaboration and networks. In: Clegg

Harris

Höpfl

(eds) Managing Modernity: Beyond Bureaucracy. Oxford: Oxford University Press, pp. 130–153.

76.

Kallinikos

Aaltonen

Marton

(2013) The ambivalent ontology of digital artifacts. MIS Quarterly 37(2): 357–370.

77.

Kearns

Nevmyvaka

(2013) Machine learning for market microstructure and high frequency trading. In: Easley

de Prado

O’Hara

(eds) High Frequency Trading—New Realities for Traders, Markets and Regulators. London: Risk Books, pp. 115–122.

78.

Kiesow

Schomaker

Thomas

(2016) Transferring continuous auditing to the digital age —The knowledge base after three decades of research. Available at: https://aisel.aisnet.org/ecis2016_rp/42/ (accessed 16 April 2020).

79.

Kilcullen

(1996) Max Weber: On bureaucracy. Available at: https://philpapers.org/rec/kilmwo (accessed 16 April 2020).

80.

Kim

Yang

(2004) What makes circuit breakers attractive to financial markets? A survey. Financial Markets, Institutions and Instruments 13(3): 109–146.

81.

King

Lyytinen

(2005) Automotive informatics: Information technology and enterprise transformation in the automobile industry. In: Dutton

Kahin

O’Callaghan

(eds) Transforming Enterprise: The Economic and Social Implications of Information Technology. Cambridge, MA: MIT Press, pp. 283–333.

82.

Kirilenko

Kyle

Samadi

, et al. (2017) The flash crash: High-frequency trading in an electronic market. The Journal of Finance 72(3): 967–998.

83.

Kling

(1980) Social analyses of computing: Theoretical perspectives in recent empirical research. ACM Computing Surveys (CSUR). Available at: https://dl.acm.org/citation.cfm?id=356806 (accessed 16 April 2020).

84.

Koopman

Wagner

(2017) Autonomous vehicle safety: An interdisciplinary challenge. IEEE Intelligent Transportation Systems Magazine 9(1): 90–96.

85.

Kreiss

Mcgregor

(2018) Technology firms shape political communication: The work of Microsoft, Facebook, Twitter, and Google with campaigns during the 2016 U.S. Presidential Cycle. Political Communication 35(2): 155–177.

86.

Kreiss

Mcgregor

(2019) The “arbiters of what our voters see”: Facebook and Google’s struggle with policy, process, and enforcement around political advertising. Political Communication 36: 499–522.

87.

Kulshrestha

Eslami

Messias

, et al. (2017) Quantifying search bias: Investigating sources of bias for political searches in social media. Available at: http://arxiv.org/abs/1704.01347 (accessed 16 April 2020).

88.

Kurzweil

(2000) The Age of Spiritual Machines: When Computers Exceed Human Intelligence. London: Penguin Books.

89.

Kyriakou

Nickerson

Sabnis

(2017) Knowledge reuse for customization: Metamodels in an open design community for 3D printing. MIS Quarterly 41(1): 315–332.

90.

Lake

Salakhutdinov

Tenenbaum

(2015) Human-level concept learning through probabilistic program induction. Science 350(6266): 1332–1338.

91.

Lake

Ullman

Tenenbaum

, et al. (2017) Building machines that learn and think like people. The Behavioral and Brain Sciences 40: e253.

92.

Lambrecht

Tucker

(2019) Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads. Management Science 65(7): 2966–2981.

93.

Latour

(1996) On actor-network theory: A few clarifications. Soziale Welt-Zeitschrift fï¿½r sozialwissenschaftliche Forschung Und Praxis 47(4). Nomos Verlagsgesellschaft mbH: 369–381.

94.

Latour

(2005) Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford: Oxford University Press.

95.

Lawrence

(2017) Living together: Mind and machine intelligence. Available at: http://arxiv.org/abs/1705.07996 (accessed 16 April 2020).

96.

Lazer

DMJ

Baum

Benkler

, et al. (2018) The science of fake news. Science 359(6380): 1094–1096.

97.

Lee

Kusbit

Metsky

, et al. (2015) Working with machines: The impact of algorithmic and data-driven management on human workers. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ’15), Seoul, South Korea, 18–23 April, pp. 1603–1612. New York: ACM.

98.

Lenglet

(2011) Conflicting codes and codings: How algorithmic trading is reshaping financial regulation. Theory, Culture & Society 28(6): 44–66.

99.

Lenglet

(2019) Algorithmic finance, its regulation, and Deleuzean Jurisprudence: A few remarks on a necessary paradigm shift. Topoi: An International Review of Philosophy. Epub ahead of print 16 May. DOI: 10.1007/s11245-019-09653-6.

100.

Leonardi

(2011) When flexible routines meet flexible technologies: Affordance, constraint, and the imbrication of human and material agencies. MIS Quarterly 35: 147–167.

101.

Leonardi

(2012) Materiality, sociomateriality, and socio-technical systems: What do these terms mean? How are they different? Do we need them. In: Leonardi

Nardi

Kallinikos

(eds) Materiality and Organizing: Social Interaction in a Technological World. Oxford: Oxford University Press, pp. 25–28.

102.

Lieto

Bhatt

Oltramari

, et al. (2018) The role of cognitive architectures in general artificial intelligence. Cognitive Systems Research 48: 1–3.

103.

List

Pettit

(2011) Group Agency: The Possibility, Design, and Status of Corporate Agents. Oxford: Oxford University Press.

104.

Lowe

Tamar

, et al. (2017) Multi-agent actor-critic for mixed cooperative-competitive environments. In: Guyon

Luxburg

Bengio

, et al. (eds) Advances in Neural Information Processing Systems 30. New York: Curran Associates, pp. 6379–6390.

105.

Lyons

Sadler

Koltai

, et al. (2017) Shaping trust through transparent design: Theoretical and experimental guidelines. In: Savage-Knepshield

Chen

(eds) Advances in Human Factors in Robots and Unmanned Systems. Cham: Springer, pp. 127–136.

106.

Lyytinen

Yoo

Boland

, et al. (2016) Digital product innovation within four classes of innovation networks. Information Systems Journal 26(1): 47–75.

107.

Mackenzie

(2017) Machine Learners: Archaeology of a Data Practice. Cambridge, MA: MIT Press.

108.

MacKenzie

(2018) Material signals: A historical sociology of high-frequency trading. The American Journal of Sociology 123(6): 1635–1683.

109.

MacKenzie

(2019) How algorithms interact: Goffman’s ‘interaction order’in automated trading. Theory, Culture & Society 36(2): 39–59.

110.

Marcus

Davis

(2019) Rebooting AI: Building Artificial Intelligence We Can Trust. New York: Knopf Doubleday Publishing Group.

111.

Markus

Benjamin

(1996) Change agentry—The next IS frontier. MIS Quarterly 20(4): 385–407.

112.

Markus

Rowe

(2018) Is IT changing the world? Conceptions of causality for information systems theorizing. MIS Quarterly 42(4): 1255–1280;.

113.

Martinez

(2014) Education as the Cultivation of Intelligence. Abingdon: Routledge.

114.

Mason

Mitroff

(1973) A program for research on management information systems. Management Science 19(5): 475–487.

115.

Masterson

(2019) Protecting election infrastructure: A view from the federal level. In: Brown

Hale

King

(eds) The Future of Election Administration: Cases and Conversations. Cham: Springer, pp. 55–60.

116.

Mathiassen

(1998) Reflective systems development. Scandinavian Journal of Information Systems 10(1): 12.

117.

Mattli

(2019) Darkness by Design: The Hidden Power in Global Capital Markets. Princeton, NJ: Princeton University Press.

118.

Mestre

(2006) Transfer of Learning from a Modern Multidisciplinary Perspective. Charlotte, NC: Information Age Publishing.

119.

Mingers

Mutch

Willcocks

(2013) Critical realism in information systems research. MIS Quarterly 37(3): 795–802.

120.

Mitchell

Cohen

Hruschka

, et al. (2018) Never-ending learning. Communications of the ACM 61(5): 103–115.

121.

Mnih

Kavukcuoglu

Silver

, et al. (2015) Human-level control through deep reinforcement learning. Nature 518(7540): 529–533.

122.

Möhlmann

Henfridsson

(2019) What people hate about being managed by algorithms, according to a study of Uber drivers. Harvard Business Review, 30 August. Available at: https://hbr.org/2019/08/what-people-hate-about-being-managed-by-algorithms-according-to-a-study-of-uber-drivers (accessed 1 December 2019).

123.

Mortensen

Haas

(2018) Perspective—Rethinking teams: From bounded membership to dynamic participation. Organization Science 29(2): 341–355.

124.

Moyer

(2016) How Google’s AlphaGo beat a Go world champion. Atlantic, 28 March. Available at: https://www.theatlantic.com/technology/archive/2016/03/the-invisible-opponent/475611/ (accessed 16 April 2020).

125.

Nair

Srinivasan

Blackwell

, et al. (2015) Massively parallel methods for deep reinforcement learning. Available at: http://arxiv.org/abs/1507.04296 (accessed 16 April 2020).

126.

O’Hara

(2015) High frequency market microstructure. Journal of Financial Economics 116(2): 257–270.

127.

Ohlsson

Händel

Han

, et al. (2015) Process innovation with disruptive technology in auto insurance: Lessons learned from a Smartphone-based insurance telematics initiative. In: vom Brocke

Schmiedel

(eds) BPM—Driving Innovation in a Digital World. Cham: Springer, pp. 85–101.

128.

Orlikowski

Scott

(2015) Exploring material-discursive practices. Journal of Management Studies 52(5): 697–705.

129.

O’Rourke

(2017) The Kangaroo conundrum. Risk Management: An International Journal 64(7): 40.

130.

Panait

Luke

(2005) Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-Agent Systems 11(3): 387–434.

131.

Parasuraman

Sheridan

Wickens

(2000) A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics: Part A, Systems and Humans 30(3): 286–297.

132.

Parker

Ritson

(2005) Revisiting Fayol: Anticipating contemporary management. British Journal of Management 16(3): 175–194.

133.

Pedro

SDS

Hruschka

(2012) Conversing learning: Active learning and active social interaction for human supervision in never-ending learning systems. In: Advances in Artificial Intelligence—IBERAMIA, Cartagena de Indias, Colombia, 13–16 November, pp. 231–240. Berlin; Heidelberg: Springer.

134.

Perrow

(2010) The meltdown was not an accident. In: Michael

Paul

(eds) Markets on Trial: The Economic Sociology of the U.S. Financial Crisis: Part A (Research in the Sociology of Organizations). Bingley: Emerald Group, pp. 309–330.

135.

Zhang

Sukul

, et al. (2016) Automated coding of political video ads for political science research. In: International symposium on multimedia (ISM), San Jose, CA, 11–13 December, pp. 7–13. New York: IEEE.

136.

Ram

Prasad

Khatri

, et al. (2018) Conversational AI: The science behind the Alexa prize. Available at: http://arxiv.org/abs/1801.03604 (accessed 16 April 2020).

137.

Redmiles

Nakakoji

(2004) Supporting reflective practitioners. In: Proceedings of the 26th international conference on software engineering, Edinburgh, 28 May, pp. 688–690. New York: IEEE.

138.

Retelny

Robaszkiewicz

, et al. (2014) Expert crowdsourcing with flash teams. In: Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST ’14), Honolulu, Hawaii, 5–8 October, pp. 75–85. New York: ACM Press.

139.

Rizzolatti

Fogassi

(2014) The mirror mechanism: Recent findings and perspectives. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 369(1644): 20130420.

140.

Robbins

(2010) GPS navigation. . . but what is it doing to us? In: International symposium on technology and society, Wollongong, NSW, Australia, 7–9 June, pp. 309–318. New York: IEEE.

141.

Rosen

(1991) Life Itself: A Comprehensive Inquiry into the Nature, Origin, and Fabrication of Life. New York: Columbia University Press.

142.

Rozario

Vasarhelyi

(2018) How robotic process automation is transforming accounting and auditing. The CPA Journal 88: 46–49.

143.

Rusu

Rabinowitz

Desjardins

, et al. (2016) Progressive neural networks. Available at: http://arxiv.org/abs/1606.04671 (accessed 16 April 2020).

144.

Salimans

Chen

, et al. (2017) Evolution strategies as a scalable alternative to reinforcement learning. Available at: http://arxiv.org/abs/1703.03864 (accessed 16 April 2020).

145.

Santana

Hotz

(2016) Learning a driving simulator. Available at: http://arxiv.org/abs/1608.01230 (accessed 16 April 2020).

146.

Sarker

Chatterjee

Xiao

, et al. (2019) The sociotechnical axis of cohesion for the IS discipline: Its historical legacy and its continued relevance. MIS Quarterly 43(3): 695–719.

147.

Schlechner

(2017) Why do gas station prices constantly change? Blame the algorithm. Wall Street Journal, 8 May. Available at: https://www.wsj.com/articles/why-do-gas-station-prices-constantly-change-blame-the-algorithm-1494262674 (accessed 16 April 2020).

148.

Schneider

(2018) Automating drug discovery. Nature Reviews. Drug Discovery 17(2): 97–113.

149.

Schön

(1983) The Reflective Practitioner: How Professionals Think in Action. New York: Basic Books.

150.

Scopino

(2015) Do automated trading systems dream of manipulating the price of futures contracts-policing markets for improper trading practices by algorithmic robots. Florida Law Review 67: 221.

151.

SEC Staff (2014) Equity market structure literature review Part II: High frequency trading. Available at: https://www.sec.gov/marketstructure/research/hft_lit_review_march_2014.pdf (accessed 16 April 2020).

152.

Seidel

Berente

Martinez

, et al. (2018) Autonomous tools in system design: Reflective practice in Ubisofts Ghost Recon Wildlands project. Computer 51(10): 16–23.

153.

Seidel

Berente

Lindberg

, et al. (2019) Autonomous tools and design: A triple-loop approach to human-machine learning. Communications of the ACM 62(1): 50–57.

154.

Sergeeva

Huysman

Faraj

(2018) Losing touch: How robots transform the practice of surgery. Academy of Management Proceedings 2018(1): 11429.

155.

Shapiro

(2016) Accelerating the race to autonomous cars. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, San Francisco, CA, 13–17 August, pp. 415–415. New York: ACM.

156.

Sibi

Ayaz

Kuhns

, et al. (2016) Monitoring driver cognitive load using functional near infrared spectroscopy in partially autonomous cars. In: Intelligent Vehicles Symposium (IV), Gothenburg, 19–22 June, pp. 419–425. New York: IEEE.

157.

Siering

Clapham

Engel

, et al. (2017) A taxonomy of financial market manipulations: Establishing trust and market integrity in the financialized economy through automated fraud detection. Journal of Information Technology 32(3): 251–269.

158.

Silver

Hubert

Schrittwieser

, et al. (2018) AlphaZero: Shedding new light on the grand games of chess, shogi and Go. Available at: https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go (accessed 16 April 2020).

159.

Simon

(1983) 2 - Why should machines learn? In: Michalski

Carbonell

Mitchell

(eds) Machine Learning. San Francisco, CA: Morgan Kaufmann, pp. 25–37.

160.

Simon

(1997 [1947]) Administrative Behavior (4th edn). New York: The Free Press.

161.

Singh

Best

Bojilov

, et al. (2013) Continuous auditing and continuous monitoring in ERP environments: Case studies of application implementations. Journal of Information Systems 28(1): 287–310.

162.

Sinz

Pitkow

Reimer

, et al. (2019) Engineering a less artificial intelligence. Neuron 103(6): 967–979.

163.

Smith

(1937) The wealth of nations (1776). Available at: https://mi01000971.schoolwires.net/cms/lib05/MI01000971/Centricity/Domain/440/Primary%20Source%20Articles%20Smith%20and%20Marx.pdf (accessed 16 April 2020).

164.

Smith

(2019) The Promise of Artificial Intelligence: Reckoning and Judgment. Cambridge, MA: MIT Press.

165.

Solaiman

(2017) Legal personality of robots, corporations, idols and chimpanzees: A quest for legitimacy. Artificial Intelligence and Law 25(2): 155–179.

166.

Srnicek

Williams

(2015) Inventing the Future: Postcapitalism and a World Without Work. New York: Verso Books.

167.

Stroulia

Goel

(1995) Functional representation and reasoning for reflective systems. Applied Artificial Intelligence: AAI 9(1): 101–124.

168.

King

Grudin

(2017) Staying alive: The IS field at the half century mark. In: Galliers

Stein

M-K

(eds) The Routledge Companion to Management Information Systems. Abingdon: Routledge, pp. 490–503.

169.

Sutherland

(2008) Directive decision devices: Reversing the locus of authority in human–computer associations. Technological Forecasting and Social Change 75(7): 1068–1089.

170.

Sutton

Barto

(2018) Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.

171.

Tan

Sun

Kong

, et al. (2018) A survey on deep transfer learning. In: Artificial neural networks and machine learning—ICANN, Rhodes, 4–7 October, pp. 270–279. Cham: Springer.

172.

Taylor

(1911) The Principles of Scientific Management. New York: Harper & Brothers.

173.

The Economist (2019) The stockmarket is now run by computers, algorithms and passive managers. The Economist. Available at: https://www.economist.com/briefing/2019/10/05/the-stockmarket-is-now-run-by-computers-algorithms-and-passive-managers (accessed 30 December 2019).

174.

The Washington Post (2019) Twitter’s new rules ban political ads from candidates but not ads about causes. The Washington Post, 15 November. Available at: https://www.washingtonpost.com/technology/2019/11/15/twitters-new-political-ads-rules-bans-them-candidates-not-about-causes/ (accessed 25 December 2019).

175.

Thomas

(2017) At BlackRock, machines are rising over managers to pick stocks. The New York Times. Available at: https://advtechconsultants.com/Machines%20are%20Rising%20Over%20Managers%20to%20Pick%20Stocks_%20NYTimes%20Mar%202017.pdf (accessed 16 April 2020).

176.

Thorndike

(1932) The Fundamentals of Learning. Teachers College Bureau of Publications. Available at: https://psycnet.apa.org/record/2006-04535-000 (accessed 16 April 2020).

177.

Tran

Hoffman

Saurous

, et al. (2017) Deep probabilistic programming. Available at: http://arxiv.org/abs/1701.03757 (accessed 16 April 2020).

178.

Treleaven

Batrinca

(2017) Algorithmic regulation: Automating financial compliance monitoring and regulation using AI and blockchain. Journal of Financial Transformation 45: 14–21.

179.

Trist

(1981) The evolution of socio-technical systems: A conceptual framework and an action research program. Occasional paper no. 2. Toronto, Ontario, Canada: Ontario Quality of Working Life Center.

180.

Valentino

King

Hill

(2017) Polling and prediction in the 2016 presidential election. Computer 50(5): 110–115.

181.

Venkatesh

Morris

Davis

, et al. (2003) User acceptance of information technology: Toward a unified view. MIS Quarterly 27(3): 425–478.

182.

Vickers

(1985) Delegation and the theory of the firm. The Economic Journal of Nepal 95(Suppl.): 138–147.

183.

Vinge

(1993) Technological singularity. In: VISION-21 symposium sponsored by NASA Lewis Research Center and the Ohio Aerospace Institute, Westlake, OH, 30–31 March, pp. 30–31. Washington, DC: NASA.

184.

Vosoughi

Roy

Aral

(2018) The spread of true and false news online. Science 359(6380): 1146–1151.

185.

Wang

Zhang

Zheng

, et al. (2016) Where does AlphaGo go: From Church-Turing thesis to AlphaGo thesis and beyond. IEEE/CAA Journal of Automatica Sinica 3(2): 113–120.

186.

Wegner

(1998) Interactive foundations of computing. Theoretical Computer Science 192(2): 315–351.

187.

Wenger

(2014) Artificial Intelligence and Tutoring Systems: Computational and Cognitive Approaches to the Communication of Knowledge. Morgan Kaufmann.

188.

Wexler

Pushkarna

Bolukbasi

, et al. (2019) The what-if tool: Interactive probing of machine learning models. IEEE Transactions on Visualization and Computer Graphics 26: 56–65.

189.

Winter

Berente

Howison

, et al. (2014) Beyond the organizational ‘container’: Conceptualizing 21st century sociotechnical work. Information and Organization 24(4): 250–269.

190.

Yampolskiy

(2016) Taxonomy of pathways to dangerous artificial intelligence. In: Workshops at the thirtieth AAAI conference on artificial intelligence. Available at: https://www.aaai.org/ocs/index.php/WS/AAAIW16/paper/viewPaper/12566 (accessed 16 April 2020).

191.

Yates

(1988) Creating Organizational Memory: Systematic Management and Internal Communication in Manufacturing Firms, 1880–1920. Cambridge, MA: Sloan School of Management, Massachusetts Institute of Technology. Available at: https://dspace.mit.edu/bitstream/handle/1721.1/47068/creatingorganiza00yate.pdf (accessed 16 April 2020).

192.

Yoo

Henfridsson

Lyytinen

(2010) Research commentary—The new organizing logic of digital innovation: An agenda for information systems research. Information Systems Research 21(4): 724–735.

193.

Zheng

Albano

Vora

, et al. (2019) The roles bots play in Wikipedia. Proceedings of the ACM: Human-Computer Interaction 3: 215.

194.

Zuboff

(2015) Big other: Surveillance capitalism and the prospects of an information civilization. Journal of Information Technology 30(1): 75–89.

195.

Zuboff

(2019) The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. London: Profile Books.