Artificial Intelligence’s new clothes? A system technology perspective

Definitions

There exist several definitions of AI. We start with OECD (2019) definition, as we find it the most encompassing: ‘an AI system is a machine-based system that is capable of influencing the environment by producing an output (predictions, recommendations, or decisions) for a given set of objectives. It uses machine and/or human-based data and inputs to (i) perceive real and/or virtual environments; (ii) abstract these perceptions into models through analysis in an automated manner (e.g. with machine learning), or manually; and (iii) use model inference to formulate options for outcomes. AI systems are designed to operate with varying levels of autonomy’. Note that the OECD definition is method-agnostic, meaning that it doesn’t prescribe how the AI system achieves the result. This allows encompassing various approaches to AI such as symbolic, cellular automata and connectionism. (Boden, 2016)

We take this definition and specify the methods that currently prevail in AI discourse, systems and solutions, resembling the notion of ‘modern AI’ introduced by Russell (2019). Our working definition of current AI is as follows: a variety of systems each consisting of virtual machine(s) (algorithm(s)) performing statistical learning and inference, using data of different modalities (e.g. visual and audio) and types (e.g. text, cross–section and panel) and relying on dedicated computing capacity. Taddy (2019) develops a similar definition to outline the technological dimensions of AI. Our definition is also in line with what Aleksander (2017, 2021) calls ‘smart computing’: a well-bounded (though evolving) category of algorithmic processes that yet require human design as well as interconnected supporting technologies. Current AI, as smart computing, is unrelated to intelligence at large as found in humans. Importantly, rather than a monolithic technology, current AI is a label that envelops a range of information technologies working according to a common principle: that of predicting output based on patterns inferred from input data. In sum, current AI systems are statistical engines with unprecedented capacity to identify and encode correlations in training data.

State of AI

Artificial intelligence, being a technological mirror of ourselves, is inevitably compared to natural intelligence. A seemingly philosophical question of whether or not AI possesses a ‘true’ intelligence has very tangible technical and socio–economic implications. For instance, cognition and understanding are among the criteria as well as fields of ongoing research that separate the so-called weak AI from strong AI. The distinction is based on the fact that the former only emulates intelligent behaviour, while the latter aims at re-creating it. Emulation of intelligence can be achieved through various methods; when a set of methods with common theoretical underpinnings gains traction, it forms an approach to AI, such as the symbolic and connectionist approaches mentioned earlier on. However, the question of how to re-create intelligence to reach emergence of cognition and understanding in algorithms remains yet unanswered. Hence, we can claim that the current state of AI belongs to the weak type.

For what concerns the technical side, to be considered strong, AI must exhibit understanding. In what follows, we dissect this concept within the natural language domain due to its inherent familiarity to any reader. Understanding, as found in humans, is the capability to infer communicated meaning given natural language expression (Bender and Koller, 2020). This definition of understanding clearly separates meaning (the former) and the form of language (the latter). In turn, Language Models (LM) (regardless of their architectural complexity) are systems that perform string prediction – that is, statistical prediction of a character, word, or phrase – given preceding and/or surrounding text (Bender et al., 2021). The combination of these two insights makes it evident that LM are trained solely on the linguistic form (vast collections of text) while meaning remains beyond their grasp. Progress in leveraging context has been made to capture increasingly longer sequences – from few preceding tokens (e.g. N-gram model) to paragraphs of text (e.g. Transformers) – with architectural innovations in LM as well as with increasing size of models and training datasets that allows the detection and encoding of finer and finer details of linguistic form. In other words, LMs are getting increasingly good at reproducing the form of language, but come nowhere near its meaning and, hence, to understanding. The limits of the statistical approach appear even more prominent when put in perspective: if capturing longer dependencies in text based on co–occurrences and distributions to emulate understanding (i.e. reproduce linguistic form) is a challenge, the construction of a robust and functioning representation of an even simple world using this approach is an intractable task. Instead, human-like understanding ‘is based on concepts – internal mental models of external categories, situations, and events, and of one’s own internal state and “self”. In humans, understanding language (as well as nonlinguistic information) requires having the concepts that language (or other informa-tion) describes, beyond the statistical properties of linguistic symbols’ (Mitchell and Krakauer, 2023).

There is no doubt that statistical learning and inference play some role in the process of achieving understanding. However, human understanding is by far more parsimonious and prudent: humans do not require such vast data and do not update the entire mental model of the world every time new data arrives; it seems to be more modular and relational. Apparently, new approaches and their combinations with existing ones are required to achieve genuine understanding by AI systems and, thus, strong AI. Perhaps, the conclusion that AI needs various approaches to modeling understanding and eventually intelligence to become stronger can be conveyed in the following quote of Marvin Minsky: ‘The power of intelligence stems from our vast diversity, not from any single, perfect principle’. (Minsky, 1988)

Given that we discuss the socio-economic side of AI at length in the remainder of the paper, here we limit ourselves to only few points that directly descend from the technical side. As weak AI inherits the same limits of the statistical methods it relies on, it poses challenges for development and creates risks of harm once deployed. One example of a challenge current AI faces is dealing with non-transitive alternatives, that is, dilemma, in various applications. One of the proposed solutions in such cases is of probabilistic nature: the usage of mixed strategies and/or fuzzy ranking. However, if an application is characterised by high-stake loss function, that is, the cost of mistake is high, probability-based solutions are dubious both legally and ethically. The Moral Machine experiment illustrates this challenge perfectly (Awad et al., 2018). In a nutshell, a large scale survey was conducted to elicit preferences of people across the globe with regard to variations of the trolley problem in the context of autonomous driving. Here, we want to clarify that the problem in question is not the incapability of AI to find the solution of a fundamental moral dilemma; instead, it is the potential to scale up a certain level of damage that is inherent to the probabilistic solution as the deployment scales up. For what concerns risks, current AI systems are sensitive to representativeness and quality of data (the ‘garbage in, garbage out’ principle) and (adversarial) attacks aimed at distorting or ‘polluting’ statistical co-occurrences in the data teaching the system to behave oddly and/or to produce harming output (Marjanovic et al., 2021; Shumailov et al., 2021). Even in presence of high-quality data and no malicious intervention, optimisation along a ‘flat’ target function might lead to a sub–optimal or detrimental outcomes (Lambrecht and Tucker, 2019). The identification of unintended harms and the monitoring of purposeful attacks in order to remedy the situation require (i) commitment on the side of AI–proprietors and (ii) high costs in terms of time, programming effort, computing power and energy, new tests, and even environmental toll (Bender et al., 2021; Strubell et al., 2019).

System components

Regardless of the methodological approach, any AI technology necessarily consists of the following components: (i) algorithms or virtual machines, (ii) computing power (and related physical devices delivering it), (iii) data, and (iv) domain structure as the problem environment and search space of actions an AI system is working with. This description builds on Taddy (2019) with some variations, and is in line with the OECD definition provided earlier, now stressing the components the AI system consists of.

Each domain is characterised by its own market structure, business models, regulations and pace of development. For instance, the semiconductor industry as the producer of hardware for (ii) is distinguished by well-defined but complex value chain (VC) with pockets of market power concentration and hence high entry barriers; it leverages clear technical opportunities (e.g. Dennard scaling principle) ² and economic mechanisms such as capacity races and economies of scale (Prytkova and Vannuccini, 2022).

The data domain is less defined and, therefore, less constrained (Koutroumpis et al., 2020a). Data markets are younger, less regulated, and by the virtue of intangibility more dynamic. In the increasingly digital economy, the importance of data as a commodity or production factor is growing so does the need to define data markets and understand their mechanisms. This is particularly visible in the context of and due to (i) privacy (Sveinsdottir et al., 2020) and (ii) competition in and for data markets (Jenny, 2021). For what concerns the latter, it is worth stating explicitly that data is frequently the source of supply–side network effects, spreading gains across related markets.

Last but not least, the software domain of AI resembles something in–between the hardware and data domains in terms of its market conjuncture. On the one hand, software markets existed before, so the mechanisms at work are well-studied; however, AI programs are not a usual download–install piece of software; at the moment, they require a whole stack of libraries and interfaces to be produced, integrated, and utilised. ³ On the other hand, this domain is still less monopolised and more open-sourced than the other two, even though this property might vanish as the presence of big firms induces smaller players to pursue the so-called ‘innovation-for-buyout’ strategy (namely, innovating just for the sake of signalling value and be acquired) (Cabral, 2018) to scale-up effectively.

Outlook

Concluding the entire Section, it is worth repeating our main claims: our object of study is current AI, which belongs to weak AI with rather statistical underpinnings above all other existing approaches. The system-ness of AI must be accounted for as a research priority because it has far reaching technical, social and economic implications, which we continue to unpack and discuss in the remainder of this paper.

Getting the data

First, in order to deploy current AI to support any given application, an established and systematic process of data collection is required. In other words, the implementation of AI requires a good representation of business processes (essential or not for a firm) in data – namely, their digitisation. This is why pioneering industries in AI adoption have been the likes of Fintech and logistics, which are characterised by highly digitised and measurable processes and had forms of algorithmic automation and optimisation already in place. The current AI systems expanded the set of ‘digestible’ data by improving capabilities of processing of sensory data such as images, video, audio, making activities that involve these capabilities cheaper (and thus economically viable), and less labour and time consuming. Thus, the current AI expanded the frontier of automation.

The existence of data does not automatically make the case for an AI application. Sometime data might exist but its accessibility could be either hindered, inefficient or even welfare-damaging. This is partially due to unresolved data ownership and absence of mechanisms such as data markets to coordinate data supply and demand which would ensure the lawful and effective exchange of data ownership rights. An insightful summary of the situation with data markets is expressed in a quote of Edward Snowden: ‘there is no property less protected and yet no property more private than data’ (Snowden, 2019). In some applications, data is a mere representation of an environment’s state or processes (e.g. temperature control in data centres). However, when data is an imprint of activities conducted by actors, individuals or organisations that are external to the owners of AI systems, then data might be considered as a property of the actors that created it (Jones and Tonetti, 2020). Said differently, when data is a public good, ownership issues do not emerge, while the elaboration of data, which has the nature of a private good, requires solutions that address simultaneously consensual data transfer and privacy concerns (personal data that owners might either sell at a very high price or not to sell at all).

In sum, the collection of data that reflects business processes including demand’s feedback loops and the establishment of data markets is a necessary, though not sufficient, prerequisite for AI deployment.

Choosing monetisation strategy

Second, to persist as a useful technology within an economic activity, data elaboration performed by AI has to bring returns. The value of data elaboration can lie in harnessing vast amounts and/or complexity of data detecting patterns in time infeasible for humans. Retrieving information about, for example, highly non-linear relations between a set of covariates and whether or not a person has clicked on an ad is undoubtedly a useful insight, but in order to systematically turn this information into profit a firm has to build a sustainable business model to monetize on it. Monetisation strategies can vary across applications, which in turn are characterised by different payoffs from the implementation of AI. For example, for online retail, the monetisation strategy would involve the structuring of pricing and versioning of the offer given the association revealed by data elaboration. This strategy allows obtaining profit directly and from each offer independently. Differently, an AI algorithm that controls an industrial robot through the processing of sensory data and producing an adequate response in order to perform a task creates value added that is more implicit and grows in a non-linear way with the scale of deployment of the AI technology.

In sum, all kinds of data elaboration done by AI has to produce either valuable/unique result in the firm’s production process or contribute to a valuable offer to the consumers, in both B2B and B2C markets, to ensure retention and generate profit.

Investing in complementary assets

Third, sustaining the monetisation strategy requires investments into complementary assets that constitute and/or support the AI system. The costs of primary collection or acquisition of data from third parties (e.g. the purchase of database licences, cookies or data appends – see Bergemann and Bonatti (2019)), the storage within a firm or purchasing cloud space in order to further elaborate the data with AI, or even contracting micro-work to conduct data annotation (Tubaro et al., 2020), constitute yet another part of the data–related decision-making. Depending on the revenue stream from the activity that involves AI, a firm has to choose between investments into the development of AI systems at least in part in–house (including all domains – data, hardware and software) or in partnership with AI solutions providers at various links of the AI value chain. The choice between the two alternatives shapes the distribution of market power among actors in the nascent AI industry. Obviously, small and medium-sized enterprises gravitate towards outsourcing option to minimise costs. Even big companies for which AI performs a side function would be prone to purchase customised but ready–made AI solution, benefiting from sharing the risks and legal responsibilities with the developer. Indeed, among AI–users the emergent strategy of ‘join-and-share’ AI-as-a-service solutions due to the high costs of every component of AI systems steers AI development towards a form of infrastructure, with the most powerful actors (AI–producers) meticulously building and piecing together the AI stack. Moreover, the burden of high costs is coupled with cross-domain network effects. For example, depending on the application, the nature of data might vary – pixel matrix for images, text corpus for legal disputes, or panel data for consumer databases. This affects the choices and developments in the hardware domain (bandwidth capacity, memory size and placement, parallel or sequential processing and so on), programming framework (programming language, libraries), and algorithms themselves (loss function, optimization procedure). Together, the initial costs of implementation and cross-domain network effects increase the switching costs of an alternative to any component and lead quickly to hard lock-ins for both supply and demand in the software and hardware domains.

In sum, AI adopters make a choice on how to deploy AI-based solutions and invest in the respective complementary assets. This creates a demand–pull effect steering the innovative efforts of AI–producers further along existing technological trajectories. The opportunity costs in this situation might be substantial, as alternative trajectories are locked out by prohibitively high switching costs.

Outlook

Now we have an understanding of the role the data domain plays in businesses that aim to use AI systems. An economic activity performed with AI (partially or entirely) immediately implies value creation tied to data through the several channels we described above. The degree of reliance on AI systems differentiates users in their preferred mode of AI consumption. Policies that aim to ensure a balanced and beneficial AI development have to account for the preferences of various user markets and, simultaneously, of AI producers. The task for policy-makers is to make sure that arguments related to high costs and strong network effects (both supply– and demand–side) are not used as a justification to tilt the development of AI systems towards an inefficient and/or monopolised realisation. Inefficient in technological sense might mean avoiding following the principle ‘the bigger the better’ in terms of ever-increasing size of data and algorithms, amount of compute, number of processors, and so on for the sake of marginal improvement in performance. In socio–economic terms, an inefficient instantia-tion would drain resources and resemble a skewed representation of stakeholders’ interests – AI–users, AI–producers, society, public institutions – creating dead-weight losses, violating rights, damaging competition, and producing an asymmetric distribution of gains. A variety of measures is available to policy–makers to address the challenges: from incentivising R&D investment in scalable AI techniques (e.g. federated learning and neural network compression) to promoting compatibility among already existing components of the system (e.g. consortium of AI software developers). The choice of measures to maintain an healthy competition and divert from hard lock-ins is crucial and should account for both (i) the mechanisms at work in AI component markets and (ii) the mechanisms that connect AI component markets; in other words, the system nature of AI is an eminent property that substantially influences the way AI should be governed.

In the sections to follow, we move from the specificity of the data domain and return to the entire AI system to assess how a framework used to map and study infrastructural technologies fares in capturing the essence of current AI.

System builders

System builders are the actors that strive to extend the reach of the system and perform the ‘sociotechnical integration’ necessary to its deployment (Van der Vleuten, 2009). These can be inventors–entrepreneurs or managers with engineering capabilities, individual actors or large firms. In different phases, system builders can work to align the interests and objectives of the different actors involved, allowing an LTS to grow and achieve its goal(s).

Reverse salients

Reverse salients ‘are components in the system that have fallen behind or are out of phase with the others. Because it suggests uneven and complex change, this metaphor is more appropriate for systems than the rigid visual concept of a bottleneck. Reverse salients are comparable to other concepts used in describing those components in an expanding system in need of attention, such as drag, limits to potential, emergent friction, and systemic efficiency’ (Hughes, 1987). Reverse salients, emerging from the uneven development of the system’s components, are sources of critical problems and, given that problems are typically focusing devices to allocate innovative efforts (Rosenberg, 1969), they are also potential loci of innovation.

Load factor

Load factor is ‘the ratio of average output to the maximum output during a specified period’ (Hughes, 1987) and, hence, an indicator of performance, here meant as use or deployment of the technology at full potential over time. The distribution of load factor indicates when and where the system is under stress. Knowing that can guide investments in capacity expansions or adjustments, as well as policy interventions.

Technological style

As for the common use of the word, style indicates a type of fashion: the specific design of a particular LTS that descends from choices regarding which features are emphasised, and in which way. An LTS technological style emerges from the particular choice and combination of its elements, given their relative importance and the specific role they play in the whole system. LTS executing the same function and aiming at the same goal can differ in style in different contexts. For example, the organisation and control structure of energy distribution systems can change across countries while the fundamental function and goal they pursue are comparable.

Momentum

Momentum, or dynamic inertia, is the degree of autonomy the LTS acquires once it reaches a certain stage of development and a ‘mass’ in terms of relevance for the economic system. Systems with high momentum are less sensitive to pressures for change – they continue their motion undisturbed.

The LTS elements we listed have an eminent socio-technical flavour. For example, the concept of system builder has mostly a social dimension, while reverse salient and load factor are aspects related to engineering and technology. Many of these concepts have closely related siblings in the other sub-fields of innovation studies. For example, reverse salients approximate bottlenecks; momentum approximates path dependence and cumulative change. However, their engineering or social angle makes them more sophisticated categories to label complex phenomena, and useful to capture the features of system technologies that are uniquely embedded in specific epistemic communities, regulatory settings, and cultural contexts. A system builder can be an entrepreneurial actor, but also a carrier of a rare combination of technical and social skills (and, potentially, power). Momentum is close to path dependence, but path dependence is a process that emerges from chance and choices, while momentum is a later-phase property of a system that keeps existing and functioning due to mass and acquired autonomy, thus refusing any role to chance.

In terms of the phase an LTS experiences from its birth to maturity, Hughes (1987) lists (i) invention, (ii) development, (iii) innovation, (iv) growth, competition and consolidation, and (v) technology transfer. The latter is characteristic of LTS: technology transfer occurs when a system developed in a given context is replicated in other environments, and can happen in parallel to other phases. More recent work added new phases experienced by mature LTS, such a stagnation, reconfiguration and decline (Sovacool et al., 2018). Gokalp (1992) stresses another key property of LTS: they develop by layering up over existing systems, creating a superposition of systems that influences the LTS configuration. The superposition of systems is characteristic of infrastructural projects and is, therefore, an essential feature to look for when assessing whether a technology can be considered an LTS.

In summary, the LTS categories can guide the analysis of a given system technology. For example, one might want to know: where is the ‘locus of control’ in the system? Which actors store and hold the relevant technological (and market) knowledge to ‘produce’ the system technology? Who advances and builds the system out of its components? Who has power on the factors constraining the development of the LTS? Which elements of the systems and related actors can facilitate the process of convergence around standards and protocols in order to improve communication and control at large? What happens if the LTS becomes so large to be unmanageable? Joerges (1988) quotes Aristotle, reminding us that when things get too small or too large ‘they either wholly lose their nature, or are spoiled’. A very timely point, when endless accounts of misuses, biases, discriminatory and malicious deployments suggest that we might be already spoiling AI.

Artificial Intelligence is large

LTS as a framework draws its specificity from the use of the attribute large. Following Joerges (1988) and Gokalp (1992), large can be considered in terms of territorial or user coverage, involving large-scale actors in the production of technology, or generating far– reaching socio-economic and/or environmental impacts. In this sense, large is used to label a technology that is encompassing, infrastructural, impactful, costly or global, or a combination of these properties. The attribute large is partially overlapping with that of pervasiveness, which is usually associated to General Purpose Technologies, and that we discuss in Comparing AI to other Epistemic Devices. However, largeness of LTS is a broader concept, more malleable and adaptive, and more apt to describe an infrastructural technology.

The way in which large is measured in the LTS framework allows for a degree of flexibility with respect to the dimensions employed to measure the largeness of current AI. Current AI spreads large in terms of user base and territorial coverage given the diffusion of its end-user applications such as recommendation systems, IoT devices (e.g. Amazon’s Alexa), or prompt– able interfaces. Stand-alone artefacts can have a large user base as well – think about classic rival private goods as a fridge or a hammer. However, in the case of AI, as in the case of the Internet, users ‘append’ themselves to AI solutions, in the same manner as they rely on other infrastructural services such as utilities (Steinhoff, 2018). AI is large also because it is developed and promoted by a large community of actors (developers and vendors), across its constituting domains, in complex technology stacks that feature both vertically integrated and specialised companies. The frontier of this large community is represented by large actors – large companies (the tech giants), national and supranational institutions, industrial consortia, global networks of Universities and organisations, as well as dedicated academic conferences. This creates a situation in which a large actor invests substantially into AI and provides access to it to a large user base for sharing. For example, this is the case of sharing computing facilities and storage via cloud, AI–powered software–based services (AI–as–a–service) such as visual recognition systems for airports, or API access to proprietary image and language models for developers. The economies of sharing (Shapiro et al., 1998) at work with AI make it similar to classic LTS, such as transport and energy supply systems. Finally, the societal traction of AI is large: ‘AI has seen itself elevated from an obscure domain of computer science into technological artefacts embedded within and scrutinised by governments, industry, and civil society’ (Mohamed et al., 2020); the AI debate has been popularised and is universal and touches all changes that AI might bring to contemporary societies, from its immediate effects on labour, inclusiveness, and exploitation, to more medium and long terms issues of sustainability and environmental toll, and even the value ‘alignment’ of AI systems.

In sum, AI is large according to various criteria identified by the LTS framework. This characteristic is extensively defined, encompassing and, hence, convenient for both the identification of AI as a system and its empirical analysis.

Artificial Intelligence is a technical system

The system nature of AI is the cornerstone to understand the working of the technology, and the rationale for us to propose the LTS framework to encapsulate it. The system–ness of AI is already suggested by the fact that the technology is shaped and evolves as a joint product of the three constituent domains we introduced in Our object of study: (i) the domain of AI algorithms and models that, in terms of actors involved and specific system builders engaged, is a subset of the software industry; (ii) the domain of computation, in practice constituting a subset of the hardware industry; and (iii) the domain of data generation, collection, storage, analysis, and transaction. As in a Venn diagram, at the intersection of these three domains one can find current AI. These three domains are large in their own right, along the criteria outlined in the previous paragraph. The functioning of the AI system relies on compatibilities and standards precisely as in the case of the Internet, railway networks, and energy distribution systems. Recent examples of how AI workloads are managed and services delivered globally are Microsoft’s Singularity service (Shukla et al., 2022), Meta’s AI Research SuperCluster, and Amazon’s AWS Lambda. Flexible AI workload delivery captures well the distributed, and technology–stack–embedded nature of AI.

Beyond the techno-economic interdependence among its constituting domains, the system–ness of AI arises from additional features of the technology. The first is the inner complexity of its components. The hardware and computation domain of AI is certainly complex. Consider, for instance, the decision-making involved in production of microchips capable to deliver on the increasing demand of AI compute: chipmakers need to resolve a complex technical trade-off among delivering processing speed, energy efficiency, and heterogeneous computing (Prytkova and Vannuccini, 2022). The algorithm domain is complex as well, with an emerging ecosystem of actors competing by developing proprietary AI solutions or, alternatively, relying on hubs that make available for re-use pre-trained models. As we have seen, the data domain is also complex. For example, the configuration of data marketplaces depends on whether the different actors (i.e. data buyers, sellers, exchanges and third–party service providers) are connected via arms’ length transaction or are merged in a handful of players (Koutroumpis et al., 2020a; Spiekermann, 2019; Srinivasan, 2019).

The second feature is that AI development clearly relies on a superposition of systems (Gokalp, 1992), a property that is characteristic of LTS. Similarly to infrastructural technologies as well as utilities (Steinhoff, 2018), current AI layers up on a broader technical stack of ICTs, such as telecommunication and Internet infrastructure. Within public and private organisations, elements of their information systems are upgraded by integrating AI while remaining integrated in existing organisational routines. For example, AI solutions integrate with existing database technologies, a trend already started decades ago (Brodie, 1989). Specific AI solutions might be seen as stand-alone products or services. However, their production requires the joint work of the software, hardware, and data domains, whose interdependence is made possible by the underlying digital and ICT infrastructure. The notion of superposition of systems is useful to illustrate the layered construction of the AI system: AI domains are complex systems in their own right, linked to produce the AI system technology which, in turn, is enabled by strata of system infrastructures.

The third feature relates to how AI solutions are implemented within actors’ existing processes. Using the term introduced by Arthur (2009), AI implementation is a form of ‘re-domaining’, which happens when a certain activities continues to be performed as usual, while the way in (the inputs through) which it is delivered changes. Re–domaining for AI means bringing certain activities under the umbrella of prediction technologies. This is achieved through capital deepening, that is, through the replacement of existing (digital) capital goods – software technologies – with more sophisticated (digital) capital goods – AI solutions. Capital deepening implies that, inside organisations, AI works as an ‘upgrade’ to ICT capital that enables the provision of better or new capabilities. Bresnahan (2021) elaborate on this idea, suggesting that AI solutions are implemented through system-level substitution. System-level substitution conveys the fact that entire modules of a business process (hence, a system enabling given functions) are replaced in block with different ones. In other words, substitution takes place mostly between production systems, rather than in production systems. For example, online retail replaces brick–and–mortar one, apps substitute interaction via websites, automated user support or algorithmic fraud check replace the computer-aided but human-controlled version, recommendation and search engines are substituted by chatbot–based text conversations. System level substitution naturally aligns with our arguments, as it illustrates that for what concerns AI, the system dimension permeates all levels of analysis.

Artificial Intelligence system builders

AI and its constituting domains are constructed by a variety of actors that actively initiate, support and shape developments of the system. The AI system builders are AI–producing and AI–using companies, dedicated regulatory bodies, industrial consortia, non-profit research organisations. Every system builder exerts efforts to influence the selection of their priorities and problems for the system to implement and address. This can be done by trying to weave in a particular manner the network of elements in the systems (an example are tech giants hiring AI pioneers and leading figures to lead their AI programmes) or by forcing the very system to converge on new standards, protocols, and shared practices. The latter can be achieved by making obsolete or ineffective the status quo through, for instance, forking decisions (Simcoe and Watson, 2019).

Consider AI–producers. They have the power to design the system and to decide which bridges between actors and subdomains to build or cut–off. AI–producing companies are an ecosystem of firms that conduct AI research, develop AI solutions, and participate in the AI value chain. ⁴ Among them, key system builders are the already mentioned tech giants, and established software and hardware companies, ⁵ vendors, startups, and platforms. ⁶ In terms of main line of business activity and industrial classification (NAICS), the majority falls under the codes ‘software publishers’ (49%) and ‘Computer Systems Design and Related Services’ (17%) ⁷ . Jacobides et al. (2021) offer a typology of AI actors along different modes of production and consumption. Next to the tech giants, the AI ecosystem features ‘AI creators’, that produce AI solutions but rely on some basic input from the giants, such as pre-trained models or programming frameworks; ‘AI integrators’, intermediaries that re–sell off–the–shelf AI solution adding some value in this step; ‘AI-powered operators’, that produce AI for their internal operations; and ‘AI takers’, that source AI solutions to be used in-house. This typology is useful to capture the ongoing process of specialisation in the industry – a signal of the ‘coming of age’ of the commercial side of AI. One example that illustrates how system builders exert their power is the ongoing issue revolving around handling harmful AI. In this context, current commercial system builders have supported the establishment of ethical boards and voluntary guidelines rather than regulation. While regulation would impose common and accountable rules on the development of the system, commitment–based solutions can be considered strategic concessions ⁸ to other stakeholders (in particular regulators, consumers of AI services, and the society at large). In practice, these initiatives represent a ‘seductive diversion’ that allow AI–producing companies to show engagement while retaining full power over the design of the system. Another leverage AI system builders acquire with their role is their ‘knowledge holding’ (Steinmueller, 2006). In the production of AI solutions, novel know–how is created and knowledge about it settles in the hands of AI–producers. Through this process, AI system builders become knowledge gatekeepers that can facilitate the diffusion of knowledge and expertise through co-invention activities as well as strategically withhold it. For example, machine learning platforms – open or closed source (Isdahl and Gundersen, 2019) – might improve access to and reproducibility of AI solutions, however at the cost of product development knowledge being held for a larger share by the platform owner. The mode of access to AI input is strongly linked to monetisation and value capture strategies: for instance, API access favours the appropriability of the AI engines behind specific commercial solutions and, thus, the concentration of knowledge holding upstream.

System builders in AI are changing over time, and their variety is increasing as the system gains complexity. AI companies superseded individual AI pioneers and universities’ computer science departments; at the same time, they are accompanied by governments and ‘lateral’ organisations. The latter are, for example, advocating to make the system more inclusive and less harmful (e.g. AI Now and DAIR research), pursuing technical advancements through non-profit or research organisations (e.g. Anthropic and Adept), facilitating coordination on principles and standards (e.g. the Partnership on AI), focusing on the existential risk linked to the possible emergence of strong AI (e.g. the Future of Humanity Institute), or acting as basic research labs for larger actors (e.g. OpenAI for Microsoft or DeepMind for Google), in a fashion resembling the Bell Labs (Gertner, 2012). Another type of system builders, much less empowered than the ones mentioned earlier on, are the (platform) workers that support the deployment of AI systems and that are subjects of processes of ‘heteromation’ (Tubaro et al., 2020). ⁹ These workers operate at the margins of AI and fill gaps in the working of the technology – they run the so-called ‘AI last mile’, labelling or moderating the data necessary for the training of algorithms, verifying their performance or even emulating the functioning of AI systems when these are not yet functional or cost-effective to use.

Artificial Intelligence reverse salients

As the system scales and becomes larger, tensions appear. These fault lines are the reverse salients of the system.

One recurring source of reverse salients in AI is the scarcity of resources in AI’s constituent domains: being a nascent industry, the provision of AI capabilities can be subject to shortages of input resources. The shortage is relative among domains, that is, the worst performing domain is a source of reverse salient, which can be of quantitative or qualitative kind, that is, respectively, delivering an insufficient amount of an input resource, or a qualitatively inadequate input. This might hold back or derail the evolution path(s) of AI. Quantitatively speaking, AI is data-hungry, but other resources whose demand grows faster than supply can become constraining factors as well: among them, AI talent, and management trained to lead AI-powered companies. Qualitatively speaking, computation is a reverse salient because the delivery of compute might not be tailor-made to the needs of AI algorithms (Hooker, 2020; Prytkova and Vannuccini, 2022). Atop of the purely technical challenge, there is also a techno–economic tension: the competition between cloud and edge modes of organisation of the computing infrastructure. These two modes of delivery of compute have different degrees of appeal to different system builders. Actors already specialised in value capture from devices might bet on AI built around edge features and constraints; actors that already invested in cloud business models could push for AI development tailored to cloud–based service provision. The relative economic success of one or the other model will feed back on the direction of progress in AI. More specifically, the cloud mode entails allocation of resources with priority on coverage and speed of access networks, and on computing capabilities of cloud–providers. The edge mode emphasises connectedness (which is not the same as coverage), compatibility, and computing capabilities of edge devices. The resolution of this qualitative reverse salient from the computation domain will shape AI system with regard to the organisation of computation: inside computing devices (chips) among its components, as well as inside the industry among producers and users.

Another source of qualitative reverse salients is the absence of compatibility and interoperability within and among domains. One example is the lack of a well-designed and regulated architecture for data troves. AI systems can be trained on (i) public dataverses and open data, (ii) AI–producers’ proprietary data, or (iii) data supplied by data marketplaces. These alternatives carry with them different implications in terms of reverse salients. In the case of (i), the reverse salient of data supply entails setting up clear rules for fair access to (and profiting from) data commons. For example, if to the exploitation of publicly available data by a private actor (i.e. a company training image generation AI models) corresponds the release of AI models to developers and final users either open source or via API, AI developments might accelerate on the algorithm domain, following a non-linear circuit that is characteristic of a complex, system technology. In the case of (ii), the limiting factor to AI development is the siloing of the data domain and the emergence of an oligopolistic market structure that can give rise to collusive behaviours, concentration of benefits, and even slowdown of AI developments if the data assets are hoarded and made un–accessible. In the case of (iii), the brake to AI progress can be related to the strategically extractive behaviour of data intermediaries.

Other examples of compatibility–based reverse salients can be found in the domain of AI algorithms. One lies in the proliferation of AI models, as more models equal multiple lines of research, duplication of efforts, less pooling of resources, and possibly coordination failures in the direction of technical advances. The reverse salient can be addressed in different ways, many of which are bottom–up initiatives, such as the organisation of contests ¹⁰ , or the launch of open-source initiatives to assist the coherence of the community and the development of cross–compatibilities, the establishment of standardised libraries and programming frameworks, and more fundamental theoretical and technological advances (Ben-David et al., 2019; Geirhos et al., 2020; Marcus, 2020). This reverse salient creates a disequilibrium but also an opportunity for entry in the AI value chain. For example, model hubs such as the platform Hugging Face ¹¹ favour experimentation and evolvability, lower the cost of coordination, and induce convergence towards a smaller set of underlying model architectures. This is precisely what seems to be happening in the domain of large language/foundation models, with the Transformer architecture becoming the established underlying structure of many widely used models (Amatriain, 2023).

The discussion around reverse salients in the data and algorithm domains makes a good case for the idea of treating AI as a system technology. An even stronger case is made by stressing the cross-domain (hence, systemic) influence of reverse salients. This was already implicit in the discussion so far; we now discuss it explicitly. For instance, despite the stubborn focus of the industry on ‘the bigger the better’ principle behind the launch of ever–larger neural networks, performance improvement is function of both model scale and data size. Hoffmann et al. (2022) find that, for Transformer–based large language models, the focus on model scaling (increase in parameters) shifts away attention from the required amount of training data. The authors were able to outperform some of the largest models at the time (e.g. Gopher or GPT–3) with a compute–optimal, much smaller model (dubbed Chinchilla) trained on a data set four times larger than those used by the competitors. Hence, improvements in one AI domains (algorithms performance) are bounded by the data reverse salient.

Similarly, reverse salients originated in the domain of algorithms have implications for the hardware domain: ad hoc AI algorithms appeal to smaller demand and have short-lived returns, quickly becoming obsolete. At the same time, the design and production of hardware that caters the needs of an ad hoc AI solution has high sunk costs. Therefore, the resolution of reverse salients in the algorithm and hardware domains is entangled, and both remain in a turbulent state until a dominant design emerges in either of the domains. Though connected, the development of the two domains remains driven by rather separate criteria, treating each other as a source of constraint for its own performance rather than a tandem partner with a common goal. The current AI shock shed light on this issue, exposing the shortcomings of such divided approach; several studies call for an expansion of performance criteria into various joint efficiencies, beyond separately accuracy in the software domain and processing speed in the hardware domain (Chen et al., 2019; Hooker, 2020; Prytkova and Vannuccini, 2022; Strubell et al., 2019).

The resolution of cross–domain reverse salients will produce positive feedback and virtuous cycles of AI progress: for example, the already mentioned foundation models (Bommasani et al., 2021) based on the Transformer architecture seem to be an area of convergence across AI communities – and, thus, as a potential dominant design – given their performance in treating multimodal data such as text and images combined, so much that some research groups envisage the emergence of ‘generalist agents’ grounded on multi-modality (Reed et al., 2022). The more these models area adopted and developed, the less binding the algorithm–related reverse salient. As a result, this increases the incentive for researchers and companies to compile multimodal dataset, which addresses in part the data related reverse salient, and open room for the advent of new AI applications, such as generative AI (Radford et al., 2021; Ramesh et al., 2021) or the embedding in physical technologies such as robots (Ahn et al., 2022; Driess et al., 2023).

Artificial Intelligence load factor and momentum

A system evolves by solving tensions created by reverse salients, producing what Sahal (1985) defines learning by scaling. Learning occurs by accumulating understanding on which changes (innovations) to implement, and how to redistribute the load in the system while it scales up. This means that mapping the load factor in the system can guide directed interventions to address salients and release resources to accelerate the system’s evolution, as well as monetisation. For example, AI companies can design third–party access to their models in order to maximise value extraction from the load distribution. One way to do that is to offer price differentiated services, with free access with queuing, and paid access for premium performance. Nightingale et al. (2003) suggest the notion of ‘economies of system’ to explain the gains a system can enjoy by redistributing activities according to the load factor, dynamically balancing the stress. ¹² Economies of system in AI would occur by rearranging the structural dependencies among its elements when some of them develop unevenly or are overloaded. The shift to federated learning architectures (Li et al., 2020) represents a system re-arrangement towards a design potentially capable of addressing the computation–related reverse salient: this would be done by distributing workload over the edge components of the system rather than leaving few giant actors to route (and control) finite computing power in the cloud. Scaling up an infrastructural technology is not a trivial task. For the case of the Internet (the LTS that in our opinion resembles current AI), Monteiro (1998) described the ‘sociotechnical negotiations’ required to revise the Internet Protocol IP as the whole information infrastructure scaled. Similar sociotechnical negotiations take place between AI system builders, as their roles and powers to shape the system change together with the nature and application of AI itself.

Concerning momentum, the growing number of actors jumping on the bandwagon of AI successes, as well as the grandiose media coverage of AI advances and the expectations of further ubiquitous diffusion of AI, build up a considerable one. However, expectations can work in both positive and negative direction. On the one hand, they channel large investments in public and private AI R&D and venture. On the other hand, the expectations of a large and ubiquitous impact of AI risk remaining unfulfilled: sustained commercialisation and growing competition among system builders make them race against each other, undertaking myopic steps in AI development leading to short-term payoffs. Stagnating diversity of AI research is among the early signs of such dynamics (Klinger et al., 2020). When the expectations that a new AI winter might be at the horizon will start to be considered plausible and the AI hype will begin slowing down, the momentum of the system might follow a similar path.

Current phase

The ‘invention’ phase of AI is a contested territory, as the very understanding of what AI is shifts over time; this is why in Our object of study we drew the boundaries of our unit of analysis to current AI. However, following the impressive results in the ImageNet visual recognition competition in 2012 and the subsequent media interest in AI – mostly due to the shadows AI seemed to cast on the future of work – the AI system was launched into the phases of intensive invention and development. From the perspective of compute alone, Sevilla et al. (2022) distinguish three phases: the pre Deep Learning era, the Deep Learning era, and the Large-Scale era. From the overall LTS perspective, we can place AI in-between the phases of innovation and growth, competition and consolidation, with commercialisation accelerating its pace and increasing technology transfer from academia to business, including a sizeable talent drain of professors and graduate students (Zhang et al., 2021). The process of growth by expanding to novel application fields generates continuous feedback into the phases of innovation and development. Technology transfer has also accelerated with the increasing efforts of national, supranational and sub-national institutions to govern AI developments as the technology has acquired geo-strategic significance. ¹³ An interesting question is whether AI experiences a process of so-called ‘technological convergence’. Technological convergence, a concept introduced by Rosenberg (1963), is a form of ‘upstreaming’, a process occurring when an activity embedded within diverse sectors or/and tasks exhibits some common features and principles that eventually mature and unbundle into a fully-fledged sector on its own. ¹⁴ We see signs of this process at work in the evolution of the AI: AI–producing companies – key system builders – emerge as specialised suppliers of AI–as–a–service tools, business automation services (e.g. recruitment), scientific discovery tasks (‘science–as–a–service’) and data analytics tasks. Traces of technological convergence signal that we are entering the consolidation phase of the AI system: the system reached the critical size that induces vertical specialisation along the supply chain. The latest stage of this process is the development of interfaces to facilitate the direct interaction of final users with AI solutions. This is the natural consequence of the emerging prevalence of language models amongst AI applications, as interactions based on natural language are key to the seamless integration of AI into ‘the fabric of everyday life’. The discussion of the phases of the AI system is important as it allows to place the technology in context, along an evolutionary trajectory. Moreover, as we already mentioned, LTS phases can be used as a predictive device to foresight possible scenarios. Sovacool and co-authors (2018) point out how LTS can experience later–stage phases of stagnation, reconfiguration, and decline. In these phases, some system builders might decide to adopt extractive strategies, sacrifice product quality for the sake of immediate returns, and concentrate further the control over the key input factors. Hence, anticipating the possible outcomes of upcoming phases of the LTS might inform the actions of policy makers and other system builders that prioritise the rebalancing and relaunching of the system.

Control mode

A challenge for any LTS is the possibility that diseconomies of scale appear during the growth phase, leading the system to lose internal coherence and to face ‘crises of control’ (Beniger, 1986). Under control we understand mechanisms of coordination on progression and management of the deployed applications. These mechanisms can feature the participation of different actors, and among these policy makers can play a fundamental, facilitating role. The issue of control arises when these mechanisms are non-existent or do not function efficiently. Nightingale et al. (2003) study and provide examples of innovations in control technologies as tools to retain control of LTS as they scale up. The issue of control does not appear as prominent in AI as in LTS such as railway or telecommunication networks. This is due to the heterogeneity of AI system’s components, which is therefore less coherent as an ensemble, but also due to the fact that AI is yet a ‘young’ infrastructure. At the moment, from the control mode perspective, AI is closer to the Internet than to integrated transport systems. As with the Internet, AI displays a mix of centralised and decentralised mechanisms of control and a layering up of commercial and non-commercial areas of development (Greenstein, 2020). Coordination among the system builders is achieved through the convergence on standards and interfaces, which is a non-frictionless process. In AI, there are not yet fully agreed standards for progression, nor an essential need for the centralised maintenance of coherence among the system’s parts; also, the actors do not have to be explicitly aware of the infrastructural nature of AI in order to conduct their operations. Therefore, the minimum requirements to make the AI system work and to avoid the system falling apart are lower than for other infrastructural technologies, while the social impact is potentially larger for AI. This does not mean that AI evolution will continue to follow the same loosely coordinated path: as the AI system consolidates and technological convergence takes place, control will become an increasing relevant challenge.

In the current phase, the system already gained significant momentum, but ‘diseconomies of scale’ have not occurred and internal coherence has not been upset due to the compartmental structure of AI. Instead of the ‘death’ of the AI system, the failure of control mechanisms could steer AI towards compartmentalisation, proprietary integration of layers of the technological stack, and detrimental turns at crucial points of its evolution path.

Technological style

AI will display different technological styles in different environments. This is evident when considering national (and supra–national) implementations of AI systems. The ‘division of labour’ and the direction of AI development and deployment depend in part on the structure of AI– and (as discussed in Unpacking the AI data domain) data value chains, but is also strongly affected by government strategies, resulting in rather distinctive styles.

Let’s consider the role of the latter in giving AI its style. Technological style can emerge as a result of the particular policy levers and priorities the regulator decides to pursue. This is reflected in public budget allocations, which can channel funds to AI through the university system, the military sector ¹⁵ , or directly to private actors – for example, in form of financial support to AI startups. Beyond the sheer amount of expenditures and the broad direction imposed on the system’s evolution, different technological styles for AI can emerge as a result of the specific tools of technology policy used (Steinmueller, 2010). Here, top–down command and control policy actions share the stage with more bottom–up governance initiatives. Horizontal interventions, such as the design of regulatory frameworks against AI harms, misuses and biases, are part of a specific style. The creation of new dedicated institutions (Calo, 2014) and intermediate bodies to facilitate coordination in the system is another, potentially complementary, option. Policy designs such as the proposal to establish a National Artificial Intelligence Research Resource (NAIRR) in the US, namely, a cloud infrastructure to pool and share computing power for AI compute workloads, is a further direction. ¹⁶ Examples of policies that can influence technological style include the efforts made by governments to attract, retain and develop AI talent through the visa regime ¹⁷ , or the alignment of macro policy levers (e.g. immigration and trade policy) with AI–related strategic priorities. A relevant case of the latter option are export controls policies targeting the semiconductor industry, as this is the producer of key components for AI–tailored hardware and its productive capabilities are a fundamental strategic asset. The use of policy levers in strategic technologies such as AI is not a novelty: the just cited semiconductor industry has been subject of trade policy interventions to shield domestic companies against emerging competitors (Langlois and Steinmueller, 2000). This point highlights the tight link existing between the actions leading to the development of a technological style and the competition between institutional actors at the international level. AI becomes one of the territories over which geopolitical forces compete, so much that some authors discuss whether an AI ‘arms race’ is ongoing (Asaro, 2019). In particular, Ding and Dafoe (2021) highlight how the ‘infrastructure–strategic logic’ underlying the identification and provision of strategic assets is relevant to frame AI because AI solutions are used as a platform to upgrade other important technologies or services.

Goal orientation

We can exploit the LTS framework to assess whether AI as a system technology has an identifiable goal orientation. LTS are goal–oriented systems; this means that all components coordinate – easily or at the cost of frictions, negotiations and forced adjustments – to achieve an overarching aim. This characteristic is easy to detect in LTS such as transport systems or water distribution networks (the goals being, respectively, mobility and water supply), while it is less evident for system technologies such as telecommunications, the Internet, or AI. One reason for that is these are highly heterogeneous technology systems, less prone to have centralised control being exerted on their applications and market conditions. We posit that AI as a system has a characteristic goal–orientation, even if latent yet. The goal of the AI is a ‘cybernetic’ one: to re–domain the ‘fabric of control’ of socio–technical systems based on human decisions into an automated one, starting with the transformation of existing activities into instances of adaptive prediction. In other words, the end game of current AI deployment is not the reproduction of cognition in vitro, as discussed in Our object of study. Rather, it is the expansion of the reach of autonomous system to every aspect of social life. This idea goes in line with recent explorations in the field of information systems, where the emergence of ‘metahuman systems’ as ‘new, emergent, sociotechnical systems where machines that learn join human learning and create original systemic capabilities’ (Lyytinen et al., 2021).

If we consider the necessary step to complete any given task as described by Agrawal et al. (2019), namely the elaboration of input data, judgement, and decision-making, and action and feedback, the AI system must be able to permeate all of them and perform them in a closed loop in order to pursue its cybernetic goal. This requires capital deepening, as we discussed earlier on, and depends on coordination and advances in all AI domains. An example of task controlled by AI along all stages is the industrial control system of cooling facilities in Google’s data centres that went completely autonomous in 2018. ¹⁸

In summary, to use the terminology of Flueckiger (1995), the goal of AI as a system is to shift further the balance from economies based on operations of transformation to economies based on operations of control – and to automate these. Whether this is a desirable outcome in terms of societal welfare as well as human progress is a normative question – one that requires democratic and inclusive discussion and evaluation.

Outlook

The study of AI we conducted through the LTS lenses is useful in at least three ways. First, it allows to take snapshot of the forces at work that are shaping AI as we write. This exercise can be repeated in the future, and further studies can go more in depth in specific aspects of the AI system, while maintaining a networked view on the interlocking nature of AI domains. For instance, it is possible to focus on those system builders advancing open source business models in the algorithm domain to single out the competitive challenges they introduce in the system, and the imbalances they can create in the other domains.

Second, our analysis provides a tool to identify emerging dynamics that descend from the fundamental mechanisms we outlined. For example, cross–domain constraints might relax or tighten-up, but will not disappear as they are essential features of the system. Consider the following scenario: what if a few system builders integrate their activities across all domains, relaxing cross–domain reverse salients but siloing AI developments and shaping them to mirror the priorities set by their business models and monetisation strategies?

Third, the bird’s eye view analysis we introduced can help detecting signals on possible future scenarios. Again, the progressive consolidation of the industry and the co–existence of vertical specialisation along the AI value chain with many specialised companies together with a full-stack integration as a strategic move by the largest actors suggests that the AI system might move towards a bi-modal structure. This will be characterised by a handful of dominant actors shaping the rules of the game across all domains of AI, and a parallel universe of small actors capturing (less) value by delivering specific AI solutions. The ‘language turn’ of AI sets the stage for an increasing focus on the interfaces between algorithms and final users. This might hide in the background the dynamics happening in the other components, giving leeway to the most powerful system builders to craft the system while the presence of AI becomes an accepted fact, an essential facility indistinguishable from the fabric of everyday life.