Technology transfer from procurement relationships in big science contexts

Abstract

Big Science is a fertile learning environment for the transfer of knowledge and technology to suppliers since it boosts economic performance, innovation and reputation. However, despite the growing importance of this topic in the literature, empirical evidence is still limited. This paper investigates technology transfer from Big Science through procurement relationships related to the need to develop front-end technologies for Big Science research infrastructures. The results show that the benefits generated for companies include intangible dimensions, such as the acquisition of technical knowledge and improved reputation. In addition, this procurement relationship leads to organizational changes such as the introduction of new specific organizational units and improved investment in R&D. In turn companies are able enter new markets thanks to the newly acquired competencies.

Keywords

Big Science public procurement technology transfer Italy

Introduction

Collaboration of both universities and public research centres with industry has gained attention from scholars and policy makers (Cheng et al., 2020; OECD, 2014; Perkmann and Walsh, 2009; Perkmann et al., 2011; Scandura, 2016; Schaeffer et al., 2020; Steimno and Rasmussen, 2018; Tseng et al., 2020). Studies in this field mainly focus on applied research (for example, He et al., 2021) and university–industry collaborations (Bruneel et al., 2010; Freitas et al., 2013; Isaeva et al., 2021; Lee 2000; Mascarenhas et al., 2018; Siegel et al., 2003), while interactions between Big Science centres (BSCs) which carry out curiosity-driven research and suppliers have received less attention (Castelnovo and Dal Molin, 2021; Li-Yin et al., 2022; Rådberg and Löfsten, 2022; Scarrà and Piccaluga, 2020).

As highlighted by several scholars (Autio, 2014; Autio et al., 1996; Autio et al., 2004; Bastianin et al., 2022; Calvert and Martin, 2001; Czarnitzki and Thorwarth, 2012; Florio, 2021; Li-Yin et al., 2022; OECD, 2014, 2001; Rådberg and Löfsten, 2022; Scarrà and Piccaluga, 2020), BSCs present very specific characteristics since they often need to design and build large-scale and capital-intensive research infrastructures (RIs) (e.g., the Large Hadron Collider, LHC and the Einstein Telescope, ET) that require completely new and challenging technologies developed in collaboration with suppliers through different kinds of contracts and collaboration.

Of the various kinds of collaborations that BSCs establish with suppliers (e.g., contract research, joint research, training, conferences and the creation of physical infrastructures) (D’Este and Patel, 2007), procurement relationships act as a powerful tool to support suppliers’ innovation through the exchange of technology and tacit knowledge (Autio et al., 2004; Bastianin et al., 2022; Bastianin and Del Bo, 2021; Castelnovo and Dal Molin, 2021; Castelnovo et al., 2018; Dal Molin and Previtali, 2019; Florio et al., 2018; Ghisetti, 2017). Such procurement favors technological spillovers and innovation impacts from BSCs to suppliers (Castelnovo and Dal Molin, 2021; Dal Molin and Previtali, 2019; Castelnovo et al., 2018; Autio et al., 2004). BSCs are thus “learning environments'” (Autio et al., 2003) with different characteristics from university contexts, in which such large-scale and capital-intensive RIs are rare (Florio, 2021).

The extant literature on the impact of Big Science public procurement focuses mainly on the Conseil Européen pour la Recherche Nucléaire (CERN) (Rådberg and Löfsten, 2022), typically investigated through a quantitative methodology such as a cost–benefit analysis. Less attention has been paid to the specific characteristics of the procurement relationship and how such relations impact on suppliers.

This paper thus studies: i) the characteristics of the procurement relationships between BSCs and suppliers, and ii) the impacts on suppliers as a result of such activities. To achieve these research objectives, we focus on three RIs related to three experiments (CUORE, Virgo and AMS) performed by the Italian National Institute for Nuclear Physics (Istituto Italiano di Fisica Nucleare, INFN), the largest Italian research centre in the field of physics. We use multiple case studies (Yin, 1993, 1994), examined mainly through direct interviews with key informants from suppliers, triangulated with the analysis of official documents (e.g., INFN technology transfer reports and procurement information on the order placed by the INFN). The case studies consist of 13 companies that have collaborated or that still collaborate with INFN for the development and upgrading of INFN RIs.

The paper is organised as follows. The second section, below, reviews the literature on both technology transfer and the impact of Big Science in order to highlight important concepts that have emerged. The third section describes the framework of analysis, and the research methodology and setting are presented in the fourth section. The results are described and discussed, respectively, in the penultimate section and final sections.

Literature review

In this section we summarize the literature on the collaboration of Big Science with suppliers in order to identify the impacts to be investigated in the empirical analysis.

Most studies on collaboration between science-based institutions and suppliers are based on university–industry interactions (for example, Al-Tabbaa and Ankrah, 2016; He et al., 2021; Isaeva et al., 2021; Lee, 2000, 1996; Mascarenhas et al., 2018; Rajalo and Vadi, 2017; Scandura, 2016). It is also widely analyzed in the literature on knowledge and technology transfer, which recognizes the paramount importance of transferring knowledge and technology beyond the academic and research boundaries to suppliers (Bozeman, 2000; Gerbin and Drnovsek, 2016; Piva and Rossi-Lamastra, 2013).

However, interactions between suppliers and BSCs are often neglected and, to the best of our knowledge, the processes of collaboration between BSCs and suppliers have been understudied. According to different authors, the context of BSCs is different from that of universities (Bastianin et al., 2022; Li-Yin et al., 2022; Rådberg and Löfsten, 2022; Scarrà and Piccaluga, 2020). As was first highlighted by Weimberg (1961), Big Science is a particular kind of research, mainly funded by national governments and international agencies, and characterized by the use of large-scale RIs that develop cutting-edge technologies (e.g., the Large Hadron Collider, LHC, at CERN). BSCs, which tend to perform curiosity-driven research, need to design and develop large RIs in order to achieve their scientific mission and support the advancement of scientific knowledge. However, as pointed out by Florio et al. (2018), it would be inappropriate to consider the impact of BSCs only in terms of their scientific dimension. In fact, these RIs develop large, capital-intensive and cutting-edge technologies that are often the result of collaborative procurement relationships between them and suppliers, involving technological spillovers and knowledge diffusion (Autio et al., 2003; Bastianin et al., 2022; Bastianin and Del Bo, 2021; Castelnovo and Dal Molin, 2021; Castelnovo et al., 2018; Dal Molin and Previtali, 2019; Florio et al., 2018).

There is thus a difference between technology transfer processes in Big Science and universities which makes BSCs a unique learning environment for suppliers because of the need to develop new technologies to conduct innovative experiments.

These assumptions are supported by empirical evidence, most of which relates to CERN. At the beginning of the 2000s, studies conducted by Autio et al. (2003, 2004) started to provide evidence on the ability of Big Science to impact suppliers. Autio et al. (2003) first defined a framework to explain how BSCs represent learning environments for their suppliers. They adopted a grounded theory method with in-depth case studies on CERN, focusing on the technological learning achieved by suppliers. They examined the relationship between CERN and firms as a dyad and found that firms’ absorptive capacity was related to the social capital built into the dyad. A survey was then submitted to CERN suppliers and CERN personnel involved in procurement for the LHC project (Autio et al., 2004) to identify and understand the possible benefits for suppliers in terms of technological innovation. With regard to outcomes of the relationship, they highlighted the frequency of interactions between firms and CERN personnel and the social capital built into the relationship. In addition, they found that the social capital depends on the frequency of the interaction, the importance of the technology used in the project, the number of technical employees of the firm involved, and CERN’s switching costs. The study provided evidence of positive returns for firms. The most frequent benefits appeared to be the development of new products, international exposure, technological and market learning, and improved economic performance. Another study (Autio et al., 2005) confirmed the importance of the social capital built into the relationship and highlighted that suppliers investing more time and resources – building more “intense” relationships – had gained more benefits. Similarly, suppliers that already had relationships with CERN had achieved better results.

Florio et al. (2018) analyzed the performance of CERN suppliers and conducted a survey to investigate improvements in performance due to their procurement relationship. They found positive effects related to market penetration and innovation outcomes and highlighted that firms involved in a cooperative relationship with CERN had obtained better results than firms with less collaborative relationships. In another study Castelnovo et al. (2018) explored the economic impact on CERN’s suppliers, looking at financial data, and provided evidence of positive effects on revenues, productivity, profits, R&D and innovation outcomes, especially in the case of high-tech suppliers.

To sum up, the empirical evidence confirms that BSCs do generate benefits for their suppliers through their public procurement activities. However, from the existing studies, two main research gaps emerge. First, prior studies have explored the relationships between BSCs and suppliers by focusing mainly on the specific context of CERN (Autio et al., 2003, 2004, 2005; Castelnovo et al., 2018; Florio et al., 2018) and, to the best of our knowledge, there is very limited evidence of the impact generated by other BSCs. Second, extant studies mainly focus on the benefits generated by Big Science public procurement, without examining the specific characteristics of the procurement relationship between Big Science researchers and suppliers.

Starting from these two gaps, this study investigates the characteristics of the procurement relationships and the related impact on suppliers, also verifying and generalizing the findings of prior studies in a previously unexplored context: the Italian National Institute for Nuclear Physics (INFN).

Framework of analysis: the dimensions investigated

To explore the characteristics of the procurement relationship between BSCs and suppliers we adapted various dimensions from the literature, looking mainly at a framework used to study interorganizational relationships between research and industry (Bonaccorsi and Piccaluga, 1994), other studies on university–industry relationships (Lee, 2000) and specific cases of BSCs and suppliers (Autio et al., 2003, 2005; Castelnovo and Dal Molin, 2021; Dal Molin and Previtali, 2019). We adopted the following dimensions:

1. duration of the relationship, operationalized as “episodic” (i.e., sporadic contact that occurs only when the order is in place) or “systematic” (i.e., recurring and cyclical, not only when a specific order is in place) (Bonaccorsi and Piccaluga, 1994; Dal Molin and Previtali, 2019);

2. type of relationship, divided into two main categories: “regular procurement” (i.e., purchase of product on the shelf) and “public procurement for innovation (PPI)” (i.e., the development of new products, systems or technologies; the modification of existing off-the-shelf products and/or development, or co-development of new products) (Castelnovo and Dal Molin, 2021; Dal Molin and Previtali, 2019);

3. frequency of contacts, operationalized as “daily”, “once a week”, “more than once a week”, “once a month”, “more than once a month” (Autio et al., 2003); and

4. Channels of collaboration, “formal” (formal agreements) or “informal” (email, phone calls, personal meetings) (Autio et al., 2003; Link et al., 2007; Perkmann and Walsh, 2009; Perkmann et al., 2011; Rybnicek and Königsgruber, 2019).

As reported in the literature, the relationships between BSCs and suppliers generate different kinds of impacts for suppliers. We identified the following impact dimensions (operationalized, in turn, in specific sub-dimensions):

1. Organizational and commercial: “introduction of new business units”, “entry in new market”, “acquisition of new clients”, “development of new partnerships” (Autio et al., 2003; Castelnovo and Dal Molin, 2021; Dal Molin and Previtali, 2019; Florio et al., 2016; Puliga et al., 2019);

2. Innovation impacts: “R&D benefits” (e.g., increased investment in R&D activities and improved R&D processes), “new patents” and “development of new products” (Alexander et al., 2020; Autio et al., 2003; Caloghirou et al., 2021; Castelnovo and Dal Molin, 2021; Dal Molin and Previtali, 2019; Florio et al., 2016; Hobbs et al., 2017; Lee, 2000; Puliga et al., 2019);

3. Learning impacts: technological and technical learning (e.g., “acquisition of new technical skills”), organizational learning (e.g., “acquisition of new capabilities” in managing the production and design processes), market learning (e.g., “acquisition of new market knowledge”) (Autio et al., 2003; Florio et al., 2016; Lee, 2000; Martin and Tang, 2007; Salter and Martin, 2001);

4. Reputational impacts: companies’ perceptions of the improved reputation due to the collaboration (Castelnovo and Dal Molin, 2021; Dal Molin and Previtali, 2019; Puliga et al., 2019; Autio et al., 2003); and

5. Economic impacts, operationalized as improved economic performance (Florio et al., 2016; Coccia and Rolfo, 2008; Autio et al., 2003; Mansfield, 1980, 1981, 1991, 1998). (Table 1)

Table 1.

Framework of analysis.

Investigated dimensions	Dimensions’ operationalization (where present)	References
Characteristics of the relationships
1. Duration of the relationship	Episodic	Dal Molin and Previtali, 2019; Bonaccorsi and Piccaluga, 1994
	Systematic
2. Type of relationship (PPI or regular procurement)	Regular procurement	Castelnovo and Dal Molin, 2021; Dal Molin and Previtali, 2019
	PPI
3. Frequency of contacts	Daily	Autio et al., 2003; the operationalization is defined by the authors
	More than once a week
	Once a week
	More than once a month
	Once a month
4. Channels of collaboration	Formal	Rybnicek and Königsgruber, 2019; Perkmann et al., 2011; Link et al., 2007; Perkmann and Walsh, 2009; Autio et al., 2003
	Informal
Impact(s) of the relationship on suppliers
1. Organizational and commercial impacts	Introduction of new business units	Castelnovo and Dal Molin, 2021; Dal Molin and Previtali, 2019; Puliga et al., 2019; Autio et al., 2003
	Entry in new market
	Acquisition of new clients
	Development of new partnerships
2. Innovation impacts	R&D benefits	Caloghirou et al., 2021; Castelnovo and Dal Molin, 2021; Dal Molin and Previtali, 2019; Puliga et al., 2019; Hobbs et al., 2017; Autio et al., 2003; Lee, 2000
	New patents
	Development new products
3. Learning impacts	Technological and technical learning	Martin and Tang, 2006; Autio et al., 2003; Salter and Martin, 2001; Lee, 2000
	Organizational learning
	Market learning
4. Reputational impact	Improved reputation	Castelnovo and Dal Molin, 2021; Dal Molin and Previtali, 2019; Puliga et al., 2019; Autio et al., 2003
5. Economic impact	Improved economic performance	Coccia and Rolfo, 2008; Autio et al., 2003; Mansfield, 1991.

Research design and methodology

Research design and data collection process

To achieve the study objective, we adopted a qualitative research strategy based on case studies (Eisenhardt, 1989; Tellis, 1997). This approach is particularly useful for responding to “how” and “why” research questions (Yin, 1993, 1994).

The case study approach also enables a complex phenomenon to be studied (i.e., the collaboration between BSCs and suppliers) in its natural environment, thus gaining a holistic view (Meyer, 2001; Silverman, 2000). The research design consists of four steps.

Step 1

Selection of INFN experiments. INFN experiments were selected according to three criteria. The first was the experiment duration, in view of the complexity of some of the new technologies involved. To include in the analysis multi-annual collaborations, we selected experiments that had lasted for at least 10 years. Second, although almost all INFN experiments require new technologies, we focused on experiments that needed the most technologically demanding research infrastructure, pushing the technological capabilities of suppliers. In the selection process, we were supported by INFN researchers and by the Head of the Technology Transfer Committee. The third criterion concerned the participating institutions and the main infrastructures involved. We focused on experiments conducted within INFN laboratories and that involved only INFN researchers in order to identify the contribution to the technological advancements of the individual institutions. A preliminary list of potential experiments was thus identified thanks to the support of the Head of the INFN Technology Transfer Committee. In a second stage, details of these experiments were recorded. Information collected included the scientific objectives, the phase of construction and operations as well as the technology involved in each research infrastructure. In accordance with these criteria, three experiments were selected (Cuore, Virgo, Ams).

Step 2

Data collection of procurement contracts of the three experiments. In this phase, we collected information about the orders (i.e., the procurement contracts with supplier companies) related to the three selected experiments carried out by each division involved in the experiments. Such information included: the name of the company, a description of the order and amount of the procurement contracts.

Step 3

Supplier selection. Starting with the order information provided by INFN, we selected 13 Italian suppliers. In this selection process, we adopted a theoretical sampling strategy (Eisenhardt, 1989), with companies selected according to the products they supplied for the three experiments.

Step 4

Direct interviews. Cases were investigated through direct semi-structured phone interviews following a semi-structured protocol. Interviews lasted about 1 hour. Each was transcribed, and a cross-case database was created to analyze and compare findings. After the transcription, a coding procedure was used to identify relevant themes emerging from the interview. Several researchers were involved in the transcriptions (i.e., investigators triangulation – Tellis, 1997): each of the authors individually analyzed the interviews and the results were then collectively discussed in order to enhance the internal validity of and confidence in the findings (Eisenhardt, 1989).

The research setting: Istituto Italiano di Fisica Nucleare

Istituto Italiano di Fisica Nucleare is the Italian BSC dedicated to experimental and theoretical studies in the fields of nuclear, sub-nuclear and astro-particle physics and fundamental interactions. To coordinate and conduct research in this field, INFN invests approximately 300 million euros per year, involving about 1700 employees, 500 PhD students and post-doctoral fellows and 4000 researchers from about 40 universities in Italy and other research centres which collaborate with and actively participate in INFN’s scientific programs. Research is also conducted at CERN, where around 1000 INFN scientists take part in different experiments connected to the LHC.

Being at the forefront of research, INFN sets up, conducts and contributes to research activities – namely experiments – that address fundamental complex research questions. In long-term national and international partnerships such experiments require dedicated, specialized infrastructures and instruments with highly demanding performance based on cutting-edge technologies. INFN conducts a significant proportion of its experimental activities in its four national laboratories (i.e., Laboratori Nazionali del Gran Sasso, Laboratori Nazionali del Sud, Laboratori Nazionali I Frascati e Laboratori Naionali di Legnaro), hosting big RIs such as particle accelerators or housing ad-hoc underground equipment for studying rare events.

Selected INFN experiments and related research infrastructures

The following three INFN experiments were selected: Cuore, Virgo, and Ams.

Cuore

The Cryogenic Underground Observatory for Rare Events (Cuore) is an international collaboration of physicists from many countries (primarily Italy and the USA) for the study of neutrinos and rare decay. The Cuore experiment is installed at the Gran Sasso National Laboratories, one of the most important facilities in the world for underground studies and providing an ideal infrastructure for very precise measurements with more than 1400 m. Of natural shielding from cosmic radiation.

The research infrastructure of the Cuore experiment is composed of 988 bolometers, arranged in 19 towers consisting of 13 floors with four crystals for each floor. These bolometers are composed of ultra-cold¹ tellurium dioxide crystals. To collect data, Cuore detectors must be cooled to around 10 mK using one of the largest and most powerful cryostats that provides the coldest cubic meter in the universe. For this reason, Cuore has also become famous in the field as ‘the coldest heart in the known universe’. All the parts of the Cuore detectors were constructed with a special radio-pure material with particular chemical and electropolish procedures, while the cryostat was assembled through a specific robotic arm aimed at ensuring the same space between each crystal and the thermometers. Lastly, Roman lead is used in the Cuore infrastructure to protect the RI from the naturally occurring radioactivity.

Virgo

Virgo is a large Michelson interferometer designed to detect gravitational waves, based on Einsten’s General Relativity theory. The experiment is located in Cascina (Italy) at the European Gravitational Observatory (EGO), founded in 2000 as a result of collaboration between INFN and Centre National de la Recherche Scientifique (CNRS). Virgo involves scientists from several European countries, while outside Europe an affiliation with the Laser Interferometer Gravitational-Wave Observatory (LIGO) in the USA has been established.

The instrument has two long arms and is extremely sensitive. It can detect variations in the length of its 3 km arms equal to one billionth of the diameter of an atom. To achieve this high sensitivity, the Virgo infrastructure is based on cutting-edge technologies from different fields, ranging from optics to electronic and ultra-high vacuum. Virgo uses high-quality mirrors (surface irregularities limited to one billionth of a meter), laser beams operated in ultra-high vacuum conditions to avoid air refraction and a super-attenuator system to suspend and isolate the instrument from external disturbances. For example, the substrate of the mirror is made with the world’s purest synthetic quartz while the mirror profile is polished at an atomic level.

Ams

The Alpha Magnetic Spectrometer (AMS), whose instrumentation was installed in 2011 on the International Space Station (ISS), is designed to find tangible traces of primordial antimatter, but also information on dark matter (which is very difficult to observe as it does not emit radiation). In addition, cosmic rays from deep space can carry important information on the sources that generated them far away in time and space.

Among the 16 countries involved in AMS, Italy is one of the main contributors with INFN and the Italian Space Agency (Agenzia Spaziale Italiana, ASI), also supported by national companies and laboratories. The AMS infrastructure is a particle physics detector that consists of a permanent magnet composed of other two particle detectors. The Transition Radiation Detector (TRD) identifies electrons and positrons by transition radiation. The Ring Image Cherenkov counter (RICH) has 10,880 photosensors that measure the charge and velocity of particles. Lastly the Electromagnetic Calorimeter (ECAL) is a three-dimensional imaging instrument that measures the energy and direction of trillion electronvoll (TeV) positrons and electrons with high precision. AMS is also known as the ‘Hubble Space Telescope for cosmic rays’, given its sensitivity (which is between a hundred and a thousand times greater than any other instrument in orbit).

Summary of the selected suppliers

The selected suppliers, whose key characteristics are reported in Table 2, differ in type of procurement relationship and type of products commercialized, thus providing a significant range of cases (Eisenhardt, 1989).

Table 2.

Selected case studies.

Case ID	Main products commercialized	Type of procurement relationship
A	Informatic, electronics, ICT	Regular procurement
B	Fiber opctic, radio frequency, high voltage cables, automation, electromechanics	PPI
C	Ad hoc electronic circuits	PPI
D	UHV components, nuclear fusion, vacuum vessels, beamlines	PPI
E	Informatics components, ICT	Regular procurement
F	High performing computing, container platform, open source cloud platform, artificial intelligence, virtualization solution	PPI
G	Environmental monitoring system, spectrometry systems, liquid effluent monitors, scanning system, radiation portal monitors	PPI
H	Electronic sensors	PPI
I	Informatics components, ICT	Regular procurement
L	Electronic components, wiring, adhesives, lighting display	Regular procurement
M	Screws, bolts, product of precision mechanical details, integration systems, software, middleware	PPI
N	Spectrum Analyser, Network Analyser, oscilloscope	Regular procurement
O	Mechanical components, measurement instruments	Regular procurement

Results

This section describes the results according to the framework presented in Section 3. We start by discussing the process of collaboration and related characteristics and then proceed with the impact dimensions.

Characteristics of the procurement relationships

All companies in the study have had a long-lasting (4–50 years) procurement relationship with INFN, both for regular procurement and PPI. Second, the intensity of contacts varies depending on the type of procurement relationship. In the case of PPI, the relationship is continuous and systematic, including during the initial phase of defining the technical detail of the order. This is because the companies need to understand the technological requirements of the researchers and then, after the new product has been developed or customized, they need to test and retest it together with the researchers to understand whether the technical requirements have been met. By way of example, companies B and C reported that:

“We normally develop the product, or we customize it according to the design provided by the researchers. Sometimes, however, this documentation is not complete, or the design could not be made. In this case we need to cooperate and collaborate with researchers to understand how to create a product that can fulfil the expectations of INFN researchers.” (Company B)

“The relationship is ongoing and systematic before and after the order is delivered. This is particularly the case in the very early stages, because we need to understand what INFN researchers need and how to customize the product. There is constant technological updating with researchers.” (Company C)

This is unsurprising since this type of procurement relationship is related to the acquisition of products that are not already part of the company’s catalogue. In the case of regular procurement, on the other hand, although long-lasting, the procurement relationship is more sporadic and limited to when the order needs to be defined.

To proceed with the third dimension, related to the characteristics of the relationships, we found that the contact between researchers and companies, before delivery of the order, was frequent in both regular procurement and in PPI. In the latter case, however, the contact was more frequent, particularly for the development of new products – again, to understand their technical characteristics. Frequency of contacts is an important way of transferring knowledge from INFN to the company, both in the case of regular procurement and PPI, as reported by companies A and B:

“The frequency of contacts is, in fact, the most important part of the transferal process.” (Company A)

“Frequent contact with INFN researchers, and informal contact such as visits and discussions in their laboratory, are extremely important to understand their specific technological needs.” (Company B)

Lastly, evidence from most cases shows an intense use of informal channels of contact, particularly email, phone calls and personal meetings. These are facilitated by the fact that the relationship is long-lasting supporting the creation of personal contacts. For example, companies F and B affirmed that:

“After several years of collaboration, we keep in touch using personal contact, particularly phone calls. This enables us to be more efficient in understanding the specific requirements of the products and to clearly understand the needs of the researchers.” (Company F)

“Speaking directly with researchers in their laboratory is the most effective way for us to understand the technology they need and also, most importantly, to acquire new technical knowledge.” (Company B)

To sum up, the collaborative relationship between INFN researchers and suppliers for the three selected research infrastructures is generally long-lasting and characterized by frequent contact through informal channels. On the other hand, the intensity of the relationship varies depending on the type of procurement contract, being systematic in the case of PPI and more episodic in regular procurement relationships.

Benefits generated for supplier companies

The benefits generated for supplier companies are described according to the dimensions that make up our conceptual framework.

Organizational and commercial benefits

At a general level, the collaboration with INFN led to what we refer to as the “organizational innovation impact”, in the case of regular procurement as well as in PPI procurement relationships.

Two important impacts were highlighted due to the suppliers' collaboration with INFN: the introduction of new business units (five companies) and entering new markets (eight companies). Regarding the creation of new business units, this was a benefit both in general procurement (cases I and E) and in PPI (cases B, C, D), where the new business units were a result of the company’s need to respond quickly and efficiently to manage the requests of INFN researchers, as reported by companies I and D:

“In order to manage the requests of INFN researchers and to understand their needs in term of products, we did not introduce a new business unit as such but rather an internal division completely devoted to INFN orders.” (Company I)

“We created a group of dedicated people to evaluate the technical requirements of INFN orders.” (Company D)

On the other hand, entering new markets characterized PPI more than regular procurement. In fact, the interviews suggested that entering new markets was facilitated by the development of new products or the customization of existing ones which led to acquiring new clients and thus entering new markets. These new markets typically involved the research market – favoured not only by the new products developed but also by the newly acquired competencies of the companies, which were increasingly able to fulfil the technical requirements of the researchers and also to respect their deadlines, which were often different from those required by more commercial market segments. As described by companies C, B and D:

“Now the research market represents an important part of our annual income. But we started working in this market only after collaborating with INFN. It was the first public research centre we worked for.” (Company C)

“At the beginning we didn’t work for public research centres and we didn’t know anything about the research market. But, after developing new products for INFN it was easier to enter that market and to satisfy the needs of other researchers by selling the products previously developed to meet INFN’s technical requirements.” (Company B)

“Before working as an INFN supplier we had not worked for any research centres and we did not know anything about the research market and its specific requirements. We worked mainly for the civil sector and for the aviation market. Today, thanks to the procurement relationship with INFN, we work on all of the synchrotrons in Europe and also on proton therapy. Also, the research market now provides our biggest revenues.” (Company D)

Market penetration refers to the acquisition of new clients, which represents a cross-case impact dimension. In fact, regardless of the type of procurement relationship, the majority of our selected suppliers reported that they had acquired new clients, particularly public research centres and universities. Companies A, D and F, for example, reported that they used to work for the civil sector (particularly in aeronautics), but after the first procurement relationship with INFN they started to obtain procurement contracts with other research centres. The Italian National Research Council, the Italian National Institute of Astrophysics, the Italian National Agency for New Technologies, Energy and Sustainable Economic Development, the Italian National Institute of Geophysics and Vulcanology and CERN were cited among the science-based institutions that had become new clients. However, the acquisition of new clients was not limited to research institutions, as reported by company D:

“Very often we develop new products for INFN, and then these products are also sold to other public or private clients. This enables us to acquire new clients from both the public and private market. In one line our slogan is ‘from the world of research to the world of companies’.” (Company D)

Innovation benefits

Innovation benefits involve three main dimensions: i) benefits in R&D (i.e., increases in financial or human investment related to procurement activities), ii) new patents and iii) new products. Four companies affirmed that they had experienced a positive impact on R&D investment. This is an important impact particularly for companies that have a PPI relationship with INFN, since R&D represents a core activity of their business. In fact, as a consequence of the procurement relationship with INFN, the investment in R&D activities increased, as described in various cases:

“Working with INFN has definitely changed and improved our internal R&D processes. Frequent discussions and exchanges of ideas with INFN researchers have enabled us to improve our internal R&D processes and activities.” (Company G)

“Of course there has been an increase in R&D investments. This is mainly the indirect effect of the collaboration with INFN but, undoubtedly, after this collaboration, our investment in R&D has increased.” (Company F)

“Thanks to INFN we have understood the importance of R&D activities in order to be competitive on the market. For this reason, the investment in such activities has increased, especially from a human resources perspective.” (Company D)

The most significant innovation benefit is related to the development of new products, which was reported by nine companies. For example, to meet the needs of INFN researchers, company F developed a high-performing storage system for a very high quantity of data, company B designed specific cables, and company M elaborated new rare metals and related applications. This is not surprising in the case of PPI, in which customized products are made for INFN researchers which are then also sold to other clients, mainly public research centres, as reported by companies C and D:

“We have modified our existing products, since the requests by INFN researchers had some particular features that we wanted to make. Once we have customized products, we also sell them to other clients. In a nutshell, INFN pushed us to increase our range of products.” (Company C)

“Our procurement relationship with INFN is characterized by the development of completely new products with high precision mechanical elements.” (Company D)

What was more unexpected was the fact that suppliers linked to INFN by a general procurement relationship identified “new product development” as an important benefit due to the collaboration. This was not about the development of a completely new product but rather was related to the customization of already existing products. Company L reported:

“We typically commercialize hardware and software. With respect to these products, the requests by INFN involve high specifications and very often we then added these products to our catalogue and we also sold them other research centres.” (Company L)

Learning impact

The learning impact has three different dimensions: “technological learning”, “organizational learning” and “market learning”. Technological learning is a cross-case impact that occurs both in regular procurement and PPI (seven companies identified that this impact was important). In the case of PPI, and in particular when the development of a new product was required by INFN researchers, the collaboration with researchers was aimed at understanding the technical characteristics of the new product, the acquisition of new technical knowledge and a deeper understanding of the technology to be developed. For example, Company F stated that, to understand and then develop a customized product, it needed to understand a specific computational language and a related unit of measurement. This newly acquired knowledge then facilitated the development of a specific storage system which was also sold to other research centres, such as CERN. Other companies reported that:

“From a technological point of view, for us INFN is a very important client since they always ask for cutting-edge technologies that we develop in collaboration with the researchers. This requires frequent meetings to understand the technical requirements and we have then benefited from the acquisition of new technical knowledge.” (Company D)

“During meetings with researchers, they explain to us the technical requirements of the technology that we then develop. This gives us the benefit of in terms of acquiring technical knowledge.” (Company B)

“To develop the cutting-edge technologies required by INFN experiments, several meetings with researchers were necessary. During these meetings we discussed the technicalities and the particular features of the new technology which then represented for us a source of new technical knowledge.” (Company G)

Market learning is a second cross-case benefit that appears both in regular procurement cases and in PPI (six companies reported this market impact). In both cases, companies reported that they had acquired new knowledge concerning the research market, particularly when INFN was the first client among the national and international public research centres. In fact, company M reported:

“Thanks to the collaboration with INFN, we entered the research market. The first collaboration with INFN was particularly useful for understanding the characteristics not only of public research centres, but also of the research market.” (Company M)

Finally, organizational learning, referring to the acquisition of new knowledge related to the productive processes, occurred only in three cases, characterized by a PPI relationship, as described by company O:

“Collaborating with INFN improved our competencies in the productive processes of highly specialized technologies. Without this collaboration, it would have been very difficult to improve these processes.” (Company O)

Reputational benefits

Reputational benefits refer to improvements in company image, brand and reputation. The findings from our cases show that this was the most common benefit: all the companies interviewed reported that being a supplier of INFN had improved their reputation from both an internal and an external perspective:

“Being an INFN supplier has improved our reputation for external and new clients, but it is also important from an internal point of view. The fact that we work with INFN has a ‘motivational’ effect on employees; that is to say that we are more confident in the quality of our products and in our competencies.” (Company A)

“The fact that we are one of the suppliers of INFN acts as a kind of ‘quality stamp’ that has also led to the acquisition of new clients.” (Company C)

Only one case reported that the reputational benefit was limited, because the interviewee believed that the reputation of his company was already excellent before the procurement relationship with INFN had started.

Economic benefits

In eight cases companies acknowledged they had increased sales thanks to the procurement relationship with INFN.

Summary of the findings

Table 3 summarizes our findings concerning the procurement relationship and the related impact on suppliers.

Table 3.

Summary of findings.

Investigated dimensions	Dimensions’ operationalization (where present)	Findings - Companies	Tot. of companies
Characteristics of the relationships
1. Duration of the relationship	Episodic	L	1
1. Duration of the relationship	Systematic	A; B; C; D; E; F; G; H; I; M; N; O	12
2. Type of relationship (PPI or regular procurement)	Regular procurement	A; E; I; L; O	5
2. Type of relationship (PPI or regular procurement)	PPI	B; C; D; F; G; H; M; N	8
3. Frequency of contacts	Daily	F; O	2
	More than once a week	A; B; C; H; M	5
	Once a week	G; I;	2
	More than once a month	D; E; N	3
	Once a month	L	1
4. Channels of collaboration	Formal	0	0
4. Channels of collaboration	Informal	A; B; C; D; E; F; G; H; I; L; M; N; O	13
Impact(s) of the relationship on suppliers
1. Organizational and commercial impacts	Introduction of new business units	B; C; D; E; I	4
	Entry in new market	A; B; C; D; E; F; M; O	8
	Acquisition of new clients	A; B; C; D; E; F; H; I; M; O	10
	Development of new partnerships	D; F	2
2. Innovation impacts	R&D benefits	D; F; G; M	4
	New patents	-	0
	Development new products	A; B; C; D; F; G; H; M; N	9
3. Learning impacts	Technological and technical learning	B; D; C; F; G; H; M	7
	Organizational learning	F; O	2
	Market learning	B; D; E; G; M; O	6
4. Reputational impact	Improved reputation	A; B; C; D; E; F; G; H; I; L; M; N; O	13
5. Economic impact	Improved economic performance	A; D; E; F; G; M; I; L	8

First, concerning the characteristics of the procurement relations, two features emerged as the most common among the suppliers investigated. The majority of relations are both systematic and characterized by frequent and informal contacts. These are the two elements that, according to the interviewees, have facilitated the process of knowledge transfer from the INFN to its suppliers.

Second, with respect to the impacts generated, all the dimensions considered seem to be relevant. Regardless of the type of relationship (PPI or general procurement), all the suppliers interviewed reported that there had been a reputational impact. A second relevant impact was the acquisition of new clients, which occurred in 10 companies. This was mainly related to the fact that, after the initial collaboration with INFN, these suppliers then also worked for other research centres, in Italy and abroad. Similarly, working with INFN in turn facilitated entry into a new market, typically the research market, thanks to new product development.

Discussion and conclusion

As highlighted in the introduction, since research infrastructures are publicly funded and incur high costs, demonstrating the ability of BSCs to generate positive impacts has become extremely important in the current context of budget constraints (Ancaiani et al., 2015; Autio, 2014; Autio et al., 2003).

BSCs also represent a very particular context because, for their scientific experiments, they need to design and build large, capital-intensive, complex and technological research infrastructures. This process leads to the transfer of knowledge and technology to suppliers.

This study highlights both the characteristics of the relationships between BSCs and suppliers and the impacts generated for suppliers.

In line with the current literature (Castelnovo et al., 2018; Dal Molin and Previtali, 2019), this research confirms that Big Science public procurement acts as a technology transfer mechanism for suppliers. Firstly, our case studies show a link between the type of procurement relationship (PPI and “regular procurement”), the kinds of contact between suppliers and researchers and the type of benefits generated for the suppliers. More intense procurement relationships, as in the case of PPI, lead to a greater impact for suppliers compared to regular procurement from the suppliers' catalogues.

The literature on PPI confirms the different nature of the relationship in the case of regular procurement or PPI (Dal Molin and Previtali, 2019); only in PPI does it act as a technology transfer mechanism. This is true especially in the case of “radical PPI”, which entails the creation of radically new innovations (Edquist and Zabala-Iturriagagoitia, 2015), as for procurement relationships in BSCs for the development of research infrastructures. Public procurement has already been highlighted as a valuable way of fostering innovation (Aschhoff and Sofka, 2009; Edquist and Zabala-Iturriagagoitia 2012, 2015; Georghiou et al., 2014), and this also applies to procurement relationships in the case of BSCs (Autio et al., 2003, 2004; Dal Molin and Previtali, 2019; Florio et al., 2018).

Our research also shows that suppliers adopt informal channels to keep in contact with INFN researchers, particularly through phone calls and meetings. In line with recent studies by Alexander et al. (2020) and Hobbs et al. (2017), our results confirm that personal relationships facilitate the transfer of technological learning (i.e., from the BSC to suppliers). In addition, frequent contact, particularly for PPI, is often necessary to develop the final products effectively. This leads to increased opportunities for the transfer of technical knowledge (Autio et al., 2003). The frequency of contact, especially when new products need be developed or need to be customized, is a characteristic of Big Science technology transfer, based on non-standard and new products. In fact, companies advanced their technical knowledge thanks to the relationship with INFN and to investments in R&D, which led to the creation of new products and the improvement of existing products. Alongside the technological learning gained in the relationship, the creation of new products is favored by the role of the BSC, which acts as a first client as well as the investor, taking the risk of the technological development (Dal Molin and Previtali, 2019). This, together with reputational benefits, leads to higher market shares and to the entry into new markets, mainly in research, which has a positive economic impact.

When the relationship between firms and INFN is based on regular procurement with moderately frequent contacts, suppliers mainly gain technological learning benefits, which turn into new or improved products, promoting market penetration. Market penetration benefits have clearly emerged also in general procurement, and these have led to improved economic performance thanks to reputational benefits.

Secondly, the “learning impact” emerged as a cross-impact. In fact, collaboration with INFN led to technological learning for PPI, in the case both of customized products and of the development of completely new products. Market learning is also relevant for the companies analyzed and this learning especially refers to the research market.

Thirdly, innovation benefits emerge as important, especially in the development of new products, which was recognized as a benefit from the collaboration with INFN. This is not surprising for relationships characterized by the provision of customized technological products. However, it is interesting that in the case of regular procurement too the collaboration with INFN led to innovation and the development of new products.

Our results confirm the findings of two streams of literature. First, concerning the Big Science literature, the results are in line with studies on CERN; for example, Autio et al. (2003), Florio et al. (2018) and Castelnovo et al. (2018). Based on a survey of 154 CERN suppliers, Autio et al. found that 38% of respondents had designed new products, 42% had gained reputational benefits, 44% reported technological learning and 36% reported market learning. More recently, thanks to the procurement relationship with CERN, Castelnovo et al. (2018) found three outcomes: “innovation” (i.e., development of new products), “learning” (i.e., acquisition of technical know-how and improved quality of products and services) and “market penetration” (i.e., acquisition of new customers and improved reputation). Similarly, Florio et al. (2018) found that the procurement relationship with CERN favored suppliers in terms of the improvement in technical knowledge (55% of respondents), improvement in products and services (48%), and reputational benefits (62%). Second, concerning the public procurement literature, our study supports the finding that public procurement represents an effective tool for supporting companies’ innovation, especially when PPI is concerned.

Study contributions and limitations

This research contributes to the literature on the impact of Big Science, and in particular public procurement, as a mechanism of technology transfer. Specifically, in line with the current literature, we found that Big Science public procurement may generate a set of positive benefits for companies. Considering the three outcome categories for supplier companies provided by Autio et al. (2004) – “innovation outcomes”, “learning outcomes” and “market penetration outcome” – this study shows that INFN public procurement favored “learning”, “innovation” and “market penetration”.

Concerning learning outcomes, we found that the main impact was related to the acquisition of technical knowledge, due to the strong collaboration with researchers that is necessary for the construction and upgrade of research infrastructures. During this collaborative process, characterized by continuous interaction between the two parties and a cyclical process of testing and refinement, company personnel can acquire new knowledge regarding the technology itself. As far as market penetration is concerned, being an INFN supplier improves the image and reputation of the company as a whole, giving it a market advantage over competitors. In addition, collaborating with INFN favors the acquisition of new clients, again providing a competitive advantage. Therefore, the impact generated refers mainly to intangible knowledge and benefits, while traditional R&D output-based indicators (such as patents and publications) are not able to fully capture the positive impact on suppliers, as already highlighted in the literature (Dal Molin and Previtali, 2019; Puliga et al., 2019). This is in line with previous studies in the field (see, for example, the work of Castelnovo et al., 2018), which highlighted that Big Science public procurement generated important learning and technological spillovers to suppliers, producing, in turn, a set of indirect positive effects for companies. For instance, the acquisition of technical knowledge generates indirect effects: increasing sales, entry into new markets and the creation of new collaborations and partnerships. Similarly, the positive effect on reputation favors the acquisition of new clients, increasing sales and the creation of new collaborations.

The study is not without limitations. First, it analyzes a single case, that of INFN, thus limiting the generalization of the results. In the future it would be useful to study other Big Science research centres to provide further comparative analyses using the same methodological approach. In addition, since there is a considerable time lag between a collaboration and its effects, longitudinal case studies would help towards an understanding the long-term benefits and how they evolve and impact on suppliers. It is also important to consider that the impact generated is related mainly to the transfer of intangible knowledge which continues to evolve, sometimes over a long period. This highlights the inability of output-based performance measures to fully capture the positive impacts of Big Science on supplier companies at a precise point in time.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Martina Dal Molin

Note

References

Alexander

Martin

Manolchev

, et al. (2020) University-industry collaboration: using meta-rules to overcome barriers to knowledge transfer. The Journal of Technology Transfer 45: 371–392.

Al-Tabbaa

Ankrah

(2016) Social capital to facilitate ‘engineered’ university–industry collaboration for technology transfer: A dynamic perspective. Technological Forecasting and Social Change 104: 1–15.

Ancaiani

Anfossi

Barbara

, et al. (2015) Evaluating scientific research in Italy: the 2004–10 research evaluation exercise. Research Evaluation 24: 242–255.

Aschhoff

Sofka

(2009) Innovation on demand—Can public procurement drive market success of innovations? Research Policy 38(8): 1235–1247.

Autio

(2014) Innovation from Big Science: Enhancing Big Science Impact Agenda. London, UK: Imperial College Business School. Available at: http://dera.ioe.ac.uk/19649/1/bis-14-618-innovation-from-big-science-enhancing-big-science-impact-agenda.pdf

Autio

ErkkoBianchi-Streti

Hameri

(2005) Technology transfer, technological learning and innovation from the large hadron collider procurement activities: main outcomes and recommendations for future projects. International Journal of Technology Management 2005: 1–14.

Autio

Hameri

Bianchi-Streit

(2003) Technology transfer and technological learning through CERN's Procurement Activity. (No. CERN-. 2003-005).

Autio

Hameri

Nordberg

(1996) A framework of motivations for industry-big science collaboration: a case study. Journal of Engineering and Technology Management 13(3–4): 301–314.

Autio

Hameri

Vuola

(2004) A framework of industrial knowledge spillovers in big-science centers. Research Policy 33(1): 107–126.

10.

Bastianin

Del Bo

(2021) Public procurement in big science: politics or technology? The case of CERN. Industrial and Corporate Change 30(3): 740–761.

11.

Bastianin

Castelnovo

Florio

, et al. (2022) Big science and innovation: gestation lag from procurement to patents for CERN suppliers. The Journal of Technology Transfer 47: 531–555.

12.

Bonaccorsi

Piccaluga

(1994) A theoretical framework for the evaluation of university-industry relationships. R&D Management 24(3): 229–247.

13.

Bozeman

(2000) Technology transfer and public policy: a review of research and theory. Research Policy 29(4–5): 627–655.

14.

Bruneel

D’Este

Salter

(2010). Investigating the factors that diminish the Barriers to University–Industry collaboration. Research Policy 39(7): 858–868.

15.

Caloghirou

Giotopoulos

Kontlamou

, et al. (2021) Industry-university knowledge flows and product innovation: how do knowledge stocks and crisis matter? Research Policy 50: 104195.

16.

Calvert

Martin

(2001) Changing conceptions of basic research. In: Brighton, England: Background document for the Workshop on Policy Relevance and Measurement of Basic Research. Oslo SPRU, Science and Technology Policy Research, University of Sussex, Falmer, Brighton BN1 9RF, UK 29–30.

17.

Castelnovo

Dal Molin

(2021) The learning mechanisms through public procurement for innovation: The case of government-funded basic research organizations. Annals of Public and Cooperative Economics 92(3): 411–446. DOI: 10.1111/apce.12311

, et al. (2018) The economic impact of technological procurement for large-scale research infrastructures: Evidence from the Large Hadron Collider at CERN. Research Policy 47(9): 1853–1867.

, et al. (2020) The effect of university–industry collaboration policy on universities’ knowledge innovation and achievements transformation: based on innovation chain. Journal of Technology Transfer 45: 522–543.

20.

Coccia

Rolfo

(2008) Strategic change of public research units in their scientific activity. Technovation 28: 485–494.

21.

Czarnitzki

Thorwarth

(2012) Productivity effects of basic research in low-tech and high-tech industries. Research Policy 41(9): 1555–1564.

22.

D’Este

Patel

(2007) University–industry linkages in the UK: what are the factors underlying the variety of interactions with industry? Research Policy 36(9): 1295–1313.

23.

Dal Molin

Previtali

(2019) Basic research public procurement: the impact on supplier companies. Journal of Public Procurement 19(3): 224–251.

24.

Edquist

Zabala-Iturriagagoitia

(2012) Public procurement for innovation as mission-oriented innovation policy. Research Policy 41(10): 1757–1769.

25.

Edquist

Zabala-Iturriagagoitia

(2015) Pre-commercial procurement: a demand or supply policy instrument in relation to innovation? R&D Management 45(2): 147–160.

26.

Eisenhardt

(1989) Agency theory: an assessment and review. Academy of Management Review 14(1): 57–74.

(2016) Forecasting the socio-economic impact of the large hadron collider: a cost–benefit analysis to 2025 and beyond. Technological Forecasting and Social Change 112: 38–53.

, et al. (2018) Big science, learning, and innovation: evidence from CERN procurement. Industrial and Corporate Change 27(5): 915–936.

29.

Florio

(2021) La privatizzazione della conoscenza, La Terza. ISBN 978-88-581-4537-1.

(2013) University–industry collaboration and innovation in emergent and mature industries in new industrialized countries. Research Policy 42(2): 443–453.

31.

Georghiou

Edler

Uyarra , et al. (2014) Policy instruments for public procurement of innovation: choice, design and assessment. Technological Forecasting and Social Change 86: 1–12.

32.

Gerbin

Drnovsek

(2016) Determinants and public policy implications of academic-industry knowledge transfer in life sciences: a review and a conceptual framework. The Journal of Technology Transfer 41: 979–1076.

33.

Ghisetti

(2017) Demand-pull and environmental innovations: estimating the effects of innovative public procurement. Technological Forecasting and Social Change 125: 178–187.

34.

von Krogh

Sirén

, et al. (2021) Asymmetries between partners and the success of university-industry research collaborations. Research Policy 50(10): 104356.

(2017) Science and technology parks: an annotated and analytical literature review. The Journal of Technology Transfer 42(4): 957–976.

(2021) How firms use coordination activities in university–industry collaboration: adjusting to or steering a research center? The Journal of Technology Transfer 47: 1308–1342. DOI: 10.1007/s10961-021-09886-x

37.

Lee

(1996) Technology transfer and the research university: a search for the boundaries of university-industry collaboration. Research Policy 25(6): 843–863.

38.

Lee

(2000) The sustainability of university-industry research collaboration: an empirical assessment. The Journal of Technology Transfer 25(2): 111–133.

(2007) An empirical analysis of the propensity of academics to engage in informal university technology transfer. Industrial and Corporate Change 16: 641–655.

(2022) Managing innovations ecosystems around Big Sciece organizations. Technovation 116: 102253. DOI: 10.1016/j.technovation.2022.102523

41.

Mansfield

(1980) Basic research and productivity increase in manufacturing. The American Economic Review 70(5): 863–873.

42.

Mansfield

(1981) Composition of R and D expenditures: relationship to size of firm, concentration, and innovative output. The Review of Economics and Statistics: 610–615.

43.

Mansfield

(1991) Academic research and industrial innovation. Research Policy 20(1): 1–12.

44.

Mansfield

(1998) Academic research and industrial innovation: an update of empirical findings. Research Policy 26(7–8): 773–776.

45.

Martin

Tang

(2007) The benefits from publicly funded research. SPRU Electronic Working Paper Series NO 161(161): 44.

(2018) University–industry cooperation: a systematic literature review and research agenda. Science and Public Policy 45(5): 708–718.

47.

Meyer

(2001) A case in case study methodology. Fields Methods 13(4): 329–352.

48.

OECD (2001) Workshop on basic research: policy relevant definition and measurement. Available at: http://www.oecd.org/sti/sci-tech/2674203.pdf

49.

OECD (2014) The impacts of large scale research infrastructures on economic innovation and on society: case studies at Cern. Available at: https://www.oecd.org/sti/sci-tech/CERN-case-studies.pdf

50.

Perkmann

Walsh

(2009) The two faces of collaboration: impacts of university-industry relations on public research. Industrial and Corporate Change 18: 1033–1065.

51.

Perkmann

Neely

Walsh

(2011) How should firms evaluate success in university-industry alliances? A performance measurement system. R&D Management 42(2): 202–2016.

52.

Piva

Rossi-Lamastra

(2013) Systems of indicators to evaluate the performance of university-industry alliances: a review of the literature and directions for future research. Measuring Business Excellence 17(3): 40–54.

53.

Puliga

Manzini

Batistoni

(2019) An industry and public research organization joint effort for ITER construction: Evaluating the impact. Fusion Engineering and Design 146 (PartA): 187–193. DOI: 10.1016/j.fusengdes.2018.12.015

54.

Rådberg

Löfsten

(2022) Developing a knowledge ecosystem for large-scale research infrastructure. The Journal of Technology Transfer 29: 209–220. DOI: 10.1007/s10961-022-09945-x

55.

Rajalo

Vadi

(2017) University-industry innovation collaboration: reconceptualization. Technovation 62: 42–54.

56.

Rybnicek

Königsgruber

(2019) What makes industry-university collaboration succeed? A systematic review of the literature. Journal of Business Economics 89: 221–250.