Reaping the fruits of co-creation through design experiments

Abstract

Co-creation has received a growing attention in the public sector as a tool for mobilizing resources, spurring public innovation, and enhancing democratic legitimacy. Reaping the fruits of co-creation largely depends on the attempts made by local leaders to facilitate effective collaboration, overcome emerging problems and obstacles, and cope with the emerging dilemmas and paradoxes. There is limited knowledge about how this is done in practice. Hence, public administration researchers must team up with local leaders of co-creation to jointly test different ways of improving the processes and results of co-creation processes. This endeavour calls for an interactive research strategy where researchers and practitioners work together to conduct design experiments aimed at improving the design of a process by iteratively diagnosing problems and obstacles that prevent goal achievement, seeking to remove the hindrances through targeted interventions, and finally measuring the impact of the interventions vis-à-vis the stated goals. This article serves the dual purpose of producing new context-dependent insights into how leaders of co-creation can tackle emerging problems and obstacles and evaluating the use of design experiments in the field of public administration. It analyses and compares six design experiments conducted in relation to local cases of co-creation that are studied using mixed methods. The main finding is that: (1) the problems emerging in co-creation processes are for the most part manageable; (2) the leadership interventions are relatively undemanding; and (3) the positive impact of most of the interventions finds support in the extant research literature.

Keywords

Co-creation design experiments interactive research leadership local public administration

Co-creation is a subspecies of collaborative governance aimed at involving public and private actors, including lay actors such as users, citizens, community leaders, and civil society associations, in distributed processes through which the differences among the participants are constructively managed so that they can foster joint solutions to common problems (Gray, 1989; Torfing et al., 2021). Hence, co-creation can be defined as a collaborative process whereby two or more actors identify and define problems and challenges and design and implement improved and often innovative solutions, which the actors may subsequently co-evaluate and co-adjust (Ansell and Torfing, 2021). The development and testing of prototypes in the co-creation process helps to break down the boundary between design and implementation when prototypes that are seen to work in practice are scaled up.

Co-creation is a relatively new “magic concept” (Pollitt and Hupe, 2011; see also Torfing et al., 2022) aiming to link political and administrative leaders and frontline staff with citizens and private stakeholders, thereby forging a new state‒society relation. Recently, co-creation has gained a stronger foothold as a local-level governance practice, where the need for new and better public solutions becomes increasingly evident and there is a notable proximity between public authorities, business actors, and citizens. The main attraction of co-creation is that it helps to mobilise societal resources into a cash-strapped public sector; foster needs-based solutions by involving users in the creation of solutions that hit the target; enhance societal cohesion by bringing different (and even polarised) groups of public and private actors together in pragmatic problem-solving; create innovative solutions that break with common wisdom and established practices; build common ownership over joint solutions; and democratise public governance by stimulating participation on the output side of the political system (Ansell and Torfing, 2021).

Reaping the above-mentioned fruits of co-creation largely depends on the attempts made by local leaders—be it public leaders, community leaders, or third-party facilitators—to facilitate effective collaboration based on trust and principled engagement, overcome emerging problems and obstacles, and cope with the dilemmas and paradoxes that arise when co-creation clashes with traditional forms of bureaucratic governance and/or the New Public Management. Many things can go wrong when aiming to co-create innovative public value outcomes: the group of participating actors may be wrong for the specific purpose; interdependence and trust may be too weak to prevent destructive conflicts; the actors may suffer from tunnel vision that prevents creativity; it may be all talk and little action; politicians may be risk-averse and block the implementation of innovative solutions, etc. These and other problems and obstacles must be removed, circumvented, or mitigated by exercising co-creation leadership (Hofstad et al., 2021).

Since there are limited experiences with how this is done in practice, public administration and governance researchers must team up with local leaders of co-creation to jointly test different ways of improving the processes and results of co-creation. Such a practical engagement of scholars in public governance processes flies in the face of the traditional image of researchers as detached spectators sitting in their ivory tower and making neutral and disengaged observations that they analyse, write up, and disseminate to a strictly scientific audience. However, if researchers want to be relevant to society while improving the quality of their research, they must climb down from their elevated position, actively engage with their object of research, recognise the validity of lay-actor knowledge, and deliver practical knowledge that is directly applicable in the public sector. This achievement may come at the expense of producing generalisable results, as insights produced through practical engagement tend to be highly context-dependent. As the number of context-dependent insights increases, however, researchers will be able to identify emerging patterns and formulate hypotheses for further testing.

Public administration and governance researchers may collaborate with local leaders of co-creation to conduct design experiments aimed at improving the design of the process or solutions by iteratively diagnosing problems and obstacles that prevent goal achievement, aiming to remove the hindrances through targeted interventions, and finally measuring the impact of the interventions vis-à-vis the stated goals based on a triangulation of available data from focus group interviews, surveys, observations and documents including qualitative and quantitative self-evaluation reports (Cobb et al., 2003; Stoker and John, 2009). As such, design experiments bear an element of social engineering, as the aim is to improve existing processes and solutions based on well-deigned interventions and measurement of their impact on pre-established goals. Design experiments have some key advantages compared to other experimental strategies. They are more theory-driven than Darwinian experiments aimed at “letting a thousand flowers bloom” and seeing which ones survive or fare better in the long run. At the same time, they are less focused than random control trials on establishing causal inference in artificial lab-like environments and more on identifying what works in real-life contexts, where it is impossible to control for the influence of all kinds of factors (Ansell and Bartenberger, 2016). As such, design experiments may help us to reap the fruits of co-creation by improving co-creation leadership based on theory-driven interventions in open social systems. In addition, design experiments neither require the presence of an ecology of many parallel trials since one trial with several iterative interventions is enough; nor do they require the construction of a control group sheltered from the planned intervention since the impact of the interventions is measured by comparing the participants assessment of goal achievement before and after their application. This tends to make design experiments less demanding, but potent in discovering what works in a given context.

Design experiments emerged in aeronautics but have recently been used to improve classroom teaching pedagogy (Cobb et al., 2003). Despite a growing interest in design experiments in the social sciences (Blok, 2013; Romme and Meijer, 2020; Stoker and John, 2009), there are very few experiences with the use of design experiments in public administration and governance research (but see Bos and Brown, 2012). In a deliberate effort to change that, this article aims to use design experiments to find ways of improving co-creation leadership and reap the fruits of co-creation at the local government level.¹ Hence, the article serves the dual purpose of producing new context-dependent insights into how leaders of co-creation can tackle emerging problems and obstacles and evaluating the use of design experiments in the field of public administration. As such, the research question is: How can we learn to better reap the fruits of co-creation through local design experiments focused on the exercise of co-creation leadership?

To answer this question, the article first defines co-creation and reflects on the need for co-creation leadership. It then explains the steps in carrying out a design experiment and accounts for our practical and interactive use of the design experiment method, including the choice of cases, the joint planning, and the collection and analysis of data. The empirical findings from a series of design experiments in a broad range of European countries are presented and briefly discussed with reference to the relevant literature, and the use of design experiments is evaluated before the conclusion summarises the results and lays out some future research avenues.

Co-creation and co-creation leadership

Co-creation can be usefully defined as a collaborative process through which public and/or private actors, including lay actors, aim to produce innovative public value outcomes based on creative exploration, mutual learning, and experimental prototype testing (Ansell and Torfing, 2021). The emphasis on innovation and citizen participation distinguishes co-creation from its parent concept, collaborative governance, which focused more on alignment and the interaction between organised stakeholders (Ansell and Gash, 2008). The emphasis on the broad-based involvement of relevant and affected actors and effort to go beyond existing public service solutions (Osborne and Strokosch, 2013) distinguishes co-creation from co-production aimed at allowing service producers and service users to use their respective skills, competences, and experiences in the production of pre-determined service solutions (Brandsen and Honingh, 2016). As such, the unique feature of co-creation is the idea that a diverse group of actors with dense interactions will produce innovative solutions that none of the actors would be able to produce singlehandedly (Dente et al., 2005; Huxham and Vangen, 2013). Hence, in essence, co-creation builds on collective wisdom and swarm creativity (Gloor, 2006; Wise et al., 2012).

Co-creation comes from the private sector, where service producers saw how they could enhance value creation by actively engaging customers (Prahalad and Ramaswamy, 2004), but it seems to play an increasing role in the public sector, especially in countries with strong traditions for citizen participation (Voorberg et al., 2015). As a tool of governance, co-creation is mostly used at the local and regional levels and most frequently in health care, environmental policy, and planning (Torfing et al., 2019). It is by no means a universal, all-purpose tool, however, as its practical relevance and feasibility appears limited in crisis situations and highly contested policy areas, as well as in routine public regulation and service production, where there is little or no need for innovation. Nevertheless, co-creation appears to have a broad relevance in relation to policymaking, governance reforms, the development of new forms of regulation and service delivery, and the realisation of new societal goals and missions (Brandsen et al., 2018). The United Nations recommends that the efforts to achieve the Sustainable Development Goals (SDGs) by 2030 are based on the co-creation of innovative solutions in networks and partnerships (Ansell et al., 2022).

Strategic management may help to make room for and prompt co-creation in the public sector (Ongaro et al., 2021; Torfing et al., 2021). In countries where co-creation is part of a new vision for the public sector aiming to mobilise societal resources, spur public innovation, and foster needs-based solutions, executive public leaders may design organisational strategies aiming to enhance co-creation, and they may also build digital and/or physical platforms that support the formation of arenas for co-creation while lowering the transaction costs of collaborating. In addition, public leaders and managers can spur co-creation by replacing command- and control-based management with a more trust-based leadership style, thereby giving frontline personnel a “license to innovate” and more time and space for collaboration with each other and external actors. Finally, co-creation requires a change in leadership style away from the traditional forms of command and control to more facilitative forms of leadership aimed at empowering employees, relevant stakeholders, and affected citizens to participate in the co-creation of innovative public value outcomes. Instead of imposing solutions on other actors, solutions should be a result of joint exploration. Changing the leadership style of public leaders will require some re-training, as scholarship inspired by the New Public Management (Hood, 1991) has told public leaders to lead downward and inward, focusing on using their own budget, staff, and organisational tools to achieve a specific set of goals measured by key performance indicators.

Strategic management can be helpful in framing and facilitating co-creation in public organisations, but it is of little use when it comes to designing, facilitating, and improving actual co-creation processes at the intersection between state, market, and civil society. Here, the responsibility for driving specific co-creation processes lies with local public leaders and frontline staff who frequently will be sharing the leadership responsibilities with the involved actors or perhaps also some specially hired consultants. However, on-the-ground learning about how to design, orchestrate and improve co-creation processes based on leadership may feedback into strategic management.

Although co-creation processes are characterised by a horizontal and ideally egalitarian relationship between the different public and private participants, who are bound by resource interdependencies and common aspirations to solve problems and attain some overall goals, leadership is essential to ensuring constructive and trust-based interaction and progression towards attaining common objectives. Co-creation does not arise spontaneously when needed; it may be undermined by destructive conflicts and fail to promote outside-the-box thinking. Hence, co-creation leaders must act as conveners, facilitators, and catalysts of innovation (Torfing, 2016). The responsibility for performing these leadership roles may be shared between formally appointed leaders of a co-creation process and some of the participants that are involved in performing leadership tasks (distributed leadership, se Bolden, 2010).

As pointed out by Hofstad and colleagues (2021), the exercise of leadership in local co-creation arenas must take the distinctive features of co-creation into account, not least the self-managing character of co-creation processes. If this is done and the co-creation leaders manage to avoid the twin pitfalls of over-steering and understeering, there is no contradiction between co-creation and leadership.

The co-creation leaders’ toolbox includes many tools, including: stakeholder analysis aiming to identify relevant and affected actors; the formulation of persuasive storylines that motivate participation; the issuing of formal mandates that empower the co-creation arena to foster innovative solutions; design-thinking tools to map problems and needs; conflict-mediation tools to constructively manage differences; engagement in boundary-spanning activities and construction of boundary objects that facilitate focused interaction; promotion of rapid learning through experimentation; soliciting of external inputs to combat the development of tunnel vision; the use of developmental evaluation that asks critical questions to stimulate mutual learning; process management that selectively activates passive participants and foster alignment; engagement in risk management; production of small wins; and more (Ansell et al., 2022; Crosby and Bryson, 2010; Hofstad et al., 2021). While this list is long, it is by no means exhaustive, as new tools are constantly invented to match challenging situations, deal with tensions, and maintain the participants’ commitment to the co-creation process and the emerging results.

For public leaders and managers to exercise co-creation leadership, several important conditions must be fulfilled (Sørensen and Torfing, 2009, 2017). First, the leaders of co-creation must be centrally placed in the network of actors involved in the co-creation process. A formal role is important but not enough since co-creation leaders must have prior contact to several of the leading actors to influence network decisions. Second, they must have authority in the sense that they are respected for the process skills by the participants in the co-creation process and are listened to when making suggestions about how to move forward. Third, they must have access to resources that enable them to host meetings, pay for excursions, and contribute towards the financing of final solutions. Fourth, they must have sufficient organisational backing to be able to monitor and evaluate the ongoing co-creation process and make informed reflections about how to intervene constructively to ensure progress towards goal achievement. Finally, they must be trained and experienced facilitators, as bringing together a diverse group of public and private actors in a process of collaborative innovation can be a genuine challenge.

Methodology: design experiments, interactive research, case selection, and data gathering

Design experiments have gained increasing popularity in the social sciences because they allow researchers to test the impact of discrete interventions in unique real-life settings through an iterative refinement of a prototypical design aimed at solving some context-related problems and challenges and, ultimately, to generate desirable results measured against a predefined goal (Stoker and John, 2009).

The procedure for conducting design experiments is relatively simple (see Kelly and Lesh, 2002; Cobb et al., 2003; Stoker and John, 2009). The first step in design experiments involves selecting a testing ground that allows the researchers to study the process and results they want to improve. For ethical reasons, consent from the case actors concerning their willingness to participate in the design experiment must be ensured. It should also be made clear that all interventions will be negotiated with and endorsed by the responsible actors in the case. Hence, the actors themselves will be responsible for making the agreed-upon interventions, albeit in close collaboration with the research team that will make theory- or experience-based suggestions for interventions.

The second step is to discuss and agree on the goal of the design experiment. What should be attained in and through the iterative design and redesign of the process or solution? When selecting the goal of a design experiment, it is important that it is something that can be clearly measured to determine the impact of one or more interventions.

The third step is to carefully diagnose the actual and emerging problems in terms of the problems and challenges preventing goal attainment and to design an intervention aiming to mitigate or remove the obstacles. A precise diagnosis must be made based on empirical observations, and a hypothesis must be formulated about what can be changed, removed, or added (the intervention) to enhance goal attainment.

The fourth step is to make the intervention in close collaboration with field actors and then measure its impact on the process and/or solution to see whether the intervention helped to alleviate the identified problems and challenges and bring the actors involved in the process closer to goal attainment.

The final step is to draw lessons by using a combination of theory and practical knowledge to explain why the interventions helped to remove the obstacles to goal achievement and by reflecting on the case-, sector-, and country-specific scope conditions for the impact of the interventions.

The whole procedure of diagnosing problems, designing a possible solution, making an intervention, and measuring its impact on goal attainment may have to be iterated a few times to obtain the pre-specified goal of the co-creation process.

One thing is the ideal procedure for carrying out design experiments – another is the complex and messy reality, where the steps in the process are less clear-cut, the measurement of the impact of interventions is difficult, and the responsibility for diagnosing problems and making interventions varies. Hence, in some cases, the researchers play a key role in suggesting and making appropriate interventions, while in others the researchers are either part of the project or merely observing practitioners developing and testing new formats by themselves with little researcher interference.

That said, it follows from the above procedure that researchers and practitioners must interact on a continuous basis to conduct a design experiment. Researchers must teach the practitioners the stepwise procedure in design experiments. Researchers and practitioners must agree on the over-all goal and purpose of the design experiment, jointly formulate a diagnosis of the problems and challenges, agree on an intervention, and plan its implementation, working together to measure the impact of the intervention against the jointly agreed goal and reflect on the possible learnings. Fostering agreement and engaging in joint orchestration of design experiments is challenging due to both the different frames of reference and unequal power relations between researchers and practitioners. Hence, misunderstandings and lack of critical feedback from the practitioners pose a serious problem that must be mitigated by a close and trust-based interaction.

While the purpose of the continuous researcher‒practitioner interaction is to facilitate the experimental improvement of real-life processes and results in line with traditional forms of action research, we prefer to describe the joint efforts as an “interactive research process”. Hence, it is a research process because it is a systematic procedure for producing in situ knowledge on what works when it comes to leading and managing co-creation, and it is interactive in the sense that researchers and practitioners work together based on a mutual respect for each other’s theoretical and practical knowledge. Hence, the researchers do not possess any privileged insight into the truth about what is going on that they try to convey to the practitioners to make them see the light; rather, the researchers and practitioners explore practical processes jointly to learn about how to remove the obstacles to goal attainment. In this sense, design experiments are instances of co-creation.

To investigate the potential of using experimentation as a tool for studying and improving the processes and results of co-creation, the EU Horizon project GOGOV conducted design experiments in different corners of Europe, including Croatia, Denmark, England, France, the Netherlands, and Wales. The six cases were selected based on advice from country-wise panels of researchers and practitioners who nominated interesting cases of local co-creation. We selected ongoing cases with a strong commitment to co-create innovative solutions, willingness to participate in the design experiment, and the absence of extraordinary circumstances that would render research results irrelevant for other cases (the cases are described in Appendix 1). In most of the cases, the public leaders’ wish to improve their strategic management of co-creation was a key motivator for the cases to accept our invitation to be part of our research. Each case study stretched over several months to allow time to plan interventions with practitioners, to carry them out, and to study and evaluate their impact. The studies all took place between May 2020 and December 2021. Case reports were drafted and validated in the spring of 2022.

To diagnose the problems and challenges in the co-creation processes and to assess the impact of the interventions, we collected and analysed available documents, observed meetings, repeatedly interviewed the leaders of and the participants in the co-creation processes, and administered mini surveys to the participants. Documents including mandates, agendas, minutes, reports, proposals, etc. helped us to understand what the case was all about, how the process was structured, and what the outcomes were. Observations of meetings helped us to evaluate the nature and character of the interactions, the ability of the actors to identify and define problems and find and implement novel solutions, and the efforts of the co-creation leaders to support the collaborative innovation processes. The interviews were semi-structured, recorded, and transcribed, and they were essential to identifying the obstacles to goal attainment and evaluating the impact of the interventions based on personal opinions. Finally, the use of mini surveys helped to develop a broader picture of the impact of the interventions. While the size of the cases and the number of participants create some variation, the collected data in the cases typically consisted of 15‒20 documents, 15‒25 interviews, 3‒6 observations, and 1‒2 mini surveys. All of the material was based on a joint research protocol. In addition to the collected data, the local design experiments were based on a series of meetings with the co-creation leaders to agree on an overall goal, create joint problem diagnosis, design interventions, and discuss their impact. Intercoder reliability was improved through the cross-coding of the data material, and the result of the data analysis was reported in a standardised report facilitating comparative cross-case analysis.

Both the co-creation processes and the data collection were seriously affected by the COVID-19 pandemic, which meant that physical meetings were replaced by online meetings and that many of the observations and interviews were made online. Moreover, the distribution of survey questionnaires at the end of meetings to get a high response rate was in many cases impossible, thus leaving us with much lower response rates than planned. To solve this problem, we increased the number of interviews and asked similar questions concerning the impact of different interventions.

Empirical findings

In all six cases, the goal of the design experiment was discussed in meetings between researchers and practitioners to ensure that everybody was aligned. The goal should either relate to the process or result of the co-creation process or a combination of the two. It should be realistic and allow the identification of problems and challenges hindering its achievement. In most cases, the formulation of the goals was motivated by the practitioners’ motivation to embark on the co-creation process, but the researchers played a key role in tying the goal to the co-creation process. Table 1 provides an overview of the goals of the six design experiments, all of which aim to promote the creation of public value through enhanced participation, improved quality, enhanced effectiveness, and the pursuit of innovation.

Table 1.

Overview of the goals to be attained in and through the six design experiments.

Case	Goal to be attained in and through the design experiment
City of Cardiff, Wales	The goal is to enhance community well-being through reduced stigma, improved confidence, lowered education barriers, and stronger interpersonal relationships
City of Rijeka, Croatia	The goal is to improve public service and utility quality in Rijeka by means of increasing the co-creation aspect of the Rijeka Local Partnership Program (RLPP)
Gentofte Municipality, Denmark	The goal is to facilitate the creation of an innovative solution (turning libraries into cultural hubs based on co-creation) that respects and retains the traditional functions of the extremely well-functioning libraries
Flevoland Region, the Netherlands	The goal is to help the participants in the Regional Work Chamber Flevoland (RWF) to foster innovative solutions to social and employment problems that respect the functions and responsibilities of the participating organisations
Newcastle upon Tyne, England	The goal is to enhance the effectiveness of the summer holiday camp organised by a local church parish in meeting the physical, entertainment, and emotional needs of children, many of whom come from disadvantaged backgrounds
Territory of Vitrolles, France	The goal is to enhance citizen participation in cultural policy and activities to build a new territorial identity and enhance social cohesion against a background of polarisation and fragmentation caused by rapid and forced development

With the different goals in hand, researchers and practitioners diagnosed problems and challenges, designed and crafted interventions aiming to overcome or mitigate the problems and challenges, and, finally, measured whether the interventions improved goal attainment. The results of these crucial steps in the design experiments are displayed in Table 2.

Table 2.

Overview of the diagnoses, interventions, and impacts in the six design experiments.

	Problem diagnosis	Intervention content	Intervention impact
City of Cardiff, Wales
Intervention #1	COVID-19 lockdown:The lockdown halted face-to-face co-creation activities at the Hillfort heritage site, demotivated school children, and enhanced social isolation	CAER organised self-managed “test pit” excavations of people’s backyards with the help of an online film and an illustrated guide. They also created activity packs for schoolchildren	Positive impact on the overall goals as 36 families participated and children worked with their parents, learned about cultural heritage, and communicated with neighbours. The activity delivered public value and promoted heritage. It won prizes and generated positive media attention
Intervention #2	Anti-social behaviour: A co-created playground at the heritage site was subject to anti-social behaviour and vandalism that had the potential to attract negative media attention and undermined the idea that the neighbourhood deserved good public facilities	A co-created response involved a meeting with the park ranger, the local high school, citizens, the local councillors, and local community police. They decided to speak to the young people involved to find out how they could collectively ensure that people would care for the communal space rather than wreck it	Positive impact on the overall goals, as the threat to the co-creation process ended up extending and strengthening it. Today, the Hillfort Heritage Center and the playground are used by many local community members. Being adjacent to the newly launched heritage/community centre means that the building itself is attracting locals and other visitors
City of Rijeka, Croatia
Intervention #1	Knowledge deficit:The lack of open-access data regarding past projects prevents local actors from obtaining a comprehensive picture of what the RPLP programme has done so far	The RPLP programme has created a data repository, which is part of the City of Rijeka e-Services portal. It has also built a GIS database that publishes data on a web map	Positive impact on the democratic legitimacy of future co-creation, since the availability of project descriptions combined with GIS data tends to assure a fairer spatial distribution of the local projects funded by the City of Rijeka
Intervention #2	Exclusion of citizen voices:The RPLP project selection committee consisted of 3 representatives of non-governmental organisations and the media, but ordinary citizens had no say in the selection process	Two active citizens with previous experience from RPLP projects will be added to the selection committee (starting in 2022)	Positive impact on the democratic legitimacy of co-creation in the future, as the democratic anchorage of the whole RPLP process is solidified
Intervention #3	Lack of formal evaluation:The RPLP programme lacks a formal evaluation procedure that assesses and compares the success and sustainability of the projects	A broad range of participants were involved in developing, testing, and refining a new framework for project evaluation in several stages and training people in its use	Positive impact on future co-creation projects in the RPLP programme since there is now a simple and effective procedure for project evaluation with a broad-based ownership
Gentofte Municipality, Denmark
Intervention #1	Present bias:Many participants prefer to maintain what they have got and avoid new and uncertain future developments	Change the format of a meeting to allow people to articulate what they want to maintain and if they may want to add something new	Positive impact on the participants’ motivation to try something new and to pursue innovation
Intervention #2	Over-steering:Facilitators took up lots of space in highly structured meetings and they were the nodal point in the star-shaped debates, thus reducing the space for free and open-ended discussion	Create a new type of open meeting with no fixed agenda, a more withdrawn facilitation, and plenty of room for brainstorming and discussion	Positive impact on the number of new, innovative ideas formulated by the participants
Intervention #3	Knowledge gap:Highly asymmetrical distribution of knowledge about the libraries among the participants	Organise site visits to all 6 libraries, where professional librarians provide information about the libraries	Unintended negative impact on the participants’ belief in their ability to propose new ideas and activities, despite massive knowledge transfer
Intervention #4	Talk-centrism:The meetings in the co-creation arena tend to be all talk and no action	Practical testing of a co-created idea of cultural events with local authors in one of the libraries	No discernible impact on the innovation ambition of the participants, due partly to failed implementation
Flevoland Region, the Netherlands
Intervention #1	Criticisms of collaboration:The common purpose was unclear, meetings were uninspiring, and there was tension between central goals and local needs	Allowing people to articulate what they want to maintain and what they want to change, eventually leading to the formulation of a 9-point plan	Positive impact on participant awareness of what can be achieved by engaging in collaborative governance innovation
Intervention #2	Overemphasis on structure:Endless debates about the governance structure prevented the participants from discussing policy content	Open meeting used to develop a new substantive agenda based on a well-prepared overview of past performance	Mixed/negative impact on the engagement of the participants, due partly to failed implementation of the intervention by an external bureau
Intervention #3	Limited ownership of agenda:The participants expressed a need for a stronger, shared ownership of their joint policy agenda and projects	Form mixed thematic sub-groups to discuss goals and ideas based on a review of current projects and white spots	Positive impact on collaborative engagement and creativity but disconnected from the review of ongoing projects and white spots
Intervention #4	Talk-centrism:There was a general, discouraging feeling that the meetings were all talk and little action	Hold board meetings focusing on practical implementation and the planning of concrete activities and next steps	Limited positive impact on the innovation ambition of the participants: mutual relations were strengthened but action decisions were postponed
Newcastle upon Tyne, England
Intervention #1	Food poverty as key problem:In the 2019 and 2020 holiday camps, it was discovered that hunger in holidays without school food was a problem and that proper meals for the schoolchildren rather than just snacks and a limited amount of food packages were needed	To provide proper meals for the participants in the 2021 summer holiday camp, the project enhanced its reliance on donations from private companies, citizens, other churches, and FareShare	Positive impact on meeting the physical needs of the children participating in the holiday camp
Intervention #2	Restricted food distribution:Relying on external public financing meant that food packages could only be given to certain families and not to all families in need	The project decided to enhance the reliance on volunteers and donations, employing only one seasonal employee to gain financial independence	Positive impact on the fulfilment of the physical needs of local families and the holiday camp participants due to enhanced self-management
Intervention #3	Low trust in external actor:In 2020, an external, national organisation was hired to do sports activities with the children, but low trust developed because it was not a team player in the local collaboration	Gaining financial independence through voluntarism and donations meant that collaboration of the external, national-level actor was no longer needed	Positive impact on the fulfilment of needs due to more effective planning, communication, and execution based on trust relations between local actors who know each other well
Territory of Vitrolles, France
Intervention #1	Defensive public actors:The city councillors and directors of cultural facilities took up a defensive posture in the co-creation workshops and, more generally, there was some confusion among the participants about their roles	Use of persons with different socioeconomic and cultural participation profiles provides a boundary object for the participants to jointly focus on when interacting (thus downplaying their different individual roles and interests)	Moderate positive impact on stimulating the co-creation of cultural policy solutions, as the actors gradually loosen up and participate in the discussions and end up finding solutions to the lack of cultural participation

Comparing the problems and challenges encountered in the different cases, it is striking how the problems are highly contextual and reflect different political and administrative attitudes towards co-creation as well as different local conditions. Hence, in the Territory of Vitrolles, the problems reflect a deep administrative reluctance to embrace co-creation as a governance tool, whereas in many of the other cases (e.g., Gentofte Municipality and the Flevoland Region), the problems are about how to improve the quality of the co-creation process and spur collaborative innovation.

The kind of problems and challenges that are encountered vary considerably but can be divided into four broad categories. First, several problems relate to the scope of participation; relevant and affected actors may be excluded or only involved late in the process (City of Rijeka). Some participants may be reluctant to participate and engage in discussions with other actors (Territory of Vitrolles). Participants may also feel that trust between the involved actors is low or that there is a lack of common purpose that reduces their commitment (Flevoland Region).

A second group of problems concerns the collaborative process itself. In two cases, the participants lack knowledge about the subject matter and what is going on (City of Rijeka, Gentofte Municipality). There are also problems with oversteering (Gentofte Municipality) and a predominance of discussions about structure and procedure over discussions of content. Two cases also bear witness to the risk of talk-centrism, where co-creation becomes all talk and little action (Gentofte Municipality, Flevoland Region).

A third group of problems relates to the generation of results that is hampered by the lack of financial autonomy (Newcastle) and lack of joint procedures for evaluation (City of Rijeka). In one case, the potential of generating better results increased by talking to the left-out stakeholders; that is, speaking to the young people and reducing the anti-social behaviours (City of Cardiff). The City of Cardiff case also shows how external events (e.g., the COVID-19 lockdown) may jeopardise the production of tangible results of co-creation; in fact, most of the cases struggled to produce results during the COVID-19 pandemic.

The problems emerging in relation to the attempt to co-create innovative public value solutions generally seem to be manageable. They do not completely block or undermine the co-creation process, and the leaders of the co-creation projects have managed to find countermeasures, often with active help from engaged researchers.

Comparing the content of the interventions aiming to counteract the emerging problems and challenges, it is remarkable how, for the most part, the interventions are incremental in the sense that they make small-scale changes to the modus operandi of the co-creation projects. Only the interventions in the City of Cardiff deviate from this pattern, as we here witness how problems are solved through a clever reinvention of parts of the project. Otherwise, the content of the interventions tends to fall into three different groups: One group of interventions aims to enhance participation by including a group of young people in the co-creation process (City of Cardiff) or adding two ordinary citizens to the selection committee (City of Rijeka). A second group of interventions aims to tinker with the process design. This is clear in Gentofte Municipality and Flevoland Region. Also, the decision to use the persona method in the Territory of Vitrolles introduces a process tool to foster a more wholehearted participation. Finally, some interventions aim to develop new governance tools in response to emerging problems. Examples of this strategy are found in the City of Rijeka (data repository and evaluation tool) and Newcastle (donations and voluntarism as funding sources).

None of the interventions seem to have been very demanding on the researchers and practitioners, and they all find theoretical support in the literature on network management (Kickert et al., 1997), empowered participatory governance (Fung, 2003), metagovernance (Sørensen and Torfing, 2009), collaborative innovation (Torfing, 2016), co-creation (Ansell et al., 2022), and design thinking (Stickdorn and Schneider, 2011), which highlights the importance of involving and motivating the right group of actors, improving process design and developing appropriate governance tools to support collaboration and drive the processes to successful conclusion.

Finally, when comparing the impact of the various interventions, it is noteworthy that a large majority of the interventions had a positive impact and help to solve the problem or challenge encountered, or at least mitigate its negative effect on the co-creation process. Only three of the 20 interventions had no impact or a negative impact. In two of these three interventions, the problem was the failed implementation of the intervention, while in the third, the negative impact was unintended and caused by the failure to anticipate what would happen.

Looking at the nature and character of the various impacts, the typical impact of the interventions seems to be enhanced support for and legitimacy of the co-creation process. There are also examples of service improvement (Newcastle) and service innovation (Municipality of Gentofte). In two cases, the result was better policy solutions (Flevoland Region, Territory of Vitrolles) and in several cases the fulfilment of citizens’ needs for services and public solutions (The City of Cardiff, City of Rijeka, Gentofte Municipality, Newcastle).

Lesson-drawing and reflections on the use of design experiments

Design experiments aim to produce new knowledge of what works in practice. This begs the question: What have we learned from the six design experiments beyond the general but important point that the problems encountered in co-creation processes are relatively minor and perfectly manageable? The following concrete lessons stand out when considering the six design experiments:

First, faced with severe externally imposed constraints, co-creation projects may reinvent themselves and robustly deliver on key goals by producing innovative solutions. The heritage project in Cardiff faced severe constraints, including the COVID-19 lockdown and anti-social behaviour of local youth vandalising a co-created playground. Instead of collapsing, however, the heritage project cleverly reinvented itself through the creation of self-managed “test pit” excavations of people’s backyards and the expansion of the range of actors involved in improving and protecting the neighbourhood. The obstacles were overcome with a positive outcome. According to Stoltz (1997), there are three typical reactions to adversity: quitters resign and leave the project, campers wait to see if conditions improve, and climbers aim to turn obstacles into opportunities through creative problem-solving. Research indicates that collaboration stimulates and helps to foster innovative solutions to problems and challenges (Torfing, 2016). As such, there seems to be solid backing for our empirical findings.

Second, countering the lack of formal evaluation of joint efforts to improve local governance and living conditions, actors may co-create a simple and effective procedure for evaluating collaborative projects. Hence, in Rijeka, the development of new urban co-creation projects was hampered by the lack of information about past projects, their content, performance, and impact. Moreover, ongoing projects lacked feedback that could stimulate performance improvement. The co-creation of a procedure for evaluating all stages of development solved these problems. The need to base co-creation projects on solid data and ensure the provision of critical feedback is highlighted by the developmental evaluation approach recommended by Patton (2010), and the need for keeping track of results in collaborative projects is underscored by Crosby and Bryson (2010). The collective impact methodology is also very insistent that co-creation must be supported by ongoing evaluation and the measurement of performance and impact, for example, through the use of a dashboard displaying results (see Kania and Kramer, 2011). Hence, the Rijeka experience is well-supported by scholarly insights.

Third, the common risk of oversteering co-creation processes can be significantly reduced by the introduction of an open meeting format with no fixed agenda, more withdrawn facilitation, and plenty of room for brainstorming and open-ended discussion among the participants. This is evidenced by the Gentofte case, where the facilitators seemed to take up much space and constitute the nodal point in star-shaped discussions with the participants, which prevents diamond-shaped, all-to-all communication facilitating learning and innovation. The Vitrolles case further showed how loosening the process with more down-to-earth activities benefits co-creation. The danger of oversteering is identified by Sørensen and Torfing (2009), and the need for facilitation and framing (rather than steering and imposition) is emphasised by scores of researchers who call for collective, relational, horizontal, and distributive leadership aimed at making room for participant-driven interaction (Bolden, 2010; Bolton et al., 2021; Kinder et al., 2021; Ospina, 2017). As such, the positive effect of a decentred leadership and the creation of opportunities for more self-organised interaction comes as no surprise.

Fourth, the lack of a common purpose among diverse actors engaged in co-creation processes can be countered by a new meeting structure that allows the participants to articulate what they want to maintain and what they want to change, and the input generated in this manner may be used to develop a joint plan. The co-creation process in the Flevoland Region shows that network meetings with long PowerPoint presentations by network leaders and endless discussions about structure rather than policy content lead to a lack of common purpose and joint ownership. However, the soliciting of participant inputs and the integration of their views and ideas can create momentum. This finding is supported by Young (2002), who warns us about the detrimental effects of the internal exclusion of actors in collaborative arenas, and by Skelcher and Torfing (2010), who insist that the active involvement of network actors is crucial for creating joint policy ownership. In line with this, the network management approach of Klijn and Koppenjan (2015) has repeatedly recommended the use of different tools to activate and integrate the participants in collaborative arenas. Changing the meeting format is one way of translating the resource interdependency between the manifold actors into an engaged dialogue capable of fostering a common purpose in networked policymaking.

Fifth, fiscal autonomy obtained through increased reliance on donations and volunteers can enhance the flexibility of co-creation projects and help them to better meet local needs. This is well-demonstrated by the Newcastle design experiment, where the broadening of the funding sources and less dependence on public money facilitated the expansion of activities and allowed more discretion in decisions on how to use the project funding. Interestingly, this finding is supported by new literature on the blended financing of co-creation projects aiming to achieve one or more SDGs (Andersen et al., 2021; Ansell et al., 2022; Pereira, 2017). The new literature stresses the importance of combining multiple funding streams instead of relying on public money that often falls short of the funding needs and imposes restrictions on the use of the funds.

Finally, the use of fictive personas as boundary objects in a co-creation process can help to engage reluctant participants in joint discussions of new and better solutions. This is the lesson from the co-creation process in the Territory of Vitrolles, where the city councillors and administrative directors from the municipality assumed a defensive posture, and many of the citizen participants were confused about their roles. The solution was to organise the discussion around fictive personas. This helped to shift the attention from the participants’ own private interests, feelings, and worries to a joint concern for what kind of cultural activities that different groups of citizens would like to have more of, which was easier to discuss because it was detached from the participants themselves. The research on technological design recognises the benefits of this type of externalisation technique (Hjalmarsson et al., 2015; Pruitt and Grudin, 2003), and the research on collaborative networks praises the use of boundary objects as a toll for generating a joint focus on how to achieve public value (Montin and Johansson, 2014).

When considering all these learnings, we should bear in mind that design experiments merely produce context-dependent knowledge of what works in real-life settings. What works in our six design experiments may not work in other contexts. That said, there is nothing to prevent learning obtained in one design experiment from being included in a “leading and managing co-creation toolbox” if the context-dependence of the new learnings is clearly specified, thus allowing leaders and managers in other contexts to reflect on the similarities and differences between their own context and situation and that from which the learning was drawn.

The studies summarised and discussed in this article show that design experiments work well as a research method in public administration studies focusing on co-creation. Diagnosing problems and barriers for achieving a particular goal in and through co-creation has allowed researchers and practitioners to formulate hypotheses about the kind of solution that could possibly alleviate the problem. Testing the hypotheses in practice by implementing the various interventions and assessing their impact has not only helped to reveal what works in practice, but also to understand the importance of thinking the interventions through to avoid unintended negative effects and to ensure proper implementation. In short, our study demonstrates that design experiments offer a near-perfect tool for advancing a positive public administration research that aims to find and enlarge solutions to pressing governance problems (Douglas et al., 2021).

That said, the experiences of the researchers engaged in the six design experiments point out some important preconditions for using design experiments in the public governance research. Hence, at various points, they struggled to fulfil four crucial demands. First, researchers must have proper access to the selected environment and be actively involved in formulating ideas for how to intervene to solve the diagnosed problems. Furthermore, researchers must assure that the interventions have solid theoretical backing and that testing them will contribute to testing or building theory. Second, the co-creation process should not be constrained by external events and regulations such as we have seen during the COVID-19 pandemic, which in some cases ruled out the use of optimal experimental designs. Third, researchers must be able to influence the implementation of the interventions to prevent implementation failure and enhance the chance of producing desired outcomes. Finally, researchers should be able to rigorously measure the impact of the interventions in a valid and reliable way. Failure to fully meet these demands (e.g., due to the impact of lockdowns and regulations during the pandemic) can somewhat reduce the validity of the results obtained in the studies.

Another important reflection concerns the use of design experiments to enhance co-creation research. On the negative side, we should not be blind to the high costs in terms of time and resources that are inherent to the use of design experiments as a research method. On the positive side, design experiments may help to identify real-life problems and obstacles that are not captured and described by existing research and thus contribute to expanding the stock of knowledge about how emerging challenges can be overcome through concrete interventions. In the long run, accumulating knowledge about what works in specific contexts and situations may facilitate the development of some generic tools that can be used in specific contexts and situations.

Finally, yet importantly, our evaluation of design experiments as a strategy for reaping the fruits of co-creation is positive. Focusing on emerging problems and challenges, ways of removing, circumventing, or mitigating the problems, and the actual impact of these interventions seems to appeal to the practitioners, who in most cases are highly concerned with improving processes and producing results. Hence, the interactive research strategy where researchers work closely together with practitioners to conduct design experiments may slowly but steadily stimulate the development of a self-critical reflexivity on the part of the practitioners through the internalisation of the design experiment method, which can be boiled down to three important steps: diagnose problems, make a well-argued intervention, and measure the impact vis-à-vis the stipulated goal.

Conclusion and further research avenues

After decades of focusing on the introduction of competition in the public sector through privatisation, outsourcing, and the commercialisation of public service production, there is growing interest in collaborative governance and its sub-species, co-creation. Co-creation aims to bring relevant and affected actors together in creative problem-solving processes where collaboration becomes a driver of innovation. While the co-creation literature is pregnant with promises highlighting the many benefits that can be achieved in and through co-creation (Brandsen et al., 2018), we know little about how we can reap the fruits of co-creation processes that go haywire due to the failure to motivate actors and build trust relations, the emergence of destructive conflicts, the lack of competence, the development of tunnel vision, or something else. The solution to this problem seems to be the exercise of hands-on or hands-off leadership that combines facilitation with institutional design (Hofstad et al., 2021). To avoid being stuck with some abstract theoretical guidelines about how to lead and manage co-creation processes and develop some concrete leadership tools that allow us to reap the fruits of co-creation, this article has reported the results of a series of design experiments in which researchers and practitioners have worked together to test the impact of different leadership strategies in real-life contexts to create in situ knowledge of what works in practice. The results of the analysis are: (1) that the problems emerging in co-creation processes are for the most part manageable; (2) the leadership interventions are relatively undemanding; and (3) the positive impact of most of the interventions finds support in the extant research literature.

The design experiments have usefully contributed to expanding the number of tools available to the leaders of co-creation. However, since design experiments are carried out in real-life testing grounds, the knowledge and insights that they produce are highly context-dependent and the prospect for generalisation non-existent. Still, the new tools in the co-creation leadership toolbox may provide inspiration for other leaders who are facing obstacles to co-creation and looking for ways of reaping the fruits of co-creation. As long as other leaders are prepared to carefully match context-dependent tools with their own context and, if necessary, adapt the tool they have discovered to the specific circumstances, there is no problem. Moreover, once we have accumulated a growing number of concrete experiences with leadership tools that work well in specific contexts, we may be able to observe some patterns and distil some general principles about how different problems in co-creation processes can be solved by using different kinds of tools in different contexts.

Further research is needed to advance the knowledge about how to reap the fruits of co-creation through the exercise of leadership. Such research may draw up an inventory of problems and challenges typically encountered in co-creation processes and link the problems and challenges with different sets of tools while specifying their scope conditions. Future research may also disentangle the tools that are based on hands-off process designs and tools based on hands-on facilitative leadership. Finally, it may seek to match concrete tools with existing theories of co-creation leadership. On a final note, design experiments have proven to be a useful tool for producing new learnings about how to lead and manage co-creation processes based on sustained interaction between researchers and practitioners. If the use of design experiments is to be further expanded, the procedure for how to conduct a design experiment may have to be refined and simplified, and the robustness of the experimental method in the face of procedural deviations must be further investigated.

Footnotes

Acknowledgements

This article is based on case studies conducted by the following principal investigators: Martin Kitchener (Cardiff), Josipa Cvelic (Rijeka), Eva Sørensen (Gentofte), Nicolette van Gestel (Flevoland), Jamie Harding (Newcastle) and Anais Saint Jonsson (Vitrolles).

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by The EU Horizon Project COGOV, grant number 770591.

ORCID iDs

Jacob Torfing

Mitja Dečman

Note

Appendix

Case	Case summary
City of Cardiff, Wales	Located on the Western fringe of Cardiff, 11 neighbourhoods in the Careau Ely area are home to around 27,000 people. Once a thriving post-war garden suburb, it went into decline when several local industrial employers, including a paper mill and brewery, closed in the 1980s. Today, it ranks among the most disadvantaged neighbourhoods in Wales. Formed in 2011, CAER is a collaboration between archaeological and historical researchers from Cardiff University, and community development specialists from the non-profit organisation Action Careau Ely (ACE). In 2016, the core team (two researchers and a development specialist from ACE) started the Hidden Hillfort Project, which aimed at reconstructing a derelict place of worship for community and heritage activities, alongside a playground for local children and trails around a local heritage site. CAER involved a broad variety of local actors in a 4-year-long, co-created “Activity Plan”. The actors included representatives from the Hidden Hillfort project team, heritage organisations, universities, primary and secondary schools, volunteers, and residents (including young people). Given that the design experiment was a method that appeared to be a part of how CAER operated, the COGOV partner decided to follow rather than engage directly in designing the experiments and studied the co-creation process over several months in 2021. The co-creation activities unfolded in a number of ways (e.g., in 5 working groups, 7 open days, and participatory activity-based consultations). The core group also conducted a community survey, a school survey, and a number of semi-structured interviews with non-users to collect important input to the process. There was a separate series of meetings designated to negotiate land ownership issues with a farmer and two divisions of the local authority
City of Rijeka, Croatia	The Rijeka Local Partnership Program (RPLP), initiated by the City of Rijeka, has supported more than 100 community initiatives since 2005, many of which are products of co-creation between a variety of public and private actors. The projects have included playgrounds/parks for children, green areas, and improvements to local schools. From 2005 to 2020, the total value of projects implemented is HRK 6.2 million (approx. $850,000), with 2.6 million from the City budget and 3.6 million as the contribution of the local community through donations and volunteer work in the projects. In 2020, when the local COGOV partners expressed interest in conducting design experiments to promote co-creation in the projects, RPLP formed an expert panel consisting of two COGOV researchers, a local government manager, an external expert consultant, and an NGO consultant with prior involvement in RPLP development and insight in participatory governance. The case under scrutiny was the expert panel, and the purpose of the interventions was to create detailed, contextual knowledge of what works when it comes to the strategic efforts of public actors to spur co-creation. Over the next year, the panel met regularly to identify current obstacles to co-creation and to discuss and test ways to further improve the RPLP programme
Gentofte Municipality, Denmark	Gentofte Municipality and its 75,000 inhabitants are located in a wealthy suburb north of Copenhagen, Denmark. In 2015, headed by its highly entrepreneurial and committed mayor and municipal CEO, the City Council decided to change how the politicians prepare and develop new policies. The City Council found that they spent too much time doing casework in political committees rather than developing policies and that they needed input to policy development from the citizens and other local actors. They decided to introduce a new type of political task committee in which politicians prepare a policy proposal in dialogue with a group of local actors. Evaluations of the by now 39 Task Committees declare them a success, albeit admitting that problems do tend to occur along the way. The Danish research team and the local leadership agreed to carry out a design experiment aiming to try out ways to overcome some of these problems and selected the committee tasked with developing the local libraries into cultural hubs for the purpose. A Task Committee in session from Nov 2020 to Oct 2021 consisted of five politicians and 10 carefully selected citizens. There were eight ordinary meetings, one open meeting, three on-site meetings, and a number of meetings in three working groups. Most of the meetings were online. The Task Committee presented its policy proposal to the City Council in Nov 2021. The proposal was formally endorsed shortly thereafter
Flevoland region, The Netherlands	In 2013, the national government and the social partners (employer organisations and trade unions) took a series of new initiatives aiming to create more jobs for disadvantages citizens laid down in a Social Agreement in the Netherlands. A national Work Chamber with 35 regional counterparts was set up as a coordinating platform to implement the Social Agreement. Each Chamber includes the national and local public agencies for social benefits, social assistance and public employment services, and the social partners. The design experiment focused on the Regional Work Chamber of Flevoland (RWF). Some of the lead actors were worried about the level of collaboration in the Chamber and contacted the local COGOV partner. In Nov 2020, the researchers and the lead organisation decided to carry out a design experiment to promote co-creation between the Chamber members
Newcastle upon Tyne, England	The Church of England parish system makes it difficult for churches to carry out some activities. Each parish has its own full-time minister. Despite frequent initiatives designed to encourage cross-parish work, parish boundaries remain difficult to break down, with many church members seeing their parish church as working quite separately from others. In 2021, the Parish Church of Thyme organised a holiday camp with leisure activities for local children. It involved two parishes, a Christian organisation called Scripture Union, and Fare Share, which provided food at low cost. Moreover, a private firm donated bread, and a local school loaned a minibus free of charge to transport the children to and from the club. A number of other individuals also donated food. The co-creation of the holiday camp itself lasted a week in Aug, but the securing of funding started months earlier and the practical preparations lasted several weeks
Territory of Vitrolles, France	In the 1970s, a rapid and forced growth from 5000 to 35,000 inhabitants in the town of Vitrolles in Southern France produced a fragmented community with a weak local identity, intense social and political conflicts, and deep social divisions. This contributed to the right wing populist party “Front National” coming to power in the years 1997‒2002. Since 2002, when a left-wing government regained power, efforts have been made to develop a more positive image of the city and to develop social cohesion and a sense of belonging among the inhabitants by involving them in co-created cultural activities. The municipality did already have some experience with involving citizens in co-creation processes but wanted to find a way to involve more young people and recent newcomers to Vitrolles. For that purpose, the municipality and the research team decided to work together to find ways to involve these actors in co-creation processes and to overcome the problems that occurred along the way

References

Andersen

Hansen

Rand

(2021) Evaluating Financial and Development Additionality in Blended Finance Operations. OECD Development Co-operation. Working Paper #91 .

Ansell

Bartenberger

(2016) Varieties of experimentalism. Ecological Economics 130: 64–73.

Ansell

Gash

(2008) Collaborative governance in theory and practice. Journal of Public Administration Research and Theory 18(4): 543–571.

Ansell

Torfing

(2021) Public Governance as Co-creation: A Strategy for Revitalizing the Public Sector and Rejuvenating Democracy. Cambridge: Cambridge University Press.

Ansell

Sørensen

Torfing

(2022) Co-creation for Sustainability: The UN SDGs and the Power of Local Partnerships. Bingley: Emerald Publishing Limited.

Blok

(2013) Urban green assemblages: an ANT view on sustainable city building projects. Science and Technology Studies 26(1): 5–24.

Bolden

(2010) Distributed leadership in organizations: a review of theory and research. International Journal of Management Reviews 13(3): 251–269.

Bolton

Logan

Gittell

(2021) Revisiting relational coordination: a systematic review. The Journal of Applied Behavioral Science 57(3): 290–322.

Bos

Brown

(2012) Governance experimentation and factors of success in socio-technical transitions in the urban water sector. Technological Forecasting and Social Change 79(7): 1340–1353.

10.

Brandsen

Honingh

(2016) Distinguishing different types of co-production: a conceptual analysis based on the classical definitions. Public Administration Review 76(3): 427–435.

11.

Brandsen

Steen

Verschuere

(2018) How to encourage co-creation and co-production: some recommendations. In: Brandsen

Verschuere

Steen

(eds), Co-production and Co-creation: Engaging Citizens in Public Services. New York: Routledge, 299–302.

12.

Cobb

Confrey

DiSessa

, et al. (2003) Design experiments in educational research. Educational Researcher 32(1): 9–13.

13.

Crosby

Bryson

(2010) Integrative leadership and the creation and maintenance of cross-sector collaboration. The Leadership Quarterly 21(2): 211–230.

14.

Dente

Bobbio

Spada

(2005) Government or governance of urban innovation? disP - The Planning Review 41(162): 41–52.

15.

Douglas

Schillemans

‘t Hart

, et al. (2021) Rising to Ostrom’s challenge: an invitation to walk on the bright side of public governance and public service. Policy Design and Practice 4(4): 441–451.

16.

Fung

(2003) Survey article: recipes for public spheres: eight institutional design choices and their consequences. The Journal of Political Philosophy 11(3): 338–367.

17.

Gloor

(2006) Swarm Creativity: Competitive Advantage through Collaborative Innovation Networks. Oxford: Oxford University Press.

18.

Gray

(1989) Collaborating: Finding Common Ground for Multiparty Problems. San Francisco: Jossey-Bass.

19.

Hjalmarsson

Gustafsson

Cronholm

(2015) Exploring the use of personas in user-centered design of web-based e-services. In: Proceedings of the iConference 2015 Proceedings, Newport Beach, CA, USA, 24–427 March 2015.

20.

Hofstad

Sørensen

Torfing

, et al. (2021) Leading Co-creation for the Green Shift. Public Money & Management. e-publication ahead of print.

21.

Hood

(1991) A public management for all seasons? Public Administration 69(1): 3–19.

22.

Huxham

Vangen

(2013) Managing to Collaborate: The Theory and Practice of Collaborative Advantage. Abingdon: Routledge.

23.

Kania

Kramer

(2011) Collective impact. Stanford Social Innovation Review 9(1): 36–41.

24.

Kelly

Lesh

(2002) Understanding and explicating the design experiment methodology. Building Research Capacity 3: 1–3.

25.

Kickert

WJM

Klijn

Koppenjan

JFM

(1997) Managing Complex Networks: Strategies for the Public Sector. London: Sage Publications.

26.

Kinder

Stenvall

Six

, et al. (2021) Relational leadership in collaborative governance ecosystems. Public Management Review 23(11): 1612–1639.

27.

Klijn

Koppenjan

(2015) Governance Networks in the Public Sector. Abingdon: Routledge.

28.

Montin

Johansson

(2014) Understanding innovative regional collaboration: metagovernance and boundary objects as mechanisms. In: Ansell

Torfing

(eds), Public Innovation through Collaboration and Design. London: Routledge, 124–142.

29.

Ongaro

Sancino

Pluchinotta

, et al. (2021) Strategic management as an enabler of co-creation in public services. Policy & Politics 49(2): 287–304.

30.

Osborne

Strokosch

(2013) It takes two to tango? Understanding the co‐production of public services by integrating the services management and public administration perspectives. British Journal of Management 24(S1): S31–S47.

31.

Ospina

(2017) Collective leadership and context in public administration: bridging public leadership research and leadership studies. Public Administration Review 77(2): 275–287.

32.

Patton

(2010) Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. New York: The Guilford Press.

33.

Pereira

(2017) Blended Finance: What it Is, How it Works and How it Is Used. London: Oxfam International.

34.

Pollitt

Hupe

(2011) Talking about government: the role of magic concepts. Public Management Review 13(5): 641–658.

35.

Prahalad

Ramaswamy

(2004) Co-creation experiences: the next practice in value creation. Journal of Interactive Marketing 18(3): 5–14.

36.

Pruitt

Grudin

(2003) Personas: practice and theory. In: Proceedings of the 2003 Conference on Designing for User Experiences, San Francisco California, 6–7 June 2003, pp. 1–15.

37.

Romme

AGL

Meijer

(2020) Applying design science in public policy and administration research. Policy & Politics 48(1): 149–165.

38.

Skelcher

Torfing

(2010) Improving democratic governance through institutional design: civic participation and democratic ownership in Europe. Regulation and Governance 4(1): 71–91.

39.

Sørensen

Torfing

(2009) Making governance networks effective and democratic through metagovernance. Public Administration 87(2): 234–258.

40.

Sørensen

Torfing

(2017) Metagoverning collaborative innovation in governance networks. The American Review of Public Administration 47(7): 826–839.

41.

Stickdorn

Schneider

(2011) This Is Service Design Thinking: Basics, Tools, Cases. Amsterdam: BIS Publishers.

42.

Stoker

John

(2009) Design experiments: engaging policy makers in the search for evidence about what works. Political Studies 57(2): 356–373.

43.

Stoltz

(1997) Adversity Quotient: Turning Obstacles into Opportunities. Hoboken: John Wiley and Sons.

44.

Torfing

(2016) Collaborative Innovation in the Public Sector. Washington, DC: Georgetown University Press.

45.

Torfing

Sørensen

Røiseland

(2019) Transforming the public sector into an arena for co-creation: barriers, drivers, benefits, and ways forward. Administration & Society 51(5): 795–825.

46.

Torfing

Ferlie

Jukić

, et al. (2021) A theoretical framework for studying the co-creation of innovative solutions and public value. Policy & Politics 49(2): 189–209.

47.

Torfing

Sørensen

Breimo

(2022) When Norway met co-creation: the import, diffusion, and onboarding of a magic concept in public administration. International Public Management Journal 1–20. DOI: 10.1080/10967494.2022.2128127

(2015) A systematic review of co-creation and co-production: embarking on the social innovation journey. Public Management Review 17(9): 1333–1357.

(2012) Value co‐creation through collective intelligence in the public sector: a review of US and European initiatives. Vine 42(2): 251–276.

50.

Young

(2002) Inclusion and Democracy. Oxford: Oxford University Press.