What makes public sector innovations survive? An exploratory study of the influence of feedback,accountability and learning

Abstract

The question whether public sector innovations last, and what determines their chances of survival, remains a gap in the public management literature. This exploratory study focuses on the winners and nominees of public sector innovation awards in Belgium, France, the Netherlands, Romania, Slovakia and the UK. Through a survey covering 220 cases, it examines whether feedback loops, accountability mechanisms and learning processes (FAL) can explain the survival of public sector innovations. The conclusion is that a culture of feedback, accountability and learning seems to be positively linked with the survival of innovations.

Points for practitioners

It is one thing to innovate, but it is another to make innovations last. A culture of feedback information, learning processes to interpret this feedback information, and a culture of accountability seem to improve the chances of innovations to stand the test of time. Instruments to measure innovation’s performance on its own do not seem to affect innovation’s survival chances.

Keywords

continuous improvement innovation public sector survival sustainability

Introduction

One of the many gaps in the public sector innovation literature is addressed in this article: a focus on the non-survival of public sector innovations. Pollitt (2011) and De Vries et al. (2016) both mention the lack of research conducted on innovations after their initiation. ‘Much of the research on innovation has … focused on the early days – on the moment of innovation itself’ (Pollitt, 2011: 42). Therefore, new research is suggested to focus on questions such as ‘What proportion of administrative innovation is short-lived?’ (Pollitt, 2011: 42). Few examples of research on the survival-rate of public sector innovation exist. One of them was conducted by Pollitt et al. (2007). They found that over 60 percent of their respondents did not know what had happened to ‘good practices’ which had been presented at a conference only 2 years prior. Borins (1998) and Farah and Spink (2008) find that about 10 percent of the innovations in their samples have been terminated since they were awarded. None of these studies, however, goes into the reasons why these innovations did not survive.

This article focuses on the question of which factors might be linked to the survival and non-survival of public sector innovations. Considering its exploratory nature, this article draws up a first possible theoretical framework based on organizational characteristics, to explain the survival of innovations. This framework focuses on three concepts: Feedback, Accountability and Learning (FAL for short). It is hypothesized that the extent to which organizations (a) gather feedback information on the performance of their innovations, (b) are subject to accountability mechanisms, and (c) use this information through learning processes to improve the innovations, will influence the survival chances of innovations. FAL thus focuses on managing and improving the innovation after its initiation. The main question can be formulated as followed:

To what extent can feedback, accountability and learning explain the survival of public sector innovations?

In this article the focus lies on the survival of innovations (temporal sustainability), and not, for example, on the survival of their outcomes. Innovation awards are used as the case selection method. Award schemes form an accessible, in-depth, innovation-focused source for public sector innovation research.

A database of 220 nominees and winners of public sector innovation awards in Belgium, France, the Netherlands, Romania, Slovakia and the United Kingdom was created to investigate the hypothesized link between FAL and innovation survival. Surveys sent to the organizations in which the innovation found their origin measured the extent to which feedback, accountability and learning were present in the organization, and investigated the life of the innovations. Control variables included organization size, country of origin, innovation type, GDP, Hofstede’s cultural dimensions and governmental level of the organization. The results indicate that there is indeed a link between the culture of FAL and the survival of innovations. Items measuring FAL instruments, performance measurement systems for example, are not linked to survival.

Theoretical framework

Although most public sector innovation research focuses on the early stages of innovation development and implementation, research has also been carried out on what happens past this point. First, there is a considerable body of work on the diffusion and adoption of innovations (e.g. Damanpour and Schneider, 2006; Lapsley and Wright, 2004; Walker, 2006). Second, the effects of innovations past their initiation on organizational performance have been studied (Arundel et al., 2015; Damanpour et al., 2009; Hughes et al., 2011; Torugsa and Arundel, 2016), but not with a focus on the survival of the innovations. Since the determinants of public sector innovation survival have not been studied yet (to the best of our knowledge), there is little to no directly related previous work to base oneself on in deciding which explaining variables might be relevant. In this article, based on a previous review of the literature (Frees et al., 2015), the focus lies on three concepts believed to be potentially of importance in explaining the survival of innovations: feedback, accountability, and learning (FAL). After briefly introducing these three concepts, the potential explanatory model and hypothesis will be elaborated upon below.

Feedback

Feedback information allows an organization to correct its errors, to adjust its goals, to restore its performance levels, and to align itself with its environment (Down, 1966; Katz and Kahn, 1978). One stream of thought concerning feedback information focuses on a closed signaling loop of cybernetic systems. Its main principle is this: if a system is capable of obtaining feedback information about the outcomes and effectiveness of its actions through data gathered from its environment, it is capable of correcting its errors and improving its overall functioning (Morgan, 2006).

A strongly related concept is that of the performance gap. Whenever the performance level of the organization drops below the norm, observed through the above-mentioned cybernetic process, the organization will be motivated to implement changes (Walker, 2013). Performance gaps can exist both internally (efficiency) and externally (effectiveness and appropriateness) (De Peuter, 2011). Feedback about the internal design of the organization is preoccupied with techniques and making techniques more efficient. Could the organization do what it’s doing in more productive ways, do it cheaper, or use alternative methods for the same objectives? Other forms of feedback are more concerned with the functioning of the system in relation to its changing environment. Attention will then be focused on the societal needs and the societal effects of policies (De Peuter, 2011; Katz and Kahn, 1978).

In short: feedback could be the foundation for the constant fine-tuning of public sector innovations, laying the groundworks for a long and sustainable life.

Accountability

Accountability mechanisms, the public nature of account-giving and the possibility of sanctions may provide incentives for public officials to implement changes to improve the performance of their organization (Bovens et al., 2008). However, an accountability regime which focuses too harshly on mistakes and sanctions may discourage change (Van Loocke and Put, 2011).

Central to the concept of accountability is the idea that when decision-making power is transferred from a principal to an agent, there must be a mechanism in place for holding the agent accountable (Lindberg, 2013). Accountability can be defined as a relationship between an actor and a forum, in which the actor has an obligation to explain and justify its conduct to the forum, in which the forum can pass judgment, after which the actor may face consequences (Bovens et al., 2008). In this article, the focus lies on two examples of accountability: external audit offices and ombudsmen.

We hypothesize that an organization which will be held accountable, just as individual employees who hold themselves accountable through a sense of responsibility for example, will strive to search for opportunities to optimize its performance. Such an organization has an incentive to constantly improve, update and adjust its innovations, further strengthening its survival chances for the future.

Learning

An organization which is characterized by a learning culture has an open and receptive attitude towards different opinions and alternative ways of doing things, and has a tolerance for errors and risk-taking. At the organizational level this is supplemented with structural arrangements which allow organizations to process relevant information as a basis for change and innovation (Greiling and Halachmi, 2013). Personal factors play the largest role in our theoretical model when it comes to learning (Katz and Kahn, 1978; Morgan, 2006). Nevertheless, organizational characteristics can greatly influence the learning behavior of individuals (Shrivastava, 1983). Top management, for example, can form both a catalyst and a barrier to learning (Döös et al., 2015). Closely connected is the question whether the same top management is willing to invest in structural, procedural arrangements to facilitate learning amongst its employees (Garvin et al., 2008).

Finally, ‘learning’ can encompass three different types of learning: single loop, double loop and deutero learning. Single loop learning is closely related to cybernetic systems: an organization investigates its effectiveness, and adjusts its functioning if it finds a discrepancy. If the organization looks deeper, however, it may find that what needs to change is not the program’s functioning, but the program’s underlying norms and assumptions. This is described by Argyris and Schön as ‘double-loop learning’ (1978). The organization may, however, reflect even further. We can reflect about what prevented us from seeing that the system needed changing in the first place. This third level is called ‘deutero learning’. It entails an institutionalized capacity to reflect on the learning process itself (Argyris and Schön, 1978).

Given the feedback information provided by feedback instruments, and accountability creating an incentive to excel, learning is the last ingredient for organizations to improve the functioning of their innovations. Organizations interpret the information through learning and draw consequences from it, leading to concrete action. In this way it is hypothesized that learning will contribute to the improved survival chances of innovations.

Interactions within FAL

The three concepts discussed above are interlinked. Some authors suggest that accountability mechanisms may help to encourage and promote learning in pursuit of continuous improvement in public management and policies, since they create an incentive for organizations to learn and improve its functioning (Bovens et al., 2008; Schillemans et al., 2013). In the accountability literature, it is argued that a public accountability arrangement, if organized in an appropriate way, confronts public managers on a regular basis with feedback information about their own organization (Bovens et al., 2008). Accountability also stimulates organizations to find feedback information themselves to prevent critical accountability reports, and not merely rely on the feedback information they gather from accountability mechanisms (Pollitt et al., 1999). Finally, learning and feedback interact as well. In a nutshell: feedback information (in whichever form, and from whichever source) could start a learning curve by pointing out possible improvements. At the same time, if a learning culture is embedded in the organization, a need for feedback information will arise consequentially.

Combining these three concepts creates a comprehensive model to test as the explanation of innovations’ survival. Considering their interlinked nature, it is possible for the three to be correlated in the empirical results. They are, however, at least intellectually separate.

Hypothesis and heuristic model

Based on the above, it is hypothesized that the extent to which FAL is present in an organization might predict an innovation’s survival. FAL exists separately from the innovation, however: it is present in the organization before, during and after the innovation’s existence. The influence of FAL on innovations is seen as a cyclical process of (a) feedback information on performance, (b) pressures from accountability mechanisms to excel, and (c) learning processing this information. This is then followed by (in)decisions on whether to continue, alter or stop the innovation. The FAL-process is depicted as a heuristic model in Figure 1.

Figure 1.

Heuristic model of FAL process.

After an innovation has become operational, the FAL dimensions will lead to an explicit decision about its future (maintain/reform/terminate) or, in other cases, the innovation will wither away. After a decision has been made to change or maintain the innovation, the FAL dimensions will again influence its future, as they are factors which constantly influence all processes, products and services. The process depicted in Figure 1 is therefore, theoretically, never ending. Ideally, one would use a longitudinal research design in order to investigate the effect of FAL on the process after the innovation’s implementation. For this study, however, FAL will be measured at one instance: at the time of the survey. We thus treat the time between the award or nomination of the innovation and the survey as one FAL cycle (‘Assumed Observation’ in the figure). It is possible that FAL has changed since the inception of the innovation and the decision about its future, and that is has undergone multiple cycles. The results of the analysis should be interpreted in the light of these limitations. However, considering the exploratory nature of this study and the difficulty of designing longitudinal innovation research in the public sector, it is a useful point of departure to investigate its influence on the survival of innovations.

Operationalization and methodology

To investigate the survival of public sector innovations, public sector organizations from six European countries were surveyed who have been awarded for their innovations between 2003 and 2013. This survey focused on the life of the innovation since its award, and measured to what extent FAL was established in the organization. Before turning to the results, the operationalization of the dependent, independent and control variables will be discussed, as well as the research sample and the pros and cons of the innovation award method.

Innovation awards as selection method

On several occasions award winners and nominees have been used as proxies for public sector innovations (for example: Bernier et al., 2015; Borins, 2001b, 2008; Glor, 1998). Although this method has been criticized in the past (see Borins, 2001a, for a discussion), we see it as an apt proxy for public sector innovation, with several advantages. First, it moves the focus from the organization to include the innovation itself. Second, a quality check on the cases is carried out by the award assessors in the jury. Third, the project description and contact information provide a good starting point for further research. Finally, many different definitions of innovation exist in the academic literature. This article lets practitioners themselves decide what innovation is, checked by a jury, instead of formulating a definition ourselves based on conceptual arguments. By side-stepping the conceptual discussion on innovations, at least for now, we end up with a definition which is potentially much closer to the definition used by practitioners.

The award schemes used in this article, summed up in Table 1, all consist of a jury determining which cases are innovative, and thus suitable to be nominated for the award. Although the awards are different in their focus (innovation, quality, good practices), they all share the fact that innovation is a central criterion in selecting nominees. Likewise, all awards are open for all types of public sector organizations.

Table 1.

Good practice sources.

International sources:
European Public Sector Awards
CAF Good Practices Database
Quality Conferences for Public Administrations in the EU
United Nationals Public Service Awards
RegioStars Awards
Belgian sources:	French sources:
– Quality Conferences for Public Administrations in Belgium – Belgian eGovernment Awards	– Victoire acteurs publics prix – Interconnectes France
Dutch sources:	Romanian sources:
– KING Best Gejat Prijs – Pink Roccade	– National Agency of Civil Servants Conference – National Association of Public Sector IT Specialists – Romanian Prize for Quality – J.M. Juran Foundation – Parliamentary Committee for IT and Communications
Slovakian sources:
– Slovak National Quality Prize
UK sources:
– Governance International

Dependent variable

The main dependent variable, survival of innovations, is defined as the continuation of the innovation, either in its original or in an altered form (Savaya and Spiro, 2012). To measure the innovations’ survival, respondents were asked to describe the current status of the innovations as either:

Still operational in its original form.

Still operational, but altered.

Not operational anymore, actively stopped.

Not operational anymore, not actively stopped (withered away/disappeared).

I don’t know.

The answers were derived from Hogwood and Peters’ (1982) work on policy termination. The first two answers are considered as survival. Answers (c) and (d) constitute non-survival. Respondents answering (e) were removed from the dataset.

Independent variables

Feedback, accountability and learning are complex concepts. We have operationalized FAL based on several sub-topics, listed in Table 2. Each sub-topic had several specific survey-times attached to them. The questions are based on previous work carried out by Garvin et al. (2008), and are based on a previously conducted review of the literature (Frees et al., 2015; Van Acker and Bouckaert, 2016).

Table 2.

FAL sub-topics.

Feedback
Active search for and processing of feedback information
From staff
From customers
From ombudsmen
From internal audit
From external audit
From evaluations
Systems to monitor performance
Evaluations of goals and objectives
Quality management systems for continuous improvement
Knowledge management
Deliberate measurement practices
Accountability
Information and reporting
Debate, explanation and justification
Possibility of sanctions
Responsibility for performance
Transparency about performance
Subject to ombudsman review
Subject to external audit
Focus of external audit
Learning
Using the feedback information
From staff
From customers
From ombudsmen
From internal audit
From external audit
From evaluations
Psychological safety / Transparency and culture of adversarial debate and openness for alternative perspectives
Tolerance for errors, risk-taking and experimentation
Time for reflection – slack learning
Diversity of staff’s educational background

Each of these sub-topics is measured through one or more survey items on Likert-scales (1–5), and a few yes/no questions¹ (with a score of 1 for a ‘no’ answer, and 5 for a ‘yes’ answer). Per FAL dimension these scores are added up at the end of the survey. This gives every organization a total score for each of the three main concepts. This sum is then divided by the highest possible score for that factor, creating a scale from 0.2 (if all items receive a 1 on the Likert scale) to 1 (if all items receive a 5). A final score of 1 would indicate very strong feedback, accountability and/or learning mechanisms, whereas 0.2 would indicate the opposite (see Van Acker and Bouckaert, 2016, for the full survey). The hypothesis is that higher scores would lead to a higher chance of innovation survival. As mentioned earlier, an exploratory factor analysis will be carried out to test whether these factors were indeed measured by our survey.

Control variables

Since there is little research on the survival of innovations, we control for factors found to be influential in the innovation process in general. There will be controls at the organization, innovation and country levels. Considering the modest sample size (N = 220, see Table 3), it is necessary to be conservative in the number of variables to add to the analysis (Hosmer and Lemeshow, 2000). The control variables are: country of origin, Hofstede’s cultural dimensions, GDP per capita, organization size, governmental level of the organization, the year of the award/nomination ceremony, and innovation type.

Table 3.

Case overview.

	BE	FR	NL	RO	SK	UK	Total
Identified awarded cases	97	470	34	53	28	163	845
Valid responses	72	79	23	19	13	14	220
Surviving cases	63	73	23	16	12	13	200
Non-surviving cases	9	6	0	3	1	1	20

National innovation culture could have a significant impact on what happens to public sector innovations after their initiation. However, little is known about the innovative culture of national public sectors. It is often assumed that the administrative tradition of a country could determine its innovativeness, although this has not been exhaustively researched (Bonsón et al., 2012). The sample of countries in this article is therefore diverse. Belgium and France are considered Napoleonic administrative traditions, the UK an Anglo-Saxon tradition, and the Netherlands a Germanic tradition (Kickert, 1997). Romania and Slovakia are added to this as former Soviet nations, with still developing administrative cultures.

How is the national innovation culture measured? The characteristics of individuals have often been used to form aggregate measures of national innovations culture. Kasaa (2013), for example, uses Hofstede’s dimensions of culture² (Hofstede, 2001), and finds that the cultural dimensions combined explain the differences in countries’ innovation performance quite well. It should be noted, however, that these findings are based on private sector innovation indicators, and that the respondents were not just public servants. Kasaa’s findings can therefore only serve as a remote indicator for public sector innovation. It was decided to include Kaasa’s combined scores for innovation culture of the six countries in our sample as control variables for innovation culture. As a final national factor we control for GDP per capita, as this is repeatedly found as an important factor in innovation research (Arundel et al., 2015). This variable is based on data from Eurostat (2016), and uses the average GDP per capita between 2003 and 2014.

At the organizational level we will control for the size of the organization, as larger organizations are expected to have more resources to sustain their innovations (Rogers, 1983). At the innovation level we will control for the innovation type (Damanpour et al. 2009; De Vries et al., 2016) and the year in which the award ceremony took place. The typology of innovations will be based on De Vries et al.’s (2016) systematic review of the literature, although slightly altered, differentiating between service/product, administrative, technological and governance innovations. The age of the innovations will be coded as the time between the year of the award ceremony and the year of the survey.

Survey methodology and sample

The survey was first designed in English, and subsequently translated into Dutch, French, Slovakian and Romanian, each time by team members who were native speakers. Suitable respondents were found using the information gathered via the award programs. If the contact person mentioned was no longer working in the organization, we asked to be directed to a different employee with in-depth knowledge of the innovation. Innovations awarded or nominated before 2003 were not considered. The surveys in Belgium, France, the Netherlands and the UK were sent digitally, whereas the Slovakian and Romanian surveys were filled in on paper copies. Finding enough respondents proved to be a time-, energy- and resource-consuming activity. Since not all teams had the same budgets at their disposal, and due to the different number of yearly nominees per award, the number of cases and response rates differ strongly between countries.

The year of the award has been used as a proxy for the age of the innovation. Although this brings along a certain degree of inaccuracy, this data was available for a large majority of the innovations, whereas the exact year of the ‘birth’ was not.

One would intuitively expect that more recently awarded innovations would be more likely to exist than those initiated earlier on. At the same time the sample is likely to be skewed towards more recent innovations since it is difficult to find innovations which have been awarded long ago. Figure 2, however, shows that this skewedness is not present in our sample, mostly due to a large number of Belgian BKC cases in 2003, 2005 and 2007. Controlling for this issue in the analysis will determine what the effect of it is on the survival chances of innovations. Figure 3, finally, displays the number of cases per award program.

Figure 2.

Age distribution.

Figure 3.

Distribution over award programs.

Results

As a first step, an exploratory factor analysis was carried out to test whether the survey indeed measured the independent variables: feedback, accountability and learning. As many questions in the survey were dependent on having either an internal audit office, external audit office or ombudsman assigned to the respective organizations, many cases had missing values on those subsequent items if they had none assigned to them. This caused problems for the factor analysis since pairwise exclusion was used for missing values, deleting many cases from our sample, and leaving a sample far too small. It was decided to delete the most ‘damaging’ items from analysis: items focusing on the impact of audit and ombudsman institutions. Afterwards, the KMO-value was found to be .855, exceeding the recommended value of .6 (Kaiser, 1974). Bartlett’s Sphericity Test was significant at the 1 percent level. The exploratory factor analysis, with maximum likelihood as extraction method and promax rotation, revealed the presence of 10 factors with eigenvalues over 1, for 38 variables. Loadings under a score of .5 were deleted from the respective factors. The first factor explained 27 percent of the variance, with a drop to 8 percent for the second factor. Considering that the third factor (6.2 percent of variance explained) contained items spanning different parts of the survey, it was decided to continue with both the first and the third factors in further analysis. These two were thought to be of most intellectual interest, even though the third factor had a low percentage of explained variance. The results for these two factors of the factor analysis are presented in Table 4. A closer look at the variables grouped together revealed that these contained cultural variables in feedback, accountability and learning on the one side, and internal instrument variables (procedures, systems, etc.) on the other side. The Cronbach’s alpha of the culture factor was 0.859, whereas the instrument factor had a score of 0.659.

Table 4.

Exploratory factor analysis results.

FAL dimension	Question	Factor: Culture	Factor: Instruments
Learning	Within my organization, people are usually comfortable talking about problems, disagreements and differences in opinion.	.827
Feedback	My organization encourages staff members to express their concerns, ideas and suggestions about the functioning of the organization.	.767
Learning	My organization is characterized by a culture of adversarial debate and openness for constructive criticism.	.687
Learning	If a creative attempt to solve a problem fails, the responsible staff members are penalized.^a	.632
Learning	My organization encourages productive conflict and debate during internal discussions.	.631
Learning	My organization encourages experimentation and alternative ways of getting work done.	.617
Feedback	The feedback information from staff members has great impact on the strategic decisions made by the organization.	.599
Learning	My organization is characterized by a tendency to avoid risks.^b	.540
Accountability	Towards external stakeholders, my organization is very transparent about its results.	.524
Accountability	In general, the people of my organization feel responsible for the performance of the organization.	.506
Learning	My organization has monitoring systems that allow it to monitor a wide spectrum of performances and to compare those performances with the stated goals and objectives.		.822
Learning	My organization has a quality management system that systematically strives for continuous improvements throughout the entire organization.		.694
Feedback	Does your organization have an internal audit office?		.548
Learning	If discrepancies between performances and goals are detected, my organization will take action in order to reduce these discrepancies.		.542
Learning	My organization has formal procedures to ensure that lessons learned in the course of a project are passed along to others doing similar tasks.		.519

This is a ‘negative’ statement. Before the analysis of the data the scores for this item were reversed.

Where it was hoped and expected to find three factors (feedback, accountability and learning), there turned out to be two major factors reaching across all three FAL dimensions: a culture of FAL and instruments of FAL. They are two different ways in which feedback, accountability and learning are expressed in an organization. Further analysis will be focused on the effect that these two factors have on the survival of public sector innovations. This forces us to change the initial research question into:

To what extent can cultures and instruments of feedback, accountability and learning explain the survival of public sector innovations?

In order to further analyze the cultural and instrumental dimensions of FAL, an aggregate score for both was calculated per case, in the same fashion as was intended for FAL: adding the scores of the answers, then dividing the aggregate by the maximum possible score. The scores on FAL culture and FAL instruments (using Spearman’s correlation method) are correlated to each other with a medium-low coefficient of .31, significant at the .01 level.

The dependent side of the equation shows that 27 (see Table 3) of the innovations had not survived. Through analyzing the qualitative responses on the question why these innovations no longer existed, five answered that it was due to the temporal nature of the innovations, or because it had reached its goal. After deleting these cases we came to the total of 22 innovations which were terminated before either their goal was attained or before their planned end-date. This shows that the number of survivors exceeds the number of non-survivors by about 10:1. This is in line with results found by others (Borins, 1998; Farah and Spink; 2008), and the fact that the sample consists of ‘top of the class innovations’, given that they were awarded or nominated. This asymmetry creates limitations on the type of statistical methods which are possible to use. This is further aggravated by the finding that none of the separate items measuring FAL culture and FAL instruments are normally distributed, nor are the aggregate scores.

The next step in the analysis was a logistic regression, with FAL culture’s and FAL instruments’ aggregate scores as independent variables, survival as the dependent variable, and controlling for the variables mentioned previously. As the variables country, GDP and the Hofstede score perfectly match (e.g. each Belgian innovation has the exact same score on all three variables), we ran the logit regression for three models, each including a different variable. Table 5 shows the model which included the Hofstede score. Model 1 with the categorical country variable was not significant, with a p-value of obtaining a chi-square statistic given that the null-hypothesis is true of 0.0763. Model 2 with the GDP score was significant, but considering it had a predictive value of 0.000, it was decided not to include the model here. The correlations (Spearman correlation test) between the independent variables (Table 5) show that there do not seem to be problems regarding possible multicollinearity.

Table 5.

Correlation table.

	Culture	Instruments	Hofstede	Time since award	Governmental level	Organizational size	Innovation type
FAL culture	1.000
FAL instruments	0.341***	1.000
Hofstede	0.032	−0.268***	1.000
Age	−0.045	0.254***	−0.263***	1.000
Government level	−0.003	0.221*	−0.227***	0.431***	1.000
Organizational size	−0.285***	0.282***	−0.139*	0.099	0.115	1.000
Innovation type	0.014	0.124	0.024	0.171*	0.098	0.157*	1.000

*p ≤ .050

**p ≤ .010

***p ≤ .001

As Table 6 shows, the only significant independent factors in Model 3 (containing the Hofstede score) are the FAL culture score, together with the score for age, and the Hofstede score itself. For an increase in FAL culture we expect an increase in the chances of survival, ceteris paribus. Likewise, for an increase the numbers of years since the award or nomination, we expect a decrease in the odds of the innovation’s survival. Finally, with an increase in the Hofstede score comes an increase in the chances of survival. The effects are most likely exaggerated due to the unbalanced dataset, with one terminated case for every 10 surviving cases. Nonetheless, the results show that a culture surrounding feedback, accountability and learning seems to have a significant impact on an innovations’ survival chances. Although the age of the innovation and the Hofstede score also play a significant role, these effects are a lot smaller than the effect of the FAL culture score. Highly interesting is the finding that FAL instruments do not seem to be directly linked to the survival of public sector innovations. It is furthermore noteworthy that the probability of survival of public sector innovations is the same across governmental levels, organization sizes, innovation types and countries. The finding that the Hofstede score is linked to the survival of innovations ought to be analyzed with caution. As mentioned before, the Hofstede score is only a remote indicator for innovation culture. It is most of all an important sign that the impact of national innovation cultures on public sector innovations (in combination with administrative regimes) warrants further investigation. Finally, a reversal of the dependent variable labeling did not influence the outcomes of the logistic regression.

Table 6.

Model 3 – output logistic regression.

Survival	Coefficient	Std. err.	z	Sign.	(95% Conf. interval)
FAL culture	5.500	2.140	2.57	0.010	1.306	9.694
FAL instruments	0.928	1.923	0.48	0.629	−2.841	4.696
Hofstede	2.363	0.753	3.14	0.002	0.887	3.840
Age	−0.256	0.098	−2.60	0.009	−0.449	−0.063
Organizational size^a
25–100 FTE	0.976	1.090	0.90	0.370	−1.160	3.113
100–250 FTE	0.840	1.141	0.74	0.462	−1.397	3.077
250–500 FTE	1.367	1.437	0.95	0.341	−1.448	4.186
> 500 FTE	0.046	0.820	0.06	0.955	−1.562	1.654
Innovation type^b
Administrative	−0.411	0.824	−0.50	0.617	−2.025	1.203
Technological	−0.122	0.805	−0.15	0.880	−1.701	1.457
Governance	0.729	1.084	0.67	0.501	−1.396	2.854
Governmental level^c
Regional/Provincial	1.159	0.879	1.32	0.187	−0.564	2.882
Local	0.876	0.754	1.16	0.245	−0.602	2.354
Constant	−2.235	1.563	−1.43	0.153	−5.298	0.828

Reference category: < 25 FTE

Reference category: Product/Service

Reference category: National/Federal

Observations 211

LR Chi Square 33.12

Prob > Chi square 0.002

Pseudo R square 0.250

Log likelihood −49.584

Discussion and conclusion

This article presents an exploratory study into the survival chances of public sector innovations after their initiation. It is possible to conclude that a culture of FAL is indeed important for public sector innovations to survive. Based on the findings, our implications for practice focus on cultural aspects of the organization. Instruments for feedback, accountability and learning alone do not seem to be enough to make innovations survive. More important is the way in which the information coming from these instruments is dealt with, and the context in which this happens (see Table 4 for the separate items).

However, it is important to emphasize the exploratory nature of this research. The shortcomings of the dataset and imperfections of the research design force modesty in the conclusions. The results do not show in which way FAL cultures or instruments might have changed over the course of the innovations’ lifetimes. Starting out with a theoretical framework indicating that feedback, accountability and learning (FAL) might be relevant explanatory factors in innovation survival and termination, an exploratory factor analysis showed that what was measured through the survey were cultures and instruments of FAL. Using these two factors and the variables which constituted them in a logistic regression, it was found that a culture of FAL was indeed linked with the survival of the innovation, as well as the time which had passed since the innovation had been awarded or nominated.

Further research is necessary to substantiate and refine the findings presented here. This would ideally be done with a longitudinal research design. With regard to qualitative research designs the focus could be put on developing narratives surrounding the ‘lives’ of innovations, from crib to casket. Both provide improvements to the fact that the independent variables in this research were measured at the moment of the survey, instead of longitudinally. The results presented here, however, are an important first step in unraveling the many questions which still remain about what happens to public sector innovations after their initiation. Furthermore, important factors which could not be incorporated here still need further investigation: the complexity and radicalness of the innovation, combined with the idea of sunk costs, for example. National public sector innovation cultures, although touched upon in this article, deserve more attention in the future as well. Do they exist? Are they indeed linked to administrative traditions? Finally, innovations, like all policies, can have a strong political component. Issues of power and politics should receive equal attention in the study of public sector innovations.

Footnotes

Funding

The research leading to these results has received funding from the European Union’s Seventh Framework Programme under grant agreement No. 320090 (Project Learning from Innovation in Public Sector Environments, LIPSE), Socioeconomic Sciences and Humanities. LIPSE is a research programme under the European Commission's 7th Framework Programme as a Small or Medium-scale Focused Research Project (2011–2014). The project focuses on studying social innovations in the public sector ().

Notes

Wouter van Acker is a researcher at the KU Leuven Public Governance Institute.

Geert Bouckaert is full professor at the KU Leuven Public Governance Institute, and president of the IIAS

References

Argyris

Schön

(1978) Organizational Learning: A Theory of Action Perspective, Reading, MA: Addison Wesley.

Arundel

Casali

Hollanders

(2015) How European public sector agencies innovate: The use of bottom-up, policy-dependent and knowledge-scanning innovation methods. Research Policy 44: 1271–1282.

Bernier

Hafsi

Deschamps

(2015) Environmental determinants of public sector innovation: A study of innovation awards in Canada. Public Management Review 17(6): 834–856.

Bonsón

Torres

Royo

Flores

(2012) Local e-government 2.0: Social media and corporate transparency in municipalities. Government Information Quarterly 29: 123–132.

Borins

(1998) Innovating with Integrity, Washington, DC: Georgetown University Press.

Borins

(2001a) Innovation, success and failure in public management research: Some methodological reflections. Public Management Review 3(1): 3–17.

Borins

(2001b) The Challenge of Innovating in Government. PricewaterhouseCoopers Innovations in Management Series, Arlington, VA: PricewaterhouseCoopers Endowment for the Business of Government.

Borins

(2008) Introduction. In: Borins

(ed.) Innovations in Government: Research, Recognition and Replication, Washington, DC: Brookings Institution Press, pp. 1–12.

Bovens

Schillemans

’t Hart

(2008) Does public accountability work? An assessment tool. Public Administration 86(1): 225–242.

10.

Damanpour

Schneider

(2006) Phases of the adoption of innovation in organizations: Effects of environment, organization and top managers. British Journal of Management 17: 215–236.

11.

Damanpour

Walker

Avellaneda

(2009) Combinative effects of innovation types and organizational performance: A longitudinal study of service organizations. Journal of Management Studies 46(4): 650–675.

12.

De Peuter

(2011) Prestatiemanagement. In: Bouckaert

Hondeghem

Voets

Op de Beeck

Cautaert

(eds) Handboek Overheidsmanagement, Brugge: Vanden Broele, pp. 103–132.

13.

De Vries

Bekkers

Tummers

(2016) Innovation in the public sector: A systematic review and future research agenda. Public Administration 94(1): 146–166.

14.

Döös M, Johansson P and Wilhelmson L (2015) Beyond being present: Learning-oriented leadership in the daily work of middle managers. Journal of Workplace Learning 27(6): 408–425. https://dx-doi-org.web.bisu.edu.cn/10.1108/JWL-10-2014-0077.

15.

Down

(1966) Inside Bureaucracy, Boston, MA: Little, Brown.

16.

Eurostat (2016) GDP per capita in PPS. Online at: http://ec.europa.eu/eurostat/tgm/table.do?tab=table&init=1&language=en&pcode=tec00114&plugin=1.

17.

Farah

MFS

Spink

(2008) Subnational government innovation in a comparative perspective: Brazil. In: Borins

(ed.) Innovations in Government: Research, Recognition and Replications, Washington, DC: Brookings Institution Press, pp. 71–92.

18.

Frees W, Van Acker W and Bouckaert G (2015) The role of feedback, accountability and learning in organizational change and innovation: A theoretical framework. Learning From Innovations in Public Sector Environments (LIPSE) Working Paper. Online at: https://lirias.kuleuven.be/bitstream/123456789/497510/1/LIPSE+Working+5+Frees+et+al..pdf.

19.

Garvin

Edmondson

Gino

(2008) Is yours a learning organization? Harvard Business Review 86(3): 109–116.

20.

Glor

(1998) The role of innovation awards. The Innovation Journal: The Public Sector Innovation Journal 3(1): 1–7.

21.

Greiling

Halachmi

(2013) Accountability and organizational learning in the public sector. Public Performance & Management Review 36(3): 380–406.

22.

Hofstede

(2001) Culture’s Consequences: Comparing Values, Behaviors, Institutions, and Organizations across Nations, Thousand Oaks, CA: Sage.

23.

Hogwood

Peters

(1982) The dynamics of policy change: Policy succession. Policy Sciences 14: 225–245.

24.

Hosmer

Lemeshow

(2000) Applied Logistic Regression, Danvers, MA: Wiley.

25.

Hughes A, Moore K and Kataria N (2011) Innovation in public sector organizations: A pilot survey for measuring innovation across the public sector, NESTA Index report, London.

26.

Kaiser

(1974) An index of factorial simplicity. Psychometrika 39: 31–36.

27.

Kasaa A (2013) Culture as a possible factor of innovation: Evidence from the European Union and neighbouring countries. Search WP 5/05 Working Paper. Online at: http://www.ub.edu/searchproject/wp-content/uploads/2013/01/WP-5.5.pdf.

28.

Katz

Kahn

(1978) The Social Psychology of Organizations, New York: John Wiley & Sons.

29.

Kickert

(1997) Public management in the United States and Europe. In: Kickert

(ed.) Public Management and Administrative Reform in Western Europe, Cheltenham: Edward Elgar, pp. 15–38.

30.

Lapsley I and Wright E (2004) The diffusion of management accounting innovations in the public sector: A research agenda. Management Accounting Research, https://dx-doi-org.web.bisu.edu.cn/10.1016/j.mar.2003.12.007.

31.

Lindberg

(2013) Mapping accountability: Core concept and subtypes. International Review of Administrative Sciences 79(2): 202–226.

32.

Morgan

(2006) Images of Organization, London: Sage.

33.

Pollitt

(2011) Innovation in the public sector: An introductory overview. In: Bekkers

Edelenbos

Steijn

(eds) Innovation in the Public Sector: Linking Capacity and Leadership, New York: Palgrave Macmillan, pp. 35–43.

34.

Pollitt C, Bouckaert G and Löffler E (2007) Making quality sustainable: Co-design, co-decide, co-produce, co-evaluate. Report by the Scientific Rapporteurs of the 4th Quality Conference, Ministry of Finance, Finland.

35.

Pollitt

Girre

Lonsdale

Mul

Summa

Waerness

(1999) Performance or Compliance? Performance Audit and Public Management in Five Countries, Oxford: Oxford University Press.

36.

Rogers

(1983) The Diffusion of Innovation, New York: The Free Press.

37.

Savaya

Spiro

(2012) Predictors of sustainability of social programs. American Journal of Evaluation 33(1): 26–43.

38.

Schillemans

Van Twist

Vanhommerig

(2013) Innovations in accountability: Learning through interactive, dynamic, and citizen-initiated forms of accountability. Public Performance & Management Review 36(3): 407–435.

39.

Shrivastava

(1983) A typology of organizational learning systems. Journal of Management Studies 20(1): 7–28.

40.

Torugsa

Arundel

(2016) Complexity of innovation in the public sector: A workgroup-level analysis of related factors and outcomes. Public Management Review 18(3): 392–416.

41.

Van Acker W and Bouckaert G (2016) A survey measuring feedback, accountability, learning and the sustainability of innovations. Learning From Innovations in Public Sector Environments (LIPSE). Online at: http://lipse.org/upload/publications/LIPSE%20Survey%20on%20sustainability%20of%20public%20sector%20innovations%20(003).pdf.

42.

Van Loocke

Put

(2011) The impact of performance audits of SAIs: A review of the existing evidence. In: Lonsdale

Wilkins

Ling

(eds) Performance Auditing: Contributing to Accountability in Democratic Government, Cheltenham: Edward Elgar, pp. 175–208.

43.

Walker

(2006) Innovation type and diffusion: An empirical analysis of local government. Public Administration 84(2): 311–335.

44.

Walker

(2013) Internal and external antecedents of process innovation: A review and extension. Public Management Review 16(1): 21–44.