Abstract
This article examines the use of performance information by public managers. It reviews literature on the impact of attitudes and social norm and puts forward a psychological-cognitive model based on the theory of planned behavior. The article finds support for this model emphasizing that performance data use is a goal-directed, reasoned action. Another critical result is that managers who consciously intend to use performance data also make sure that the data in their division are of good quality which, in turn, fosters information use. These findings indicate that—in addition to organizational routines—cognitive factors are promising starting points for interventions to foster managers’ data use. The article is based on survey data from German cities.
Keywords
Introduction
Performance management practices have spread across Organisation for Economic Co-operation and Development (OECD) countries (Bouckaert & Halligan, 2008; Organisation for Economic Co-operation and Development, 2009; Pollitt & Bouckaert, 2004). Academics and practitioners have now posed the question of what happens to performance information once it is collected and reported (Moynihan, 2008; Van Dooren, Bouckaert, & Halligan, 2010; Van Dooren & Van de Walle, 2008). There are different scenarios for the use of performance data (Moynihan, Pandey, & Wright, 2012a). Public managers can just passively comply with set reporting standards (Radin, 2006), or they actively exploit and politicize the data as evidence for their claims (Moynihan, 2008). The collection of performance information can even lead to dysfunctional management behavior known as gaming or effort substitution (Bevan & Hood, 2006).
Most researchers have devoted their attention to studying the purposeful use of performance information (Askim, Johnsen, & Christophersen, 2008; de Lancer Julnes & Holzer, 2001; Dull, 2009; Ho, 2006; Kroll, 2013; Melkers & Willoughby, 2005; Moynihan & Pandey, 2010; Yang & Hsieh, 2007). That is, the utilization of these data by public managers to evaluate target achievements and take action after they have identified performance gaps, to allocate resources based on achievements or simply use trend or benchmarking results for trial-and-error learning.
One way to make managers use performance information purposefully is changing decision-making routines. Formal rules can be established that require the collection and reporting of performance information to create organizational routines that continuously provide decision makers with performance data and thus make their incorporation in management decisions more likely. For the United States, the Governance Performance and Results Act (GPRA) and the Program Rating Assessment Tool (PART) are two examples. Though a recent study by Moynihan and Lavertu (2012) found that the mere involvement with GPRA and PART only makes managers work with performance information passively but not purposefully, there is a good deal of evidence that measurement and reporting routines make purposeful data use more likely when ensuring a certain quality, relevance, and regularity of the produced information (Ammons & Rivenbark, 2008; de Lancer Julnes & Holzer, 2001; Ho, 2006; Kroll & Vogel, 2013; Melkers & Willoughby, 2005; Yang & Hsieh, 2007).
Another way to foster purposeful use is the establishment of learning routines. Such routines can be created if organizations offer their employees permanent opportunities to reflect on their core processes and outcomes to improve them (Argyris & Schön, 1996; Dosi, Nelson, & Winter, 2009). There are a growing number of studies which found that the establishment of learning forums—where performance information is discussed and collectively made sense of—can be positively associated with purposeful data use (Moynihan, 2008; Moynihan & Lavertu, 2012).
So far, we can conclude that routines can shape decision making in public administration. Evidence comes from the organizational sciences which have pointed to the importance of meta-routines on the organizational level (Becker, 2005; Levitt & March, 1988) as well as from the field of social psychology that has focused on the cognitive aspects of habits and automaticity (Hassin, Aarts, Eitam, Custers, & Kleiman, 2009; Ouellette & Wood, 1998). Changing administrative routines in ways that embrace the measurement, reporting, and interpretation of performance information therefore seems to be a promising strategy to foster the data use.
This article takes an alternative approach to studying the utilization of performance information. Though routines seem to have an impact, research in social psychology has shown that factors, such as behavioral intention, environmental constraints, social pressure, individuals’ skills, attitude, self-image, and self-efficacy, are also of great importance to predict human behavior (Bagozzi & Warshaw, 1990; Bandura, 1986; Campbell, 1963; Sherman & Fazio, 1983; Stretcher, Champion, & Rosenstock, 1997). This article argues that there are at least two other categories of influences that might be good predictors of a behavior such as data use—the managers’ attitude and the existing social norm. Though this argument by itself is not entirely new, the article suggests a specific mechanism that can be used to predict data use based on the theory of planned behavior. The theory states that behavior is highly intentional if the effects of attitude and norm are not direct but entirely mediated by the intention to perform this behavior. Hence, if performance data use is a reasoned action, this behavior should be well predictable by the antecedents suggested by the theory, and the managers’ intention should be a critical mediator. If this is the case, we could draw important conclusions. One major implication would be that we should not only focus on changing rules and routines when we want to foster performance data use. Instead, public managers need to be convinced of the added value of using performance information and there needs to be a critical mass of promoters for such a steering philosophy.
This article builds on the theory of planned behavior because this is to date the most prominent and one of the most successful approaches to model behavior from a reasoned action perspective (Ajzen, 1991; Fishbein & Ajzen, 2010). It has been used to predict several behaviors in different fields, including the behavior of managers in the private sector (Cordano & Frieze, 2000; Hill, Mann, & Wearing, 1996; Kidwell & Jewell, 2008). By testing a mediation model of data use, the article contributes to research that has examined important moderators and mediators of performance information use (see Kroll & Vogel, 2013; Moynihan, Pandey, & Wright, 2012b; Yang & Hsieh, 2007). Though research testing direct effects of various potential impact factors has been helpful to distinguish between more and less critical variables, moderation and mediation models will help to further qualify our hypotheses and contribute to the maturity of this research field.
The article proceeds by introducing Ajzen’s theory of planned behavior and adapt it to the case of performance information use. It then presents the data and applied methods. This is followed by a discussion of the empirical results, their limitations, and the potential for further research. It ends with drawing the major conclusions from the findings, including practical implications.
Ajzen’s Theory of Planned Behavior
The foundations of the theory of planned behavior were laid in articles by Icek Ajzen in 1985 and 1991. Ajzen (2005) elaborated on this research in a monograph and in a joint book with Martin Fishbein (2010). The theory suggests that the performance of a behavior is contingent on three factors: an individual’s attitude toward a behavior, the subjectively perceived social norm, and the behavioral control. A positive attitude and a social norm that embraces a particular behavior will make its performance more likely. To account for factors beyond an individual’s volition, the theory introduces the concept of perceived behavioral control which refers to people’s perceptions of their ability to perform a behavior. A major claim of this theory is that these factors do not directly predict behavior but that their effects are mediated by a person’s behavioral intention. That is, a positive attitude and norm make people want to show a certain behavior, and the direct effects of both predictors disappear once we have accounted for people’s intentions. This is a little different for a person’s perceived behavioral control. Though the assessment that an individual is able to perform a behavior might also increase their intention to do so (the indirect effect), behavioral control might also have an effect that is independent from a person’s willingness (see Figure 1). In other words, being in control over a behavior can make its performance more likely, even if we leave aside behavioral intentions.
A General Model of the Theory of Planned Behavior.
A second major claim is that not all intentions are eventually carried out. Being willing to show a certain behavior and actually performing it have been found to be highly correlated but remain distinguishable constructs (see a meta-analysis by Sheeran, 2002). Even within the same survey, respondents might provide different answers to questions with regard to whether they would like to show a certain behavior and whether they did so in the past.
Though the mechanism the theory suggests might seem intuitively appealing, the planned-behavior approach is not by definition true. The theory is considered falsified if the factors attitude, norm, and control are not able to explain a good portion of the variance in a certain behavior. Second, the theory might not work for a particular behavior if attitude, norm, and control only show direct behavioral effects, and the intention variable turns out to be an insignificant mediator. If the theory can be verified, the behavior under investigation can be characterized as a consciously made, reasoned action which is likely to be influenced through interventions that address the factors attitude, norm, and control. Though conscious intention plays a major role in this theory, the reasoned action approach should not be confused with the assumption of perfect rationality because intentions can be formed on an inaccurate, biased, or irrational basis.
The next two sections apply the theory of planned behavior to the issue of performance information use. The first section addresses the factors “attitude” and “social norm” and their meaning in the field of performance management. The second section deals with the concept of “perceived behavioral control” and a potential indirect effect of intention on the behavior of performance data use.
Performance Information Use as a Function of Attitude and Social Norm
There is widespread evidence for a direct effect of attitude on data utilization. Ammons and Rivenbark (2008) found support for the importance of the managers’ positive attitude toward the use of performance information. They compared the data use of municipalities participating in a benchmarking project. A commonality that all municipalities reporting a frequent use of the benchmarking data shared was that their managers showed an “enthusiasm to compare” and thereby had a positive attitude toward performance information. Ho (2006) surveyed mayors in the American Midwest and similarly found that the mayors’ interest in performance and benchmarking data is one of the most crucial impact factors on their actual data use. Taylor (2011) also reported that the managers’ attitude toward performance indicators has been an important determinant to explain data use. Even after controlling for sociodemographic, technical, cultural, and environmental variables, the managers’ attitude showed a significant impact. Only one study by de Lancer Julnes and Holzer (2001) tested for the influence of attitude and found no significant results. However, this study used a joint measure for the attitude of managers and staff. This measure is not individual—as is common in psychological studies—but collective and can thereby only provide distorted results for the managers’ attitudes.
Most research that focused on the impact of social norms focused on the pressure that comes from external stakeholders (Bourdeaux & Chikoto, 2008; de Lancer Julnes & Holzer, 2001; Ho, 2006; Moynihan & Hawes, 2012; Moynihan & Ingraham, 2004; Yang & Hsieh, 2007). The involvement of politicians, interest groups or citizens seems to encourage public managers to seriously consider using their performance information for decision making. Other studies found that the active involvement of employees in performance management practices creates general support for performance-based steering and facilitates managers’ data use (Boyne, Gould-Williams, Law, & Walker, 2004; Melkers & Willoughby, 2005). There is, however, only little research on the influence of the immediate peers of public managers which is the main focus of the theory of planned behavior. In this case, these are the managers’ supervisors and colleagues. Research in this direction has been conducted by Folz, Abdelrazek, and Chung (2009) who found that a critical mass of line-managers supporting the use of performance measures is essential for the success of performance systems.
In line with the theory of planned behavior there is evidence in favor of the important role of attitudes and social norm for data use, though we do not know much about the impact of the managers’ direct peers. What is still controversial, however, is whether the mechanism suggested by the theory of planned behavior is able to explain information use and thus qualifies this behavior as a conscious, reasoned action. To do so, the effects of attitude and norm should be fully mediated by the managers’ intention indicating that performance data use is highly intentional.
Hypothesis 1 (H1): Attitudes and social norm, fully mediated by the manager’s intention, will affect the manager’s use of performance information.
The Indirect Effect of Intention
Studies on performance management have identified poor data usability as an important factor that limits data use (Ammons & Rivenbark, 2008; Brun & Siegel, 2006; Dull, 2009). I therefore include this factor as a constraint to the behavioral control of the managers. A high usability of the data is treated as a proxy for a high behavioral control. Once the quality of the data has reached a reasonable level, the actual use of these data is only dependent on the intention of the managers and is therefore in their control.
In contrast with what is suggested by the theory of planned behavior, this article does not treat data usability as a fully exogenous variable. Public managers have the possibility of influencing the quality of the data that are produced in their departments. They can readjust measures that are of little use, and they can influence their staff’s enthusiasm with regard to the performance measurement process. I expect managers, who have firm intentions to use performance information (due to a positive attitude and social pressure), will engage in activities that lead to data quality improvements. Better data usability in turn will foster information use by managers. Therefore, I not only assume a direct effect of the intention to use performance data on the actual behavior but also an indirect effect that is mediated by increased data usability (see Figure 2). Information usability, which might limit the manager’s behavioral control, is therefore considered to be an endogenous and not exogenous variable. The following paragraphs elaborate on the theoretical explanation of how enthusiastic managers can increase the usability of the data that is provided to them.
A Modified Planned-Behavior Approach to Explain Performance Information (PI) Use.
Performance information use is often conceptualized as a scenario where managers passively receive information and can then choose to incorporate it in their decisions. Such an understanding neglects the whole process that takes place before managers receive performance reports as well as their role in shaping the reports. In this phase, managers can make a difference by taking ownership of the performance measurement and encourage their staff to do the same (Behn, 1994). To claim ownership of a performance measurement system, which is often externally initiated, it has to become customized. The focus must be shifted from the external reporting requirements to the utilization for an improved steering of the managers’ unit. Studying the field of the U.S. Correction Administration, Moynihan (2008) found that performance information is used when managers at the agency level were able to make sense of the state-wide performance measurement systems and make them usable for agency-internal management purposes.
A strategy to help ensure that internally relevant management information is produced is to get staff to buy-in and thereby increase their commitment to performance measurement. One way to achieve this is for managers to use symbolic behavior and to show credible commitment. Dull (2009) has made this point and argued that public officials only truly engage in the data collection and reporting if they perceive that this is an important priority for the organization leaders: “If they perceive a lack of credible leadership commitment to results-model reform, managers may see reform as a threat or a nonissue, gathering less information from fewer, less diverse sources, engaging fewer people, and leaning on familiar ideas and practices” (p. 260). Only if managers continuously signal that performance measurement is important to them, will their staff be willing to devote their scarce resources to an improvement of the collection, analysis, and visualization of the data. The U.S. General Accounting Office (GAO, 2004a, 2008) found that this is not only an issue of will but also of skill. Managers, who invested in capacity building with regard to performance measurement, also benefited from better usable documents and reports provided by their staff. Further evidence for the important role of leadership support for the prioritization of performance information by subordinates comes from studies by Askim et al. (2008), Moynihan and Ingraham (2004), and Moynihan et al. (2012b).
This section has shown that managers can shape performance measurement in their organizations. They can engage in this process and positively impact its outcomes long before a performance report has reached their desk. Making externally initiated performance measures internally useful and continuously showing credible commitment requires effort on the part of the managers.
Hypothesis 2 (H2): Managers who firmly intend to make use of performance information will make sure that the performance data in their division are of good quality. Improved information usability, in turn, positively affects the manager’s use of performance information.
Data and Method
To test the hypotheses, I use data collected through an online survey in 2011. It was addressed to the middle managers of eight divisions in all 130 German cities with county status (including the districts of the city states Berlin, Bremen, and Hamburg). 1 The overall sample consisted of 1,040 (130 × 8) managers. Due to practical problems, such as invalid email addresses and vacant positions, the sample population was reduced to 954. Middle managers were surveyed as they play a key role in the successful implementation of public management reforms. This group of managers is defined as the divisional heads who are supervised by the heads of the departments. In contrast with the latter, division managers are not politically appointed and can therefore be regarded as actual administrative managers rather than executive politicians. The performance data they deal with include aggregated, quantitative information on internal processes, efficiency, outputs, and outcomes. The survey was addressed to middle managers from the following divisions: organization and staff management, finance, citizen service, facility management, cultural issues, building authority, social welfare, and youth welfare. These divisions include all services that municipalities spend the most money on and a good variety of functions performed by public administration.
The response rate of the survey is 29.8%. Not all of these responses could be considered for the statistical analysis of this article due to missing data. 2 To conduct a nonresponse analysis, all managers who did not respond were asked via email about their reasons for not participating in the survey. At that stage, 164 responses were collected, and 36.6% of these former nonrespondents stated that the main reason for nonparticipation was that their division does not systematically collect and analyze performance information. To study the use and nonuse of performance information, a necessary condition is that this type of data is existent. Therefore, the 36.6% (extrapolated to all nonrespondents) could theoretically be excluded from the population of interest, and the adjusted response rate could be considered as 39.8%. 3
The operationalizations for all variables can be found in Table 1. As indicated earlier, I used measures of purposeful performance information that fall in the categories of “steering and controlling” and “learning” (Moynihan et al., 2012a; Van Dooren et al., 2010). The measures for intention, attitude, and norm were developed according to the recommendations for direct planned-behavior indicators by Fishbein and Ajzen (2010). However, since these measures were part of a greater public administration survey, the items had to be adapted to a certain extent. For example, each construct was measured by only two or three indicators, and to ensure the consistency of the entire survey, 7-point Likert-type scales were used for all questions. All items are perceptual measures, which is not problematic as the studied latent concepts attitude, norm, and behavioral control are cognitive factors and thus highly subjective in nature.
CFA Factor Loadings and Correlations of the Latents.
Note: Agreement and disagreement with these items has been measured via 7-point Likert-type scales. To measure “information usability,” the Likert-type scale ranged between very low = 1 and very high = 7. All correlations are significant at a .01 level. N = 226.
CFA = confirmatory factor analysis; PI = performance information.
Intention is measured by only one item which, however, is the concept’s most prominent and straightforward indicator. The first attitude measure classically provides an assessment of the behavior’s value whereas the second item accounts for the bipolar choices (costs-benefits) attitude measures usually provide. To operationalize social norm, I used descriptive rather than injunctive measures. The latter would be based on statements that measure what the survey participants think that their peers want them to do. But this is usually difficult to capture and rather uncommon for public administration surveys. The former, in contrast, directly measures the behavior or attitude of the peers as it is perceived by the survey participants. Information usability is the only construct not based on Fishbein and Ajzen (2010). As this article focuses on data usability as one specific external restriction of data utilization, the items used aim to capture the understandability, reliability, and overall quality of the existent performance information.
Table 1 shows the factor loadings of all items (λ). Their average value is 0.78 and the mean variance explained in the empirical indicators is 61% which is evidence for good convergent validity. The relatively low factor loading of the second attitude item indicates that even managers who generally have a positive attitude toward using performance information (and score high on the first item) might be at least slightly critical of the cost-benefit ratio of the existent data in their divisions. This implies that the second item only partially contributes to the operationalization of the attitude construct. The differences between the loadings of the two social norm items indicate that supervisors (0.64) are slightly more influential than other peers (0.59) and that both groups of actors can signal divergent expectations. Though factor loadings below 0.7 are far from perfect, they are not overly problematic when confirmatory factor analysis (CFA) is applied. In contrast to other approaches, such as principal component factoring or simple additive indices, CFA accounts for every item’s measurement error and corrects for it when estimating relationships between latent factors (Kline, 2005; O’Leary-Kelly & Vokurka, 1998).
The mean correlation between all factors is 0.56 and can be considered as relatively high. This is not surprising because we are dealing with perceptive measures of behavioral attitudes, behavioral intentions, and behavioral performance that are by definition highly correlated (for similar results see the planned-behavior studies by Kassem & Lee, 2004; McMillan & Conner, 2003; Orbell, Blair, Sherlock, & Conner, 2001). To ensure discriminant validity, I compared the goodness of fit of different measurement models. One concern with the high correlation among the latents in Table 1 could be that items highly cross-load on factors they are not supposed to be theoretically associated with. This can be tested if we compare the suggested measurement model with alternative model specifications where highly correlated factors have been merged. Comparing the p value tests for a close fit, it turns out that the suggested model fits the data better (0.29) than models where we treat intention and data use (0.00), intention and attitude (0.06), intention and norm (0.12), attitude and norm (0.01), or norm and use (0.14) as one factor instead of separate constructs. Table two displays that the suggested measurement model also has a good model fit if we take various fit indices into account.
A way to deal with higher levels of collinearity is to use structural equation model (SEM) rather than simple regressions. Instead of ignoring intercorrelation between the independent variables, SEMs take this into account when estimating regression weights. Further advantages of this technique are that its measurement model is based on CFA (see the discussion above), it allows for systematically testing indirect effects (“mediations”), and it provides information about the goodness of the overall model instead of just single coefficients (Byrne, 2010; Kline, 2005).
Results and Discussion
The results of the structural model are displayed in Figure 3. These findings tell us that the chosen planned-behavior approach was able to explain a good deal (76%) of the variation in performance information use. Also, there is evidence in favor of hypothesis 1: A positive attitude toward performance information (β = 0.50) and the existence of a social norm embracing regular data use (β = 0.40) positively affect the managers’ intention to use performance information. A high intention to use performance data (β = 0.31) also makes the actual behavior more likely. However, the managers’ intention fully mediates only the effect of attitude whereas the effect of social norm is partially mediated. In addition to the indirect effect, social norm also has a direct impact on the use of performance information (β = 0.29). This is an interesting finding as it helps to better understand the role of immediate peers. Though a social norm seems to make managers want to use performance data, coworkers and supervisors also foster data use independently from influencing managers’ intention. That is, the effect of social norm seems to be twofold: It creates readiness for data use but at the same time also pressures managers to do so, even if this is not consistent with their intentions.
Results Structural Equation Model.
There is clear support for hypothesis 2. Managers who intend to use performance information also highly engage in the improvement of the data usability (β = 0.35). An increased usability, in turn, fosters the actual use of the data (β = 0.47). The fit indices in Table 2 (see SEM) indicate that the data fit with the overall SEM quite well. Furthermore, the indirect effects of attitude, norm, and intention are all significantly different from zero as examining the bias-corrected bootstrap confidence intervals reveals.
Fit Indices.
Note: *The standard has been defined based on Hair, Black, Babin, Anderson, & Tatham (2006), Hu and Bentler (1998), Marsh, Balla, and Hau (1996).
AGFI = adjusted goodness of fit index; CFA = confirmatory factor analysis; CFI = comparative fit index; DF = degrees of freedom; GFI = goodness of fit index; PCLOSE = p value test for close fit; RMSEA = root mean square error of approximation; SEM = structural equation model; SRMR = standardized root mean square residual; TLI = Tucker-Lewis coefficient.
The results also consist of a few more interesting findings that need to be discussed. If we compare the standardized total effects (direct plus indirect ones) of attitude (0.30), social norm (0.48), and perceived data quality (0.47), it turns out that the first two effects combined will be able to explain more variance in data use than the third one. This finding suggests that cognitive factors like attitude and social norm are more important overall than technical ones, such as the data quality. This result has a major implication: To foster performance information use, it is certainly not enough to improve technical features like the sophistication of the measurement instruments, the readability of reports, or the validity of the data. Public managers have to be convinced of the added value of a performance measurement (attitude). In addition, there needs to be a coalition of people who promote a performance-based steering philosophy (social norm).
Furthermore, the direct and the indirect effect of intention deserve to be analyzed in more detail. On one hand, we can see that the direct effect of intention (β = 0.31) is stronger than its indirect effect (β = 0.35 × .47 = 0.16). This indicates that performance information use can be explained by the managers’ intention even if we do not consider the variable information usability. On the other hand, the inclusion of the described indirect effect completes our understanding of performance data use. Including this partial mediation improves the explained variance of the dependent variable performance information use by notable 13% (from 63% to the reported 76%). It is important to note that I have not claimed that the indirect effect of intention will be a substitute for its direct one. Instead, I have argued that both effects complement each other. Though the results show that public managers can influence data usability, the majority (88%) of the variation in this variable remains to be explained by other factors.
The model that has been presented is a modification of Ajzen’s theory of planned behavior because information usability (as a limitation to the managers’ behavioral control) has been treated as an endogenous and not exogenous factor. That is, information usability does not determine the managers’ intention to use performance data but the managers’ intentions determine how well they engage in strategies for data usability improvements. To test this argument, I ran another model where information usability, as suggested by Ajzen’s theory, was treated as an exogenous variable (similarly to Figure 1). Here, information usability was correlated with the other two exogenous factors attitudes and social norm, and there was one regression path going to the intended use and one directly going to the actual performance of the behavior. It turned out, however, that the path between information usability (exogenous) and intended use (endogenous) was not significant (p = .42), which provides further support for the argument put forward by this article.
Limitations and Further Research
There are limitations that need to be acknowledged. These caveats do not invalidate the results but suggest the need for additional research to determine the generalizability of this article’s findings. One issue is the context of this study. The main argument was that public managers can positively influence the use of performance information if they take on ownership of these data and engage in improving their quality. However, this only seems to work where managers have some discretion to choose and change indicators. This article is based on data from German local government where performance measurement works bottom-up, and middle managers have a great deal of discretion to decide which data to collect and report. But does this theory also hold for a country like the United States where performance measurement can rather be characterized as top-down? I think it does, but we have to consider a few adjustments. First of all, the theory might be more easily transferable to the local level in the United States than to the state or federal government. Second, we might not expect that enthusiastic managers—unlike in this study—will be able to account for 12% of the variation in data quality but for a smaller share. Third, ownership might even differ on the federal level dependent on the performance regime we look at. For example, GPRA granted much more discretion to agencies regarding choosing goals and measures than PART where the Office of Management and Budget (OMB) took on an active role as the agenda setter. But even within the PART evaluations the GAO (2004b) noticed that there were negotiations and even struggles with regard to the definition and interpretation of goals, measures and data between the OMB and the federal agencies.
Another limitation comes from the statistical analysis. As I surveyed more than one manager per city, these responses were not fully independent from each other, and there could be nested city effects. I could not control for these effects and track survey IDs as there was an agreement with the cities to grant the participating managers full anonymity. Though this is certainly a limitation of the statistical results, this issue does not seem overly problematic. One reason for this is again related to the discretion that middle managers in German cities have. Due to this discretion many performance-management-related factors as well as data use itself vary from division to division. Thus, the differences between divisions within a city might be almost as great as between cities. To further control for city-specific variables, I ran a stepwise regression accounting for several impact factors which provided general support for the findings from the SEM. 4 Moynihan and Pandey (2010) used the same approach to control for several city influences when dealing with the NASP data that provided more than one response per city.
Another statistical limitation that comes with the use of survey data is a potential common-source bias (Chang, Witteloostuijn, & Eden 2010). Such a bias could produce false correlations between variables as their “common measurement contains a source of error that shows up in both measures and thus can contribute to spurious results” (Meier & O’Toole, 2010, p. 4). Sources of common method effects are potentially common raters (e.g., consistency motifs, implicit theories, social desirability), item characteristics (e.g., common scale formats, item wording), item context and measurement context (for a discussion, see Podsakoff, MacKenzie, Lee, & Podsakoff, 2003). To diagnose whether common method variance is likely to appear in the data, I conducted Harman’s single factor test in the post hoc analysis. The idea behind this test is to examine whether all used items can be explained by one single factor. If one factor accounts for most of the variance in all items, a common method variance seems to be present. To run this test, I conducted a CFA that showed a poor fit and therefore indicates that common method variance was not a serious problem (see the one-factor model in Table 2).
Future research should test the suggested model using more indicators for every latent factor and maybe even different items. This article reports that a cost–benefit assessment of performance data is not an optimal measure to reflect the general attitude toward performance information use. It might also be promising to use injunctive measures for social norm because the descriptive indicators used in this study only shared a limited portion of variance. More research is needed on the factor “perceived behavioral control” and its meaning in the field of performance management. My application of this construct focused on data usability as one factor limiting information use. Another constraint to the manager’s behavioral control that has not been considered in this article could be their ability to use performance data. There might be managers who have developed better data analysis skills than others. The suggested model could also be enhanced by using expectancy-value products to model the factors attitude, norm and control instead of using direct item measures. This way, we could learn more about the substantive beliefs behind the behavior of performance data use.
Conclusion
This article contributes to the discussion on performance information use by offering a social psychological perspective on the subject. Using Ajzen’s theory of planned behavior, I have shown that the managers’ data use can be modeled as a function of their attitude and the existing social norm, mediated by their intention. This explanation could be labeled psychological-cognitive because it emphasizes the importance of the managers’ mindsets as well as the social pressure they experience from their most important peers. Compared with a technical explanation of performance data use (“data quality matters”), it turns out that the cognitive factors are more important. To improve information use, it seems to be promising to convince managers of the advantages of performance management and to form a critical mass of promoters.
If conscious intention matters and intention is a function of attitude and social norm, then it does not seem to be enough to foster data use through only technical improvements. For public management in practice this means that fine-tuning indicators, regularly updating data bases, and designing more appealing reporting formats will only increase data use to a certain extent. At the same time, workshops, testimonies, best-practice examples, and powerful narratives are needed to convince managers of the value added by performance measurement practices. Only if the managers are intrinsically convinced that they need performance measures, will they engage in time-consuming practices to make these measures usable and eventually use them for decision making. The results have shown that it is equally important to get a coalition of managers to buy-in to performance management practices. These promoters contribute to the creation of a social norm that is in favor of performance-based steering by encouraging other managers (influencing their intentions) and even pressuring them (independently from their intentions) to consider performance data for decision making.
The article also shows that cognitive and technical factors interact. Managers who firmly intend to use performance information also strongly engage in the improvement of the collected data. A higher data quality, in turn, fosters the information use by the managers. My empirical examination of this effect furthermore indicated that the cognitive factors determine the technical ones and not vice versa. It seems that it is not high data quality which makes managers willing to use performance information but it is the enthusiastic manager who is willing to engage in data usability improvements which eventually lead to a higher data use.
Comparing the findings of this article with previous research we can conclude that, in addition to organizational routines, cognitive factors are also critical starting points for management interventions to foster data use. Though this study was not able to examine the effects of attitude, norm, and intention while controlling for organizational routines, its results still suggest that establishing routines and making rules is only one way to foster actual data use. Another promising strategy is to address the cognitive aspects of this behavior, and it seems likely that performance managements systems will succeed if both routines and cognition will be considered. What has yet to be examined is how social cognition and organizational routines interact—that is, how both variables affect each other and to what extent the existence of one factor increases the impact of the other one.
Footnotes
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.