Abstract
Performance information (PI) is only valuable when decision makers use it to improve their organization. Despite its importance, evidence on PI use is still limited, particularly about PI use in different contexts and for specific purposes. This article examines managers’ use of PI for specific purposes (purposeful PI [PPI]) and compares cross-sector differences in reported PPI use with a survey of nearly 1,000 top-level hospital managers in public, private, and nonprofit hospitals in the United States, while controlling for task. Findings show significant cross-sector differences in reported PPI use in three decision areas: personnel, day-to-day operations, and service efficiency, and that public managers use PI significantly less than their private and nonprofit counterparts in most decision areas. Results are discussed within the literature on performance management and cross-sector differences.
Introduction
The ultimate value of performance information (PI) rests in its expected use by managers to build and enable organizational capacities with the goal of performance improvement (Moynihan, 2008). Researchers have focused on empirically identifying the organizational and individual factors that explain either the use or non-use of PI in managerial decision making (Ammons, Liston, & Jones, 2013; Moynihan et al., 2011; Moynihan & Ingraham, 2004; Moynihan & Lavertu, 2012; Moynihan & Pandey, 2010). These studies predominantly take one of two approaches. The first approach focuses on one type of agency, organization, or government level such as schools or school districts. The second approach pools managers across many different government agencies to examine the factors that influence performance management activities.
Studies have rarely examined the use of PI by managers to make decisions in specific decision areas. Rather, researchers focus on the use of PI in decision making more generally. This study examines purposeful PI (PPI) use, which is PI use by managers for specific purposes, such as personnel or allocation decisions. This study focuses on top organizational leaders’ general and purposeful PI use in the policy area of health care. With the exception of research on schools, most research focuses on managers but not top organizational leaders.
Given that public services are often provided by public, private, and nonprofit organizations due to co-production or the contracting out of services, this study fills a gap in the literature by examining the impact of institutional operating context (e.g., sector) on managers’ use of PI, while controlling for task. Research on organizational publicness and public–private comparisons finds significant cross-sector differences in managerial autonomy, goal clarity, and incentives that can influence managerial activities, such as managerial responses to political constraints and market pressures (Johansen & Zhu, 2014; Rainey & Bozeman, 2000). These sector differences can also affect PI use. We argue that to better realize the value of PI to improve public services, it is necessary to consider PI use by managers in the nonprofit and private sectors as well.
After a discussion on the literature on PI use and sector differences, this article presents hypotheses about factors that can influence managers’ PPI use, including sector, while controlling for task. These hypotheses are tested using data from a survey of nearly 1,000 top-level hospital managers from public, private, and nonprofit hospitals, which includes self-reported measures of PPI use in six decision areas. We find that the factors that influence general PI use also influence reported PPI use in specific decision areas. We also find that sector affects reported PPI use in certain decision areas (decisions related to personnel and service efficiency) directly and indirectly influence PPI use in other areas (decisions related to managerial strategies and resource allocation). The article concludes with a discussion of the findings within the literature on performance management and cross-sector differences.
Performance Management as PI Use
Organizations that provide public services are increasingly required to measure performance and report their measures to justify their actions by the outcomes they produce (Moynihan, 2008; Rainey, 2014). Many government agencies and other public service organizations, such as public schools, hospitals, long-term care facilities, and nonprofit charitable organizations, have adopted performance management systems (Hatry, 2006; Van Thiel & Leeuw, 2002). Performance management is a “system that generates performance information through strategic planning and performance measurement routines” and that connects this information (PI) to decision venues, where ideally the information influences a range of possible decisions (Moynihan, 2008, p. 5).
Despite the adoption of performance measurement in public and nonprofit organizations, researchers often find no or limited use of PI by decision makers (Ammons et al., 2013; Melkers & Willoughby, 2005; Pollitt, 2006; Ter Bogt, 2004). For example, Ammons et al. (2013) in their examination of city and county performance management practices noted that “most venture little beyond the reporting of measures” (p. 172). Scholars have also found limited use of PI by politicians and public managers (Pollitt, 2006; Ter Bogt, 2004). For instance, in a survey of aldermen in Dutch municipalities, Ter Bogt (2004) found limited use of PI in decisions and attributed this to the poor quality of information as well as a lack of information relevance to their day-to-day work or external stakeholders’ interests.
Basically, the measuring of performance is transformed into performance management only when PI is used to inform capacity or allocation decisions and strategic decisions that would involve making changes in goals, rules, or management structures for inducing performance improvement (de Lancer Julnes & Holzer, 2001; Hatry, 2006; Moynihan & Ingraham, 2004; Poister & Streib, 1999; Van Dooren, 2008). In other words, generating PI does not necessarily mean its utilization; measuring performance merely for internal or external reporting purposes will not necessarily lead to PI use for guiding management decisions and improving operations and services (Ammons et al., 2013; Moynihan & Ingraham, 2004).
Thus, scholars have raised questions about how and why public managers engage in performance management (Behn, 1995; Heinrich, 1999), and have tried to identify organizational or individual factors that could explain either the use or nonuse of PI in decision making (Ammons et al., 2013; Moynihan et al., 2011; Moynihan & Ingraham, 2004; Moynihan & Lavertu, 2012; Moynihan & Pandey, 2010).
In regard to individual decision makers’ characteristics, leaders with a higher level of public service motivation or transformational leadership, as well as a commitment to and engagement in measuring and managing performance, can result in greater use of PI in decision making (Ammons & Rivenbark, 2008; Moynihan et al., 2011; Moynihan & Ingraham, 2004; Moynihan & Pandey, 2010). Studies have also examined the exogenous political factors that drive PI use such as legislative involvement or citizen participation (Bourdeaux & Chikoto, 2008; Moynihan & Ingraham, 2004).
PI use is also framed as an issue of organizational support, because PI use is dependent on the commitment and support of those who use PI in decisions. For instance, organizational support for PI use in decision making can mean that the organization has a developmental culture, a risk-taking culture, or has learning forums (de Lancer Julnes & Holzer, 2001; Moynihan, 2008; Moynihan & Landuyt, 2009; Moynihan & Pandey, 2010).
However, scholars have paid less attention to the broader institutional context in which managers work, where “situational opportunities and constraints [can] affect the occurrence and meaning of organizational behavior as well as functional relationships between variables” (Johns, 2006, p. 386). The broader operating context is particularly salient to the management of public services provision, as many public services are increasingly provided by nonprofit and private organizations through contracting out or partnering across sectors (Frederickson & Frederickson, 2006). As such, this article examines the effect of an important institutional determinant, sector, on managers’ PI use for specific purposes.
Explaining Managers’ Use of PPI
Defining PPI Use
We conceptualize PPI as the deliberate, instrumental use of PI by managers for decision making in specific decision areas (Moynihan et al., 2012a, 2012b). In other words, PI use that is “purposeful” means that the manager uses PI for specific tasks, such as managing their programs or making resource-allocation decisions. This is a different approach from most performance management research, which focuses on managers’ use of PI for decision making more generally.
Cross-Sector Differences in PPI
The institutional context in which managers work is likely to affect managers’ use of PPI (Johns, 2006). There are many documented cross-sector differences such as red tape rules, formalized processes and procedures, stakeholders, goal clarity, career motivation, and organizational constraints and design that influence managerial behavior (Amirkhanyan, Kim, & Lambright, 2008; Boyne & Chen, 2007; Eggleston & Zeckhauser, 2002; March, 1991; Nutt & Backoff, 1992; Salamon, 2002). These factors can influence manager’s use of PPI as well.
One difference particularly relevant to PI use is sources of funding, which can have an impact on managerial incentives to use PI. For instance, public organizations have a dedicated revenue stream from the government, insulating them from the need to make a profit. However, government funding of public (and nonprofit) organizations is often accompanied by performance reporting mandates, which can create incentives to use PPI (Ben-Ner & Ren, 2015; Moynihan, 2008). The incentive to use PI to justify funding and be accountable to funders may be of particularly greater concern to nonprofit organizations (Hendricks, Plantz, & Pritchard, 2008). Private (and possibly also nonprofit) organizations, however, have to compete in the market for resources, and therefore have greater economic incentives to reduce costs and operate efficiently and effectively (Boyne & Chen, 2007; Johansen & Zhu, 2014). These differences in funding streams may influence managers’ PPI use due to differences in incentives to maximize organizational performance and effectiveness.
Another relevant difference is what Hvidman and Andersen (2014) refer to as managerial “capacity, including the autonomy, for making decisions and acting on them when PI is available” (p. 4). For instance, public organizations have civil service and purchasing and procurement systems that diminish public managers’ decision-making authority and flexibility (Rainey, 2014; Rainey, Pandey, & Bozeman, 1995). Compared with managers in private sector organizations, public managers have relatively low autonomy in making decisions due to a large number of procedural rules and regulations, and “public managers’ discretion on personnel issues is especially low because rules on hiring, firing and promotion are inflexible” (Boyne, 2002, p. 102).
Indeed, managerial capacity has an effect on PI use in the public sector; greater flexibility is associated with greater general PI use (Moynihan & Pandey, 2010). Nielsen (2014) finds in Danish schools that the lack of managerial authority, especially over personnel decisions, is a major impediment to benefiting from performance management even when managers are attentive and committed to tracking performance and developing measures. Similarly, lower levels of managerial autonomy and higher levels of bureaucracy that are often observed in public sector organizations may cause a lack of PI use despite its availability (Hvidman & Andersen, 2014). However, considering that government performance results are often provided through contracting or partnerships with third-party actors, public managers may be more likely to use PI in daily decision making as a means to control and manage this relationship (Moynihan & Hawes, 2012).
Many nonprofit organizations have challenges in developing and using performance management systems and information due to the lack of organizational/staff capacity or technological resources (Carman, 2007; Carnochan, Samples, Myers, & Austin, 2014). Even when they do measure performance, it may be mainly to comply with funder-mandated reporting requirements, which can lead to overly simplified performance data. Moreover, nonprofit organizations may find PI less useful for daily operations as it is often not compatible with actual service providers’ views of performance (Carnochan et al., 2014; Zimmermann & Stevens, 2006).
Given these cross-sector differences, we would expect to see cross-differences in the use of PPI related to variations in economic incentives and capacity (i.e., flexibility/autonomy). In decision areas where there is a higher economic incentive to use PI, we would expect to see greater PPI use. Considering theoretical arguments and empirical findings about cross-sector variations in economic incentives, we propose that private managers are more likely than their public and nonprofit counterparts to use PPI related to decisions regarding efficiency.
There is a similar relationship between variations in capacity and PPI use. In decision areas where managers may have less capacity, we would expect to see less PPI use. Taking into account both theoretical arguments and empirical findings, there are significant cross-sector differences in personnel and procurement processes, which result in differences in the managerial capacity of public, nonprofit, and private managers (Rainey & Bozeman, 2000). As a result, we propose that in decision areas where managers have less managerial capacity, such as areas related to personnel and procurement, managers are less likely to use PPI related to those decision areas. Specifically, we hypothesize the following:
Our expectations about nonprofit managers’ PPI use are less clear. This is because in some cases, nonprofit managers behave more like private managers in their responses to economic incentives, but more like public managers in their responses to the external political and community environment (Johansen & Zhu, 2014). The capacity of nonprofit managers to collect and use PI may be limited in multiple decision areas due not to processes and procedures but rather to lack of staff, resources, or technology (Carman, 2007; Carnochan et al., 2014). However, LeRoux and Feeney (2013) find that compared with public managers, nonprofit managers report greater discretion over their job functions and work schedules. Given this, we do not have specific propositions about PPI use by nonprofit managers apart from the expectation that there will be some differences in nonprofit managers’ reports of their PPI use compared with their private and public counterparts.
Perceived Self-Efficacy
In addition to the effect of sector on PPI use, we examine the influence of other factors, identified in the literature as affecting general PI use, on managers’ reported PPI use. One factor is perceived self-efficacy, or that leaders have power over clients. Perceived self-efficacy is a combination of the managers’ belief that PI “will help them to improve the outcomes of their work” (Kroll, 2015a, p. 473) and perceived social impact, which is the managers’ belief that their work meaningfully affects clients (Moynihan et al., 2012a, p. 468). In other words, managers with greater perceived self-efficacy are more likely to see the broader benefits of PI use to both their organization and the clientele they serve (Moynihan et al., 2012a).
In regard to PPI use, perceived self-efficacy may influence the use of PI to track and make decisions related not only to effective management of the organization but the organization’s social impact as well. PPI use for specific decision areas would provide the manager with a variety of means to benefit the organization and its clients (Moynihan et al., 2012a). Given this, it is reasonable to expect that managers who believe their actions can positively affect clients would be more likely to use PPI. Therefore, we hypothesize the following:
Receptivity to Feedback
Feedback plays a critical role in improving performance through ensuring greater goal clarity and inducing positive behavioral changes. Mahler, Casamayou, and Mahler (2009) note that in response to environmental changes or external forces, managers may engage in learning by analyzing relevant information to find solutions and directions for necessary changes. Feedback from clients or staff may signal to managers if the current performance level is acceptable, or what is working or not working. This feedback can motivate managers to seek more information to analyze the given information and find opportunities for performance improvement. However, some managers may hesitate to use feedback because they discredit negative feedback, or are unwilling to redesign programs or services in response to feedback (Barzelay, 1992; Mahler et al., 2009).
Given this, receptivity to feedback is related to one’s predisposition to seek, process, feel accountable, and use feedback (London & Smither, 2002). Managers that are more receptive to feedback are goal oriented and hold a higher expectancy value for feedback with the belief that it helps them get direct or indirect cues about the environment or areas that need correction for performance improvement (Van de Walle, 2004; Van de Walle & Cummings, 1997). For instance, Kroll (2015b), using a survey of managers in German cities, finds that managers’ receptivity to feedback, in which they believe using PI is important and brings more benefits than costs, predicts both their intention to and use of PI.
Given that managers who are more receptive to feedback place a higher value on PI, we expect that managers’ receptivity to feedback would also influence their PPI use. Therefore, we hypothesize the following:
Sensitivity to the External Environment
Previous research finds that pressure from external stakeholders heavily affects PI use (Kroll, 2015b; Kroll & Vogel, 2014; Moynihan & Ingraham, 2004; Yang & Hsieh, 2007). The influence of external interest groups such as elected officials, citizens, or unions greatly shapes PI use, and the lack or minimal use of PI may be attributable to a lack of commitment and interest from external stakeholders (de Lancer Julnes & Holzer, 2001). For example, Moynihan and Hawes (2012, p. 97) find that external stakeholders “solicit strategic responses from managers,” and that an increase in external stakeholders’ interest in performance data is associated with a greater likelihood of managers’ use of PI.
Moreover, literature on strategic management suggests that managers need to be sensitive to the external environment to better understand the organization’s weaknesses and strengths. Through this external intelligence gathering, managers seek information about newly developed services, economic trends, or any changes in the environment that may affect an agency’s ability to achieve their outcomes (Ireland, Hitt, Bettis, Porras, & Auld, 1987; Poister & Streib, 1999). In their external operating environment, managers seek the perceptions of service recipients and stakeholders on the organization’s performance, and use this information to interpret any emerging strategic issues, find a better fit between their environment and organization, and seek strategic alternatives (Bryson, 2011; Thomas, Clark, & Gioia, 1993).
Given the relationship between managers’ PI use and their attentiveness to external environmental changes facing their organization, it is reasonable to expect the same relationship with PPI use. Therefore, we hypothesize the following:
Valuing Employee Input
In addition to the external environment, managers rely on the internal environment, namely employees, for information on their current status and performance. For instance, Moynihan and Lavertu (2012) find that the existence of learning routines led by management, where employees periodically meet with management and discuss performance results, highly predicts managers’ PI use. The organizational learning literature also stresses the importance of giving employees opportunities to engage in inquiry efforts (Argyris & Schön, 1996; Torres & Preskill, 2001). Boyne, Gould-Williams, Law, and Walker (2004) find that the degree to which managers value and engage employees in evaluation activities predicts higher self-evaluation, which relates to the monitoring of performance and revision of performance goals if necessary.
The participation of internal stakeholders is a major factor in getting employees to buy-in to performance management processes as well as fostering managers’ PI use (de Lancer Julnes & Holzer, 2001; Moynihan & Landuyt, 2009). Valuing and getting employee buy-in on performance management practices and giving them ownership can facilitate managers’ use of PI in decisions (Boyne et al., 2004; Melkers & Willoughby, 2005). Therefore, it is plausible to expect a similar relationship with PPI use:
Empirical Context, Data, and Measures
Our empirical research context is the American hospital system, which for several reasons is a suitable setting for examining our questions about the key determinants of managers’ reported PPI use and to further compare reported PPI use across the three sectors. First, the U.S. health care system is a decentralized fee-for-service system that includes hospitals in the public, nonprofit, and private sectors. By focusing on hospitals, we can examine cross-sector differences while also controlling for task specialization. Second, all hospitals collect the same performance measures, as required by the U.S. Medicare Prescription Drug, Improvement, and Modernization Act of 2003. Under the Patient Protection and Affordable Care Act of 2010 (ACA), the U.S. federal government has linked its Medicare payments to hospitals to their performance, as measured by the efficiency of service delivery, quality of care, and patient outcomes. Moreover, several provisions of the ACA emphasize public reporting of hospitals’ performance data.
Third, national health care reform resulted in an increase in the visibility of hospital PI, which has made performance management an important priority for hospital managers. Despite facing similar accountability pressures in improving performance, hospitals across the three sectors differ in legal property ownership, revenue streams (particularly the level of dependence on government funding), clientele characteristics, and the sale of their services. Thus, although managers in the three sectors have comparable training and backgrounds, and are required to collect the same PI, managers face different incentives and internal and external institutional constraints due to sector differences, which may affect managers’ PPI use.
Public management scholars mostly rely on surveys to study managers’ PI use. For example, Moynihan and Pandey (2010) use survey data of U.S. city managers’ self-reported PI use. Kroll (2015b) relies on self-reported survey responses from German city managers. Our design is no exception. Data on hospital managers’ PPI use were collected from a mail survey of top-level managers (CEOs) in more than 6,000 private, public, and nonprofit hospitals, which covers general medical and surgical hospitals as well as a full set of specialized hospitals (e.g., specialized surgery, heart, cancer, chronic disease, children’s hospital, psychiatric, etc.). Almost one thousand responses were collected, rendering an overall response rate of 16%, which is considerably high compared with other large-scale surveys on American hospitals (e.g., Goldstein & Naor, 2005). 1 The survey supplies self-reported measures of managers’ PPI use in six decision areas, managerial tenure, community conditions, and demographics of top-level hospital managers.
While surveys are the most commonly used data-collection method, self-reported responses may be prone to reporting bias, such as common source bias (Meier & O’Toole, 2013; Favero & Bullock, 2015). Common source bias is particularly salient when researchers link subjective performance measures to self-reported management practices that have been collected from the same survey. To mitigate potential common source bias, it is better to use archival data for measures of objective organizational performance and organizational characteristics (Favero & Bullock, 2015; Meier & O’Toole, 2013). Our concerns about common source bias are minimal, given that our study does not involve any subjective organizational performance measures, and we use archival data from the American Hospital Association’s (AHA) Annual Hospital Survey Database to create variables for hospital ownership, size, service specialization, and other important organizational characteristics.
Dependent Variables: Managers’ PPI Use
To measure PPI use, we use survey responses to the following question: “Hospital administrators are provided with substantial detail on the performance of their hospital’s processes and units. To what extent do you use this type of performance data to: make personnel decisions, make strategic decisions, make day-to-day management decisions, advocate for my hospital with stakeholders, allocate resources, and learn how to make services more efficient?” For each decision area, respondents were asked to rate their PI use on a scale of 1 to 4, with 4 meaning very frequently and 1 meaning never. Our two tests for interrespondent reliability did not find evidence that different respondents read and interpreted the choice categories differently. 2 The descriptive statistics for these six measures, Personnel, Strategic, Day-to-Day, Advocate, Allocate, and Efficiency, and all other variables are provided in Table 1.
Descriptive Statistics.
Note. PI = performance information.
As shown in Table 1, on average, hospital managers more frequently use PPI in three decision areas: to make strategic decisions (mean PI use score = 3.588), to make resource-allocation decisions (mean PI use score = 3.419), and to improve service efficiency (mean PI use score = 3.388). Reported PPI use in the other three areas (personnel, advocate, and day-to-day) is somewhat lower. As a point of reference, the U.S. Government Accountability Office (GAO) found a comparable level of reported PI use in U.S. federal agencies based on its 2013 survey of agency managers. On a 5-point scale, the government-wide average PI use score is 3.41, which means that reported PI use among federal agencies is between “a moderate extent” and “a great extent.” 3
A seventh measure was created, PI Use Index, which is a factor score of the six reported PPI use measures. 4 This measure allows for easier comparison with findings in the literature, particularly when considering the impact of sector on managers’ PI use.
Independent Variables
Sector is conceptualized as ownership, and a hospital’s ownership is defined and coded based on the AHA classification of hospitals, which places all registered hospitals into one of three categories: public, private for profit, or nonprofit. We measure sector using dummy variables. Public hospitals are the baseline category and include hospitals that are owned by federal, state, county, city, or hospital districts. The Nonprofit hospital dummy variable includes church-operated hospitals and all other secular nonprofit hospitals that are tax exempt under Section 501c(3) of the Internal Revenue Code. The Private hospital dummy variable includes investor-owned for-profit hospitals, partnership and hospital corporations, and individual-owned hospitals. Our full sample includes 347 public, 476 nonprofit, and 186 private hospitals in 49 states and the District of Columbia. 5
A manager’s perceived self-efficacy in achieving outcomes is defined as the manager’s belief that his or her decisions influence the outcomes of their organization, which critically affects their clients’ health status. Our measure for Self-Efficacy is the response to the survey question: “My decisions directly affect my hospital patients’ well-being and health outcomes.” This measure was coded on a 1 to 4 scale, where 4 means strongly agree.
A manager’s Receptivity to Feedback is defined as an individual’s overall receptivity to feedback and the extent to which he or she considers feedback meaningful to his or her decisions (London & Smither, 2002). The measure is a factor score of two measures created from two survey prompts: “Non-manager feedback is taken into consideration when revising policies” and “Patient feedback and outcomes are taken into consideration when revising policies.” We use receptivity to feedback from staff and patients as our measure because feedback from these groups produces performance data critical to understanding challenges in providing quality service or areas that need improvement. Consideration of this type of feedback is a proxy of their receptivity to all feedback. Higher values indicate greater receptivity.
We define a manager’s Sensitivity to changes in the external environment by how the manager responded to the ACA health care reforms. During the time of the survey, the enactment of the ACA was a major change to the external political and market environment of American hospitals. In our survey, we gave hospital managers the prompt, “Our hospital is preparing for health care reform now by making specific changes in operations or financial strategies.” This measure ranges from 1 to 4, with 4 meaning strongly agree. A manager who is sensitive to changes in the external environment would be more proactive in leading his or her organization to prepare for the coming health care reform.
Valuing employee input is defined as the value a manager places on including internal stakeholders in the decision-making process. Therefore, our measure for Valuing Employee Input is a factor score of two measures created from the responses to two survey prompts: “I involve line staff (non-managers) in my hospital’s decision-making process” and “I involve practicing physicians in my hospital’s decision-making process.” Higher values indicate that a manager places a higher value on employee input.
Controls
We include three groups of variables to control for other factors that may affect PPI use. We first control for managerial and organizational factors internal to a hospital. Managerial Tenure is measured based on the number of years a hospital CEO has been in his or her position. Based on our sample, the average managerial tenure of hospital managers is around 13 years. Female manager is coded 1 if a CEO is female. Minority manager is coded as a dummy variable, with 1 referring to non-White managers. As shown in Table 1, hospital CEO is a White, male-dominated occupation. Female managers account for about 23% of the sample, and minority managers account for only about 5% of the sample. We control for these factors as they can influence managerial practice (Gonzalez Juenke, 2005; Johansen, 2007; Karnig & McClain, 1988; Molina, 2016).
We also control for a manager’s professional training as that training can affect managerial behavior (Teodoro, 2014). We measure managers’ Professional Training and Background by the type of school from which a manager earned his or her highest degree. Based on our survey responses, there are three major types of professional backgrounds for hospital CEOs: (a) professional health administrators trained by public health schools (e.g., MA or PhD in public health), (b) medical professionals who hold a medical degree (e.g., MD), and (c) professional managers trained by business schools. In our data, managers with these different professional backgrounds can be found across the three sectors, as none of the three sectors are uniquely linked to one professional tradition. In our analysis, we use a dummy variable for managers with a business school background and a dummy variable for managers with a medical school background, leaving managers with a public health school background as the baseline group. In our sample, about 26% of hospital CEOs have business school degrees, about 29% hold medical-school-related degrees, and the rest are trained by public health schools or public administration programs.
Hospital Size is measured based on the AHA categorization of hospitals’ inpatient bed capacity. This variable is measured with a 1-to-8 ordinal scale. Category 1 includes small hospitals that have less than 25 inpatient beds, and Category 8 includes large hospitals that have more than 500 inpatient beds. Table 1 shows that the average hospital size is roughly 4, which refers to about 100 inpatient beds. 6 We control for size as larger hospitals may have more performance measurement processes making it easier to collect and use PPI, and size can affect both managerial incentives and capacity to use PPI (Kroll, 2015a).
We also control for how managers perceive various community factors that may affect their use of PPI. Active Community is measured with a 1-to-4 ordinal scale about how active citizens are in community issues and public affairs, with 4 meaning the manager considers the community very active. The variable Generous Community is measured with a 1 to 4 scale about the base of charitable contributions to local foundations that fund health care, where 4 means a manager considers the area very generous. The variable Low-Income Community is measured with a 1-to-4 ordinal scale regarding whether the hospital’s service area is in a low-income area, where 4 means a manager considers the area an extremely low-income area. 7
Results
Before looking at the results obtained from the regression analysis, we first compare reported average PPI use across sectors with a simple t test to examine if the cross-sector differences are statistically significant. Table 2 reports these comparative analyses. Overall, public hospital managers’ reported PPI use is lower compared with nonprofit and/or private hospital managers for most decision areas, and the cross-sector differences are statistically significant. 8 The only exception is for Advocacy, where both public and private hospital managers have lower averages than nonprofit hospital managers.
Average Managers’ Purposeful PI Use by Sector.
Note. PI = performance information.
Statistically significant from nonprofit hospitals (p = .020) and private hospitals (p = .007).
Statistically significant from nonprofit hospitals (p = .027) and private hospitals (p = .050).
Statistically significant from public hospitals (p = .007) and nonprofit hospitals (p = .031).
Statistically significant from public hospitals (p = .008) and private hospitals (p = .069).
Statistically significant from nonprofit hospitals (p = .001).
Statistically significant from nonprofit hospitals (p = .001) and private hospitals (p = .000).
Statistically significant from nonprofit hospitals (p = .001) and private hospitals (p = .006).
Specifically, as shown in Table 2, public hospital managers scored an average of 3.10 on their reported use of PPI to make personnel decisions, compared with an average of 3.22 for nonprofit managers and an average of 3.28 for private managers. These means are statistically different from public managers’ (p = .020 and p = .007, respectively). For day-to-day PPI use, public hospital managers scored an average of 3.27, nonprofit managers scored 3.31, and private managers scored 3.44. The scores for nonprofit and public managers are statistically different from private managers’ (p = .031 and p = .007, respectively). For efficiency, public hospital managers scored an average of 3.29, nonprofit managers scored 3.42, and private managers scored 3.48. Results of a t test show that the scores from nonprofit and private managers are statistically different from those obtained for public managers (p = .001 and p = .000, respectively). Thus, we find some support for H1 and H2.
Table 3 presents the results from seven linear regression models, one each for the six measures of reported PPI use and one for the PI use index measure. Regarding the influence of sector on our six PPI use measures, we find that private managers report significantly more PPI use than public hospital managers in three decision areas: making personnel decisions, making day-to-day decisions, and decisions aimed at improving service efficiency. Being a private hospital manager results in a 0.141 point increase in the reported use of PPI to make decisions regarding personnel on a 4-point scale compared with public hospital managers. A private hospital manager’s reported use of PPI to make day-to-day decisions and make decisions regarding efficiency is 0.127 points and 0.134 points higher, respectively, than public hospital managers. Nonprofit managers do not report that they significantly use PPI differently than public managers. In regard to the PI use index measure, sector is not a significant predictor of PI use. Taken together, these findings provide support for H1 and H2; sector can directly influence a manager’s use of reported PPI in certain decision areas related to incentives and capacity.
Predicting Managers’ Purposeful PI Use.
Note. PI = performance information.
p < .05.
For our other proposed predictive factors, we find consistent evidence that Self-Efficacy, Receptivity to Feedback, and Valuing Employee Input have a significant and positive impact on reported PPI use in all seven models. As a manager’s perceived self-efficacy becomes stronger, his or her reported use of PPI increases in all six decision areas. The same can be said for a manager’s receptivity to feedback; the more receptive a manager is to feedback, the more likely he or she is to report that they use PPI when making decisions. Moreover, the more a manager values the inclusion of employees in the decision-making process, the more the manager will report that he or she uses PPI. The effect of these three predictive factors is also significant for our overall index of PI use. In sum, we find support for H3, H4, and H6.
We also find some support for H5; Sensitivity to the external environment is a significant factor in three decision areas: Personnel, Advocate, and Allocate but not in the other three areas: Strategic, Day-to-Day, and Efficiency. Managerial attention to the environment may not influence the day-to-day and efficiency decision areas because the external environment may not directly affect the day-to-day workings and operations of a hospital. However, the external environment is critical for hospitals to hire and retain talent, to draw charitable donations, and to advocate public service values. Thus, it is reasonable that they are more likely to use PPI in these decision areas. Managers’ sensitivity to the environment also has a significant and positive association with the overall PI use index. We do not find evidence that sensitivity to the external environment significantly predicts PPI use in making strategic decisions.
In regard to the control variables, Hospital Size has a significant and positive influence on the PI use index measure and three of the six reported PPI use measures. The variation in the significance of organization size across our PPI use measures comports with Kroll’s (2015a) finding across multiple PI use studies that the effect of organization size on PI use is inconsistent. Given that for the measures where size may be more salient (strategy, resources, and efficiency), size matters because larger organizations are likely to have more sophisticated systems dedicated to the more efficient use of resources in both short and long term. A Business School background has a significant effect on reported PPI use in strategic decisions. An Active Community has a significant effect on reported PPI use in strategic decision making, advocating with stakeholders, and the PI use index measure.
Given that we do not observe a direct impact of sector on reported PPI use in three areas (making strategic decisions, advocating for the hospital, and making resource-allocation decisions), we further explore if sector has an indirect impact on PPI use in these decision areas by examining if our key managerial style variables affect reported PPI use differently across the three sectors.
Table 4 presents regression models for reported PPI use for making strategic decisions and resource-allocation decisions, with separate samples for public, nonprofit, and private hospitals. 9 Table 4 shows that Sector exhibits some indirect impact on reported PPI use in two decision areas (Strategic and Allocate), in that the effect of a few managerial style variables on reported PPI use have different magnitudes. For Strategic decisions, we observe that Self-Efficacy is positively associated with reported PPI use, but the positive effect is greater in the private sector (b = 0.284, t = 3.53) than in the public (b = 0.205, t = 3.47) and nonprofit sectors (b = 0.117, t = 2.37). Receptivity to Feedback also positively affects PPI use for Strategic decisions, and that positive effect varies by sector; the positive effect is smaller in the private sector than in the public and nonprofit sectors. Similarly, both Self-Efficacy and Receptivity to Feedback are positively associated with PPI use for making resource allocation decisions (Allocate), and these positive effects vary by sector. The positive effect of Self-efficacy is greater in the private sector than in the public and nonprofit sectors. The positive effect of Receptivity to Feedback on PPI use for resource allocation, however, decreases as we move from public to nonprofit and private hospitals.
The Indirect Impact of Sector on Performance Information Use for Strategic and Resource-Allocation Decisions.
p < .05.
Discussion and Conclusion
The purpose of this study was to examine whether managers use PI in various decision areas differently, and to what extent PPI use may differ across the public, nonprofit, and private sectors. Distinct from previous research that examines PI in the public or nonprofit sector (LeRoux & Wright, 2010; Moynihan & Lavertu, 2012), our study is one of the first to compare top managers’ PPI use across all three sectors while controlling for task. Looking at managers’ reported PPI use in six decision areas, results show that PPI use is explained by both sector and management styles. The sector the manager is in significantly affects the reported use of PPI in decisions involving personnel, day-to-day operations, and service efficiency. Managers’ perceived self-efficacy, receptivity to feedback, and valuing of employee input also have a consistently significant influence on PPI use.
When looking at managers’ overall PI use and PPI use in different decision areas, it is reasonable to conclude that cross-sector differences matter. Managers need both the incentive and the capacity to use PI (Boyne & Chen, 2007; Moynihan, 2008). We observe significant sector influences on reported PPI use in three decision areas where sector differences in incentives and capacity may be strongest: personnel decisions, day-to-day operations, and improving service efficiency. As we have previously argued, managers in public organizations have diminished capacity due to civil service systems that weaken autonomy over subordinates, and thus are less likely to use PPI in decision areas related to personnel issues (Feeney & Rainey, 2010; Rainey, 2014). Private sector managers often have greater capacity regarding personnel decisions and thus are more likely to use PPI in that decision area.
Economic incentives, such as those related to improving efficiency, are more likely to motivate private and to a lesser extent nonprofit managers, who have a greater capacity to act on PI (Boyne & Chen, 2007; Hvidman & Andersen, 2014; Johansen & Zhu, 2014; Moynihan, 2008). As we hypothesized, differences in PPI use for service efficiency decisions may be due to differences in incentives. In such cases, private managers are likely to have more of an incentive to focus on service efficiency than public and nonprofit managers.
Our examination of the impact of sector on PPI use provides additional evidence about the effect fundamental sector differences can have on managerial practice. Bureaucratic structures and a lack of market competition can demotivate or disincentivize public managers from engaging in effective management behaviors such as performance management. In cities where top leaders’ (like city mayors) commitment to performance management is evident, such as in New York City or Baltimore, we may be able to observe more PPI use by public managers. Our understanding of public PPI use could be improved and expanded by adding case studies of hospitals in those cities. Moreover, public managers are much more susceptible to political forces, which may be able to counterbalance any disincentivizing forces (Johansen & Zhu, 2014). One limitation of this research is that political forces were not included in the analysis.
The results contribute to our understanding of nonprofit organizations as well. Nonprofit managers sometimes behave similarly to public managers, due to their public service orientation and their responsiveness to political forces. Sometimes, they behave similarly to private managers, as they have fewer constraints due to bureaucratic processes and a similar susceptibility to market forces (Johansen & Zhu, 2014). In our study, the one area in which nonprofit managers report their use of PPI differently from both public and private managers is the area of advocacy. This may possibly be due to their heavy reliance on funders. Hendricks et al. (2008) argue that nonprofit organizations engage in performance management as a way to justify funding and to show accountability to their funders and the public. This is particularly important in the nonprofit sector as an organization needs to maintain their reputation and the trust of their community to attract funding and clients. Future research should include political and other external forces in the examination of the link between PPI use and the role of nonprofit organizations in advocacy.
With regard to management styles, a managers’ perceived self-efficacy in achieving performance outcomes, receptivity to feedback, and valuing employee input are significant predictors of PPI use. As Van Dooren (2008) notes, “Whether performance information is used and how it is used depends on the motivations of potential users and utility to performance information to their goals” (p. 27). This is because when managers are interested in improving performance outcomes, they want information to discover “why the outcomes are what they are” and what should be done (p. 64). Newcomer (1997, p. 10) suggests that managers “must venture beyond the performance data,” suggesting managers use PI to address questions of “how” and “why” that would in turn inform internal management decisions in various operational areas. Our findings offer empirical support for the importance of having managers with perceived self-efficacy to affect performance outcomes to foster PPI use. The availability of relevant information and its quality may moderate the relationship between managers’ perceived self-efficacy and their use of PPI in decisions (Bourdeaux & Chikoto, 2008; de Lancer Julnes & Holzer, 2001; Moynihan & Ingraham, 2004) but this direction can be examined in future studies.
Evidence regarding managers’ feedback receptivity and valuing of employee input also signals the importance of internal decision-making procedures in fostering PPI use. Previous studies on PI use in public agencies observe the prevalence of public managers’ use of PI to display the organization’s positive qualities to advocate for the legitimacy of the organization to external stakeholders, or just to comply with legal requirements (de Lancer Julnes and Mixcóatl, 2006; Moynihan et al., 2011). Our examination of reported PPI use in different sectors finds that PPI use may not just be driven by external political pressures from stakeholders, rules, and legal requirements. Instead, feedback receptivity is positively related to openness to new ideas and learning (London & Smither, 2002). Specifically, people with a greater receptivity to feedback find feedback constructive and instrumental to the achievement of their goals, and are more likely to use the information (London & Smither, 2002). Given our positive and significant findings about the relationship between managers’ feedback receptivity and their use of PPI in decisions, scholars should look for ways to develop managers’ receptivity to feedback. In addition, valuing the inclusion of employees in decision making and getting employee buy-in with the measurement process can foster the use of PPI in decision making (Moynihan, 2008; Moynihan & Landuyt, 2009).
To conclude, this study shows the importance of examining managers’ PPI use because it provides the nuances needed to better understand how PI use can be influenced by sector differences and various management styles. Moreover, our understanding of performance management is improved by focusing on a different policy area (health care) and a different level of management (top organizational leaders). As stated earlier, performance measurement is only valuable if managers use it. Studying PI use for specific purposes, and the instances where the context in which a manager works affects its use, can provide much needed insight into realizing the value of performance management to the provision of public services.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.