Abstract
In the face of increased accountability pressures, nonprofits are searching for ways to demonstrate their effectiveness. Because meaningful tools to evaluate effectiveness are largely absent, financial ratios are still the main indicators used to approximate it. However, there is an extensive body of literature on determinants of nonprofit effectiveness. In this study, we test the extent to which these assertions in the literature align with practitioner views. To increase the practical value of our comparative exercise, we create a self-assessment survey on the basis of the practices that find support in both academia and practice. This provides managers with a tool to assess the extent to which the identified practices are present in their organizations and with suggestions, which might lead to improvements in their effectiveness. Intermediaries can use the tool to provide better information to donors. Funders can use it in their selection of grantees, and capacity-building efforts.
Keywords
“In the land of the blind, the one-eyed man is king.”
Introduction
Nonprofit organizations (NPOs) are increasingly pressured to demonstrate their effectiveness (Carman, 2010; Ebrahim & Rangan, 2010; Saxton & Guo, 2011). Rating agencies such as American watchdogs GuideStar and Charity Navigator have picked up on this need, but their models still rely heavily on financial ratios rather than indicators of effectiveness (Bhattacharya & Tinkelman, 2009; Gordon, Knock, & Neely, 2009; Stork, 2007; Tinkelman & Donabedian, 2007). Moreover, multiple academic efforts have tried to define (e.g. Buckmaster, 1999; Herman & Renz, 1999), measure (e.g. Kaplan, 2001; Sowa, Selden, & Sandfort, 2004), and identify practices that enhance NPO effectiveness (e.g. Callen, Klein, & Tinkelman, 2003; Smith & Shen, 1996). However, these studies also often rely on financial measures to the prime indicators of NPO effectiveness (e.g., Brown, 2005; Provan, 1980).
There are a few exceptions in which conceptualizations of effectiveness are based on reputation (e.g., Chait, Holland, & Taylor, 1991; Herman & Renz, 1997, 2000), outcomes or impact (e.g., Edwards, 1999; Eisinger, 2002), or multi-dimensional models (e.g., Cameron, 1982; Kushner & Poole, 1996). As Tinkelman and Donabedian (2007) have argued, the relative emphasis on financial measures over other measures of effectiveness is probably because of the complexity of conceptualizing and measuring organizational effectiveness (OE), in particular that of NPOs, in a way that allows for meaningful comparison between organizations (Kanter & Summers, 1987; Letts, Ryan, & Grossman, 1999; Mitchell, 2013; Sawhill & Williamson, 2001; Speckbacher, 2003).
Despite these difficulties in researching OE, increased demands for transparency and accountability of NPOs make the topic a pressing concern for both nonprofit scholars and practitioners (Carman, 2010; Ebrahim & Rangan, 2010; Lecy, Schmitz, & Swedlund, 2012; Ebrahim, 2010). Several authors have warned of the detrimental effects that the current one-sided information supply can have on the NPO marketplace (Goggins Greggory & Howard, 2009; Lowell, Trelstad, & Meehan, 2005; Tinkelman & Donabedian, 2007). They call for models based on practices that relate more closely to nonprofit effectiveness (Tinkelman & Donabedian, 2007). This study provides an additional contribution to formulating a set of alternative indicators of effectiveness.
Multiple studies have looked at the past literature on OE to further our understanding of the topic. However, most of these studies have focused either on conceptualizing and operationalizing the concept of OE itself (e.g., Beamon & Balcik, 2008; Kaplan, 2001; Sowa et al., 2004), or have focused on one specific determinant of OE such as board effectiveness (e.g., Herman & Renz, 1997, 1999, 2000). To the best of our knowledge, only the literature reviews performed by Forbes (1998) and Lecy et al. (2012) have provided comprehensive overviews of the body of knowledge on OE. Where Forbes included both research on effectiveness determinants and conceptualizations of OE, the review by Lecy et al. focused only on the latter. There is a lively body of research on specific subsets of determinants of NPO effectiveness, such as strategic management (Herman & Renz, 1997; Siciliano, 1997) and board performance (Brown, 2005; Provan, 1980; Smith & Shen, 1996). However, there is no recent comprehensive overview of the academic literature on determinants of OE.
In this study, we conduct a literature review of putative nonprofit OE determinants. As Bushouse and Sowa (2012) have argued, it is also our duty as nonprofit scholars to make sure our research is relevant for practice. Therefore, we test the extent to which the list of effectiveness enhancing practices based on the assertions of academic researchers resonates with the views of a group of practitioners from Dutch NPOs. It is important to note that we do not intend to offer a substitute for the challenging and important work on the evaluation of program effectiveness with the set of practices identified in this study. Our process approach to effectiveness discusses practices that many (though not all) academics and practitioners believe contribute to effectiveness in NPOs. Rigorous research that measures the actual impact of these practices on short- and long-term OE is still needed.
Still, the potential contributions of this study are numerous. First and foremost, the literature review offers academics and practitioners an overview of the proposed determinants of OE in the nonprofit literature. For researchers, the study reveals the extent to which practitioner views parallel their findings and assertions. To increase the practical value of our comparative exercise, we created a self-assessment survey on the basis of the practices that find support in both academia and practice. This provides managers with a tool to assess the extent to which the identified practices are present in their organizations, and gives them suggestions on how they might change them to improve their effectiveness. Intermediaries, such as consultants, can use the tool to provide better information to donors. Funders might use it as an aid to decision making in their selection of grantees, and to support their capacity building efforts for grantees.
Last, we demonstrate that, despite the difficulties in defining and measuring comparative effectiveness of nonprofits there is practical value in the rich literature on organizational practices that may contribute to OE. Thus, this study is a first step in providing a counterweight to the emphasis on financial measures that are currently the dominant source of information for allocation decisions in the making for philanthropic donations (Goggins Greggory & Howard, 2009; Gordon et al., 2009; Grossman, 1999; Tinkelman & Donabedian, 2007; Wing & Hager, 2004a; Lecy & Searing, 2012).
Theoretical Background
The Elusive Concept of OE
For over a decade, complaints about the inefficiency and ineffectiveness of NPOs have been increasing (Carman, 2010; Ebrahim & Rangan, 2010; Edwards & Hulme, 1996; Murray, 2005). A decrease in public trust in NPOs has resulted in a demand for greater transparency about, and accountability for, the difference NPOs make to society (Ebrahim, 2003, 2010; Edwards & Hulme, 1996; Rey Garcia, 2008). As a result there is an urge to develop theories and tools that enable NPOs to demonstrate their OE (Alexander, Brudney, & Yang, 2010; Anheier, 1994; Carman, 2010; Mitchell, 2013; Yates, 2004; Murray, 2010). The academic literature on NPO effectiveness offers a variety of OE models (Mitchell, 2013). In his review of this literature, Forbes (1998) found that, despite the diversity in the ways effectiveness has been studied, most draw on one or a combination of three major models: (a) goal attainment model (Bernard, 1938; Price, 1972), (b) systems resource model (Georgopoulos & Tannenbaum, 1957; Yuchtman & Seashore, 1967), or (c) reputational model (Jobson & Schneck, 1982; Herman & Renz, 1997). In reviewing these models, Herman and Renz (1997, 1999) have argued that most of the actual research has focused solely on the first model of OE, the goal attainment model.
The goal attainment model defines OE as the achievement of self-selected goals. This definition of OE results in a number of problems when operationalizing the concept, including subjectivity (Mohr, 1982), incomparability between organizations (Etzioni, 1964; Forbes, 1998; Lecy et al., 2012; Mohr, 1982), and a lack of concretization and subsequent possibilities for evaluation (Fowler, 1996; Herman & Renz, 1999; Mohr, 1982; Murray & Tassie, 1994; Stone & Cutcher-Gershenfeld, 2001).
In response to these limitations, models were developed that provided what were claimed to be proxies for OE. For example, the systems resource model uses organizational survival or growth as a proxy for OE, and the reputational model relies on the various perceptions relevant stakeholders have of an organizations’ effectiveness (Forbes, 1998; Herman & Renz, 1999, 2004, 2008; Lecy et al., 2012). The reputational model is based on the post-modern belief that there can be no objective definition of a concept, as its true definition lies in the perception of individuals, regardless of how diverse they may be (Herman & Renz, 1999; Packard, 2010; Shilbury & Moore, 2006; Tassie, Murray, & Cutt, 1998). Mitchell (2013), for example, offers a model of OE that is based on the perceptions of leaders of international NPOs. In his study, most leaders defined the effectiveness of their NPO as “outcome accountability” or, to a lesser extent, “overhead minimization” (Mitchell, 2013, pp. 9, 12).
Lastly, Herman and Renz (2008) and Lecy et al. (2012) have argued that there is great value in multi-dimensional models such as Quinn and Rohrbaugh’s (1983) competing values model, Kaplan’s (2001) balanced scorecard, and Ebrahim and Rangan’s (2010) contingency model. However, according to Lecy et al., these models do not provide a novel perspective of OE but simply “incorporate aspects of the goal, system resource and reputational approaches” (Lecy et al., 2012, p. 7).
Accountable for Effectiveness But Rated by Financial Ratios
Despite the many conceptual and theoretical models of OE, “the practical challenge of measuring organizational effectiveness persists” (Mitchell, 2013, p. 3). A manifestation of the desire for comparative measures of OE in the nonprofit sector is the growing number of self-appointed watchdogs that have emerged in the past decade. Though they initially focused primarily on financial data, some have recently started to search for measures that relate to other indicators of OE (Lowell et al., 2005; Mitchell, 2013). Some of the most promising efforts include Givewell’s provision of extensive analyses of charities’ program evaluations (www.givewell.org); Charting Impact’s five questions that push NPOs to reflect on their goals and achieved impact (www.chartingimpact.org), and Philanthropedia’s crowdsourcing of expert opinions of NPOs, their impact, organizational strengths, and areas of improvement (www.myphilanthropedia.org).
Despite the ability of these initiatives to steer away from a singular focus on financial measures, there are limitations to their individual ability to provide sufficient counterbalance against the existing tools. These limitations include the descriptive nature of the information, which complicates comparisons; a focus on program effectiveness instead of organization-wide effectiveness; the labor intensity required to generate their measure; and the subjective aspects of expert opinion such as being overly impressed by the personalities of certain leaders (Deluga, 2001; De Hoogh, Den Hartog, & Koopman, 2005; Hoyt, 2000).
These limitations clearly illustrate the advantages financial measures have in terms of their scalability, collectability, level of objectivity, and comparability across organizations. The ratios that are calculated from financial data are quantitative and operational criteria are more readily available and easily comparable than more qualitative and normative approaches (Rey Garcia, 2008).
Turning from the problems of defining and measuring OE, academic research in the for-profit (e.g., Flood, 1994; Quinn & Rohrbaugh, 1983), nonprofit (e.g. Baruch & Ramalho, 2006; Carman, 2010), and public sectors (e.g., Rainey & Steinbauer, 1999; Wolf, 1993) has attempted to identify relevant organizational and managerial strategies that will impact OE. An early example is the empirical study by Cameron (1986), who found that the presence of certain managerial strategies had a strong positive relationship with certain static and longitudinal indicators of OE. Moreover, these strategies were found to be more important than the organizations’ structures, the demographics of its members, and certain indicators of its finances (Cameron, 1986). Forbes (1998) found that similar studies on determinants of OE have mostly focused on specific topics such as boards.
In Table 1, an overview of both empirical and conceptual studies of determinants of OE illustrates the frequency of a narrow focus on one specific indicator of OE (e.g., board performance). Roughly three types of OE indicators can be identified: (1) financial standards, often based on financial ratios such as overhead ratio or fundraising ratio; (2) perceived effectiveness, such as reputation or program-based outcome/impact measures; and (3) multi-dimensional indicators. The overview also illustrates the number of studies claiming to research determinants of OE that mainly look only at financial indicators. This underscores the relevance of the warning that a lack of shared definitions and agreed on ways of operationalizing OE threatens progress “in our collective understanding” of the concept (Lecy et al., 2012, p. 5).
Empirical and Conceptual Indicators of OE.
Note. IV: independent variable in the study; OE = organizational effectiveness.
Reviewed in Forbes (1998).
Method
The purpose of this study is to identify management practices that have been hypothesized as influencing NPO effectiveness from the rich base of the literature on this topic and test the extent to which practicing managers agree or disagree with them. Figure 1 shows the results of the first phase of this research. Due to the complexity in defining appropriate search terms to guide a systematic literature review of hypothesized OE determinants, a “snowball” technique was used (Greenhalgh & Peacock, 2005). The articles identified by this method were from either peer-reviewed journals with an ISI/Social Science Citation Index listing or chapters in books specifically focusing on NPOs as indicated by the word nonprofit in the title. The starting points for the snowballing were two widely used handbooks on research in NPOs: “The Nonprofit Sector: A Research Handbook,” the second edition published in 2006, and “The Jossey-Bass Handbook of Nonprofit Management and Leadership,” published in 2010.
Multi-step method selection of determinants.
For the second phase, a focus group was set up comprising 20 actors from the Dutch nonprofit sector. It included consultants, managers, chief executive officers (CEOs), in-house evaluators, and academic researchers. They were brought together because of the diversity of their perspectives relative to those of the authors. They also were a representation of the actors noted in the Dutch NPO “Knowledge Bank of Philanthropy” (kennisbankfinantropie.nl), which functions as the central online database of the Dutch nonprofit sector.
This group was asked to draw up as many management practices as they could that they believed to contribute to effectiveness in NPOs. In other words, they were asked to think of the so-called “best practices” that, when adhered to by NPOs, increased the chance that the NPO would be effective. It is important to note that the group was unaware of the results of the literature review and thus started with a tabula rasa.
The literature review resulted in a total of 52 management practices that were hypothesized to have an impact on NPO effectiveness or that were found empirically to be correlated with certain measures of OE. It is important to note that widely used measures about which there is no consensus with regard to their relation to effectiveness, such as overhead ratios, were not included (Wing & Hager, 2004b). Acting independently, the focus group, consisting predominantly of practitioners, drew up a list of 36 unique determinants that they believed would enhance OE (see Table A1 in the appendix for an overview of these practices). Interestingly, with the exception of two of them (size of the marketing budget and employee turnover), all of the practices identified by the focus group were part of the list of effectiveness enhancing practices from the literature review (although sometimes slightly differently formulated). When the two lists were merged, a set of 34 practices were identified that found support both in the literature and practitioner group (see Figure 1 for a graphic presentation of this elimination process).
On the basis of the commonalities between the two lists, a survey was designed to measure the extent to which NPOs have these practices in place.
In some cases, it was felt that certain conditions were too subjective to identify reliably and hence should not be included in the final survey. Examples of these are the perceived presence of charismatic leadership, willingness to change, ambition, and employee motivation. It was decided that to be included, an organizational process or practice had to be one that is objectively verifiable.
To check whether the questionnaire items would be understood correctly, and whether, when asked, NPOs would answer them reliably, an additional questionnaire item (control question) was added, when possible, for each practice. In most cases, this required NPOs to give an example, or to answer the same question formulated in a different way. A preliminary online questionnaire with the remaining items was then compiled based on guidelines in “Mail and Internet Surveys” by Dillman (2000).
In the fourth step, this online questionnaire was tested (Table A2). First, the expert group was invited to review each item as to whether it should be included, and suggest a new item if necessary. Next, to test the perceived level of difficulty of the items, and the time and effort required to complete the questionnaire, one representative of four NPOs of different organizational sizes, activity types, levels of professionalism, and ages since founding was asked to complete the online questionnaire following the “three-step test interview” (TSTI) process that requires respondents to think out loud while filling in a survey and give suggestions for improvements (Hak, van der Veer, & Jansen, 2008).
The final questionnaire was then uploaded to the online database of the “Knowledge-Bank Philanthropy” (kennisbankfilantropie.nl), which manages the data-profiles of NPOs in the Netherlands. This database currently contains over 5,000 Dutch NPOs, which is close to the estimated total number of active NPOs in the Netherlands. A pilot study was then conducted involving all NPOs that have a focus on improving health (N = 554).
This pilot study served five purposes. First, through its response rate it showed both the willingness and ability of NPOs to provide answers to the survey questions. Second, it revealed the extent to which NPOs provided reliable responses to the survey. Third, the data allowed for an analysis of whether some practices were clearly understood. Fourth, it was also possible to ascertain whether some practices were more or less likely to appear in organizations of a specific size or activity type (e.g., research). Fifth, to control for the presence of a non-response bias, an archival analysis was carried out in which respondents and non-respondents were compared on the basis of a number of organizational characteristics as suggested by Armstrong and Overton (1977).
Results
Of the 554 health-focused organizations, 83 fully completed the 29-item questionnaire with the matching control questions, resulting in a response rate of 15% (see Table A3 in the appendix for the questionnaire used in the pilot study). The pilot study results indicated that NPOs’ self-reports were unreliable in only 5% of the cases. This was determined by analyzing whether the NPOs that reported that they engaged in a practice could subsequently provide an example of this or could consistently give the same answer on the matching control item. The results from the pilot pointed to three management practices that provided to be unclear. First, the questionnaire item that asked about the presence of a logic model was often misunderstood. Second, only 5% of the NPOs that claimed to measure outcomes could provide a satisfactory example of such a measure. Third, it was found that the item that asked about the use of external evaluations was predominantly reported by only large NPOs. Therefore, these three variables were deleted from the list. Regarding the presence or absence of practices in different types of organizations, no substantial biases toward larger or smaller NPOs were detected. Nor were there any significant biases toward NPOs focused on specific types of activity (e.g., research) or on a specific target group (e.g., the poor). Investigating the non-response bias, no structural differences were found when using an archival analysis between the respondents and non-respondents on the basis of the available information of their organizational characteristics.
As a result of this analysis of the pilot study results, a final questionnaire was constructed consisting of 26 items that are presented in Table 2. To facilitate easier interpretation, the items are clustered into nine themes, which are grouped into three pillars that relate to transparency, program, or organizational characteristics (Sowa et al., 2004). The following section discusses the potential value of these themes in contributing to OE (even though definitive evidence showing causal connections remains to be found for the reasons already discussed).
Final Set of Determinants of Nonprofit OE.
Note. OE = organizational effectiveness.
Does not limit to a specific type of evaluation.
Refers to beneficiaries where possible or otherwise other relevant constituents or experts (please refer to the set requirements depicted in Appendix Table A4 that specifies the required group type to be involved depending on type of interventions).
Same criteria as in “design” but now with the function to create ownership.
Transparency
Transparency is often argued to be one of the most important processes by which NPOs render themselves accountable to their stakeholders (Bothwell, 2001; Edwards & Hulme, 1996). Transparency is the first step in creating the ability for stakeholders, such as donors, clients, and others to hold an NPO accountable (Bejar & Oakley, 1996; Cheng, 2009; Edwards & Hulme, 1996). Transparency not only enables stakeholders to keep NPOs accountable relative to the needs of beneficiaries, it also provides them with indicators of the extent to which the organization is fulfilling its fiduciary and legal duties (Sloan, 2009).
Reporting
Reporting is a key first step for NPOs in creating openness about their past, current, and future actions. The selected management practices in our questionnaire measure the extent to which an NPO reports on the presence of such things as a strategic plan. Another variable asks about the presence of an annual report that, at a minimum, includes (a) reporting on the extent to which planned targets are achieved, thus supporting the NPO and its stakeholders in examining its strategy in a critical way (Copps & Vernon, 2010; Edwards & Hulme, 1996); (b) financial reporting, to develop the trust of not only the public but also of staff and donors past, present, and future (Axelrod, 1994; Dalsimer, 1991); (c) a budget for the coming financial year, by which the lessons learned from the achievements of targets are illustrated in the future strategy (Edwards & Hulme, 1996; Kaplan, 2001; Sheehan, 1996). Without insight into the extent to which targets are achieved, there is “no way to distinguish whether their strategy was succeeding or failing” (Kaplan, 2001, p. 356). Without this kind of reporting, accountability between the NPO and its various stakeholders is heavily inhibited.
Documents such as these allow the organization to be open with various stakeholders, hence upholding upward accountability to donors, horizontal accountability to other NPOs, and downward accountability to beneficiaries (Ebrahim, 2003; Edwards & Hulme, 1996; Sowa et al., 2004; Ebrahim, 2010).
Accessibility
For an organization to be transparent, it must report information but also must be accessible for questions and complaints. The accessibility practices measure the extent to which NPOs are reachable by their stakeholders through such means as mail, email, and telephone. It also asks if the organization has systems in place for receiving, dealing with, and learning from feedback (Ebrahim, 2003; Edwards & Hulme, 1996; Saxton & Guo, 2011). Lastly, there must be a website, as nowadays the Internet is a key tool allowing NPOs to operate in a transparent manner (Gandia, 2011; Saxton & Guo, 2011).
Online publication
The Internet provides a simple, and often low-cost, way for NPOs to publish information and for stakeholders to obtain it (Gandia, 2011; Saxton & Guo, 2011). Considering the contemporary importance of the Internet, practices that include the online publication of the strategic plan, annual report, and the identity of all board members are usually desirable. This practice was strongly endorsed by the panel of NPOs and nonprofit experts, possibly because of concerns relating to the abuse of the legal benefits and responsibilities of certain NPOs in the Netherlands.
Organizational Characteristics
Focus
Best practices dealing with organizational focus address whether an organization operates with the aim of advancing its mission in a clear and focused manner. First, the normative literature suggests that there be a mission statement that is specific and containing (a) a specified target group that the organization is created to serve (which in some cases will be society at large), (b) a statement of the organization’s purpose, and (c) if relevant, a geographic focus. The presence of a specific mission statement such as this allows for clarity in internal direction as well as clear positioning in the landscape of other organizations with a similar social goal (Copps & Vernon, 2010; Kaplan, 2001; Minkoff & Powell, 2006; Oster, 1995). As Kaplan (2001) puts it, “Attempting to be everything for everyone virtually guarantees organizational ineffectiveness” (p. 359). In other words, the mission statement articulates the NPOs’ “reason for being” (Kaplan, 2001; LeRoux, 2009; Oster, 1995; Sowa et al., 2004, p. 717). Thereby, it creates so-called “mission motivation,” which Rainey and Steinbauer (1999) have argued to enhance OE.
The next practice asks to what extent there is a clear linkage between the mission statement and the main activities of the NPO (Copps & Vernon, 2010; Murray, 2010). As Kaplan (2001) observes, an NPO performing activities that are directly deduced from a clear and specific mission statement has higher chances of attaining the impact and goals it envisions. Research has shown that longer term planning has been a major weakness of many NPOs. It suggests that the absence of a long-term vision puts them at risk of failing to correctly prioritize their activities and a lack of strategic focus, although in the face of a rapidly changing context, this long-term vision should be counterbalanced by a continuous review of these plans and a willingness to adapt them as needed (Allison & Kaye, 2003; Anheier, 1994; Bryson, 2010; Sheehan, 1996). Therefore, a third practice related to focus asks about the presence of a continuously reviewed strategic plan spanning at least 3 years, should reflect the focus of an NPO on the state of its external environment. With such a plan in place, internal stakeholders are provided with both direction and guidance in their quest to effectively advance the organization’s mission (Copps & Vernon, 2010; Sowa et al., 2004). Moreover, a strategic plan enables consistent communication of the raison d’être of the NPO to external stakeholders such as donors (Kaplan, 2001).
In addition, the way the goals are set out in the strategic plan is considered to be an important influence on effectiveness (Copps & Vernon, 2010; Kaplan, 2001; Sowa et al., 2004). As Kaplan (2001) puts it, “By quantifying and measuring the strategy, organizations reduce and even eliminate ambiguity and confusion about objectives and methods” (p. 360). As a consequence of the consultations with the NPOs and experts in this research, the presence of SMART goals (defined as Specific, Measurable, Attainable, Relevant, and Time-bound) was selected as the most practical measure of an effective strategic plan.
Strategy
Practices that relate to how the strategic plan is formulated are also important. The first normative practice states that NPOs need to conduct a context analysis, whereby the presence of threats and opportunities in the environment and the practices of similar NPOs and other organizations are explored. This reveals where the organization needs to focus and prevents organizations from working against one another, and ensures that opportunities for cooperation are fully explored (Herman & Heimovics, 1994).
Moreover, NPOs need to remain inquisitive about activities by others that could assist in fulfilling its desired goals. This facilitates a focus on the most effective way to advance the mission (Kaplan, 2001; Sowa et al., 2004). Numerous research findings suggest that NPOs that focus mainly on their activities (such as building schools), in place of the needs of those they seek to serve (such as for education), risk paying too much attention to the make-up of their activities rather than the desired outcomes they want to achieve (Kaplan, 2001; Sheehan, 1996; Sowa et al., 2004).
The normative literature suggests also that it is desirable to carry out risk analyses in the organizations’ environment to investigate the extent to which a strategy could be affected by unforeseen future events (Carman, 2010; Copps & Vernon, 2010; Weiss, 1995). For example, an NPO investing heavily in building a school will be more likely to achieve sustainable success if it has checked that there are no/minimal risks that the school will be empty due to lack of secured funding for teacher’s salaries. Moreover, when possible, the primary target group of beneficiaries and other relevant constituents need to be involved in the development and adaptation of the organizational strategy (Copps & Vernon, 2010; Hoole & Patterson, 2008; Tandon, 1996).
The board
The board is the body that has legal responsibility for ensuring that the organization’s mission is fulfilled (Axelrod, 1994; Ostrower & Stone, 2006; Tandon, 1996). The normative literature on boards recommends that there usually be a clear separation between the duties and responsibilities of the board and the executive body. The former should be responsible for focusing on the long-term vision of the organization and ensuring that the organization achieves its mission. Such a focus is found to strongly relate to OE (Chait, 1993; Holland & Jackson, 1998; O’Connell, 1985). If the board “meddles” by engaging in micromanagement of the executives, there is a risk that less attention will be paid to the long-term focus (Axelrod, 1994; Chait, 1993). However, in practice, especially in smaller NPOs, this strict separation might be difficult to realize as the boards often function as “working boards.”
This ability to focus on the long-term vision of the NPO is possible only when another criterion is fulfilled, namely independence of the board (Bell, 1993; Ostrower & Stone, 2006; Tandon, 1996). In consultation with the sector, the criteria of board independence is defined as a situation in which all of the board members remain uncompensated (Ingram, 1989), are trusted by other stakeholders (Axelrod, 1994), and are authorized to voice their ideas and concerns (Silk, 1994).
Programs
Design
Best practice suggests that, when making decisions about programs, it is important to base them on empirical evidence including input from all relevant stakeholders (Copps & Vernon, 2010).
Ownership
Consulting those who will be affected by programs (see Table A4 in the appendix) is generally believed to be crucial in the program design phase, but this also facilitates a sense of ownership by these stakeholders (Copps & Vernon, 2010; Gibbs, Fumo, & Kubi, 1999). Ownership refers to the control of both the design and implementation of programs. This practice is based on the common sense wisdom that programs are likely to be more effective when those who have to live with them have a voice in decisions about them (Donais, 2009; Wandersman et al., 2005).
Evaluation
With the respect to the evaluation of programs, best practice suggests that, again, whenever possible, beneficiaries and other relevant constituents should be included in the design, implementation, and interpretation of the evaluation process. However, here the hypothesized link with effectiveness is not only via increased ownership but also through the possibility of obtaining increased accuracy of the information collected. Naturally, collecting this information is only valuable for effectiveness when there are structural feedback loops between the findings of the evaluations and the organization’s future strategy (Ebrahim & Rangan, 2010; White, 2010). The presence of these feedback mechanisms is thus another important practice.
Finally, research in program evaluation suggests that NPOs need to go beyond examining the intended positive impacts of programs. It is also important that they be alert to the potential presence of unintended impacts (Wainwright, 2002).
Discussion and Conclusion
In response to the increased pressures on NPOs to demonstrate their OE, and the heavy reliance on financial standards and ratios to approximate it, we have explored an alternative approach based on measuring the extent to which the organization has adopted certain management practices that researchers and expert believe to affect OE.
The extensive literature review conducted for this study provides a more or less up-to -date overview of the management best practices that have been proposed by normative writers and some (academic) researchers. What was interesting and important was the findings that a select panel of practitioners, when asked to provide their own list of effectiveness-enhancing practices, produced one that is widely comparable with that of management scholars. This could either be because the “real life” experiences of practitioners leads them to similar conclusions as those reached by the normative writers on management or it might reflect the pervasive influence of management fads that are absorbed as a result of mimetic tendencies among organizations (Abrahamson, 1996; D. Miller, Hartwick, & Le Breton-Miller, 2004).
In contrast to the one-dimensional focus of much of the research on effectiveness, the practices discussed here provide a multi-dimensional conceptual model of potential influences on OE, that concern the program and organizational levels (Herman & Renz, 1999; Sawhill & Williamson, 2001; Sowa et al., 2004). The overview of potentially effectiveness-enhancing practices provided in the second part of this study was intended to stimulate NPOs to consider their value for their own situation.
It is hoped that the practical relevance of this study is enhanced by the development of an instrument that allows for the measurement of the extent to which NPOs have adopted the best management practices covered.
It is very important to note, however, that there are several limitations to measuring the use of best practices to judge OE. First and foremost, the measurement of these practices should not replace the use of other, more traditional forms of evaluations. Second, sets of best practices such as the one developed here are, like other management and quality systems, vulnerable to a checklist or manage-to-the-measure-mentality, wherein leaders of organizations become obsessed with fulfilling the requirements, resulting in a disengagement from the overall organizational mission. This risk might be partially mitigated by the direct focus of many of the recommended practices on the organization’s mission and its beneficiaries.
Most importantly, due to the absence of consensus on the concept and measurement of OE it is not possible to empirically validate the recommended set of practices. In other words, the causal direction and explanatory power of them is based solely on inference. Because the measurement tools currently available to evaluate OE do not allow for comparisons between NPOs, there is no dependent variable to test regarding whether, or to what extent, a single or entire set of practices can predict a nonprofits’ effectiveness. This implies that an NPO could successfully implement all practices identified in this study and still be ineffective, for example, due to external factors such as economic or political crises of the kind plaguing Europe today or internal factors such as demotivating leaders.
Other limitations should be mentioned. The set of practices discussed here is neither exhaustive nor stable, and is vulnerable to context-dependency due to their partial reliance on the views of practitioners based in the Netherlands. Moreover, as discussed before, they are distilled from studies that have used a variety of conceptualizations and measurements of OE (presented in Table 1). Therefore, the use of an aggregated set of best practices, and their translation into questionnaire items risks the loss of their original validity. However, all practices and their corresponding questionnaire items have been established through the multi-step method where they were subject to the views of practitioners that did explicitly understand OE as mission advancement. As research on NPO effectiveness and its determinants progresses, the set can be expanded or altered. The progression of the research on the concept and measurement of nonprofit OE will have to show to what extent the set of practices identified in this study, and the survey specifically, can impact OE, and whether this ability varies for various cultural and sector-specific contexts. Further research could also focus on the potential role of the identified set of practices and the instrument in processes of organizational change.
Footnotes
Appendix
Required Group Type to be Involved in the Design/Monitoring/Evaluation of the Activities Depending on Type of Interventions.
| Direct psycho-social and physical improvements (e.g., sexual harassment group therapy) | Knowledge development (e.g., leukemia research) | Skill development and capacity building (e.g., obstetrics workshop) | Awareness and behavioral changes (e.g., drunk driving campaign) | Policy (e.g., human rights advocacy) | |
|---|---|---|---|---|---|
| Design | Beneficiaries | Experts | Recipients (e.g., doctors, policy makers) | Experts | Experts |
| Monitoring | Beneficiaries | Experts | Recipients (e.g., doctors, policy makers) | Experts | Recipients (e.g., doctors, policy makers) |
| Evaluation | Beneficiaries | Experts | Recipients (e.g., doctors, policy makers) | Beneficiaries | Recipients (e.g., doctors, policy makers) |
Acknowledgements
We would like to thank Jeffrey Brudney for his comments and editorial suggestions. Moreover, we would like to thank our highly devoted reviewers who have gone through great lengths in supporting the improvement of this manuscript through their incredibly helpful comments and suggestions.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
