Developing and Validating a Capacity Instrument for Chinese and U.S. NGOs

Abstract

Capacity-building initiatives are popular among nongovernmental organizations (NGOs) worldwide. In response to a lack of valid and reliable capacity measures for NGOs working on various social issues, Shumate and colleagues developed an 8-dimension, 45-item NGO capacities instrument, based on data from U.S. NGOs. However, the proliferation of international research on NGO capacity raises questions about the degree to which such an instrument equally applies in other countries. To allow meaningful comparisons of NGO capacities across countries, we examine the measurement equivalence of this NGO capacity instrument across a matched sample of Chinese (N = 119) and U.S. (N = 150) NGOs. Findings suggest a new NGO capacity instrument, which comprises seven factors and 28 items and better accounts for the capacity of both Chinese and U.S. NGOs. Based on the new instrument, a series of comparative analyses highlight the influence of institutional factors on NGO capacities.

Keywords

NGO capacity survey instrument measurement comparative analysis

Organizational capacity is increasingly important because government and philanthropic funders, clients, and the public exert pressure on nongovernmental organizations (NGOs) to more effectively tackle social problems (Despard, 2017; Minzner et al., 2014). Hence, NGO capacity has garnered considerable interest among nonprofit scholars around the globe (e.g., AbouAssi et al., 2016; Doherty et al., 2014; Suárez & Marshall, 2014). Despite this attention, limited work has quantitatively measured NGO capacity to guide more systematic investigations. To address this research gap, scholars have attempted to develop NGO capacity instruments in recent years. However, existing NGO capacity instruments are either unvalidated (Doherty et al., 2014; Minzner et al., 2014) or developed based on data from a particular type of NGOs in the United States (Despard, 2017).

Yet, questions remain about the degree to which instruments developed in one country adequately measure the same concept in other countries. In particular, we know little about whether existing NGO capacity instruments developed in the U.S. context apply to NGOs in other countries. Comparing the same NGO capacity construct in different countries can help researchers understand the extent to which NGO capacity has the same conceptual structure and underlying meaning across languages and cultures. Moreover, developing an NGO capacity scale in multiple countries ensures that “any cultural differences or similarities are not merely artifacts of measurement” (Kim et al., 2012, p. 79), allowing researchers and practitioners to make meaningful comparisons and generalize empirical findings across countries. Such a measure advances nonprofit research so that researchers can conduct fair, substantive comparisons of NGO capacity across countries, expand their inquiries to more diverse institutional contexts, and shed light on how institutional factors influence NGO capacity. Comparing NGO capacities across countries has practical benefits; international NGOs and institutional funders could use such a measure to inform tailored capacity-building strategies, training programs, or targeted funding priorities based on institutional context.

This research adopts a comparative perspective and compares the dimensions and measurement characteristics of the NGO capacity instrument among a matched set of U.S. and Chinese NGOs. The growth of Chinese NGOs has increasingly attracted the attention of nonprofit scholars (e.g., Hsu, 2010; Spires, 2011). Chinese NGOs present an ideal case to examine NGOs in the Global South because they operate in a distinct institutional environment from Western NGOs (Ma, 2002). However, our knowledge of Chinese NGO capacity is sparse.

The purpose of this research is twofold. First, this study describes the results of the first effort to develop an NGO capacity instrument for international use. Extending Shumate and colleagues’ (2017) research,¹ we develop an NGO capacity instrument that measures both Chinese NGO and U.S. NGO capacity. Second, by comparing Chinese NGOs’ capacities with U.S. NGOs’ capacities, this research highlights the influence of social origins on the sector (Salamon et al., 2017), contributing to our understanding of how institutional and contextual factors may influence NGO capacities. In particular, we test the assertion that Chinese NGOs have lower capacity than U.S. NGOs.

Literature Review

NGO Capacity: Background and Conceptualization

Scholars have defined NGO capacity from various perspectives (Table 1). But, they generally view capacity as an organization’s ability and means to achieve goals (Bryan, 2019; Christensen & Gazley, 2008; Sowa et al., 2004) and as a multidimensional concept comprised of several dimensions.

Table 1.

An Illustrative Review of Some Recent Nonprofit Capacity Typologies Available.

Instruments	Context	Capacity describes. . .	#Dimensions	Dimensions of capacity
Validated capacity instruments
Despard (2017)	Human service nonprofits	“the resources, skills, and functions an organization needs to fulfill its mission across multiple domains.” (p. 608)	4	Resource development; management; program development; and board development
Shumate et al. (2017)	General	“the processes, practices, and people that the organization has at its disposal that enable it to produce, perform, or deploy resources to achieve its mission.” (p. 156)	8	Financial management; adaptive; strategic planning; external communication; board leadership; operational; mission orientation; and staff management
Unvalidated capacity instruments
Allison & Kaye (2005)	General	–	4	Mission; finances; administrative capacity; and board governance
Doherty et al. (2014)	Community sport clubs	“the ability of an organization to draw on various assets and resources to achieve its mandate and objectives.” (p. 125S)	5	Human resources; infrastructure; finances; planning and development; and external relationships
Gupta et al. (2006)	HIV/AIDS NGOs	–	5	Organizational strength; HIV/AIDS technical capacity; promotion of participation of key populations; partnerships, referral systems, and coordination; and involvement in evidence and consultation-based advocacy
Minzner et al. (2014)	Social service nonprofits	“the skills, practices, and systems that allow NPOs to operate more effectively and sustainably.” (p. 550)	5	Organizational development; program development; revenue development; leadership development; and community engagement
Renzi (1996)	General	–	5	Oversight/vision; management resources; human resources; financial resources; and external resources
Suárez & Marshall (2014)	General	Organizational characteristics and practices	4	Infrastructure; human resources and governance; formalization and rationalization; and external relationships

Note. For a summary of instruments devised before 2008, see Shumate et al. (2017) and Christensen and Gazley (2008). NGO = nongovernmental organization.

NGO capacities have been described as the antecedents (or determinants), processes, and outcomes of organizational activities. As antecedents, capacity is the required input for NGOs and their networks partners to thrive. Scholars contend that some levels of human resource capacity are key for NGOs to engage in collaborative networks and derive benefits from them (AbouAssi et al., 2016). As a process, NGO capacity describes the organizational factors that influence organizational performance and effectiveness (Sowa et al., 2004) and the ability to achieve organizational mission (Castelloe & Watson, 2000). Finally, NGO capacity has been studied as an outcome, particularly in research on capacity building (Gupta et al, 2006; Minzner et al., 2014). For instance, NGOs may seek to build their capacity via interorganizational collaboration (Sanyal, 2006).

Similar to nonprofit capacities, nonprofit performance (or effectiveness) also encompasses multiple dimensions and includes outcomes of an organization’s management and program activities, such as financial health, employee satisfaction, client satisfaction, and social impact (Sowa et al., 2004). Bryan (2019) proposed a contingency model for the relationship between nonprofit capacity and effectiveness, positing that different dimensions of nonprofit effectiveness require different types of nonprofit capacity. In this light, better measures of nonprofit capacity are critical to advance research on the capacity-effectiveness link.

In sum, NGO capacity as antecedents, processes, and outcomes suggests that understanding NGO capacity contributes to the theoretical advancement in nonprofit research and has important practical implications for the NGO sector. Accurately measuring NGO capacity is necessary to evaluate program effectiveness, service quality, client outcomes, and community impact (Despard, 2017; Doherty et al., 2014).

Creation of the Multidimensional NGO Capacities Instrument

Although significant, academic research is hampered by a lack of valid and reliable NGO capacity instrument to guide NGO capacity-building efforts. Although scholars have recently developed various NGO capacity instruments (Table 1), these instruments have several problems. First, several instruments have not been validated using methods to confirm the reliability and validity of the items (Doherty et al., 2014; Minzner et al., 2014). Shumate et al. (2017) identify several additional instruments that are used by various development agencies; they note several problems including (a) none of the instruments have been empirically validated, (b) most require a trained facilitator to administer, and (c) and the results of the instrument are not comparable across NGOs. Only two instruments we identified were empirically validated (Despard, 2017; Shumate et al., 2017), but Despard’s (2017) instrument applies to NGOs working in one issue area (i.e., human services) in the U.S., limiting the generalizability of the capacity instruments to other types of NGOs.

This research relies on the nonprofit capacities instrument (Shumate et al., 2017), a 45-item instrument that measures eight nonprofit capacities. Shumate and colleagues (2017) developed and validated the NGO capacities measure among 949 NGOs working on various social issues (e.g., human services, health, environment, education, and arts). They used a two-study approach. In Study 1, they created an instrument from a 149-item pool from 19 pre-existing capacity instruments and used orthogonal exploratory factor analysis to examine the emergent factor structure of NGO capacity. In Study 2, to further validate the 8-factor solution, the authors conducted confirmatory factor analysis (CFA) with a second sample of NGOs. Their findings revealed eight dimensions of NGO capacities (Table 2). In addition, they found that NGO capacity is not a second-order concept composed of a higher-order singular factor; instead, NGO capacities are composed of eight interrelated dimensions. Finally, they established the discriminant validity of the instrument and concurrent validity through its relationship with established measures of nonprofit effectiveness.

Table 2.

Definitions of Eight Nonprofit Capacities by Shumate et al. (2017).

Capacity	DefinitionThe ability of an organization to . . .
Financial management	Manage accounts, including costs, surpluses, and trends in revenue over time
Adaptive	Create a culture to learn from and adapt to changes in its environment
Strategic planning	Deliberatively make decisions to guide an organization’s actions and long-term development based on its identity and mission
External communication	Engage stakeholders using public relations strategies and information communication technologies
Board leadership	Have a board of directors to commit to and involve with the organization’s vision and plan, develop good relationships with the staff, and make sound decisions about its future
Operational	Develop and use documented procedures to set goals for programs and activities and evaluate outcomes
Mission orientation	Commonly orient toward organizational mission from various groups of stakeholders
Staff management	Meet employee needs for information, training, and mentoring

Compared to other instruments in Table 1, Shumate and colleagues’ (2017) instrument shares some common factors with other instruments (e.g., financial management, staff management, board leadership, and operational capacity in Despard, 2017) but has some unique dimensions (e.g., mission orientation and external communication) that other instruments do not have. As numerous studies have demonstrated (Guo & Saxton, 2018), capacities such as external communication are pivotal for NGOs’ visibility and performance in the digital age.

In comparison to U.S. NGO capacities, little is known about NGO capacities in the Global South more generally and China more specifically. In particular, research on NGOs in the Global South contributed little to the theoretical development of the NGO sector. Specifically, our knowledge about the NGO sector in the Global South is hampered by a lack of (a) descriptive information on these organizations; (b) conceptual knowledge about the NGO sector in the Global South; and (c) suitable theories to describe, explain, and understand these organizations (Anheier & Salamon, 1998). As such, researchers (e.g., Salamon et al., 2017) call for research of NGOs in the Global South to advance the global conceptualizations of the NGO sector and the applicability and validity of prevailing theories. In this study, we employ a comparative perspective and examine the capacity of Chinese NGOs. To achieve this goal, a reliable, valid instrument for both U.S. and Chinese NGOs is necessary.

Chinese NGO Capacities

Capacity-building programs are increasingly popular among Chinese NGOs to improve their management, governance, and leadership (Spires, 2012). However, research in Chinese NGO capacity is limited. When scholars discuss Chinese NGO capacities, they often introduce this concept to illustrate other theories and concepts. Furthermore, the eight dimensions of capacities identified by Shumate et al. (2017) did not receive equal attention from researchers studying Chinese NGOs. Instead, prior scholarship focuses mostly on Chinese NGOs’ external communication capacity (Yang & Taylor, 2010; Yang, 2007).

Overall, the majority of prior scholarship suggests that the capacities of Chinese NGOs are limited. Notably, the commonly held assumptions by Chinese NGO scholars and international nonprofit practitioners alike are that Chinese NGOs have relatively low capacities, so foreign grantmakers and international NGOs have helped build Chinese NGO capacities via funding and training (Spires, 2012). For example, Schwartz (2004) pointed out that Chinese NGOs lack the capacity, such as human capital and financial resources, to evaluate programs and influence public policies. Yang (2007) found a minimal level of external communication capacity in Chinese NGOs. Yang and Taylor (2010) suggested that Chinese NGOs cannot generally advocate and/or mobilize action due to the sensitivity of activism in the Chinese context. Furthermore, Hasmath and Hsu’s (2015) interviews with employees in over 100 Chinese NGOs revealed that high staff turnover creates problems such as a lack of expertise and professionalism.

However, prior research primarily relies on case studies of Chinese NGOs and interviews with NGO leaders in China alone when making claims about the lack of Chinese NGO capacity. Without directly comparing the capacities of Chinese NGOs with those of NGOs in other countries based on a common instrument, we cannot ascertain whether Chinese NGOs indeed have lower capacities than their Western counterparts. Chinese NGOs may indeed have lower capacity, as these scholars suggest. Alternatively, researchers may have selected cases or NGOs that do not represent Chinese NGOs in general. As these studies are not comparative, scholars may be comparing the average Chinese NGOs’ capacity to an idealized Western NGOs’ capacity. Moreover, a lack of fair, substantive comparisons of NGO capacities across countries based on a common NGO capacity scale hinders our examination of how institutional contexts shape NGO capacities. To address these issues, this research develops an NGO capacity measure that can be used reliably across contexts and based on this instrument, investigates how contextual factors and social origins (Salamon et al., 2017) shape NGO capacities.

Differences Between Chinese and U.S. NGO Capacities

Sociopolitical and sociocultural contexts may influence the conceptualization and underlying meanings of NGO capacities. The capacities recognized as important, or behaviors that constitute a distinct capacity may differ from culture to culture. In particular, there are features of the Chinese context that may influence the dimensions of NGO capacity. Notably, previous research suggests that the government has a ubiquitous impact on NGOs in China’s restrictive political context and that China’s NGO sector is still in the early growth stage.

First, Chinese NGOs must adapt to subtle and delicate state-NGO relations (Hsu, 2010; Ma, 2002). Previous research highlights the coercive pressures from the government on Chinese NGOs. As a result, they focus more on nonsensitive and depoliticized social issues (Schwartz, 2004; Spires, 2011) and sometimes sacrifice their autonomy for financial resources and political legitimacy (Hsu, 2010). For example, environmental NGOs in China adopt a mild approach, instead of an activist approach that openly confronts the government, to alleviate environmental pollution (Ho, 2001). Furthermore, NGOs have to refrain from making democratic claims or “address[ing] social needs that might fuel grievances against the state”; instead, they must acknowledge the government support to gain legitimacy (Spires, 2011, p. 1).

Second, in contrast to NGOs in the United States that are obliged to obey the Generally Accepted Accounting Principles (Larkin & DiTommaso, 2011), Chinese NGOs face fewer established legal frameworks and guidelines regarding financial reporting and management. The Ministry of Civil Affairs stipulates that NGOs must meet a minimum level of financial resources (i.e., about US$15,000 for national ones and US$5,000 for local ones) to be registered, creating formidable registration barriers to smaller and community-based grassroots NGOs. As a result, a significant number of NGOs remain unregistered in China (Hildebrandt, 2011). However, NGO registration involves complex state-society dynamics, beyond sufficient funding; factors such as organizations’ social issue area, distance to Beijing, local governments’ political agenda, and organizational age also influence NGOs’ registration decisions (Hildebrandt, 2011; Spires, 2011). Moreover, due to a lack of financial transparency, many unregistered organizations lack financial independence and have difficulty attracting donations (Ho, 2001). In contrast, the government authority (e.g., Ministry of Civil Affairs, Ministry of Finance and Auditing, and NGOs’ sponsoring units) keeps the financial records of registered NGOs’ bank accounts and performs annual checks on their financial activities. In addition, registered NGOs must regularly disclose their financial activities to the public (Ministry of Civil Affairs, 1998, Article 9, 15, 18, 27–31). However, beyond these government measures, little is known about Chinese NGOs’ proactive management of their long-term financial health.

Third, Spires (2012) uniquely demonstrates that “board of directors” is mostly a regulatory requirement from the Chinese government and a funding requirement from international grantmakers for Chinese NGOs to obtain financial support. His field study reveals that the role of boards of directors in China is more symbolic and ceremonial than substantive. Many Chinese NGOs do not have a board of directors; even when they do, boards often have limited decision-making authority in comparison to Western contexts. Furthermore, both the general public and NGOs in China lack an understanding of the roles and functions of the board of directors. For instance, the boards of directors of Chinese NGOs are usually drawn from individuals (e.g., retired or current government officials and businessperson), who often lack an understanding of the NGO sector and social issues.

In summary, existing research suggests that Chinese NGOs generally lack capacity, but there has not been a comparative analysis using a validated instrument. The dimensions of NGO capacity in China may be influenced by the sociopolitical and sociocultural context. This research seeks to investigate the dimensions of Chinese NGO capacity in general and compare Chinese NGOs’ and U.S. NGOs’ levels of capacity.

Method

Sample and Data Collection

This research is part of a larger research project that examines the link between nonprofit capacity and interorganizational networks. We obtained a random sample of 5,000 U.S. NGOs from the Urban Institute 2013 list of active 501c3 organizations with revenues greater than $100,000 from B (education), C (environment), and S (community improvement and capacity building) categories. We chose these three categories to maximize diversity and because each has been areas of cross-sector partnership research (Simo & Bies, 2007). In addition, we obtained a list of Chinese NGOs from the NGO directory of the Beijing Civil Society Development Research Center (CDB), an important information platform dedicated to providing information, resources, consulting, and networking services for China’s NGO sector. At the time of the study, CDB had a total of 814 NGOs working in 26 issue areas, such as rural development, migrant workers, arts and culture, and woman welfare (see Fu & Shumate, 2016). They were headquartered in every province/municipality of China and had been operating for an average of 13.10 (SD = 12.37) years. We then invited executive directors or their equivalents from these NGOs to participate in this research via mailing, emails, and follow-up phone calls from July 2013 to June 2015. We also sent invitations to any NGOs that these organizations nominated as partners. In total, we mailed postcards to 4,825 U.S. NGOs with valid addresses and sent emails to 1,094 Chinese NGOs listed in the CDB directory and their nominated partners to invite participation. Organizational leaders received a $20 gift card as incentives to participate.

Participants had the option of completing the online survey in Qualtrics in English or Chinese. The original instrument was created in English. Two native Chinese speakers translated the survey into simplified Chinese and back-translated the instrument in English for verification. Each item asked the respondent to evaluate their opinions on statements on a 1 to 4 scale (1 = strongly disagree, 4 = strongly agree). Two additional answers—“don’t know” and “not applicable”—were coded as missing data. In total, 749 U.S. NGOs (response rate = 16%) and 119 Chinese NGOs (response rate = 11%) had valid, complete responses.

Matching Sample

To ensure that the NGOs in the U.S. and Chinese sample were similar for comparative purposes, we used a stratified sampling approach to select 150 U.S. NGOs out of 749 U.S. NGOs that matched the sample of Chinese NGOs in terms of social issue.² For example, in our sample, 21 Chinese NGOs of the 119 Chinese NGOs (17.64%) focused on education and research. Proportionally, 27 U.S. NGOs (18%) of the 89 U.S. NGOs in education and research were randomly selected from U.S. NGOs. Further t-tests suggested that the 119 Chinese NGOs and the 150 U.S. NGOs selected were balanced in other organizational attributes, such as scope of work (i.e., total number of countries where they had operations; t = 0.27, df = 261, p = .79), yearly budget (t = −0.58, df = 240, p = .72), and revenue (t = −0.30, df = 249, p = .76). Thus, 119 Chinese NGOs and 150 U.S. NGOs comprised our final sample.

Sample Characteristics

A majority of Chinese NGOs focused on social missions such as education and research (n = 21, 17.95%), environment (n = 25, 21.37%), social services (n = 29, 24.79%), and philanthropic intermediaries and voluntarism promotion (n = 15, 13.68%). The median of organizational revenue for Chinese NGOs and U.S. NGOs was US$162,459 (n = 104) and US$380,000, respectively (n = 147). The average number of countries that Chinese NGOs and U.S. NGOs had offices in was 1.63 (SD = 3.70) and 1.75 (SD = 3.85), respectively. We compared the characteristics of the NGOs with their respective NGO population in social mission, geography, and organizational type. The findings suggested that U.S. NGOs and Chinese NGOs were representative of the NGO population in their respective country. However, both U.S. and Chinese NGOs had higher revenue than those in their respective country.³

Analytic Procedures

We used CFA (Byrne, 2003; Byrne & van de Vijver, 2010) to examine whether the original NGO capacity instrument measures the same construct(s) across U.S. and Chinese NGOs. By testing for measurement invariance across groups, CFA allowed us to determine whether Chinese NGOs understood the items to measure the same capacity dimensions as U.S. NGOs did. Byrne (1998) and Vandenberg and Lance (2000) suggest that measurement invariance should be tested via several successive steps, from configural invariance (i.e., same number of factors and items under each factor) to metric invariance (i.e., similar magnitudes of correlations between a factor and an item) and scalar invariance (i.e., similar intercepts of the regression between the latent factor and items). Variances in any of the three levels may result in a different NGO capacity instrument for Chinese NGOs from the U.S. NGO instrument. If the tests show significant variances at a lower level, then there is no need to test the metric and/or scalar invariance at higher levels (Figure 1).

Figure 1.

Analytic procedure and results.

Stata was used to conduct the CFA and measurement invariance testing. Missing data accounted for 5.61% of the U.S. NGO sample and 9.98% of the Chinese NGO sample. Multiple imputation method was employed (Allison, 2003). Model fit was determined by χ², χ²/df (less than 5), root mean square error of approximation (RMSEA; less than 0.10), confirmatory fit index (CFI; greater than 0.90), and standardized root mean square (SRMR; less than 0.10) (Hooper et al., 2008; Kline, 2005). In this research, RMSEA primarily guided our evaluation of the goodness of fit (GOF) of models, since RMSEA deals with particularly smaller sample sizes and remains the most popular measure for model fit (Schreiber et al., 2006).

Results

Testing for Measurement Invariance

We first performed CFA to assess the fit of the original capacity instrument for all 269 observations. The results suggested good model fit (χ² = 2,467.95, df = 917, χ²/df = 2.69, RMSEA = .08, CFI = .79, SRMR = .07), indicating that we could proceed to test for measurement invariance (Byrne & van de Vijver, 2010) across U.S. and Chinese NGOs.⁴

We then conducted CFA using the original instrument for U.S. NGOs and Chinese NGOs separately to ascertain whether the original instrument fit U.S. NGOs and Chinese NGOs, respectively. Our findings suggested that U.S. NGOs and Chinese NGOs showed significant configural variances. Specifically, the original 8-factor solution fit the U.S. NGO sample (χ² = 1,654.11, df = 917, χ²/df = 1.80, RMSEA = .07, CFI = .84, SRMR = .07), not the Chinese NGO sample (χ² = 2,416.82, df = 917, χ²/df = 2.64, RMSEA = .12, CFI = .60, SRMR = .11). Chinese NGO capacity instrument had a different number of dimensions and a smaller set of items. Therefore, the next step was to explore the factors and items that resulted in the most variances (Byrne, 2003).

Explorations in Measurement Variance Using CFA

We explored the sources of differences in the U.S. and Chinese NGO capacity instrument. The procedure was as follows. At each step, we removed one item or several items (the minimum number of items possible) from the 45-item instrument for Chinese NGOs and determined if the removal of the item(s) lead to an improvement in GOF for the Chinese NGO sample⁵ (i.e., decrease in RMSEA). We iterated this process until the Chinese sample achieved acceptable GOF (i.e., RMSEA < 0.10). Based on the fit criterion, 17 items were removed, creating a 7-factor 28-item instrument, with four items loading onto each factor.

The revised instrument yielded improvement of GOF for both the U.S. and Chinese sample, leading to configural invariance across the Chinese and U.S. NGO model (Table 3). That is, Chinese and U.S. NGOs agreed on the dimensions and which items described those dimensions in the revised instrument. This suggested that the 7-factor 28-item capacity instrument was an appropriate instrument for both U.S. and Chinese NGOs. Although U.S. and Chinese NGOs still had variant factor loadings (i.e., metric variance; Δχ² = 257.61, Δdf = 21, Δχ²/Δdf = 12.27, p < .001), the improvement of GOF in both U.S. and Chinese NGOs indicated that we could retain the structure for interpretation. In other words, although we did not achieve strict measurement invariance, the new 7-factor 28-item capacity instrument better accounted for the variances in both Chinese and U.S. NGOs, allowing us to make meaningful comparisons across the two groups. Meanwhile, each of the items uniquely loaded onto one of the factors.

Table 3.

Measurement Invariance Testing Using the Revised Instrument.

Model	χ²	df	Δχ²	Δdf	Δχ²/Δdf	RMSEA	CFI	SRMR
U.S. NGOs	525.51	329	—	—	—	.063	.916	.063
Chinese NGOs	709.67	329	—	—	—	.099	.780	.083
Metric invariance test	1,397.38	700	257.61	21	12.27^***	.086	.828	.105
	1,654.99	721	—	—	—	.098	.770	.113

Note. GOF indices are shown before taking modification indices. After taking modification indices, the GOF for Chinese NGOs: χ² = 479.70, df = 304, RMSEA = .070, CFI = .898, SRMR = .073. RMSEA = root mean square error of approximation; CFI = confirmatory fit index; SRMR = standardized root mean square; NGO = nongovernmental organizations; GOF = goodness of fit.

***

p < .001.

The New NGO Capacity Instrument

Table 4 presents the standardized factor loadings and descriptive statistics for the 7-factor 28-item model for U.S. and Chinese NGOs. The Online Appendix presents the NGO capacity instrument in simplified Chinese. Each of the scales had good reliability, except for staff management capacity (α = 0.68) for U.S. NGOs, which had marginally satisfactory reliability, given the small sample size. To assess the validity of the instrument, we took two alternative approaches. First, the average variance extracted (AVE) for each factor suggested good convergent validity (Table 4). We also examined the heterotrait–monotrait ratio (HTMT) for the factors (Henseler et al., 2015), and the correlations among all factors were below the .85 threshold. Hence, the revised instrument also demonstrated discriminant validity. Each of the seven capacities was related to one another (Table 5).

Table 4.

Standardized Loadings From 7-Factor Solution for U.S. and Chinese NGOs, respectively.

Capacity and capacity items	Factor loading	Alpha	CR	Average	M	SD
Financial management		.83/.74	.84/.82	.58/.54	3.26/2.98	.62/.52
Financial plans are in place for the long-term sustainability of this organization’s work	.61/.82
An annual budget is updated and reviewed regularly by management	.78/.80
There are qualified personnel who manage this organization’s finances	.83/.58
The budget takes into account the long-term financial recourses of this organization	.80/.71
Adaptive capacity		.88/.88	.89/.89	.67/.68	3.38/3.16	.54/.55
There is a sense of shared values among the entire staff	.71/.68
Conflicts among staff are resolved productively	.75/.77
People at this organization work together to solve problems	.91/.93
Employees at this organization are supportive of one other	.89/.90
Strategic planning		.93/.74	.93/.82	.77/.54	2.96/2.98	.65/.46
Implementation of activities reflects the strategic plan	.90/.66
Strategic plans are actually followed	.91/.74
This organization is guided by a long-term strategic plan	.85/.78
The strategic plan is structured around this organization’s mission	.84/.74
External communication		.79/.70	.83/.79	.57/.50	2.89/3.01	.60/.49
Information technology is regularly used for communicating with external stakeholders (i.e., donors, media, and other organizations)	.55/.80
This organization has developed cause-related fundraising activities	.86/.48
Information about organizational activities is regularly disseminated to the public	.88/.64
This organization has the ability to develop key messages for potential supporters	.67/.86
Board leadership		.87/.85	.88/.86	.64/.60	3.06/3.04	.64/.51
The board members are accessible to employees	.72/.76
This organization’s board has a good working relationship with staff	.89/.91
The board takes regular steps to stay informed about the important trends in the larger environment that might affect the organization	.83/.70
The board learns from its mistakes	.75/.71
Operational capacity		.75/.75	.82/.81	.55/.53	2.87/2.85	.56/.47
Performance indicators have been identified for each program objective	.98/.71
Regular reports track each program on at least a quarterly basis	.56/.86
Programs are routinely monitored through external evaluation	.83/.53
Before a program begins, measurable objectives are set out	.50/.75
Staff management		.68/.78	.74/.81	.42/.52	2.98/3.03	.54/.54
Employees have all the information they need to do their jobs effectively	.71/.72
Management provides opportunities for regular job training activities	.71/.79
Managers have the necessary skills to run this organization	.57/.56
Staff receives adequate mentoring	.58/.78

Note. CR = composite reliability. We allowed the measurement error to covary within the same factor. The first number respresents statistics for U.S. NGOs and the second number represents statistics for Chinese NGOs.

Table 5.

Correlation Matrix for 7-Factor Solution for U.S. and Chinese NGOs.

Capacity	Financial management	Adaptive capacity	Strategic planning	External communication	Board leadership	Operational capacity	Staff management
Financial management	–
Adaptive capacity	.53*/.44	–
Strategic planning	.64*/.71	.45/.52	–
External communication	.61*/.53	.35/.49	.58/.49	–
Board leadership	.62*/.53	.42/.56	.49/.58	.46/.51	–
Operational capacity	.42*/.40	.28**/.15	.57/.53	.47**/.18	.38/.38	–
Staff management	.62*/.56	.58/.68	.56/.52	.49/.69	.49/.53	.55*/.31	–

Note. NGO = nongovernmental organizations.

p < .05. **p < .01.

A Comparative Analysis of U.S. and Chinese NGO Capacities

We compared U.S. and Chinese NGO capacities in three ways. For capacity scores, t-tests suggested that U.S. and Chinese NGOs had similar levels of capacities, except financial management and adaptive capacity. Specifically, Chinese NGOs generally had lower financial management capacity (t = 4.02, df = 267, p < .001) and adaptive capacity (t = 3.25, df = 267, p < .005) than U.S. NGOs. Second, we correlated organizations’ revenue (natural log) with each dimension of capacity. Our results indicated that no capacities were correlated with revenue for U.S. or Chinese NGOs. Third, our analysis of variance (ANOVA) test results indicated that U.S. and Chinese NGOs in different social issue areas did not have significantly different scores in all seven dimensions.⁵

Discussion

Based on the survey data of a matched sample of 150 U.S. NGOs and 119 Chinese NGOs, we created a new NGO capacity instrument for U.S. and Chinese NGOs using measurement invariance tests, compared their capacities, and studied their links to organizational revenue and social mission. First, we found significant differences between Chinese and U.S. NGOs using the original 8-factor 45-item capacity instrument. By conducting further analyses, we found that the new 7-factor 28-item capacity instrument improved the GOF for both Chinese and U.S. NGOs, compared to the GOF of the two groups using the original instrument. As such, the revised capacity instrument is an appropriate instrument for Chinese and U.S. NGOs.

However, we failed to find mission orientation capacity as one of the capacities for Chinese NGOs, although present in the U.S. sample. This represents a difference in the U.S. versus Chinese NGOs; Chinese NGOs may understand mission-orientation capacity differently from U.S. NGOs. One explanation is that the semiauthoritarian context of China imposes a stringent boundary on the normative social mission (i.e., nonsensitive and depoliticized) of Chinese NGOs for legitimacy and survival (Schwartz, 2004; Spires, 2011). As a result, Chinese NGOs may understand mission orientation as constrained by explicit and implicit government rules and mandates. In contrast, U.S. NGOs are not constrained in the types of social mission they focus on or the approaches (e.g., activist and confrontational) they employ in addressing social problems. U.S. NGOs’ mission-orientation capacity is rooted in their accountability to donors, communities, clients, beneficiaries, and the broader society; governments play a limited role in determining their mission. With the presence of state intervention and repression in China, Chinese NGOs are forced to (not) develop certain types of capacity. This finding suggests the influence of institutional contexts on the actualization of NGO capacity.

Second, the capacity scores of Chinese NGOs and U.S. NGOs exhibit striking similarities, and none of their capacity scores were related to organizational revenue or social mission. Despite an abundance of scholarship that suggests that Chinese NGOs have low levels of capacities (e.g., Hasmath & Hsu, 2015; Schwartz, 2004; Yang, 2007), our results suggest the opposite. U.S. and Chinese NGOs generally have similar levels of capacity. These results run counter to previous qualitative and case-based research that argues that Chinese NGOs lack capacity, professionalism, and expertise. One explanation for the similar level of capacity scores is that we used a matched sample to compare similar NGOs in terms of revenue, budget, size, and social issue. Alternatively, this may be because that the common diagnosis that Chinese NGOs lack in capacity is often assumed and “imagined” and “developed in a decidedly top-down fashion” by researchers, donors, and international grantmakers. Spires (2012, pp. 145–146) considers this “myopia” and “a common kernel of failure” in research on Chinese NGO capacities, advocating that,

A more realistic and ultimately more worthwhile effort would require detailed study of the experiences of the actual Chinese organizations, many of whom have acquired a great deal of wisdom and practical experience during the past 15 years or so and have much to share with their peer organizations. To do so, however, would require an investment of time and resources that most foreign donors seem as yet unwilling to provide.

Taken together, the finding that Chinese and U.S. NGOs have similar levels of capacity highlights the necessity and importance of developing and using a common instrument for comparative capacity research to make meaningful evaluations and accurate claims. Before the creation of the revised NGO capacity instrument, researchers were limited in their ability to compare capacities systematically.

However, Chinese NGOs had lower financial management capacity than U.S. NGOs. Only Chinese NGOs with sufficient revenues are able to register and subject to the financial regulation from the government. In contrast, NGOs in the U.S. are subject to financial oversight—including audits and financial reporting—regardless of their level of revenue. Consequently, U.S. NGOs may be more motivated to manage their financial health under reporting pressures. The differential government regulations on NGO financial reporting and coercive financial pressures in the two contexts may have led to the different scores in their financial management capacity.

In addition, Chinese NGOs also had lower adaptive capacity compared to U.S. NGOs. Adaptive capacity captures an organization’s shared values and supportive culture, conflict resolution, and teamwork and joint problem solving among staff members. One possibility is that the measure evokes a very flat organization, which may be counter cultural to Chinese hierarchical organizations. Alternatively, Chinese NGOs may generally lack the capacity to monitor and adapt to the environment continuously. Recent research suggests that Chinese NGOs vary in terms of their ability to adapt to an environment. For instance, Li (2019) notes that Chinese NGOs that are more responsive to their environment are more likely to have a successful leadership transfer from the founder. Guo and Zhang (2013) find that Chinese service-oriented NGOs can adapt more readily to their stakeholders than advocacy NGOs because the institutional environment is more challenging for the advocacy organizations. More research is needed to understand the reasons why Chinese NGOs may be more or less responsive to their environment.

In summary, the absence of mission-orientation capacity for Chinese NGOs and lower financial management and adaptive capacity of Chinese NGOs highlight the influence of social origins on civil society development (Salamon et al., 2017) and institutional pressures on NGOs to build capacities to achieve legitimacy (Barman & MacIndoe, 2012). In particular, this research highlights the influence of the government actors and political structures on NGO capacities in China. Although NGOs need to develop capacities to meet the expectations of multiple stakeholder groups (Bryan, 2019), this research reveals that some social actors are more influential in some institutional contexts than others. This has the potential to extend a contingency theory of NGO capacity (Bryan, 2019).

Practically speaking, this research suggests that global funders should overcome the bias that Chinese NGOs have lower capacity than U.S. NGOs, although they could focus more on improving Chinese NGOs’ financial management and adaptive capacity via various capacity-building activities, training programs, and funding priorities.

Limitations and Future Research

This research contributes to our understanding of Chinese NGO capacities from a comparative perspective. However, this research has several limitations. First, the capacity instrument of Chinese NGOs was adapted from the original U.S NGO capacity instrument. As a result, it failed to consider some unique capacities valued in the idiosyncratic institutional environment of China, such as NGOs’ capacity to manage NGO-government relations (Hsu, 2010) and guanxi with government and business entities (Fu & Shumate, 2017). Second, this research did not distinguish language invariance from cultural invariance. Future research should disentangle language and cultural variances in their models. Third, although only executive directors or their equivalents were invited to participate in this research, we did not ask their years of working in their organization. Future research may filter out responses where nonprofit leaders had worked fewer than 3 years in their organization. Finally, although Chinese NGOs and U.S. NGOs generally exhibited similar capacity levels, their scores in financial management and adaptive capacity differed. We interpret these differences with caution. Although our samples were matched based on social issue, revenue, scope, and budget, the sampled NGOs had higher revenue than the population of NGOs in China or the U.S. In addition, future research should distinguish between advocacy and service-oriented Chinese and U.S. NGOs, since institutional context influences these types of organizations differently (Guo & Zhang, 2013).

Based on the research findings, we highlight two promising directions for future research. First, future research may employ this capacity instrument to assess the capacities of NGOs in the U.S. and China and examine the determinants (e.g., organizational size, age, type, mission, and regulatory environment) of NGO capacities. In particular, future research may more vigorously examine why Chinese NGOs had lower financial management and adaptive capacities than U.S. NGOs. Moreover, future research may examine how varying levels of NGO capacities influence outcomes at the organizational, network, and community levels. This comparative approach will allow researchers to understand the influence of institutional factors on NGO capacity building and evaluation.

Second, the findings of this research highlight significant structural differences between NGOs and the necessity of comparing the same construct in different countries. Future research may compare the NGO capacity instrument in more countries and develop a global NGO capacity instrument to support the proliferation of NGO capacity research internationally.

Conclusion

This research makes two contributions to the study of NGO capacity, comparative research, and NGOs in China and the Global South. First, this research introduces a new NGO capacity instrument that can be used for both Chinese and U.S. NGOs. Despite an abundance of scholarship that suggests that Chinese NGOs have low levels of capacities (e.g., Schwartz, 2004; Spires, 2012), this study suggests that U.S. and Chinese NGOs have very similar levels of capacity in five of the seven dimensions examined. These findings highlight the necessity of understanding organizational capacities across contexts based on the same capacity instrument and examining a matched sample of NGOs to make meaning comparisons.

Second, the article furthers our understanding of how the definition and actualization of NGO capacity are related to the sociopolitical and sociocultural context. Specifically, this research demonstrates that mission orientation, as a capacity, is not understood in China in a similar way as in the U.S. Moreover, it uncovers the influence of sociopolitical and sociocultural environment on financial management and adaptive capacities. The social origins approaches suggest that the NGO sector differs because of the conditions of emergence (Salamon et al., 2017). Drawing upon this approach and the current research, we call for more NGO capacity research in the Global South and encourage researchers to take into consideration the unique institutional environments where NGOs are embedded and understand NGO capacities accordingly.

Despite a burgeoning interest in capacity building in the NGO sector, theoretical advancement remains hampered by a lack of research in NGO capacities, in particular, NGO capacities in the Global South (Salamon et al., 2017). Although significant, the current research only marks an initial step to examining the capacities of NGOs in the Global South. Much work remains to be done to understanding the capacities of NGOs in different contexts. This line of research also has the potential to guide public managers and policymakers in how they could strategically invest resources in building NGO capacities for greater social impact.

Supplemental Material

NVSQ889179_Online_Appendix – Supplemental material for Developing and Validating a Capacity Instrument for Chinese and U.S. NGOs

Supplemental material, NVSQ889179_Online_Appendix for Developing and Validating a Capacity Instrument for Chinese and U.S. NGOs by Jiawei Sophia Fu and Michelle Shumate in Nonprofit and Voluntary Sector Quarterly

Footnotes

Acknowledgements

The authors would like to thank Patrick Y. Hsieh, Yiqi Li, Ivory Mills, Jie Qin, and other members of the Network for Nonprofit and Social Impact laboratory for their help with data collection.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is funded by the National Science Foundation (grant no. SES-1264417) and the first author’s startup funding from the Rutgers University School of Communication and Information.

ORCID iD

Jiawei Sophia Fu

Notes

Author Biographies

Jiawei Sophia Fu (PhD, Northwestern University) is an assistant professor at the School of Communication and Information of Rutgers University. Her research interests center around organizational communicaiton, social networks, interorganizational collaboration, technology management, and social entrepreneurship and innovation.

Michelle Shumate (PhD, University of Southern California) is a professor at the Department of Commuunication Studies of Northwestern University. She is the founder and director of the Network for Nonprofit and Social Impact. Her current work centers on the question: How can nonprofit networks be rewired for maximum social impact?

References

AbouAssi

Makhlouf

Whalen

(2016). NGOs’ resource capacity antecedents for partnerships. Nonprofit Management & Leadership, 26(4), 435–451.

Allison

M. J.

Kaye

(2005). Strategic planning for nonprofit organizations: A practical guide and workbook. John Wiley & Sons.

Allison

P. D.

(2003). Missing data techniques for structural equation modeling. Journal of Abnormal Psychology, 112(4), 545–557.

Anheier

H. K.

Salamon

L. M.

(1998). The nonprofit sector in the developing world: A comparative analysis. Manchester University Press.

Barman

MacIndoe

(2012). Institutional pressures and organizational capacity: The case of outcome measurement. Sociological Forum, 27(1), 70–93.

Byrne

B. M.

, (1998). Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming. Mahwah, NJ: Lawrence Erlbaum Associates.

Bryan

T. K.

(2019). Toward a contingency model for the relationship between capacity and effectiveness in nonprofit organizations. Nonprofit and Voluntary Sector Quarterly, 48(4), 885–897.

Byrne

B. M.

(2003). Testing for equivalent self-concept measurement across culture: Issues, caveats, and application. In Marsh

H. W.

Craven

McInerney

D. M.

(Eds.), International advances in self research (Vol. 1, pp. 291–314). Information Age.

Byrne

B. M.

van de Vijver

F. J. R.

(2010). Testing for measurement and structural equivalence in large-scale cross-cultural studies: Addressing the issue of nonequivalence. International Journal of Testing, 10(2), 107–132.

10.

Castelloe

Watson

(2000). The participatory change process: A capacity building model from a US NGO. Development in Practice, 10(2), 240–244.

11.

Christensen

R. K.

Gazley

(2008). Capacity for public administration: Analysis of meaning and measurement. Public Administration and Development, 28(4), 265–279.

12.

Despard

M. R.

(2017). Can nonprofit capacity be measured? Nonprofit and Voluntary Sector Quarterly, 46(3), 607–626.

13.

Doherty

Misener

Cuskelly

(2014). Toward a multidimensional framework of capacity in community sport clubs. Nonprofit and Voluntary Sector Quarterly, 43(2_Suppl.), 124S–142S.

14.

J. S.

Shumate

(2016). Hyperlinks as institutionalized connective public goods for collective action online. Journal of Computer-Mediated Communication, 21(4), 298–311.

15.

J. S.

Shumate

(2017). Understanding the size and spread of Chinese NGO networks. Chinese Journal of Communication, 10(1), 72–88.

16.

Guo

Saxton

G. D.

(2018). Speaking and being heard: How nonprofit advocacy organizations gain attention on social media. Nonprofit and Voluntary Sector Quarterly, 47(1), 5–26.

17.

Guo

Zhang

(2013). Mapping the representational dimensions of non-profit organizations in China. Public Administration, 91(2), 325–346.

18.

Gupta

McPherson

Sellers

Chamreun

C. S.

Choudhary

B. N.

Levene

(2006). Assessing and evaluating community organizations’ capacity for working in HIV/AIDS responses in India, Ecuador and Cambodia. Journal of International Development, 18(4), 595–598.

19.

Hasmath

Hsu

(2015, November 19–21). Communities of practice and the NGO sector in China [Paper presentation]. Association for Research on Nonprofit Organizations and Voluntary Action Annual Conference, Chicago, IL, United States.

20.

Henseler

Ringle

C. M.

Sarstedt

(2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135.

21.

Hildebrandt

(2011). The political economy of social organization registration in China. The China Quarterly, 208, 970–989.

22.

(2001). Greening without conflict? Environmentalism, NGOs and civil society in China. Development and Change, 32(5), 893–921.

23.

Hooper

Coughlan

Mullen

(2008). Structural equation modelling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53–60.

24.

Hsu

(2010). Beyond civil society: An organizational perspective on state-NGO relations in the People’s Republic of China. Journal of Civil Society, 6(3), 259–277.

25.

Kim

Vandenabeele

Wright

B. E.

Andersen

L. B.

Cerase

F. P.

Christensen

R. K.

Desmarais

Koumenta

Leisink

Liu

Palidauskaite

Pedersen

L. H.

Perry

J. L.

Ritz

Taylor

De Vivo

(2012). Investigating the structure and meaning of public service motivation across populations: Developing an international instrument and addressing issues of measurement invariance. Journal of Public Administration Research and Theory, 23(1), 79–102.

26.

Kline

R. B.

(2005). Principles and practice of structural equation modeling. Guilford Press.

27.

Larkin

R. F.

DiTommaso

(2011). Wiley not-for-profit GAAP 2011: Interpretation and application of Generally Accepted Accounting principles. Wiley.

28.

(2019). Leadership succession and the performance of nonprofit organizations: A fuzzy-set qualitative comparative analysis. Nonprofit Management & Leadership, 29(3), 341–361.

29.

(2002). Defining Chinese nongovernmental organizations. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 13(2), 113–130.

30.

Minzner

Klerman

J. A.

Markovitz

C. E.

Fink

(2014). The impact of capacity-building programs on nonprofits: A random assignment evaluation. Nonprofit and Voluntary Sector Quarterly, 43(3), 547–569.

31.

Ministry of Civil Affairs of the People’s Republic of China. (1998). Interim regulations on registration administration of private non-enterprise units. Retrieved from http://www.mca.gov.cn/article/gk/fg/shzzgl/201507/20150715847908.shtml

32.

Renzi

(1996). An integrated toolkit for institutional development. Public Administration and Development, 16(5), 469–483.

33.

Salamon

L. M.

Sokolowski

S. W.

Haddock

M. A.

(2017). Explaining civil society development: A social origins approach. John Hopkins University Press.

34.

Sanyal

(2006). Capacity building through partnership: Intermediary nongovernmental organizations as local and global actors. Nonprofit and Voluntary Sector Quarterly, 35(1), 66–82.

35.

Schreiber

J. B.

Nora

Stage

F. K.

Barlow

E. A.

King

(2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99(6), 323–338.

36.

Schwartz

(2004). Environmental NGOs in China: Roles and limits. Pacific Affairs, 77(1), 28–49.

37.

Shumate

Cooper

K. R.

Pilny

Peña-y-lillo

(2017). The nonprofit capacities instrument. Nonprofit Management & Leadership, 28(2), 155–174.

38.

Simo

Bies

A. L.

(2007). The role of nonprofits in disaster response: An expanded model of cross-sector collaboration. Public Administration Review, 67(s1), 125–142.

39.

Sowa

J. E.

Selden

S. C.

Sandfort

J. R.

(2004). No longer unmeasurable? A multidimensional integrated model of nonprofit organizational effectiveness. Nonprofit and Voluntary Sector Quarterly, 33(4), 711–728.

40.

Spires

A. J.

(2011). Contingent symbiosis and civil society in an authoritarian state: Understanding the survival of China’s grassroots NGOs. American Journal of Sociology, 117(1), 1–45.

41.

Spires

A. J.

(2012). Lessons from abroad: Foreign influences on China’s Emerging Civil Society. The China Journal, 68, 125–146.

42.

Suárez

Marshall

J. H.

(2014). Capacity in the NGO sector: Results from a national survey in Cambodia. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 25(1), 176–200.

43.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70.

44.

Yang

Taylor

(2010). Relationship-building by Chinese ENGOs’ websites: Education, not activation. Public Relations Review, 36(4), 342–351.

45.

Yang

(2007). How do Chinese civic associations respond to the Internet? Findings from a survey. The China Quarterly, 189, 122–143.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.38 MB