Systems Antecedents for Dissemination and Implementation

Abstract

There is a growing emphasis on the role of organizations as settings for dissemination and implementation. Only recently has the field begun to consider features of organizations that affect dissemination and implementation of evidence-based interventions. This manuscript identifies and evaluates available measures for five key organizational-level constructs: (a) leadership, (b) vision, (c) managerial relations, (d) climate, and (e) absorptive capacity. Overall the picture was the same across the five constructs—no measure was used in more than one study, many studies did not report the psychometric properties of the measures, some assessments were based on a single response per unit, and the level of the instrument and analysis did not always match. One must seriously consider the development and evaluation of a robust set of measures that will serve as the basis of building the field, allow for comparisons across organizational types and intervention topics, and allow a robust area of dissemination and implementation research to develop.

Keywords

cancer prevention and screening chronic disease management health promotion research design work site health promotion

Over the past several years researchers and practitioners alike have recognized the need for more research focused on the dissemination and implementation (D&I) of evidence-based programs to promote health and manage chronic disease. Organizations (e.g., schools, workplaces, hospitals) are considered important settings for delivering health promotion interventions (Brownson, Haire-Joshu, & Luke, 2006; Fielding, 1984; Katz, 2009). There is a reasonably robust literature across organizational settings on the delivery of health promotion interventions. However, only recently has the field begun to consider features of organizations that facilitate or inhibit the D&I of evidence-based interventions.

Some of the earlier studies examining the role of organizations in the delivery of evidence-based interventions have considered primarily structural features, such as organization size, complexity, and formalization (Drazin & Schoonhoven, 1996; Emmons & Biener, 1993; Emmons et al., 2000; Emont & Cummings, 1989). Some of these features may in fact reflect less tangible but perhaps more important characteristics of organizations in influencing D&I decisions, such as organizational readiness (Weiner, 2009; Weiner, Amick, & Lee, 2008), leadership, climate (Helfrich, Weiner, McKinney, & Minasian, 2007), and organizational culture (Barnsley, Lemieux-Charles, & McKinney, 1998; Ferlie, Gabbay, Fitzgerald, Locock, & Dopson, 2001; Kanter, 1988; Van de Ven, Polley, Garud, & Venkataraman, 1999). For example, although organizational size has been well studied, it is likely a proxy for other determinants, such as extent of resources available and functional differentiation or specialization of roles (Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004). Much of the literature at this point is conceptual, with a call for increased research examining the role of these factors in D&I.

A key challenge in the transition from research focused on evidence generation to that focused on D&I is the unit of analysis. The very nature of dissemination efforts often requires an organizational perspective, moving beyond the individual as the unit of analysis and exploring how organizational factors influence dissemination efforts. Although such an approach is relatively new in the health field, other fields have historically focused on organizations as a key intervention target (e.g., organizational behavior and theory, public policy, education) and have extensively used organizational-level measures to assess factors influencing organizational behavior and outcomes.

To build the field of D&I research, we need reliable and valid measures. Recent reviews have reported a dearth of such measures even when considering the broader literature. For example, Weiner (2009) recently developed a conceptual framework of organizational readiness to change and completed an extensive review (Weiner et al., 2008) examining how organizational readiness for change has been defined and measured in health services research and other fields. Analysis of 106 peer-reviewed articles revealed conceptual ambiguities and disagreements and limited evidence of reliability or validity for most publicly available readiness measures.

As health promotion research increasingly examines organizational-level factors, the need for good operational definitions and measures of key organizational characteristics becomes clearer. The purpose of this article is to identify available measures for key organizational-level constructs that are important for D&I research, to evaluate the measures’ psychometric properties, and to determine if additional measures are needed. A key goal from the outset was to recommend measures that appear to have sound psychometric properties so that a larger body of research using common measures could develop.

The starting point was to identify and evaluate the extant measures related to organizational factors. To guide this work, we selected Greenhalgh et al.’s (2004) conceptual framework for considering the determinants of D&I within health service and delivery organizations (see Figure 1). This framework identifies three categories of system antecedents for innovation that relate to the organizational-level factors that may influence dissemination outcomes: (a) structure, (b) absorptive capacity for new knowledge, and (c) receptive context for change. Since there has been considerable attention in the literature to structural variables, for this review we focused on absorptive capacity and four features of receptive context (leadership, vision, managerial relations, and climate). We provide a brief review of the literature for each of these five constructs below.

Figure 1.

Conceptual model for considering the determinants of diffusion, dissemination, and implementation of innovations in health service delivery and organization, based on a systematic review of empirical research studies

Leadership

Although leadership has been studied since the 1930s, it is a difficult construct to define and measure. Aditya (2004) argues that leadership centers on both the creation and achievement of goals. The Global Leadership and Organizational Behavior Effectiveness program (GLOBE), a network of researchers who seek to identify effective “universal and culture-specific” leader behaviors and organizational practices, has defined leadership as “the ability of an individual to influence, motivate, and enable others to contribute toward the effectiveness and success of organizations of which they are members” (House et al., 1999).

Leadership is thought to be an important component of organizational change and may be particularly important in terms of encouraging members of an organization to think in new ways (Van de Ven et al., 1999). Leadership style also has important implications for power balances, social relationships within the organization and across systems, and institutional attitude and approach toward risk taking (Kanter, 1988; Van de Ven et al., 1999).

Vision

Although vision is often seen as a leader’s ability to create and communicate clear direction and rationale for the organization (Alexander, Zakocs, Earp, & French, 2006), it can also be conceptualized as a separate construct that cuts across multiple levels within a given organization. Vision has been defined as a guiding theme or overriding principle that steers the organization, articulating the desired direction and intentions for the future (Reh, n.d.).

West (1990) defines vision as “an idea of a valued outcome which represents a higher order goal and a motivating force at work” (see also Anderson & West, 1998). From a theoretical perspective, we would expect that having a clear vision that values innovation sets the stage for adoption and implementation of new health programs. We might also expect that a clear vision would allow members of an organization to adequately assess whether or not an innovation matches with the existing organizational values and goals of the organization, a concept known as compatibility, as described in Rogers’s (1995) diffusion of innovation theory.

Managerial Relations

A simple definition for good managerial relations is the positive alliances between groups within an organization to promote change (Pettigrew, Ferlie, & McKee, 1992; Shortell, Morrison, & Friedman, 1990). In clinical settings, Shortell et al. (1990) emphasized the significance of (a) looking for common ground, (b) involving selected physicians early on in planning, (c) carefully identifying the needs and interests of key physicians, and (d) working on a daily basis to build a climate of trust, honesty, and effective communications. Pettigrew et al. (1992) also proposed that manager–clinician relations are easier when negative stereotypes have been broken down and that it is important to understand what clinicians value. In fact, managers who were best at promoting change had semi-immersed themselves in the world of clinicians, understood the implications of medical work flow, and perhaps even helped clinicians do their own planning as a way of earning trust.

Climate

Climate describes organizational members’ perceptions of their work environment. Climate captures the meaning or significance that organizational members perceive in organizational policies, procedures, and practices. These perceptions enable organizational members to interpret current events, predict possible outcomes, and gauge the appropriateness of their own actions and those of others (Jones & James, 1979).

There is some debate as to whether climate is a homologous multi-level construct. Some argue that the construct’s meaning, measurement, and relations with other variables differ across levels of analysis. Reflecting this position, scholars distinguish between psychological climate, a property of individuals that refers to personal perceptions of work environment, and organizational climate, which is a property of the collective and refers to shared perceptions of work environment.

Climate is also a multifaceted construct. In a classic formulation, James and his colleagues identified five primary work environment facets: (a) job characteristics, (b) role characteristics, (c) leadership characteristics, (d) work group and social environment characteristics, and (e) organizational and subsystem attributes (L. R. James & Sells, 1981; Jones & James, 1979). More recently, Patterson et al. (2005) identified 17 facets of work environment. Confirmatory factor analysis provides support for both approaches (L. A. James & James, 1989; Patterson et al., 2005).

Absorptive Capacity

An important albeit quite understudied construct is absorptive capacity, or an organization’s ability to access and effectively use information. absorptive capacity was originally conceptualized as an organization’s cognitive structures and prior related knowledge that underline learning (Cohen & Levinthal, 1990). Zahra and George (2002) expanded this definition to include (a) a broad set of skills needed to deal with and modify transferred knowledge and (b) the organization’s capacity to learn and solve problems. This construct reflects the process by which ideas from outside the organization are captured, circulated internally, adapted for the organizational culture/environment, internalized, and integrated into organizational routines. absorptive capacity reflects an organization’s ability to find and use knowledge. In the theoretical literature, there is a discussion of potential versus realized absorptive capacity, with the former focusing on organizational capability for knowledge acquisition/assimilation and the latter focused on use of absorbed knowledge (Cohen & Levinthal, 1990; Daghfous, 2004), suggesting that an organization’s absorptive capacity depends on that of its individual members but is not simply a summation of individual capacity. As Caccia-Bava, Guimaraes, and Harrington (2006) point out, organizational absorptive capacity is best understood in terms of the structures that allow multiple organizational members to gather, communicate, apply, and exploit diverse knowledge within the organization.

Method

In order to identify measures of these five constructs, we searched the peer-reviewed literature using PubMed, CINAHL®, and ISI Web of Science^SM. With assistance from a reference librarian, we formulated a set of search strings that combined each of the target organizational constructs (organization, climate, managerial relations, leadership, vision, absorptive capacity) and the following terms: innovation, diffusion, adoption, implementation, and health. All search strings included the target construct and “health” and were applied to the title, abstract, and keyword list. We added wildcard characters to all terms in order to capture potentially important permutations (e.g., innovate, innovating, innovative, and innovation). We restricted the search to English-only articles. Overall, we identified 1,178 articles in PubMed, 946 articles in CINAHL®, and 1,698 articles in ISI Web of Science. The distribution of these articles by construct is outlined in Table 1.

Table 1.

Articles Identified for Each Construct, by Source

Construct	PubMed	CINAHL®	ISI Web of Science	Identified Through Article References
Unrestricted search
Leadership	549	428	303	0
Vision	215	212	234	0
Managerial relations	21	23	1001	0
Climate	375	235	141	0
Absorptive capacity	18	48	19	13
Construct	PubMed	CINAHL®	ISI Web of Science	Nonduplicative Articles Retained After Full-Text Review
Restricted search
Leadership	41	10	35	12
Vision	16	22	9	2
Managerial relations	3	1	58	9
Climate	5	7	31	14
Absorptive capacity	5	15	16	3

From the articles identified, we then reviewed the abstracts using four predetermined inclusion/exclusion criteria. First, articles had to report original empirical research and appear in a peer-reviewed journal. Second, articles had to mention the organizational construct or a close synonym in the abstract and use a quantitative measure of the construct. Third, articles had to include a measured outcome relevant to innovation, diffusion, adoption, or implementation. Examples of such outcomes include implementation of an innovation, defined as a new program, product, service, practice, or technology; the spread of an innovation through an organization or social system; the decision to use an innovation; the process of putting an innovation into practice; and the level of initial use of an innovation, or the sustained use of an innovation beyond an initial trial period. Finally, articles had to focus on a health service or health-related intervention or outcome. Research studies involving schools, businesses, or other organizational settings were included if they focused on a health-related intervention, such as an employee wellness program or a tobacco cessation program.

We assigned each article retained after full-text review a unique identification number and used a structured data abstraction form to extract key information from each article (e.g., study setting, construct name, construct dimensionality, number of items, and construct level). Table 2 shows the “dictionary” that we used to structure the data abstraction form. We then created tables to display, categorize, and analyze the information we extracted.

Table 2.

Coding Form Used to Abstract Article Information

Domain	How Coded
Author and year	Author and year
Study setting	Types of organizations included in study
	H = Heath care
	PH = Public health
	MH/SA = Mental health/substance abuse
	S = Schools
	M = Multiple (e.g., health care and public health)
Construct name	Name of construct used by authors
Measure name	Name of measure or scale used by authors
Construct dimensions	S = Single (one) dimension
	If multiple, names of construct dimensions
Number of items	Number of items comprising the climate measure
Construct level	I = Assessed at individual level of analysis
	T = Assessed at team or group level of analysis
	O = Assessed at organizational level of analysis
	M= Assessed at multiple levels of analysis
Responses per unit	If construct assessed at team or organizational level of analysis
	S = single respondent per team or organization
	M = multiple respondents per team or organization
Outcome type	G = Innovation generation
	A = Innovation adoption
	I = Innovation implementation
	S = Innovation sustainability/institutionalization
Outcome variable(s)	Name of outcome variable(s) examined in study
Outcome level	I = Individual level of analysis
	O = Organization level of analysis

We used Trochim and Donnelly’s (2007) classification of validity and reliability types (see Table 3), in which construct validity is regarded as an umbrella term that includes translational validity and criterion-related validity. Translational validity includes face and content validity. Criterion-related validity includes predictive, concurrent, convergent, and discriminant validity. Reliability includes interrater or interobserver reliability, parallel forms reliability, test–retest reliability, and interitem reliability (e.g., Cronbach’s alpha). In this review, we combined face and content validity into a single category. We did the same for the various forms of reliability assessment.

Table 3.

Types of Validity and Reliability Examined in this Review

Construct validity	The degree to which inferences can legitimately be made from an instrument to the theoretical construct that it purportedly measures
Translation validity	The degree to which an instrument accurately translates (or carries) the meaning of the construct
Face validity	A summary perception that an instrument’s items to translate or carry the meaning of the construct. Procedures for assessing face validity include informal review by experts or more formal review through a Delphi process
Content validity	A check of instrument’s items against the content domain of the construct. Examples include expert review based on a clear definition of the construct and a checklist of characteristics that describe the construct. If the construct is multidimensional, factor analysis could be used to verify the existence of those theoretically meaningful dimensions
Criterion-related validity	An empirical check on the performance of an instrument against some criteria
Predictive validity	The degree to which an instrument predicts a theoretically meaningful outcome. Examples include regression analysis in which the instrument serves as an independent variable. Predictive validity is not demonstrated if the instrument serves as a dependent variable
Concurrent validity	The degree to which an instrument distinguishes groups it should theoretically distinguish (e.g., a depression screener distinguishes depressed and nondepressed patients)
Convergent validity	The degree to which an instrument performs in a similar manner to other instruments that purportedly measure the same construct (e.g., two measures show a strong positive correlation). Convergent validity is most often assessed through confirmatory factor analysis
Discriminant validity	The degree to which an instrument performs in a different manner to other instruments that purportedly measure different constructs (e.g., the two measures show a zero or negative correlation). Discriminant validity is most often assessed through confirmatory factor analysis
Reliability	The consistency or repeatability of an instrument’s measurement. Examples include interrater or interobserver reliability, test–retest reliability, parallel forms reliability, and internal consistency reliability (e.g., Cronbach’s alpha)

Results

Leadership

Description of articles

Twelve articles examined the association of organizational leadership and health innovation dissemination. Among these, six (50%) focused on health care organizations, three (25%) on public health organizations, and two (16%) on schools; one article (8%) focused on multiple settings. Table 4 shows the diversity of dimensions included in the leadership construct and Table 5 summarizes results for leadership and the other organizational constructs.

Table 4.

Description of Research Studies

Author(s) (Year)	Study Setting	Construct Name	Construct Dimensions	Number of Items	Construct Level	Responses Per Unit	Outcome Type^a	Outcome Variable(s)	Outcome Level
Leadership
Bin Saeed (1995)	Health care	Leadership commitment	Single dimension of leadership commitment	11	Individual/organization	Multiple	I	QA implementation	Organization
Evashwick and Ory (2003)	Public health	Leadership	Leader at start, leader continuing, champion in addition to leader	3	Organization	Single	S	Sustainability of innovative community-based health programs	Organization
Kegler, Steckler, McLeroy, and Malek (1998)	Public health	Leadership	Single dimension of leadership	6	Individual, combined for team	Multiple	I	Implementation (of action plan; no. of activities implemented)	Organization
Livet, Courser, and Wandersman (2008)	Public health	Leadership	Single dimension of leadership	10	Organization	Multiple	I	Implementing computer-programming frameworks	Organization
Marchionni and Ritchie (2008)	Health care	Transformative leadership (multifactor leadership questionnaire)	Idealized influence, inspirational motivation, individualized consideration, intellectual stimulation	45	Organization	Multiple	I	Extent of guideline implementation	Organization
McFadden, Henagan, and Gowen (2009)	Health care	Leadership	Single dimension of leadership	8	Individual	Multiple	I	Patient safety initiatives	Organization
Roberts-Gray, Gingiss, and Boerm (2007)	Schools	School-based leadership	Single dimension of school-based leadership	4	Organization	Single	I	Implementation of school program	Organization
Roberts-Gray et al. (2007)	Schools	External leadership	Single dimension of external leadership	1	Organization	Single	I	Implementation of school program	Organization
Seguin et al. (2008)	Multiple	Leadership style	Organization, support, communication, conflict resolution	8	Individual	Single	I	Implementers	Individual
Seguin (2008)	Multiple	Leadership competence	Single dimension of leadership competence	2	Individual	Single	I	Implementers	Individual
Somech (2006)	Health care	Participative leadership	Single dimension of participative leadership	3	Team	Multiple	I	Team innovation	Team
Somech (2006)	Health care	Directive leadership	Single dimension of directive leadership	6	Team	Multiple	I	Team innovation	Team
Thaker et al. (2008)	Schools	School leadership and administrative support	Single dimension of school-based leadership and administrative support	3	Team, organization	NR	A, S	Adoption of program	Organization
Weiner (1996)	Health care	Governance structure	Physician involvement in governance, management involvement in governance	5	Organization	Multiple	A, S	Adoption of CQI	Organization
Weiner (1996)	Health care	Leadership for quality	CEO involvement in CQI/TQM, board quality monitoring, board activity in quality	3	Organization	Multiple	A, S	Leadership for quality	Organization
West (2003)	Health care	Leadership clarity	Single dimension of leadership clarity	11	Individual, team	Multiple	I	Team innovation	Organization
Vision
Inamdar, Kaplan, and Bower (2002)	Health care	Vision	Single dimension of vision	1	Organization	Multiple	I	Overcome barriers to implementation	Organization
Livet (2008)	Public health	Shared vision	Single dimension of shared vision	4	Organization	Single	I	Use of programming processes	Organization
Managerial relations
Lukas, Mohr, and Meterko (2009)	Health	Management support	Personal leadership support Practical management support	1 = 7, 2 = 8	Individual	Multiple	I	Implementation of advanced clinic access	Individual/organization
Organizational climate
Allen, Lehrner, Mattison, Miles, and Russell (2007)	Multiple	Organizational climate	Single dimension of climate	8	Organization	Multiple	I	Provider screening for domestic violence	Organization
Anderson and West (1998)	Health care	Proximal work group climate	Vision, participative, safety, task orientation, support for innovation	38	Team	Multiple	A	Innovativeness (various measures)	Team
Bostrom, Wallin, and Nordstrom (2007)	Health care	Organizational climate	Challenge, freedom, support for ideas, trust, liveliness, playfulness/humor, debate, conflicts, risk taking, idea time	50	Individual	N/A	I	Research utilization in daily clinical practice	Individual
Brownson et al. (2007)	Public health	Organizational climate	Single dimension of climate	2	Organization	Single	A	Presence of evidence-based physical activity interventions	Organization
Choi, Andersen, and Veillette (2009)	Multiple	Unsupportive climate	Single dimension of climate	5	Individual	N/A	G	Peer-rated creative performance	Individual
Gittelson et al. (2003)	Schools	School climate	Single dimension of climate	1	Organization	M	I	Pathways intervention implementation (various measures)	Organization
Glisson, Schoenwald, et al. (2008)	Mental health/substance abuse	Climate	Stress, engagement, functionality	46	Organization	Multiple	S	Sustained use of new clinical program, service, or treatment model	Organization
Gregory, Henry, and Schoeny (2007)	Schools	School climate	Negative relationships, administrative leadership, supportive climate	40	Organization	Multiple	I	Prevention intervention implementation (level and change)	Organization
McCormick, Steckler, and McLeroy (1995)	Schools	Organizational climate	Single dimension of climate	32	Organization	Unclear	I	Implementation, level of use	Organization
Nystrom, Ramamurthy, and Wilson (2002)	Health care	Organizational climate	Risk orientation, external orientation, achievement orientation	16	Organization	Multiple	A	Organizational innovativeness	Organization
Parcel et al. (2003)	Schools	School climate	Supportive, directive, restrictive collegial, intimate, disengaged	79	Organization	Multiple	S	Institutionalization of CATCH intervention (various process and outcome measures)	Organization
Schoenwald, Carter, Chapman, and Sheidow (2008)	Mental health/substance abuse	Organizational climate	Fairness, role clarity, role overload, role conflict, cooperation, growth and advancement, job satisfaction, emotional exhaustion, personal accomplishment, depersonalization	Unclear	Individual/Organization	Multiple	I	Therapist adherence to multisystemic therapy, client outcomes	Individual/organization
Simpson, Joe, and Rowan-Szal (2007)	Mental health/substance abuse	Organizational climate	Clarity of program mission, staff cohesiveness, staff autonomy, communication, stress, openness to change	30	Organization	Multiple	A	Trial use of therapeutic alliance workshop materials	Organization
Wilson, Ramamurthy, and Nystrom (1999)	Health care	Organizational climate	N/A	Unclear	Organization	Multiple	A	Radicalness and relative advantage of innovations adopted	Organization
Absorptive capacity
Caccia-Bava, Guimaraes, and Harrington (2006)	Health	Absorptive capacity	Evaluation and assimilation of knowledge, ability to apply knowledge internally	5	Organization	Single	A	Technology adoption	Organization
Knudsen and Roman (2004)	Mental health/substance abuse	Absorptive capacity	Environmental scanning, Collection of satisfaction data, Level of workforce professionalism	10	Organization	Single	A	Substance abuse treatment innovations	Organization
Belkhodja, Amara, Landry, and Ouimet (2007)	Public health	Absorptive capacity	Size of the unit, if there are employees who are paid to do research	2	Organization	Single	A	Research utilization	Organization

Note: QA = quality assurance; CQI = continuous quality improvement.

Outcome type: G—innovation generation; A—innovation adoption; I—innovation implementation; S—innovation sustainability/institutionalization.

Table 5.

Summary of Findings Across Constructs

Domain	No. of Measures Used	No. of Constructs	No. of Construct Dimensions	No. of Items (Range)	No. of Measures With Reliability and Some Form of Validity Measured
Leadership	16	9	24	1-45	4
Vision	2	2	2	1-4	0
Managerial relations	1	1	2	15	0
Organizational climate	14	3	45	1-79	12
Absorptive capacity	3	1	7	2-10	0

The articles examined various leadership dimensions, including commitment, clarity, champion, transformational leadership, style, competence, and administrative support, among others. The measures of leadership included 1 to 45 items. Several articles measured more than one dimension of leadership; in all, there were 16 leadership measures identified across the 12 studies reviewed.

Five articles (42%) reported measurement of the leadership construct at the organizational level of analysis. One article (8%) measured the construct at the team level of analysis, and two articles (16%) assessed leadership at the individual level of analysis; one article (8%) measured leadership at both the individual level and organizational level of analysis; one article (8%) measured leadership at the team and organizational levels; and two articles (16%) measured the construct at the individual level and then combined responses to create a team level variable. Among the articles that described leadership measured at either the team or organizational level, two used assessments of leadership based on a single response per unit, eight studies based the assessment on multiple respondents per unit, and one article did not report the number of responses per unit used.

Organizational leadership was examined in the context of several dissemination-related outcomes. Nine (75%) focused on innovation implementation, one (8%) focused on innovation sustainability (Evashwick & Ory, 2003), and two (16%) focused on innovation adoption and innovation sustainability (Thaker et al., 2008; Weiner, Alexander, & Shortell, 1996). In all but three cases (Kegler, Steckler, McLeroy, & Malek, 1998; McFadden, Henagan, & Gowen, 2009; West et al., 2003), the level of the outcome variable matched the level of the leadership variable. Thaker et al. (2008) assessed leadership at the team and organizational level and outcomes on the organizational level, and Bin Saeed (1995) looked at leadership at the individual and organizational levels and outcomes at the organizational level.

Analysis of instruments for measuring organizational leadership

There was no agreement across the articles about how leadership should be assessed; our review identified 16 different measures to assess organizational leadership in dissemination-related studies, with no measures used in more than one study.

Psychometric properties of instruments were reported in some of the studies. Four studies (33% of articles, representing five measures) reported face/content validity (Bin Saeed, 1995; McFadden et al., 2009; Roberts-Gray, Gingiss, & Boerm, 2007; Somech, 2006). Five studies (42%) reported reliability (Bin Saeed, 1995; Livet, Courser, & Wandersman, 2008; Marchionni & Ritchie, 2008; McFadden et al., 2009; Somech, 2006). Ten of the leadership constructs measured either reported on or exhibited predictive validity for dissemination related outcomes. For example, Bin Saeed (1995) collected data from 202 physicians across three hospitals in Saudi Arabia to determine factors associated with implementation of a hospital-based quality assurance (QA) program. Using factor analysis to identify dimensions related to implementing QA programs, an 11-item scale was created for “leadership commitment.” In a subsequent analysis using multiple regression, this leadership construct was significantly associated with implementation of the QA program. West et al. (2003) measured leadership clarity among 3,447 respondents from health care teams in the United Kingdom. Leadership clarity predicted levels of innovation among community mental health teams and breast cancer care teams but not among primary health care teams. Table 6 summarizes the psychometric properties assessed in each study.

Table 6.

Assessment of Validity and Reliability for Instruments Identified in Review

Instrument Name	Key Citations	Predictive Validity^a	Face/Content Validity	Concurrent Validity	Convergent Validity	Discriminant Validity	Reliability
Leadership
Self-efficacy (program specific)	Seguin (2008)	✓
Leadership style	Seguin (2008)	✓
Bridge-It Capacity Survey	Roberts-Gray, Gingiss, and Boerm (2007)	✓	✓
Participative leadership	Somech (2006)	✓	✓				✓
Directive leadership	Somech (2006)	✓	✓				✓
QA implementation	Bin Saeed (1995)	✓	✓
Multifactor Leadership Questionnaire	Marchionni and Ritchie (2008)						✓
Transformational leadership	McFaddin, Henagan, and Gowen (2009)	✓	✓		✓	✓	✓
Leadership clarity	West (2003)	✓
Partnership Self-Assessment Tool (Weiss, Anderson, & Lasker, 2002)	Livet, Courser, and Wandersman (2008)	✓					✓
Champion	Livet et al. (2008)	✓
Vision
None
Managerial relations
Personal leadership support^a	Lukas, Mohr, and Meterko (2009)						✓
Practical management support^a							No
Organizational climate^a
Organizational climate^a	Allen, Lehrner, Mattison, Miles, and Russell (2007)	✓			✓		✓
Team Climate Inventory	Anderson and West (1998)	✓	✓	✓			✓
Creative Climate Questionnaire	Bostrom et al. (2007)			✓
Organizational climate^a	Brownson et al. (2007)	✓	✓				✓
Unsupportive climate	Choi, Andersen, and Veillette (2009)	✓	✓			✓	✓
	Scott and Bruce (1994) [S]
School climate^a	Gittelson et al. (2003)	✓					✓
Organizational social context	Glisson, Landsverk, et al. (2008)		✓	✓		✓	✓
	Glisson, Schoenwald, et al. (2008) [S]
School climate	Gregory et al. (2007)	✓	✓				✓
Kettering SCS	Johnson, Johnson, Kranch, and Zimmerman (1999) [S]
Emotional triangles	Henry, Chertok, Keys, and Jegerski (1991) [S]
Organizational climate	McCormick et al. (1995)						✓
	Steckler, Goodman, McLeroy, Davis, and Koch (1992) [S]
Organizational climate^a	Nystrom, Ramamurthy, and Wilson (2002)	✓	✓			✓	✓
	Litwin and Stringer (1968) [S]
	Stern (1967) [S]
	Narver and Slater (1990) [S]
School climate	Parcel et al. (2003)	✓					✓
	Hoy, Tarter, and Kottkamp (1991) [S]
Organizational climate	Schoenwald, Carter, Chapman, and Sheidow (2008)	✓	✓	✓		✓	✓
	L. R. James and Sells (1981) [S]
TCU organizational readiness for change^a	Simpson et al. (2007)	✓	✓	✓	✓	✓	✓
	Lehman, Greener, and Simpson (2002) [S]
Organizational climate	Wilson, Ramamurthy, and Nystrom (1999)	✓	✓				✓
	Litwin and Stringer (1968) [S]
Absorptive capacity
Absorptive capacity/managerial IT knowledge, communication channels	Caccia-Bava, Guimaraes, and Harrington (2006)
	Boynton (1994)	✓	✓
	Nilakanta and Scamell (1990)	✓	✓
	Chakrabarti, Feineman, and Fuentevilla (1983)	✓	✓				✓
	Daft and Lengel (1986)
	Boynton (1994)
	Grover (1993)		✓
	Ettlie, Bridges, & O’Keefe (1984)		✓
	Cohen and Levinthal (1990)
Availability of technical specialists (boundary spanners)
Workforce professionalism	Knudsen and Roman (2004)	✓ (partial support)	✓
Environmental scanning		✓	✓
Satisfaction		✓	✓
Unit size	Belkhodja, Amara, Landry, and Ouimet (2007)	✓
Paid research staff

Note: QA = quality assurance; S = innovation sustainability/institutionalization; TCU =Texas Christian University.

For dissemination outcomes related to health services or health behavior.

Vision

Description of articles

Of the two articles examining the association of vision and innovation dissemination, one took place in a health care organization and the other in a public health organization (see Table 4 for study description).

Analysis of instruments for measuring vision

Both studies examined the association of organizational vision with innovation implementation. Livet et al. (2008) measured shared vision and Inamdar, Kaplan, and Bower (2002) measured presence of a well-defined vision as well as the extent to which vision barriers were overcome by program implementation. Shared vision was measured with four items and was based on a single response per unit. Presence of a well-defined vision was measured with a one item question. Inamdar et al. also assessed the extent to which barriers were overcome by successful implementation of the innovative strategy and had interviewees rate (from 1% to 100%) the extent to which the vision barrier (lack of vision related to adoption and implementation) was overcome. Neither study reported psychometric properties of instruments (see Table 6).

The Livet et al. (2008) study, which aimed to identify organizational mechanisms or characteristics that influence implementation of comprehensive programming frameworks, demonstrated that shared vision was correlated with the use of program planning, implementation, and maintenance processes. In the Inamdar et al. (2002) study, interviewers asked executives of health care organizations that were early adopters of an organizational strategy framework (Kaplan & Norton, 1992) whether or not their organization had a well-defined vision. All but one of these organizations reported having a well-defined vision. These same respondents later rated the extent to which the barriers of having “a lack of vision” was overcome (i.e., the strategy is understood by most of the organization); the average rating was 77%.

Managerial Relations:

Description of article

Only one article examined the association of managerial relations with implementation (Lukas, Mohr, & Meterko, 2009). This article examined management support in the context of the national implementation of an Advanced Clinical Access initiative in 78 Veterans Affairs medical centers and included the dimension of managerial support. We did not find any articles that studied the association of managerial relations and innovation adoption, implementation, or dissemination (see Table 4 for study description).

Analysis of instruments for measuring managerial relations

The single article measuring managerial relations (Lukas et al., 2009) examined two dimensions of management support: personal leadership support and practical management support. Personal leadership support was measured using seven items (Cronbach’s α = .89) adapted from scales designed to measure effectiveness of work teams (Lemieux-Charles, Murray, & Baker, 2002; Shortell et al., 2004). Personal leadership support was measured at the individual level. Practical management support, a facility-level variable, was measured using a summary index created from eight dichotomous items about the presence of specific practical expressions of management support for the advanced clinic access (ACA) initiative at the facility (Cronbach’s alpha not reported). Contrary to their hypotheses, only personal leadership support was significantly associated with greater ACA implementation (see Table 6). Of note, these two dimensions just included items on communication (“. . . talking about the ACA” at facility “town meeting” events and staff meetings as well as presentations by clinical staff at managerial meetings) and did not encompass the other characteristics of positive alliances described by Shortell et al (2004; see also Pettigrew et al., 1992; Shortell et al., 1990), which include (a) looking for common ground, (b) involving selected physicians early in planning, (c) identifying the needs and interests of key physicians, (d) daily efforts to build a climate of trust and honesty, (e) breaking down negative stereotypes, and (f) understanding what physicians valued.

Climate

Description of articles

Of the 14 articles that examined the association of organizational climate and innovation dissemination in health, 4 (29%) focused on health care organizations, 1 (7%) focused on public health organizations, 4 (29%) focused on schools, 3 (21%) focused on mental health or substance abuse organizations, and 2 (14%) focused on more than one type of organization (see Table 4). Six articles measured organizational climate as a one-dimensional construct; these measures of climate included from 1 to 32 items. Eight articles measured organizational climate as a multidimensional construct; these measures included 16 to 39 items.

The majority of articles (n = 11) assessed organizational climate at the organization level of analysis. In all but one case (Brownson et al., 2007), team-level or organization-level assessments of climate were based on the perceptions of multiple respondents per unit. The articles also examined the association of organizational climate with a variety of dissemination-related outcomes, although the majority focused on adoption (5 articles [36%]; e.g., presence of innovative medical imaging technologies), and implementation (6 articles [43%]; e.g., therapist adherence to multisystemic therapy). Two articles (14%) focused on sustained innovation use (e.g., institutionalization of obesity prevention interventions in schools). In all but one case, the level of the outcome variable matched the level of the climate variable. Only Schoenwald, Carter, Chapman, and Sheidow (2008) examined the possibility of cross-level effects—that is, the effects of organizational climate on aggregate (i.e., organization-level) and individual-level therapist adherence to multisystemic therapy.

Analysis of instruments for measuring organizational climate

Of the 14 articles reviewed, there were 14 instruments used to assess organizational climate. In other words, each article used a different instrument.

Nine (64%) of the studies reviewed used in whole or in part well-researched, standardized instruments to assess organizational climate. Examples include the Team Climate Inventory, Creative Climate Questionnaire (CCQ), the Organizational Social Context instrument, the Charles F. Kettering School Climate Scale, the Texas Christian University Organizational Readiness for Change instrument, and the Organizational Climate Questionnaire (OCQ). Not surprisingly, the psychometric properties of these instruments are better understood than those of locally adapted, exploratory instruments. Even for these well-researched instruments, however, questions remain about their validity and reliability. Mathisen and Einarsen (2004) raised questions about the translational validity of the CCQ. Although the instrument’s developer defined climate as an organizational attribute, the CCQ measured individual perceptions of the work environment. These authors also note that instrument’s developer has not presented sufficient information to support his claim that the CCQ has adequate psychometric properties. Similarly, several authors have raised doubts—based on empirical study—about the factor structure of the OCQ (Hellriegel & Slocum, 1974; Patterson et al., 2005; Simms & Lafollette, 1975). It remains unclear whether these scales developed by Litwin and Stringer (1968) and subsequently used in two of the studies included in this review (Nystrom, Ramamurthy, & Wilson, 2002; Wilson, Ramamurthy, & Nystrom, 1999) actually measure what they purport to measure. Likewise, the climate scales in the Texas Christian University instrument have displayed variable levels of reliability across types of respondents and settings. For example, Lehman, Greener, and Simpson (2002), computed alpha coefficients for each of the six climate scales for administrators, staff, and programs. Six of the 18 alpha coefficients were below the accepted .70 threshold. Other studies also report variable levels of reliability for climate scales, with some also reporting alpha coefficients well below .70 (Rampazzo, De Angeli, Serpelloni, Simpson, & Flynn, 2006; Saldana, Chapman, Henggeler, & Rowland, 2007).

The climate instruments included in this review exhibited mixed results in terms of predictive validity (see Table 6). Three (21%) instruments showed statistically significant associations with dissemination-related outcomes in health. Allen, Lehrner, Mattison, Miles, and Russell (2007) observed a positive, significant relationship between organizational climate—which they measured as perceived organizational support—and the frequency with which health care providers screened patients for possible domestic violence. In an organizational sample that included medical research and services firms, Choi, Andersen, and Veillette (2009) found that an unsupportive organizational climate inhibited employee creativity. Finally, Wilson et al. (1999) observed in correlation analysis that organizations with more risk-oriented climates tended to adopt more radical innovations and innovations that provide greater relative advantage.

Eight (57%) instruments exhibited only partial evidence of predictive validity for dissemination-related outcomes in health. In most cases, some of the dimensions of organizational climate assessed by the instrument were significantly associated with some of the dissemination-related outcomes examined in the study. For example, Anderson and West (1998) found that one dimension of team climate, support for innovation, emerged as the only significant predictor of overall innovation and innovation novelty. Another dimension of team climate, participative safety, emerged as the best predictor of the number of innovations. It also emerged as the best predictor of team self-reports of innovativeness. A third dimension of team climate, task orientation, predicted innovations’ anticipated administrative efficiency. Finally, as Table 6 shows, well-researched climate instruments exhibit no greater predictive validity than locally developed, exploratory ones, at least with regard to dissemination-related outcomes in health. It is difficult to interpret the mixed results observed in these and other studies included in this review because the statistically significant relationships between climate dimensions and dissemination-related outcomes displayed no obvious pattern.

Absorptive Capacity

Description of articles

Of the three articles that examined the association of absorptive capacity and dissemination in health, one focused on health care organizations, one focused on private substance abuse organizations, and one focused on governmental health ministries and hospitals (see Table 4).

Analysis of instruments for measuring absorptive capacity

There was no agreement across the three studies reviewed in terms of how absorptive capacity was measured. Caccia-Bava et al. (2006) defined and measured two dimensions of absorptive capacity: managerial knowledge and communication. The communication dimension encompassed communication channels, cross-function teams for knowledge integration, and communication boundary spanners. Both dimensions were related to hospitals’ IT adoption, the dependent variable. This study also examined the relationship between absorptive capacity and organizational culture; strong developmental and rational cultures and weaker hierarchical culture were significantly related to both dimensions of absorptive capacity (see Table 6).

Knudsen and Roman (2004) defined and assessed absorptive capacity across three different dimensions, including workforce professionalism, environmental scanning, and satisfaction among client referral sources and third-party payers. The dependent variable was organizational adoption of treatment innovations. They found partial support for workforce professionalism as a predictor of innovation adoption; both environmental scanning and satisfaction were associated with innovation adoption.

Belkhodja, Amara, Landry, and Ouimet (2007) defined absorptive capacity as the size of the unit, as an estimate of its capacity to acquire/absorb knowledge, and the presence in the unit of people who were paid to do research. The dependent variable was whether or not governmental health service organizations used research, assessed for seven types of activities: (a) received research results for areas of responsibility, (b) understood research results, (c) referenced research evidence, (d) adapted research results to provide information to decision makers, (e) promoted adoption of research evidence, (f) made professional decisions based on evidence, and (g) made concrete changes in services provided based on use of research evidence. Absorptive capacity was associated with research utilization. However, the individual components of absorptive capacity had different relationships with research utilization in different kinds of health organizations. Being a medium-size unit was related with research utilization in health ministries and hospitals but not in regional health authorities; being a smaller unit was associated with higher research utilization in regional authorities, as did having paid research staff.

Discussion

There is currently a growing emphasis on the D& I of evidence-based strategies at the organizational level. The standard methodologies used in the health promotion and disease prevention literature to create evidence (e.g., randomized trials) have serious limitations for research focused on how to disseminate evidence-based interventions (Mercer, DeVinney, Fine, Green, & Dougherty, 2007). Attention to measurement issues is critical if we are to make progress in both understanding the factors that influence dissemination and for developing approaches to accelerate the adoption and use of evidence-based strategies in practice and community settings (Bowen et al., 2009; Weiner, 2009). The development of this area of research will benefit from the use of good measures with adequate psychometric properties.

The purpose of this article was to identify available measures for five key systems antecedents hypothesized to be important for D& I research (Greenhalgh et al., 2004), to report on the measures’ psychometric properties, and to develop a compendium of measures that could be recommended for use. We limited our review to peer-reviewed empirical research that focused on outcomes related to innovation, diffusion, adoption, or implementation; that assessed the construct at the organizational level; and that used a quantitative measure of the construct. Some of the constructs we studied have a larger quantitative literature (e.g., climate, leadership) than others (e.g., managerial relations, absorptive capacity), and thus we expected that we might be able to make more recommendations for measures for those constructs. Overall, however, the picture was the same across the five constructs—no measure of a specific construct was used in more than one study, many studies did not report the psychometric properties of the measures, some assessments were based on a single response per unit, and the level of the instrument did not always match the level of analysis. Thus, we are unable to make any recommendations for specific instruments in any of the five constructs evaluated.

A key issue with the majority of the constructs measured is that they might not be homologous multilevel constructs. That is, the construct’s meaning, measurement, and relationships with other variables differ across levels of analysis. Construct validity issues arise when an organizational characteristic is measured at the individual level of analysis when such measures do not convey the construct’s emphasis on shared perceptions of the organization. Construct validity issues also arise when data regarding organizational characteristics are obtained from only one individual. One cannot ascertain from a single respondent the extent to which other organizational members share his or her perceptions across the organization. As more research is done using these constructs, it would significantly benefit the field if extant measures were evaluated and built on, rather than new measures being created for each inquiry. Across all of the constructs, there is a significant need for strong psychometric evaluations of the measures’ properties, and in particular a focus on predictive validity across a range of D& I outcomes. Of particular concern is the considerable diversity in how the constructs are measured, which suggests a lack of theoretical agreement on the constructs’ meaning. This highlights the need for more theoretical work, particularly for construct that do not seem homologous. The need for more careful attention to measurement practices was also apparent in this review. When theory suggests that consensus is important, then it is critical to have more than one respondent. In addition, when multiple respondents are used, it is important to conduct the appropriate statistical tests (e.g., within-group agreement statistics) before aggregating to higher levels of analysis.

Three limitations merit discussion. First, our review only covered those articles published in peer-reviewed journals. Our review could be subject to a publication bias if the “gray literature” contains reliable, valid measures of system antecedents for D& I that do not also appear in peer-reviewed articles. Second, our review included only articles that contained the term health in the title, abstract, or keyword list. Restricting the search in this way facilitated the review by eliminating articles that did not focus on health-related D& I issues, our principal concern. However, our search might have missed articles that included disease-specific terms such as diabetes, cancer, stroke, or HIV rather than the broader term health. Finally, our review includes only articles that contained a measured outcome relevant to innovation, diffusion, adoption, or implementation. Although we accepted a broad range of outcomes within these domains, we excluded articles that measured outcomes in terms of organizational members’ knowledge, attitudes, or behavioral intentions. Researchers interested in reliable, valid measures of system antecedents relevant to these important precursors to D& I should exercise care in interpreting and using the results of our review.

It is likely that the concerns raised in this review of measures of systems antecedents of D& I are relevant to other organizational constructs, as was noted in Weiner et al.’s (2008) evaluation of measures of organizational readiness. It is important for health promotion researchers to recognize that there is a large literature in other fields focusing on organizational assessment beyond the health sector. Inclusion of this literature was beyond the scope of this review, but it may include measures that would be useful in the health context. If we are to build a literature that addresses how to effectively disseminate and implement evidence-based health interventions, we must consider the development and evaluation of a robust set of measures that will serve as the basis of building the field, allowing for comparisons across organizational types and intervention topics, and will allow a robust area of research to develop.

Footnotes

Acknowledgements

The authors would like to acknowledge Glenna Dawson for her work on the literature reviews and Nancy Klockson for participation in the manuscript preparation.

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article:

This manuscript was supported in part by the Centers for Disease Control and Prevention (CDC) and the National Cancer Institute through the Cancer Prevention and Control Research Network, a network within the CDC’s Prevention Research Centers Program (U48-DP001911, U48-DP000057, U48-DP001946, U48/DP000059, U48/DP001944) and by grants from the National Cancer Institute (1K05 CA124415-01A1, R01 CA124402, R01 CA124397, R01 CA123228).

References

Aditya

(2004). Leadership. In Hersen

(Series Ed.) & Thomas

J. C.

(Vol. Ed.), Comprehensive handbook of psychological assessment: Vol. 4. Industrial and organizational assessment (pp. 216-239). Hoboken, NJ: Wiley.

Alexander

M. P.

Zakocs

R. C.

Earp

J. A.

French

(2006). Community coalition project directors: what makes them effective leaders? Journal of Public Health Management & Practice, 12, 201-209.

Allen

Lehrner

Mattison

Miles

Russell

(2007). Promoting systems change in the health care response to domestic violence. Journal of Community Psychology, 35, 103-120.

Anderson

West

(1998). Measuring climate for work group innovation: Development and validation of the team climate inventory. Journal of Organizational Behavior, 19, 235-258.

Barnsley

Lemieux-Charles

McKinney

M. M.

(1998). Integrating learning into integrated delivery systems. Health Care Management Review, 23, 18-28.

Belkhodja

Amara

Landry

Ouimet

(2007). The extent and organizational determinants of research utilization in Canadian health services organizations. Science Communication, 28, 377-417.

Bin Saeed

(1995). Implementation of quality assurance program in Saudi-Arabian hospitals. Saudi Medical Journal, 16, 139-143.

Bostrom

A. M.

Wallin

Nordstrom

(2007). Evidence-based practice and determinants of research use in elderly care in Sweden. Journal of Evaluation in Clinical Practice, 13(4), 665-673.

Bowen

D. J.

Sorensen

Weiner

B. J.

Campbell

Emmons

Melvin

(2009). Dissemination research in cancer control: Where are we and where should we go? Cancer Causes & Control, 20, 473-485.

10.

Boynton

Zmud

Jacobs

(1994). The influence of IT management practice on IT use in large organizations. MIS Quarterly, 18(2), 83-88.

11.

Brownson

R. C.

Ballew

Dieffenderfer

Haire-Joshu

Heath

G. W.

Kreuter

M. W.

Myers

B. A.

(2007). Evidence-based interventions to promote physical activity: What contributes to dissemination by state health departments. American Journal of Preventive Medicine, 33(1 Suppl.), S66-S73.

12.

Brownson

R. C.

Haire-Joshu

Luke

D. A.

(2006). Shaping the context of health: A review of environmental and policy approaches in the prevention of chronic diseases. Annual Review of Public Health, 27, 341-370.

13.

Caccia-Bava

Guimaraes

Harrington

(2006). Hospital organization culture, capacity to innovate and success in technology adoption. Journal of Health Organization and Management, 20, 194-217.

14.

Chakrabarti

Feineman

Fuentevilla

(1983). Characteristics of sources, channels, and contents for scientific and technical information systems in industiral R and D. IEEE Transactions on Engineering Management, EM, 30(2), 83-88.

15.

Choi

Andersen

Veillette

(2009). Contextual inhibitors of employee creativity in organizations the insulating role of creative ability. Group & Organization Management, 34, 330-357.

16.

Cohen

Levinthal

(1990). Absorptive capacity: New perspective on learning and innovation. Administrative Science Quarterly, 35, 128-152.

17.

Daft

Lengel

R. H.

(1986). Organizational information requirements, media richness and structural design. Management Science, 32(5), 554-571.

18.

Daghfous

(2004). Absorptive capacity and the implementation of knowledge-intensive best practices. SAM Advanced Management Journal, 69(2), 21-27.

19.

Drazin

Schoonhoven

(1996). Community, population, and organziation effects on innovation: A multilevel perspective. Academy of Management Journal, 39, 1065-1083.

20.

Emmons

K. M.

Biener

(1993). The impact of organizational characteristics on hospital smoking policies. American Journal of Health Promotion, 8, 43-49.

21.

Emmons

K. M.

Thompson

McLerran

Sorensen

Linnan

Basen-Enquist

Biener

(2000). The relationship between organizational characteristics and the adoption of workplace smoking policies. Health Education & Behavior, 27, 483-501.

22.

Emont

S. L.

Cummings

K. M.

(1989). Adoption of smoking policies by automobile dealerships. Public Health Reports, 104, 509-514.

23.

Ettlie

Bridges

O’Keefe

(1984). Organization strategy and structural differences for radical versus incremental innovation. Management Science, 30(6), 682-695.

24.

Evashwick

Ory

(2003). Organizational characteristics of successful innovative health care programs sustained over time. Family & Community Health, 26, 177-193.

25.

Ferlie

Gabbay

Fitzgerald

Locock

Dopson

(2001). Evidence-based medicine and organisational change: An overview of some recent qualitative research. Bastingstoke, England: Palgrave.

26.

Fielding

J. E.

(1984). Health promotion and disease prevention at the worksite. Annual Review of Public Health, 5, 237-265.

27.

Gittelsohn

Merkle

Story

Stone

E. J.

Steckler

Noel

. . . Ethelbah

(2003). School climate and implementation of the Pathways study. Preventive Medicine, 37(6 Pt 2), S97-106.

28.

Glisson

Landsverk

Schoenwald

Kelleher

Hoagwood

K. E.

Mayberg

. . ., Research Network on Youth Mental Health. (2008). Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health, 35(1-2), 98-113.

29.

Greenhalgh

Robert

Macfarlane

Bate

Kyriakidou

(2004). Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly, 82, 581-629.

30.

Gregory

Henry

D. B.

Schoeny

M. E.

(2007). School climate and implementation of a preventive intervention. American Journal of Community Psychology, 40(3-4), 250-260.

31.

Grover

(1993). An empirically derived model for the adoption of customer-based interorganizational systems. Decision Sciences, 24(3), 603-640.

32.

Helfrich

C. D.

Weiner

B. J.

McKinney

M. M.

Minasian

(2007). Determinants of implementation effectiveness: Adapting a framework for complex innovations. Medical Care Research and Review, 64, 279-303.

33.

Hellriegel

Slocum

J. J.

(1974). Organizational climate: Measures, research and contingencies. Academy of Management Journal, 17, 255-280.

34.

Henry

Chertok

Keys

Jegerski

(1991). Organizational and family systems factors in stress among ministers. American Journal of Community Psychology, 19(6), 931-952.

35.

House

Hanges

Ruiz-Quintanilla

Dorfman

Javidan

Dickson

& GLOBE Country Co-Investigators. (1999). Cultural influences on leadership and organizations: Project GLOBE. In Mobley

Gessner

Arnold

(Eds.), Advances in global leadership (pp. 171-233). Stamford, CT: JAI Press.

36.

Hoy

W. K.

Tarter

C. J.

Kottkamp

R. B.

(1991). Open schools/health schools: Measuring organizational climate. Newbury Park, CA: Sage Publications.

37.

Inamdar

Kaplan

R. S.

Bower

(2002). Applying the balanced scorecard in healthcare provider organizations. Journal of Healthcare Management, 47, 179-195.

38.

James

L. A.

James

(1989). Integrating work environment perceptions: Explorations into the measurement of meaning. Journal of Applied Psychology, 74, 739-751.

39.

James

L. R.

Sells

(1981). Psychological climate: Theoretical perspectives and empirical research. In Magnussen

(Ed.), Toward a psychology of situations: An interactional perspective (pp. 275-295). Hillsdale, NJ: Lawrence Erlbaum.

40.

Johnson

W. L.

Johnson

A. M.

Kranch

D. A.

Zimmerman

K. J.

(1999). The development of a university version of the Charles F. Kettering Climate Scale. Educational and Psychological Measurement, 59(2), 336-350.

41.

Jones

James

(1979). Psychological climate: Dimensions and relationships of individual and aggregated work environment perceptions. Organizational Behavior and Human Performance, 23, 201-250.

42.

Kanter

(1988). When a thousand flowers bloom: Structural, collective and social conditions for innovation in organization. Greenwich, CT: JAI Press.

43.

Kaplan

R. S.

Norton

D. P.

(1992). The balanced scorecard—Measures that drive performance. Harvard Business Review, 70(1), 71-79.

44.

Katz

D. L.

(2009). School-based interventions for health promotion and weight control: Not just waiting on the world to change. Annual Review of Public Health, 30, 253-272.

45.

Kegler

M. C.

Steckler

McLeroy

Malek

S. H.

(1998). Factors that contribute to effective community health promotion coalitions: A study of 10 Project ASSIST coalitions in North Carolina. American Stop Smoking Intervention Study for Cancer Prevention. Health Education & Behavior, 25, 338-353.

46.

Knudsen

H. K.

Roman

P. M.

(2004). Modeling the use of innovations in private treatment organizations: The role of absorptive capacity. Journal of Substance Abuse Treatment, 26, 353-361.

47.

Lehman

W. E.

Greener

J. M.

Simpson

D. D.

(2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22, 197-209.

48.

Lemieux-Charles

Murray

Baker

(2002). The effects of quality improvement practices on team effectiveness: A mediational model. Journal of Organizational Behavior, 23, 533-553.

49.

Litwin

Stringer

(1968). Motivation and organization climate. Boston, MA: Division of Research, Graduate School of Business Administration, Harvard University.

50.

Livet

Courser

Wandersman

(2008). The prevention delivery system: Organizational context and use of comprehensive programming frameworks. American Journal of Community Psychology, 41, 361-378.

51.

Lukas

C. V.

Mohr

D. C.

Meterko

(2009). Team effectiveness and organizational context in the implementation of a clinical innovation. Quality Management in Health Care, 18, 25-39.

52.

Marchionni

Ritchie

(2008). Organizational factors that support the implementation of a nursing best practice guideline. Journal of Nursing Management, 16, 266-274.

53.

Mathisen

Einarsen

(2004). A review of instruments assessing creative and innovative environments within organizations. Creativity Research Journal, 16, 119-140.

54.

McFadden

Henagan

Gowen

C. I.

(2009). The patient safety chain: Transformational leadership’s effect on patient safety culture, initiatives, and outcomes. Journal of Operations Management, 27, 390-404.

55.

McCormick

L. K.

Steckler

A. B.

McLeroy

K. R.

(1995). Diffusion of innovations in schools: A study of adoption and implementation of school-based tobacco prevention curricula. American Journal of Health Promotion, 9(3), 210-219.

56.

Mercer

S. L.

DeVinney

B. J.

Fine

L. J.

Green

L. W.

Dougherty

(2007). Study designs for effectiveness and translation research: Identifying trade-offs. American Journal of Preventive Medicine, 33, 139-154.

57.

Nystrom

Ramamurthy

Wilson

(2002). Organizational context, climate and innovativeness: Adoption of imaging technology. Journal of Engineering and Technology Management, 19, 221-247.

58.

Parcel

G. S.

Perry

C. L.

Kelder

S. H.

Elder

J. P.

Mitchell

P. D.

Lytle

L. A.

. . . Stone

E. J.

(2003). School climate and the institutionalization of the CATCH program. Health Education & Behavior, 30(4), 489-502.

59.

Patterson

West

Shackleton

Dawson

Lawthorn

Maitlis

. . . Wallace

A. M.

(2005). Validating the organizational climate measure: Links to managerial practices, productivity and innovation. Journal of Organizational Behavior, 26, 379-408.

60.

Pettigrew

Ferlie

McKee

(1992). Shaping strategic change: Making change in large organizations: The case of the National Health Service. London, England: Sage.

61.

Rampazzo

De Angeli

Serpelloni

Simpson

D. D.

Flynn

P. M.

(2006). Italian survey of organizational functioning and readiness for change: A cross-cultural transfer of treatment assessment strategies. European Addiction Research, 12, 176-181.

62.

Reh

(n.d.). Vision. Retrieved from http://management.about.com/cs/generalmanagement/g/Vision.htm

63.

Roberts-Gray

Gingiss

P. M.

Boerm

(2007). Evaluating school capacity to implement new programs. Evaluation and Program Planning, 30, 247-257.

64.

Rogers

E. M.

(1995). Diffusion of innovations. New York, NY: Free Press.

65.

Saldana

Chapman

J. E.

Henggeler

S. W.

Rowland

M. D.

(2007). The Organizational Readiness for Change scale in adolescent programs: Criterion validity. Journal of Substance Abuse Treatment, 33, 159-169.

66.

Schoenwald

S. K.

Carter

R. E.

Chapman

J. E.

Sheidow

A. J.

(2008). Therapist adherence and organizational effects on change in youth behavior problems one year after multisystemic therapy. Administration and Policy in Mental Health, 35, 379-394.

67.

Scott

Bruce

(1994). Determinants of Innovative Behavior: A Path Model of Individual Innovation in the Workplace. Academy of Management Journal, 37(3), 580-607.

68.

Seguin

R. A.

Palombo

Economos

C. D.

Hyatt

Kuder

Nelson

M. E.

(2008). Factors related to leader implementation of a nationally disseminated community-based exercise program: A cross-sectional study. The International Journal of Behavioral Nutrition and Physical Activity, 5, 62.

69.

Shortell

S. M.

Marsteller

J. A.

Lin

Pearson

M. L.

S. Y.

Mendel

. . . Rosen

(2004). The role of perceived team effectiveness in improving chronic illness care. Medical Care, 42, 1040-1048.

70.

Shortell

S. M.

Morrison

Friedman

(1990). Strategic choices for America’s hospitals. San Francisco, CA: Jossey-Bass.

71.

Simms

Lafollette

(1975). Assessment of Litwin and Stringer Organization Climate Questionnaire. Personal Psychology, 28, 19-38.

72.

Simpson

D. D.

Joe

G. W.

Rowan-Szal

G. A.

(2007). Linking the elements of change: Program and client responses to innovation. Journal of Substance Abuse Treatment, 33(2), 201-209.

73.

Somech

(2006). The effects of leadership style and team process on performance and innovation in functionally heterogeneous teams. Journal of Management, 32, 132-157.

74.

Steckler

Goodman

R. M.

McLeroy

K. R.

Davis

Koch

(1992). Measuring the diffusion of innovative health promotion programs. American Journal of Health Promotion, 6(3), 214-224.

75.

Thaker

Steckler

Sanchez

Khatapoush

Rose

Hallfors

D. D.

(2008). Program characteristics and organizational factors affecting the implementation of a school-based indicated prevention program. Health Education Research, 23, 238-248.

76.

Trochim

Donnelly

(2007). Research methods knowledge base. Mason, OH: Thomson Custom.

77.

Van de Ven

Polley

Garud

Venkataraman

(1999). The innovation journey. New York, NY: Oxford University Press.

78.

Weiner

B. J.

(2009). A theory of organizational readiness for change. Implementation Science, 4, 67.

79.

Weiner

B. J.

Alexander

J. A.

Shortell

S. M.

(1996). Leadership for quality improvement in health care; empirical evidence on hospital boards, managers, and physicians. Medical Care Research and Review, 53, 397-416.

80.

Weiner

B. J.

Amick

Lee

S. Y.

(2008). Conceptualization and measurement of organizational readiness for change: A review of the literature in health services research and other fields. Medical Care Research and Review, 65, 379-436.

81.

Weiss

E. S.

Anderson

R. M.

Lasker

R. D.

(2002). Making the most of collaboration: Exploring the relationship between partnership synergy and partnership functioning. Health Education & Behavior, 29(6), 683-698.

82.

West

(1990). Innovation and creativity at work: Psychological and organizational strategies. Chichester, England: Wiley.

83.

West

Borrill

Dawson

Brodbeck

Shapiro

Haward

(2003). Leadership clarity and team innovation in health care. The Leadership Quarterly, 14, 393-410.

84.

Wilson

Ramamurthy

Nystrom

P. C.

(1999). A multi-attribute measure for innovation adoption: The context of imaging technology. IEEE Transactions on Engineering Management, 46, 311-321.

85.

Zahra

George

(2002). Absorptive capacity: A review, reconceptualization, and extension. Academy of Management Review, 27, 185-203.