Abstract
We present evidence of the construct validity of a revised version of the Work Cognition Inventory (WCI-R). With the addition of three new scales and the reforming of two from the WCI, the WCI-R contains 12 subscales to measure the cognitive factors associated with employee work passion. Across the majority of scales, the WCI-R was found to produce scores that were conceptually distinct from the preexisting measures. The WCI-R also uniquely demonstrated factor construct validity across the full set of constructs.
Zigarmi, Nimon, Houson, Witt, and Diehl (2009) presented an operational definition of employee work passion that outlined four constructs (work cognition, work affect, employee well-being, and work intention) to help explain the formation of employee work passion. “They used the term employee work passion in order to differentiate the construct from the redundancy, confusion, and misinterpretation associated with employee engagement” (Zigarmi, Nimon, Houson, Witt, & Diehl, 2011, p. 195). In presenting their social cognitive model, Zigarmi et al. (2009) called for instrumentation that focused on the “descriptive cognitive statements of the conditions, policies, and procedures of the work and organizational setting” separate from statements that call for employees to express their feelings that may be the result of those cognitions (pp. 317–318). Consequently, Nimon, Zigarmi, Houson, Witt, and Diehl (2011) developed the Work Cognition Inventory (WCI) to measure the various schema that are associated with the cognitive appraisal of the workplace (i.e., autonomy, collaboration, connectedness with colleagues, connectedness with leader, fairness, feedback, growth, and meaningful work). Although the psychometric scale development process of the WCI was considered exemplar by Newman, Joseph, Sparkman, and Carpenter (2011), the WCI does not measure constructs such as task variety, workload, procedural fairness, and performance expectations, which may be instrumental in explaining how employees become engaged at work (Nimon, Zigarmi, Houson, Witt, & Diehl, 2011).
In this article, we present a revised version of the WCI (i.e., WCI-R) that includes an expanded set of scales that seek to more fully assess employee perceived work experiences. We also present evidence of convergent validity between scale scores and corresponding measures from established scales. Finally, we empirically validate a theoretical factor model that links WCI-R scale scores to specific types of work experiences. Several researchers have advocated that a myriad of different commitment forms, used in combination, are more predictive of employee work behavior than a single set such as job, organization, or people factors (e.g., Cooper-Hakim & Viswesvaran, 2005; James & James, 1989; Saks, 2008; Swailes, 2002). If clustering of the WCI-R scale scores can be psychometrically established, it might serve to simplify interpretation and increase the likelihood of application when used to facilitate interventions at the job, organization, or people level. This study therefore addresses documented limitations of the Nimon et al. (2011) study by expanding the set of work cognition scales and determining whether they measure anything substantively different from scales that purport to measure similar constructs and fit a higher order theoretical model.
Theoretical Underpinning of the WCI-R
The WCI-R and the employee work passion model (Zigarmi, Nimon, Houson, Witt, & Diehl, 2009) from which it stemmed are based on social cognitive theory. Social cognitive theory was founded on the basic premise that human behavior is “agentic” in nature. Because individuals are capable of forethought, vicarious anticipation, self-regulation, symbolization, and self-reflection, individuals are capable of making choices and thereby influencing how they act and what happens in the future (Bandura, 1986, 1997; Deci & Ryan, 2002). Choice making implies appraisals of one’s experiences (Lazarus & Folkman, 1984) and these appraisals give individuals meaning to their experiences in regard to their present and future well-being (Lazarus, 1991a, 1991b).
Appraisal Process
Human appraisals involve both cognitive and affective evaluations of immediate experiences (Lazarus, 1991a, 1991b), and these evaluations result in both cognitive and affective implications for what should be done as a result of those experiences (Parkinson 1997, 2007). The appraisal process has two phases. In the first, individual derives a sense of well-being from an immediate experience. In the second, individual develops appropriate coping mechanisms based on his or her sense or lacking sense of well-being (Lazarus, 1991a; Lazarus & Folkman, 1984).
When processing experiential information, individuals start by forming descriptive mental pictures of “what is” and then move to forming both affective inferences and logical conclusions concerning the meaning of the experience (Parkinson, 2007). Affective meaning can occur early in the experience, during, or after the experience. If inordinately intense, affect can interrupt cognitive processing (Lazarus, 1991a). Cognitive and affective schema (mental descriptions) can also occur simultaneously via bidimensional paths in a synchronous reciprocal relationship (Fugate, Harrison, & Kinicki, 2011). Given the interrelationship between these two evaluation modes, cognitive and affective conclusions must be assessed separately if researchers are able to differentiate between what employees think and feel about their work experiences (Zigarmi et al., 2011).
Implications of the Appraisal Process
The appraisal process is important to the development of instrumentation that seeks to explain how employees become engaged at work. If scales are not focused on specific constructs, blurring can and does happen (Bozeman & Perrewe, 2001). Therefore, instrumentation that seeks to assess the cognitive aspects of what employee thinks about workplace experiences should not be confounded by expressions of affect that may be the result of the cognitive conclusions.
Consider the difference between following items: I have a positive relationship with my leader and My boss asks for my input on decisions that affect me in my job. The first item assesses perceptions about feelings while the second item assesses what the respondents perceive that their boss does or does not do concerning inputs for decision making. The first item helps the practitioner decide whether an intervention has to be made but does little to aid the practitioner to understand what specific interventions should be made. Also, see that the second item assesses “what is” rather than assessing “what is felt concerning what is.” By separating cognition from affect through item design, researchers avoid confounded perceptions of the respondent and thereby gain an estimation of the independent effects of each (Fields, 2002).
WCI-R Constructs, Definitions, and Organizing Framework
The process used to determine the constructs of the WCI-R was similar to the process used by Nimon et al. (2011). Using EBSCOhost as a search engine, we conducted a literature review of studies published from 1995 to 2010. As in Nimon et al. (2011), we included a variety of databases in our search, including Academic Search Complete, Business Source Premier, ERIC, PsychARTICLES, PsychCRITIQUES, and PscyhINFO and used search terms associated with the outcome variables of employee work passion such as organizational commitment, job satisfaction, sense of well-being, intrinsic motivation, positive affect, and productivity. In some cases, we examined studies before 1995 when the primary source of various constructs appeared frequently in other researchers’ studies (e.g., Hackman & Oldham, 1975; Meyer & Allen, 1991).
As we reviewed the number of possible environmental factors that form what employees think about the workplace, we realized that both parsimony and breadth were important. Although our research yielded 35 factors that could be considered, we limited the WCI-R to 12 factors so as to keep the length of the instrument appropriate for use in corporate settings. We kept six of the eight WCI constructs (autonomy, collaboration, connectedness with colleagues, connectedness with leader, growth, and meaningful work) and reframed two of the WCI constructs. The original WCI construct of feedback was reframed to focus on job performance in order to avoid construct overlap with other WCI-R scales. The original WCI construct of fairness was split into two constructs—distributive justice and procedural justice—in order to avoid blurring the two types of justice into a single construct. In keeping with Nimon et al. (2011) and our review of the literature, we added three new constructs not considered in the original WCI (performance expectations, task variety, and workload balance).
Feedback
Feedback is the extent to which individuals perceive they have access to accurate information concerning their job performance (Hackman & Oldham, 1975). It considers to what degree job performance is accurate, frequent, and can be fairly judged by self and others (Sims, Szilagyi, & Keller, 1976). Previous researchers found that feedback positively correlated (rs = .41–.43) with outcome variables such as organizational commitment (Eby, Freeman, Rush, & Lance, 1999), job satisfaction (Eby et al., 1999; Fried & Ferris, 1987), pay satisfaction (Williams, McDaniel, & Nguyen, 2006), and intrinsic motivation (Eby et al., 1999).
Distributive Justice
Distributive justice is the extent to which an individual perceives that there is an equal input to output ratio and effort to reward (Colquitt, Conlon, Wesson, Porter, & Ng, 2001). Individuals calculate the ratio of their contribution or input to outcomes and then compare that ratio with other employee reward levels. Distributive justice is the individual’s reaction to the nature, level, and distribution of organizational rewards, much to the exclusion of the quality of the decisions through which those rewards are given. Distributive justice is also referred to in the literature as rewards and has positively correlated (rs = .42–.51) with outcome variables such as organizational commitment (Colquitt et al., 2001), job satisfaction (Colquitt et al., 2001; Eby et al., 1999; Griffeth, Hom, & Gaertner, 2000), and trust in authority (Colquitt et al., 2001).
Procedural Justice
Procedural justice is the extent to which an individual perceives that decision making, which distributes resources, is fair and equitable (Colquitt et al., 2001; Greenberg, 1987). This includes behaviors such as the suppression of bias in decision making, the inclusion of employee input on decisions that affect them, and equal application of rules for everyone. Procedural justice positively correlated (rs =.33–.43) with outcome variables such as job satisfaction (Colquitt et al. 2001), trust in leadership (Colquitt et al., 2001; Dirks & Ferrin, 2002), pay satisfaction (Brown, 2001; Greenberg, 1996; Williams et al., 2006), employee retaliation behaviors (Greenberg, 1996; Lee & Allen, 2002; Sprouse, 1992) as well as organizational commitment, use of organizational citizenship behaviors (OCBs), and performance (Colquitt et al., 2001).
Task Variety
Task variety is the extent to which individuals perceive that the work they do and actions they take to accomplish their work are new and different (Hackman & Oldham, 1975). It is a matter of how much formalization, routine, and rules are present in the job so as to produce boredom versus the amount of variety that individuals have in doing their job. Task variety is also referred to as skill variety (e.g. Mathieu & Zajac, 1990), routinization (Curry, Wakefield, Price, & Mueller, 1986; Griffeth et al., 2000), and challenge and variety (James & James, 1989). Task variety correlated most strongly (rs = .37–.46) with job satisfaction (Brown, 1996; Curry et al., 1986) and to a lesser degree (rs = .18–.26) with outcome variables such as organizational commitment (Brown, 1996; Curry et al., 1986; Mathieu & Zajac, 1990), positive organizational climate (James & James, 1989), and pay satisfaction (Williams et al., 2006).
Workload Balance
Workload balance is the extent to which individuals perceive that their work load is reasonably proportioned for the time they have to accomplish their work (Katz & Kahn, 1978). It is a matter of reasonable timelines, needed resources, complexity, and the feasibility of actions needed to accomplish various work outcomes (James & James, 1989). Workload balance, also referred to as role overload, correlated most strongly (rs = .25–.27) with outcome variables such as job satisfaction (Curry et al., 1986) and positive organizational climate (James & James, 1989) and to a lesser degree (r = .20) with organizational commitment (Curry et al., 1986).
Performance Expectation
Performance expectation is the extent to which an individual perceives that work outcomes must be done to a certain level of quality and quantity (Locke, 1966). Performance expectations are the employee’s perceived emphasis on the performance of in-role behaviors by management (Lee, Locke, & Latham, 1989) and are associated with the clear signals sent from many layers of management concerning performance standards for unit and individual performance (Williams & Anderson, 1991). Performance expectations are not discretionary but rather in-role behaviors connected to short-term performance periods, where in-role behaviors are separate from organizational citizenship behaviors (Williams & Anderson, 1991). Although not frequently measured, performance expectations or in-role behaviors correlated most strongly (rs = .37–.42) with outcome variables such as interpersonal helping, personal industry, individual initiative, and loyal boosterism (Tompson & Werner, 1997) and to a lesser degree (rs = .20–.22) with perceived organizational support, satisfaction, and affective organizational commitment (Randall, Cropanzano, Bormann, & Birjulin, 1999).
Organization of Constructs
We found that the categorization of work cognition constructs was inconsistent in the literature we reviewed. Based on our synthesis of the literature, we hypothesized that the WCI-R constructs could be theoretically organized into three groups: (a) constructs that focus on job experiences, (b) constructs that focus on organizational experiences, and (c) constructs that focus on experiences with people (see Figure 1). We hypothesized that the constructs of autonomy, meaningful work, task variety, and workload balance would cluster together because they were focused on job experiences (cf. Brown, 1996; Griffeth et al., 2000; Mathieu & Zajac, 1990). We hypothesized that the constructs of distributive justice, growth, performance expectations, and procedural justice would cluster together because they were focused on organizational experiences (cf. Kopelman, Brief, & Guzzo, 1990; Williams et al., 2006). We hypothesized that the constructs of connectedness with colleagues, connectedness with leader, collaboration, and feedback would cluster together because they were focused on experiences with people (cf. Eby et al., 1999; James & James, 1989; Mathieu & Zajac, 1990).
Conceptual model of the Work Cognition Inventory–Revised (WCI-R) constructs.
Development and Validation of Scales
The WCI items and new items formed the item pool for the WCI-R. Instrument authors reviewed relevant literature to ensure content validity for the work cognition constructs and developed new items for the constructs of distributive justice, feedback, performance expectations, procedural justice, task variety, and workload balance. Existing items from the remaining WCI scales (autonomy, collaboration, connectedness with colleagues, connectedness with leader, growth, meaningful work) were reviewed and revised if it were determined that the items were not clearly situated within a job, organization, or people context.
We followed the sequential exploratory–confirmatory procedure recommended by Worthington and Whittaker (2006) to examine, refine, and confirm the factor structure of the scales. Exploratory factor analysis (EFA) was used to explore factor structures, reduce the item set, and identify items that best fit a simple structure. Principal axis factoring and promax rotation was employed because we hypothesized a theoretical factor structure with correlated factors. When deciding how many factors to extract, we considered the conceptual consideration of the items (see Hair, Black, Babin, & Anderson, 2010), scree plots, eigenvalues, and Wood, Tataryn, and Gorsuch’s (1996) guidelines for overextraction. We confirmed the final factor structure that resulted from the EFA with confirmatory factor analyses (CFAs), employing the maximum likelihood estimation technique in keeping with Thompson’s (2004) recommendations.
We tested for evidence of construct validity of WCI-R scores over the course of two studies. In Study 1, we assessed the factor structure of the initial pool of WCI-R items, selected those items that best fit a simple structure, and confirmed the factor structure of the refined set of items. In Study 2, we further refined the pool of WCI-R item resulting from Study 1 and examined the nomological validity of WCI-R first-order scale scores.
Study 1
Purpose
The purpose of Study 1 was to test the underlying factor structure and reliability of the WCI-R. In keeping with traditional scale development processes, we started with “a substantially larger number of items than” was “planned to retain” (Worthington & Whittaker, 2006). Factor and reliability analyses were conducted with the intent of reducing the original item set to a total of 60 items (5 per scale). Our goal to produce a set of a scales with 5 items each was supported by Hulin, Netemeyer, and Cudeck (2001) who suggested that constructs such as those defined for the WCI-R could suffice with 4 or 5 items; recommendations from Durvasula, Netemeyer, Andrews, and Lysonski (2006) to develop multiple scale instruments with an equal number of items; and our desire to keep the instrument sufficiently concise to be reliably used in corporate settings.
Participants and Procedure
Participants included clients and client prospects of a multinational management and training consulting company. The company was selected out of convenience, as it was the affiliated institution for a one of the WCI-R authors. Using an e-mail list that the company maintains for marketing purposes, individuals were invited to participate in the study and offered a free consulting report as an incentive. Those who volunteered were provided a link to a survey that included the initial set of WCI-R items. Of the 2,282 who volunteered, 1,380 completed the study. Over half (57%) of the participants were female and supervised other employees (66%). The age of most of the participants was between 52 and 69 (44%) or between 31 and 51 (53%). Over two thirds (68%) of the respondents were from North America. The sample also included employees from Africa, Asia, Australia, New Zealand, Central America, South America, and Europe.
The initial version of the WCI-R contained 83 items. The autonomy, collaboration, connectedness with colleagues, connectedness with leader, distributive justice, growth, and meaningful work scales each had 5 items. The feedback scale had 8 items, performance expectations had 10, procedural justice had 9, task variety had 9, and workload balance had 9. All items were measured on a 6-point scale, with 1 indicating to no extent and 6 indicating to the full extent.
Analysis
EFAs were conducted using responses from a randomly chosen half of the sample (n 1 = 667), with remaining responses (n 2 = 713) saved for a series of CFA. We then computed coefficient αs for the full sample.
Results
EFA (Subsample 1)
In the initial EFA, 12 factors with eigenvalues greater than 1.0 were extracted, accounting for 70% of the variance of the 83 original items. Analysis of the pattern and structure matrices indicated that 2 items did not load on their respective factor and 2 items had pattern coefficients of less than .40. Omitting those items resulted in 12 factors with eigenvalues greater than 1 being extracted, accounting for 71% of the variance of the 79 items submitted. Our analysis of the pattern and structure coefficients indicated that 19 items could be omitted without an adverse effect on content coverage or scale validity, resulting in a desirable equal number of items per scale (Durvasula, Netemeyer, Andrews, & Lysonski, 2006). In the final EFA, 12 factors were extracted, accounting for 73% of the variance of the 60 items submitted. All items had pattern coefficients in excess of .40 on their respective factor, with no significant cross loadings on other factors. Analysis of structure coefficients indicated that all items correlated most highly with their theoretical factor. Interfactor correlations ranged from .26 to .70, suggesting a higher order factor structure.
CFA (Subsample 2)
We first tested a first-order factor structure of the 60 WCI-R items because a well-defined first-order factor structure sets the upper limit for goodness of fit and is the basis for subsequent higher order models (Marsh, 1987). We examined modification indices and item wording to determine whether or not the data warranted a complex factor structure where error terms were allowed to correlate. In 3 of the 12 scales, we correlated error terms because it appeared that the use of meaningful in 2 of the meaningful work items, procedures in 2 of the procedural justice items, and standards in 2 of the performance expectations items resulted in correlated error.
Pattern coefficient values were all .70 or greater indicating appropriate measurement structure in terms of pattern coefficients (cf. Hair et al., 2010). Analysis of structure coefficients (Graham, Guthrie, & Thompson, 2003) indicated that all items correlated most highly with their theoretical factor. The range of composite reliability (CR; .88–.96) and average variance extracted (AVE; .61–.85), respectively, provided evidence of adequate reliability and convergent validity for the first-order factors (cf. Hair et al., 2010). Shared variances between factors were lower than the AVE of individual factors, thus providing evidence of discriminant validity (cf. Hair et al., 2010). Model fit indices (comparitive fit index [CFI] = .94; Tucker Lewis index [TLI] = .93; root mean square error of approximation [RMSEA] = .048, 90% confidence interval [CI]: [.049, .053]; standardized root mean square residual [SRMR] = .051) indicated that the model fits the data reasonably well (cf. Hair et al., 2010; Hu & Bentler, 1999; Schumacker & Lomax, 1996).
We next tested a hierarchical factor model in which the 60 WCI-R items identified through the EFA were arranged in the 12 hypothesized factors, each of which was related to one of three second-order factors, which were related to an overall third-order factor (see Figure 1). The autonomy, meaningful work, task variety, and workload balance first-order factors were related to a job-cognition second-order factor. The distributive justice, growth, performance expectations, and procedural justice first-order factors were related to an organization-cognition second-order factor. The collaboration, connectedness with colleagues, connectedness with leader, and feedback first-order factors were related to a people-cognition second-order factor. The job-cognition, organization-cognition, and people-cognition second-order factors were related to a third-order work-cognition factor.
Pattern coefficient values were all .60 or greater, with the exception of the path between job cognition and work balance (.50), indicating fairly appropriate measurement structure in terms of pattern coefficients (cf. Hair et al., 2010). Analysis of structure coefficients (Graham et al., 2003) indicated that all items correlated most highly with their theoretical factor. Model fit indices (CFI = .93; TLI = .93; RMSEA = .051, 90% CI: [049, .053]; SRMR = .06) indicated that the model fits the data reasonably well (cf. Hair et al., 2010; Hu & Bentler, 1999; Schumacker & Lomax, 1996). The hierarchical third-order factor model fits the data better than a single-factor model, where all 60 items were directly loaded on a single factor, Δχ2(15) = 18,151.8, p < .001, and a second-order factor model where all first-order factors were directly loaded onto a single second-order factor, Δχ2(3) = 155.2, p < .001.
Reliability (Full sample)
Reliability coefficients for the first-order scales are presented in Table 1. All of the scales produced acceptable levels of reliability (cf. Henson, 2001).
Reliability (α), Validity (r), and Discriminant Validity (ΔR 2) Coefficients for Studies 1 and 2.
Study 2
Purpose
The purpose of Study 2 was to further refine the pool of WCI-R item resulting from Study 1 to determine whether a shorter version of the instrument could be developed that yielded a good model fit and that did not require error terms to be correlated in order to achieve a good model fit. We also tested for convergent validity by correlating scale scores from the WCI-R to scores from established scales where we expected correlations between WCI-R scales and conceptually similar scales to be in the range of |.30| and |.75| (cf. Ward, Fischer, Lam, & Hall, 2009). Finally, we tested for discriminant validity by regressing the WCI-R scales and the validity scales on two measures of employee affect and comparing the results to determine whether the WCI-R scales had more or less shared variance with measures of employee affect than the validity scales.
Method
Participants and Procedure
As in Study 1, participants included clients and client prospects of a national management and training consulting company. Individuals were invited to participate in the study and offered a free consulting report as an incentive. Those who volunteered were randomly assigned to one of the two groups (i.e., A, B). Each group received a survey that contained all of the 12 WCI-R scales along with a set of corresponding scales selected to test the convergent validity of a parcel of WCI-R scales. Group A completed a parcel containing validity scales to test the convergent validity of collaboration, distributive justice, meaningful work, workload balance, task variety and performance expectation, and procedural justice scales. Group B completed a parcel containing validity scales to test the convergent validity of autonomy, feedback, procedural justice, connectedness with colleagues, growth, and connectedness with leader.
Of the 1,714 participants who volunteered, 1,519 completed the study. Over half (66%) of the participants were female and supervised other employees (65%). The age of most participants was between 31 and 51 (56%) or between 52 and 69 (40%). Over four fifths (81%) of the respondents were Caucasian.
WCI-R
The 60-item WCI-R resulting from Study 1was refined to a 36-item set. We used standardized factor loadings from Study 1 to create an index of internal item quality and the face validity of each item to create an index of judgmental item quality (cf. Stanton, Sinar, Balzer, & Smith, 2002). Finally, we used professional judgment to evaluate the items’ quality scores to configure a 36-item instrument from among the top items, where each first-order scale contained 3 items. Each dimension was measured on a 6-point scale, with 1 indicating to no extent and 6 indicating to the full extent.
Autonomy
The autonomy scale from the Job Characteristics Inventory (Sims et al., 1976) was used to assess the convergent validity of the WCI-R autonomy scale. It contained 5 items anchored on a 5-point scale, with 1 indicating very little or a minimum amount and 5 indicating very much or a maximum amount depending on the item. Coefficient α was .78.
Collaboration
The group cohesiveness scale from Podsakoff and MacKenzie (1994) was used to assess the convergent validity of the WCI-R collaboration scale. It contained 6 items anchored on a 5-point scale, with 1 indicating strongly disagree and 5 indicating strongly agree. Coefficient α was .97.
Connectedness With Colleagues
The social support scale from Morgeson and Humphry (2006) was used to assess the convergent validity of the WCI-R connectedness with colleagues scale. It contained 6 items, anchored on a 5-point scale, with 1 indicating strongly disagree and 5 indicating strongly agree. Coefficient α was .86.
Connectedness With Leader
The leader–member exchange survey from Scandura and Green (1984) was used to assess the convergent validity of the WCI-R connectedness with leader scale. It contained 7 items, anchored on a 4-point scale, with 1 indicating never know where I stand, not at all, no chance, probably not or less than average and 4 indicating always know where I stand, completely, certainly would, or extremely effective depending on the item. Coefficient α was .92.
Distributive Justice
The distributive justice scale from Parker, Baltes, and Christiansen (1997) was used to assess the convergent validity of the WCI-R distributive justice scale. It contained 3 items anchored on a 5-point scale, with 1 indicating strongly agree and 5 indicating strongly disagree. Coefficient α was .94.
Feedback
The feedback scale from the Job Characteristics Inventory (Sims et al., 1976) was used to assess the convergent validity of the WCI-R feedback scale. It contained 3 items anchored on a 5-point scale, with 1 indicating very little or a minimum amount and 5 indicating very much or a maximum amount depending on the item. Coefficient α was .92.
Growth
The career development scale from Ivancevich and Matteson (1980) was used to assess the convergent validity of the WCI-R growth scale. It contained 5 items, anchored on a 7-point scale, with 1 indicating never a source of stress and 5 indicating always a source of stress. Coefficient α was .93.
Meaningful Work
The task significance scale from Morgeson and Humphrey (2006) was used to assess the convergent validity of the WCI-R meaningful work scale. It contained 4 items, anchored on a 5-point scale, with 1 indicating strongly disagree and 5 indicating strongly agree. Coefficient α was .89.
Performance Expectations
The role ambiguity scale from Ivancevich and Matteson (1980) was used to assess the convergent validity of the WCI-R performance expectations scale. It contained 5 items, anchored on a 7-point scale, with 1 indicating never a source of stress and 5 indicating always a source of stress. Coefficient α was .86.
Procedural Justice
The procedural justice scale from Parker et al. (1997) was used to assess the convergent validity of the WCI-R distributive justice scale. It contained 4 items anchored on a 5-point scale, with 1 indicating strongly agree and 5 indicating strongly disagree. Coefficient α was .85.
Task Variety
Three items from the job variety scale of the Job Characteristics Inventory (Sims et al., 1976) were used to assess the convergent validity of the WCI-R task variety scale. The items were anchored on a 5-point scale, with 1 indicating very little and 5 indicating very much. Coefficient α was .73.
Workload Balance
The quantitative role overload scale from Ivancevich and Matteson (1980) was used to assess the convergent validity of the WCI-R growth scale. It contained 5 items, anchored on a 7-point scale, with 1 indicating never a source of stress and 5 indicating always a source of stress. Coefficient α was .89.
Affect
We used the Positive and Negative Affect Scale (PANAS; Watson, Clark, & Tellegen, 1988) as measures of employee affect. The positive (PA) and negative (NA) affect scales contained 10 items each and were anchored on a 5-point scale with 1 indicating very slightly or not at all and 5 indicating extremely. Coefficient αs for PA and NA scores were .93 and .90, respectively.
Analysis
We assessed factor structure validity by conducting a set of CFAs as in Study 1. We assessed convergent validity by correlating scores from each WCI-R scale to scores from the respective validity scale. For example, scores from the WCI-R meaningful work scale were correlated with scores from the Morgeson and Humphrey (2006) task significance scale. We assessed discriminant validity by regressing each set of WCI-R scale scores and validity scale scores on PA and NA scores and subtracting the Multiple R 2 for each of the WCI-R scales from the corresponding validity scale. For example, WCI-R meaningful work scale scores and Morgeson and Humphrey task significance scores were both regressed on PA and NA scores. The Multiple R 2 from the Morgeson and Humphrey task significance scores was subtracted from the Multiple R 2 from the WCI-R meaningful work scores as a measure of discriminant validity. Positive values of Δ Multiple R 2 indicated that WCI-R scale scores had more variance in common with employee affect than validity scale scores and negative values indicated that the WCI-R scale scores had less.
Results
First-Order Factor Model
In the first-order factor model, the 36 WCI-R items were arranged in the 12 hypothesized factors. Pattern coefficient values were all .60 or greater, indicating appropriate measurement structure in terms of pattern coefficients (cf. Hair et al., 2010). Analysis of structure coefficients (Graham et al., 2003) indicated that all items correlated most highly with their theoretical factor. The range of CR (.82–.95) and AVE (.62–.87), respectively, provided evidence of adequate reliability and convergent validity for the first-order factors (cf. Hair et al., 2010). Shared variances between factors were lower than the AVE of individual factors, thus providing evidence of discriminant validity (cf. Hair et al., 2010). Model fit indices (CFI = .95; TLI = .94; RMSEA = .056, 90% CI: [054, .058]; SRMR = .049) indicated that the model fits the data reasonably well (cf. Hair et al., 2010; Hu & Bentler, 1999; Schumacker & Lomax, 1996).
Higher Order Factor Model
For the higher order model, the 36 WCI-R items were arranged in the 12 hypothesized factors, each of which was related to one of three second-order factors that were related to an overall third-order factor. Pattern coefficient values were all .60 or greater, with the exception of the path between job cognition and work balance (.32), indicating fairly appropriate measurement structure in terms of pattern coefficients (cf. Hair et al., 2010). Analysis of structure coefficients (Graham et al., 2003) indicated that all items correlated most highly with their theoretical factor. Model fit indices (CFI = .94; TLI = .94; RMSEA = .056, 90% CI: [054, .058]; SRMR = .06) indicated that the model fits the data reasonably well (cf. Hair et al., 2010; Hu & Bentler, 1999; Schumacker & Lomax, 1996).
In support of the low correlation between job cognition and work balance, we also tested a model where work balance was directly correlated with the third-order factor, work cognition. This model exhibited a slightly better fit (Δχ2 = 27.9, df = 0; CFI = .94; TL = .94; RMSEA = .056, 90% CI: [.054, .057]; SRMR = .06) than the theorized model and fits the data better than a single-factor model where all 36 items were directly loaded on a single factor, Δχ2(14) = 17,765.6, p < .001, and a second-order factor model where all remaining first-order factors were directly loaded onto a single second-order factor, Δχ2(3) = 432.4, p < .001.
Reliability, convergent validity, and discriminant validity coefficients for each of the first-order WCI-R scale scores are presented in Table 1. Consistent with Study 1, WCI-R reliability coefficients fell above the standard cutoff of .70 (Henson, 2001). Convergent validity coefficients ranged from .63 to .86 and with only one exception did validity scale scores not share the most variance with their corresponding set of WCI-R scale scores. Distributive justice scale scores from Parker et al. (1997) shared more variance with WCI-R growth (r 2 = .37) than WCI-R distribute justice (r 2 = .36) scale scores. In general, WCI-R first-order scale scores demonstrated convergent validity with scores from like measures, with only a few of the measures (i.e., connectedness with leader and feedback) demonstrating sufficient common variance to call into question their conceptual distinctiveness from their convergent validity counterpart (cf. Ward et al., 2009). Discriminant validity coefficients ranged from −.09 to .25. With the exception of meaningful work, task variety, and collaboration, the WCI-R scales scores and their corresponding validity scale scores shared a similar amount of variance with employee affect.
Discussion
The findings from the two studies indicate that the WCI-R yields 12 factors associated with the cognitive appraisal of the workplace from the perspective of job, organization, and people experiences. In addition to autonomy, collaboration, connectedness with colleagues, connectedness with leader, feedback, meaningful work, and growth that were the foci of the WCI, the WCI-R contains scales to measure cognitive perceptions of distributive justice, procedural justice, task variety, performance expectations, and workload balance. Additionally, the WCI-R yields four superordinate constructs that provide a more general perspective of work cognition.
Although other scales exist that consider many of the constructs measured in the WCI (e.g., autonomy, meaningful work, growth), the WCI was specifically designed to focus solely on what employees think about their work environment. Unlike scales that include items that assess employee feelings (e.g., “I feel that the work I do on my job is valuable,” May, Gibson, & Harter, 2004; “I feel overburdened by my role,” Peterson et al., 1995; “I am satisfied with the quality of training and development programs available,” Vandenberg, Hettie, Eastman, & Eastman, 1999), the WCI does not include expressions of affect that may be the result of the cognitions the scales purport to measure.
In general, the WCI-R produced scale scores that were conceptually distinct from like measures. In only 2 of the 12 WCI-R scales were correlations with scores from conceptually similar scales greater than .75. In only 3 of the 12 WCI-R scales did scores share more variance with measures of employee affect that their corresponding validity scales.
Although the conceptual distinctiveness and the discriminant validity of a few of the WCI-R scales may be questioned, we consider the WCI-R an appropriate tool to use when assessing employees’ cognitive appraisal of the workplace as it focuses on what employees think about the workplace, does not contain negatively worded items that can produce method effects, and demonstrates factor analytic construct validity across the full set of factors. A post hoc factor analysis of the validity scale items employed in Study 2 produced less than optimum evidence of factor analytic construct validity. Across the set of validity scales used in Study 2, we found instances where items loaded on the wrong factors, cross-loaded with other factors, and had factor loadings < .50 on their respective factors.
Implications for Practice
We offer two suggestions for practitioners considering the development of the work environment in which they expect the employee to develop and flourish. Administer the WCI-R or similar scales to assess the work environment as a diagnostic mechanism for planning interventions that (a) encourage employee work passion or engagement (b) increase person–environment (P-E) fit.
Environmental Assessment
When examining the environmental factors that contribute to employee work passion, practitioners must determine the granularity of their assessment. The higher order factor structure of the WCI-R allows practitioners to assess work cognition at three levels. The work-cognition factor (third level) provides a general assessment of what employees think of their environment. The second-level factors provide a more focused assessment of what employees think of environmental issues related to job, organization, and people. The first-level factors provide specific assessment of 12 environmental factors associated with employee work passion.
The multiple levels of work cognition complement each other and as with other higher order factor models, “each is needed to see patterns at a given level of specificity versus generality” (see Thompson, 2004, p. 73). Consider the analogy of hiking in the mountains versus flying over them offered by Thompson (2004, p. 72), “When you are hiking you can see rocks and trees and fish in streams. When you fly over the mountains, you can see patterns made by the mountains as a range.” In much the same way, organizational stakeholders may differ in the perspective of the WCI-R data they wish to view. Managers may be interested in a specific view of the environmental factors that contribute to employee work passion, while executive team members may only desire a general view of the data. The subordinate and superordinate constructs of the WCI-R accommodate both types of stakeholder as it yields constructs that consider both specificity and generality.
Although we recommend that practitioners consider all 12 environmental factors and associated superordinate constructs when making an organizational diagnosis relating to employee work passion, we recognize that organizational resources may prohibit such a full-scale assessment. In those cases, practitioners might consider conducting pulse checks by administering a single WCI-R scale to determine the need for an intervention along a particular environmental area. Alternatively, practitioners may want to focus on a particular type of experience and administer the set of scales relating to job, organization, or people. We note that if practitioners choose to administer scales other than the WCI-R, they must carefully examine the content of the items to ensure that the cognitive statements of the workplace are not blurred with statements of affect. They should also verify the construct validity of resulting scores no matter what assessment tool is chosen.
P-E Fit
When considering interventions to increase P-E fit, some researchers (e.g., Cable & DeRue, 2002; Kristof-Brown, Zimmerman, & Johnson, 2005) advocate indirect measures to assess fit. Such an approach involves practitioners and researchers considering various dimensions of the work environment that may influence P-E fit. This approach is by nature multifaceted as it considers job characteristics, employee relationships, and organizational elements (Kristof-Brown et al. 2005).
Simultaneously, research in career assessment and development (e.g., Dawis & Lofquist, 1984; Holland, 1985) often advocates a direct method of assessing fit. The direct measurement of fit between employee attributes and work environment involves explicitly asking employees whether they believe a good fit exists. Such items may be built around topics such as a general sense of fit with the job or organization. (e.g., “All things considered, this job suits me,” Brkich, Jeffs, & Careless, 2002; “My personal values match my organization’s value and culture,” Cable & DeRue, 2002).
Some researchers have built P-E fit items around diverse topics such as social efficacy (e.g., Fan et al., 2013) or career indecision (Hacker, Carr, Abrams, & Brown, 2013), while others (e.g., Guan, et al. 2012) have built around personal values and personal attributes such as career locus of control. When using such measures of direct fit, researchers and practitioners may learn what the quality of the fit is without gaining insight into why such a level of fit exists.
Because a considerable amount of the measurement emphasis is placed on direct perceptions of fit (Fields, 2002), the assessment of the characteristics of the work environment around which P-E fit perceptions are formed is often neglected and therefore gives little hint as to what practitioners may do to change the environment. The WCI-R, however, provides respondents the opportunity to comprehensively describe perceptions of their work environment including job, organization, and people experiences. The resulting data therefore provide practitioners with the opportunity to understand what could be changed to increase P-E fit, which could increase the possibility that employees will choose to remain and develop their careers.
Limitations and Implications for Research
We note three major limitations of this study that form suggestions for future research. In this article, we (a) presented evidence of construct validity based on convenient samples and (b) limited tests of reliability to measures of internal consistency, and (c) limited evidence of nomological validity.
Sample
The samples described in this article suffered from the similar selection bias as Nimon et al. (2011). In particular, study participants were composed of volunteers who had access to a computer and were affiliated with a particular training and management consulting company. Additionally, only half of the participants who volunteered to participate in Study 1 completed the survey. Future research should examine the construct validity of the WCI-R using random samples from defined populations. Testing for measurement invariance across different industries, job types, ethnicities, and culture would add to the body of knowledge on the WCI-R (cf. Nimon & Reio, 2011). As noted by Shuck, Reio, and Rocco (2011), cross-cultural research on constructs related to employee engagement could be of particular benefit to the broader international human resource development community.
Reliability
Evidence of reliability presented in this article is limited to internal consistency. Future research should examine the stability of items across time by administering the WCI-R at two different measurement occasions and comparing results. Examining reliability and the factor structure of the WCI-R across time would add to the body of knowledge on the WCI-R (cf. Pitts, West, & Tein, 1996).
Nomological Validity
Evidence of convergent and discriminant validity presented in this article is limited to first-order scale scores. Future research could examine the convergent and discriminant validity of second- and third-order scales scores. The job cognition, organizational cognition, and people cognition scale scores could be correlated with existing scales. For example, total scores from the Job Characteristics Inventory (Hackman & Oldham, 1975) could be used to test the convergent validity of second-order job cognition scale scores.
Footnotes
Appendix
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.