Development and Validation of a Work Mattering Scale (WMS)

Abstract

Many people desire work that matters to others and society. This study reports the development and validation of the Work Mattering Scale (WMS), which measures work mattering with two dimensions, that is, societal mattering and interpersonal mattering. Phase I utilized a review of the scholarly literature, in-depth interviews, and expert feedback to formulate 45 items. In Phase II, with a sample of 299 working adults in the United States, exploratory factor analyses were employed and produced a two-factor model with 10 items. In Phase III, with another sample of 251 working adults, confirmatory factor analyses were conducted to test the initial structure specified in Phase II. In Phase IV, the construct validity was examined by correlations and factor analysis among the total and subscale scores with existing measures. In Phase V, 4-week test–retest reliability was calculated. Results provided support for the reliability and validity of the WMS.

Keywords

mattering work mattering societal mattering interpersonal mattering scale development

Work accounts for a critical part of the identity of many people, as work often becomes a source for meaning, dignity, and mattering (Blustein, 2011). Influenced by the recent movement of ecological perspectives (Cook, Heppner, & O’Brien, 2002) and relational approaches to work experience (Schultheiss, 2007), understanding work experience within the social and relational context has gained more attention in work literature (Ali, Fall, & Hoffman, 2013; Blustein, 2011; Richardson, 2012). Consistent with this trend, mattering, which refers to “the individual’s feeling that he or she counts, makes a difference” (Rosenberg, 1985, p. 215), can be useful in understanding the dynamic structure of an individual’s work experience within the social and interpersonal context. However, the construct of mattering has received only minimal investigation in work literature (Jung, 2015).

Mattering, which was originally introduced by social psychologist, Morrison Rosenberg, consists of two dimensions: interpersonal mattering and societal mattering (Rosenberg, 1985). Interpersonal mattering refers to a person’s perception that he or she matters to a specific group of people (Rosenberg & McCullough, 1981), while societal mattering is “the feeling of making a difference in the broader scheme of sociopolitical events—of feeling that one’s thoughts and actions have an impact, create ripples, are felt” (Rosenberg, 1985, p. 215). Rosenberg emphasized interpersonal mattering over societal mattering (e.g., Rosenberg, 1985), and the results of subsequent empirical studies support the association between interpersonal mattering and psychological well-being (e.g., Dixon & Kurpius, 2008; Tovar, Simon, & Lee, 2009). Thus, the mattering literature was established with an almost exclusive focus placed on interpersonal mattering and subsequently lost its appreciation of a broader social contribution and the sociopolitical context that was represented by societal mattering (Jung, 2015).

A few empirical studies investigating mattering at work have been conducted in relation to job satisfaction and stress as well as a relational theory of working. The positive influence of interpersonal mattering on mental health has been demonstrated through a number of empirical studies (e.g., Dixon & Kurpius, 2008; Tovar et al., 2009). However, Amundson (1993) and Connolly and Myers (2003) studied working adults and hypothesized that interpersonal mattering contributes to the individuals’ ability to cope with their work-related stresses. They did not find support for the direct role of interpersonal mattering on indices of job-related well-being, such as job satisfaction. Compared to the robust literature on the positive role of interpersonal mattering in psychological well-being, Jung (2015) argues that the power of interpersonal mattering may not be sufficient to explain job-related experience, but societal mattering may be. In essence, societal mattering may contribute to a person’s psychological well-being in the work context, as the person may be interconnected to the broader society through work (Jung, 2015).

The importance of societal mattering and its influence on psychological well-being at work was highlighted in research on the care work of women (e.g., Schultheiss, 2009). Care work in the domestic area is often not considered legitimate work, so it is often excluded from vocational psychology theories and discussion (Heppner & Jung, 2013). Schultheiss (2009) argued that the economic structure that rarely rewards care work leaves most women with caregiving roles wondering if they want “to mother or matter” (p. 26) in the society. In their qualitative study of the work of full-time mothers, Jung and Heppner (2015) found that full-time mothers neither think their work matters nor are they satisfied with their work and life mainly due to their perceived lack of societal appreciation and acknowledgment for their work. That is, perceived social contribution or value of their work threatens the perceived degree of mattering, which then negatively impacts their psychological well-being (Jung & Heppner, 2015).

Indeed, investigating both societal and interpersonal mattering can expand our understanding of work experiences that exist within relational and social contexts. Vocational literature that has built a long history upon primarily understanding within-individual differences in career and work (e.g., Holland, 1987; Lent, Brown, & Hackett, 1994; Super, 1990) can be strengthened by investigating the influence of the broader sociopolitical context on the work experience of individuals, especially those from previously marginalized groups (e.g., Klehe, Zikic, Van Vianen, & De Pater, 2011; Marfleet & Blustein, 2011). Work mattering as personal interpretation and subjective perception (Jung, 2015) can add greatly to our understanding of understudied work experiences. Therefore, this study aims to develop a measurement tool that includes both interpersonal and societal mattering at work.

In her literature review on mattering, Jung (2015) provided a review and critiques of the psychometric properties and construct formations of four major assessment tools that were developed and applied to the measurement of mattering for adults: the General Mattering Scale (Marcus, 1991), Interpersonal Mattering (Elliott, Kao, & Grant, 2004), School Counselor Mattering Survey (Rayle, 2006), and Mattering to Others (France & Finney, 2009).

Although psychometric rigor varies among them, the review reveals two noticeable conceptual trends (Jung, 2015). One trend is that almost all mattering inventories measure only interpersonal mattering. While they vary regarding the range of the interpersonal context of mattering, specific others or others in general, all scales are exclusively interested in assessing interpersonal mattering. Another trend indicates that the theoretical framework of these scales were solely based on Rosenberg (1985) or Rosenberg and McCullough (1981)’s framework of interpersonal mattering, which have not been updated with new findings from empirical studies (e.g., Elliott et al., 2004; Schlossberg, 1989). Thus, the purpose of this investigation is to develop an inventory to adequately measure interpersonal and societal mattering in the work context and to examine its psychometric properties and utility in predicting psychological well-being and job satisfaction. Results of this research are presented in five phases: (a) item construction, (b) exploratory factor analysis (EFA), (c) confirmatory factor analysis (CFA), (d) construct validity (convergent and discriminant validity), and (e) internal consistency, test–retest reliability estimates.

Phase I—Item Development

Items were generated for an initial scale in four ways: (a) by reviewing the literature on mattering, (b) by initiating discussions and interviews with 10 working individuals, (c) by having graduate students reviewers, and (d) by utilizing the professional review of four vocational psychologists. This procedure is adapted from Crocker and Algina (1986). The literature review addressed a broad range of scholarly work, including journal articles, books, and dissertations by researchers who investigated mattering. This process illuminated two dimensions of mattering at work, societal and interpersonal mattering, and 40 statements that operationalize the construct. Through intensive discussion and clarification with working individuals, graduate students, and vocational psychologists, the 45 items were constructed.

Phase II—Item Analysis, EFA, and Initial Internal Reliability Test

Method

Participants

Data were obtained from 641 working adults who were recruited through four different mechanisms in the United States: word of mouth through a religious affiliation (n = 29, 4.5%); advertisement posted on Craigslist (n = 37, 5.8%); advertisement posted in a weekly bulletin for staff, hospital employees, and faculty at a Midwestern university (n = 249, 38.8%); and Amazon’s Mechanical Turk platform (n = 326, 50.9%). Then, 25 cases (3.9%) were removed due to invalid responses to the validity check items (see subsequently in Procedure subsection for more information). Adopting Byrne (2013)’s recommendation, the remaining 616 participants were divided into two subsamples using a random split procedure in order to avoid sample-specific results, which may potentially influence reliability and validity. Sample A (n = 330) was used for item analysis and EFA in the Phase II. Sample B (n = 286) was used for CFA in the Phase III.

Homogeneity was tested between samples on the demographic variables, and no difference was found: age, t(612) = 1.67, p = .10; gender, χ²(1) = .03, exact p = .93; race, χ²(6) = 4.50 p = .61; working industry, χ²(28) = 34.74, p = .18; working position, χ²(11) = 6.66, p = .83; years in a position, t(612) = 1.93, p = .05; education, χ²(7) = 8.14, p = .32, and marital status, χ²(6) = 4.57, p = .60, demonstrating no differences in the two samples from the random split.

Sample A for Phase II consisted of 299 working adults (84 men, 214 women, and 1 unknown) with ages ranging from 18 to 74 years (M = 40.83, SD = 13.12, Median = 38). Most participants were White/European American (83.6%, n = 250), followed by Asian/Pacific Islander (5.7%, n = 17), African American (5%, n = 15), Hispanic (3%, n = 9), Native American (1%, n = 3), and other (1%, n = 3). Participants represented diverse occupational backgrounds, including 22.1% of participants (n = 66) worked at colleges, universities, and adult education facilities; 21.1% (n = 63) from health care and social assistance fields; 6.7% (n = 20) from finance and insurance; and 4.7% (n = 14) from government and public administration. Participants held diverse positions, such as professional (25.4%, n = 76), administrative staff (29.1%, n = 57), support staff (16.1%, n = 48), middle management (10.7%, n = 32), skilled laborer (6%, n = 18), and upper management (3.3%, n = 10). About 66% of the sample (n = 197) had a college education, followed by master’s degree (17.1%, n = 51), doctoral degree (7.7%, n = 23), high school/general equivalency diploma (GED; 6.4%, n = 19), and professional degree (2.7%, n = 8).

Procedure

Participants completed the initial 45-item Work Mattering Scale (WMS) that was generated in Phase I. Participants were asked to respond to each item with options that ranged from 1(disagree very much) to 6 (agree very much). Participants also completed demographic questionnaire and eight scales, which were used to assess the construct validity in Phase IV. All surveys were conducted through a web-hosted survey page after a participant indicated his or her interests by clicking a button to an informed consent. Participants who were from outside the United States and were not working at the point of participation were excluded through the initial screening items. Three validity check items were included to screen out invalid responses (e.g., “Please simply select/mark [strongly agree] for this item”). Participants who completed the survey through the first three sampling methods were entered into a drawing for 1 of the 15 US$40 gift certificates, while each participant from Amazon completed the survey in exchange for US$0.50 in cash. The estimated time for the survey completion was 15–30 min.

Results

Preliminary Analysis

The original sample A with 330 respondents was examined for normality of distributions, univariate and multivariate outliers, and missing values. First, 11 cases were identified for with missing responses (less than 5% of the total responses). As these missing responses were considered to be missing completely at random (MCAR) using Little’s MCAR test, χ²(308) = 328.33, p = .20, the missing data were replaced using expectation maximization (EM) methods to retain all possible sample size for the EFA (Schlomer, Bauman, & Card, 2010). Second, 19 cases were identified as extreme univariate (z scores bigger than ±3) and were replaced with the mean of the participants’ nonextreme responses. Third, 31 cases were identified and eliminated as multivariate outliers. Therefore, responses from 299 participants were analyzed in the EFA.

Item Analysis

Item analyses were conducted to identify possible items for deletion. First, a decision was made to remove nine negatively worded items (NWI). An increasing number of studies have resulted in the methodological suggestion to not use NWI in inventories for several reasons (see Barnette, 2000; DiStefano & Motl, 2006; Motl & DiStefano, 2002; Roszkowski & Soven, 2010; Weems & Onwuegbuzie, 2001). Second, items with extreme skewness were identified, which indicates that an item may not differentiate responses well. Four items that most respondents reported highly positive responses to were deleted. Third, to avoid redundancy and multicollinearity, the interitem correlations were tested to identify any pairs of items that were too highly correlated (r > .80). Therefore, a total of 17 items were utilized for EFA.

EFA

Factor analyses were conducted in IBM SPSS (version 21) software. The Kaiser–Meyer–Olkin measure of sampling adequacy (.92) and statistically significant Bartlett’s test of sphericity (p < .001) indicated the factorability of the data set for the 17 items. The factor extraction method was principal axis factor analysis. To estimate the probable number of factors to retain, a combination of the scree plot (Cattell, 1966) and parallel analysis (Horn, 1965; O’Connor, 2000) was used. Given that the anticipated factors are theoretically expected to be correlated, an oblique rotation method (i.e., promax) that assumes factors to correlate was chosen to improve the interpretability and utility of the solution (Tabachnick & Fidell, 2001). Finally, three criteria were used to determine the number of factors and items: (a) retaining factors loadings of .71 (50% of overlapping variance) and above (Comrey & Lee, 1992), (b) retaining factors that comprise at least 3 items per factor (Comrey, 1988), and (c) eliminating items with cross-loadings over .30 after rotation.

Regarding the number of factors to retain, parallel analysis indicated a three-component solution with Eigenvalues of 7.79, 1.32, and .95, respectively. The scree plot indicated either a two-factor or a four-factor solution. A comparison of the factor structure and the number of items of a two-factor model to four-factor model was analyzed. Compared to the other models (i.e., a three-factor model and a four-factor model), the two-factor solution produced the cleanest factor structure and also provided theoretical integrity for the model. Thus, the two-factor model was retained. To conform to item selection criteria, 7 items were removed from the solution. To ensure that the deletion of 7 items does not degrade the quality of the scale in terms of the factor structure, factor intercorrelations, item communalities, factor loadings, and crossloadings (Worthington & Whittaker, 2006), a final EFA was conducted. The final EFA indicates a two-factor structure with a similar value of communalities that are higher than .40, meeting all item selection criteria above, except factor loading of 1 item with .67. Its factor intercorrelations were similar (17 items = .66, 10 items = .60). The remaining 10 items, 5 items per factor, accounted for 71.43% of the variance in the total scale (see Table 1).

Table 1.

Items, Component Loading, Communality Estimates, Means, and Standard Deviations of 10 Final Items.

Items	Factor Loadings
Items	F1	F2	h ²	M	SD
Factor 1: Societal mattering
I think that society values the work I do	.864	−.056	.69	3.97	1.38
I feel my work meets a societal need	.851	−.041	.69	4.45	1.38
I am connected to society through my work	.812	.047	.71	4.25	1.46
People say that my work influenced their life	.786	.066	.69	3.68	1.63
My work influences people’s lives	.737	.066	.61	4.62	1.34
Factor 2: Interpersonal mattering
My coworkers/colleagues would be disappointed if they knew that I may leave my job	.007	.859	.67	4.42	1.46
I feel like I matter to my colleagues/coworkers	−.036	.837	.75	4.47	1.34
My coworkers/colleagues value my ideas and suggestions	.046	.767	.63	4.59	1.18
My boss/supervisor would be disappointed if they knew that I may leave my job	.033	.727	.47	4.46	1.57
My coworkers/colleagues appreciate my support and help	.014	.677	.56	4.79	.98

Note. h ² = communality estimates.

Factor 1 was named “Societal Mattering” because the items reflect the individual’s perception of their work contribution, value, and connection to the society (5 items, accounting for 55.84% of the variance, M = 20.97, SD = 6.18, range = 5–30). Highest loading items were “I think that society values the work I do” and “I feel that my work meets a societal need.” A high score indicates a feeling of more societal mattering.

Factor 2 was named “Interpersonal Mattering” because all items reflect the individual’s experience with people with whom they are working (i.e., supervisor/boss and coworkers) that indicate that the person is valued, recognized, and appreciated (5 items, accounting for 15.59% of the variance, M = 22.74, SD = 5.43, range = 7–30). Highest loading items were “I feel like I matter to my colleagues/coworkers” and “My coworkers would be disappointed if they knew that I may leave my job.” A high score indicates a feeling of more interpersonal mattering.

The interfactor correlation coefficient between factors was .60. The internal reliability coefficient α were .91 for the total score, .91 for the Societal Mattering subscale, and .88 for the Interpersonal Mattering subscale.

Phase III—CFA

Method

Participants and Procedure

Final sample B for Phase III consisted of 251 working adults (71 men, 179 women, and 1 unknown) with ages ranging from 18 to 67 years (M = 39.39, SD = 13.42, Median = 36). Most participants were White/European American (86.5%, n = 217), followed by African American (5.6%, n = 14), Asian/Pacific Islander (4%, n = 10), Hispanic (2%, n = 5), Native American (0.4%, n = 1), and other (1.2%, n = 3). Participants represented diverse occupational backgrounds, including 21.1% of participants (n = 53) worked at college, university, and adult education; 19.5% (n = 49) from health care and social assistance fields; 5.6% (n = 14) from retail; and 4.8% (n = 12) from art, entertainment, and recreation. Participants held diverse positions, such as trained professional (23.9%, n = 60), administrative staff (18.7%, n = 47), support staff (18.7%, n = 47), middle management (8%, n = 20), skilled laborer (7.2%, n = 18), and junior management (4.8%, n = 12). About 64% of the sample (n = 161) had a college education, followed by master’s degree (19.9%, n = 50), professional degree (4.4%, n = 11), high school/GED (7.6%, n = 19), and doctoral degree (3.2%, n = 8).

Results

CFA

The original sample B with 286 respondents was examined for the normality of distributions, univariate and multivariate outliers, and missing values. First, eight cases were identified for missing responses (less than 5% of the total responses) and replaced using EM methods. Second, 14 cases were identified as extreme univariate (z scores bigger than ± 3) and were replaced with the mean of the participants’ nonextreme responses. Third, 20 cases were identified and eliminated as multivariate outliers. Fourth, 15 cases that were from self-employed individuals were removed. As a result, responses from 251 participants were analyzed in the CFA.

CFA (IBM AMOS version 21; Arbuckle, 2006) with maximum likelihood estimation was performed to test the stability of the constructs in the 10-item WMS with a different sample. Model fit was assessed using the χ² value and the associated degree of freedom, nonnormed fit index (NNFI), comparative fit index (CFI), standardized root mean square residual (SRMR), root mean square error of approximation residual (RMSEA), and RMSEA 90% confidence interval (CI), as recommended by Hu and Bentler (1998, 1999). The χ², CFI, and NNFI indices compare the specified model to the independent model: A nonsignificant χ², CFI, and NNFI values bigger than .95 reflect a desirable model fit. SRMR compares the observed covariance matrix to the estimated one: A value less than .05 is considered a good fit. The RMSEA estimates error due to the approximate fit of the model: Values less than .08 reflect a good fit. Because the χ² statistic is sensitive to sample size, assessing a ratio between χ² and the degree of freedom is also recommended as a measure of model fit. Thus, a χ² value 2–3 times greater than the degrees of freedom is considered acceptable (Hair, Black, Babin, & Anderson, 2010; Hu & Bentler, 1999).

Two models were tested: (a) a two-factor model demonstrating the two factors identified in Phase II and (b) a single-factor model in which all 10 items were allowed to load on a single latent variable. The model fit for the first model were χ² = 84.67, df = 34, p < .001, χ²/df = 2.49, CFI = .96, NNFI = .95, SRMR = .04, RMSEA = .08, and RMSEA 90% CI = [.057, .098]. The model is shown in Figure 1. In addition, all factor loadings were significant at the p < .001 level, ranging from .57 to .89 (standardized). The range of squared multiple correlations of items was from .53 to 76 for societal mattering and from 33 to 79 for interpersonal mattering. The societal mattering factor accounts for 55% of the variance in the 5 items while the interpersonal mattering factor explains 63% of the variance in the remaining 5 items. This suggests good convergent validity in that each factor well explains items that are loaded with each factor. Finally, the intercorrelation among the two factors was .66, which is very similar to that found in Phase II.

Figure 1.

The two-factor model of the Work Mattering Scale. SC_M = societal mattering; IP_M = interpersonal mattering.

To examine the possibility of a unidimensional scale, rather than a two-factor model, the second model where all 10 items load onto a single factor was tested. The model fit for the first model was χ² = 342.00, df = 35, p < .001, χ²/df = 9.77, CFI = .78, NNFI = .72, SRMR = .10, RMSEA = .19, and RMSEA 90% CI = [0.170, 0.206]. In addition, the results of the χ² difference test were $χ_{D}^{2}$ (1) = 257.33, p < .001, indicating that the fit of the two-factor model is statistically better than that of the single-factor model. In essence, the results supported a two-factor model over a one-factor model.

Phase IV—Construct Validity

This phase aimed to assess the initial construct validity of the scale by calculating the correlations between the scores of the WMS and measures of other conceptually relevant and distinct constructs. Specifically, convergent validity was examined by correlating scores from the WMS with scores from work meaning, job satisfaction, life satisfaction, organizational commitment, and withdrawal intentions from work. Discriminant validity of the WMS was also assessed by exploring the correlations with positive and negative affect and factor analysis.

First, work mattering was expected to be positively associated with the work meaning construct, as meaning and mattering are both considered as relational outcomes in the relational theory of working (Blustein, 2011). So far, existing literature investigating the relationship between these two constructs is sparse. However, in particular, it was expected there would be higher correlations between the Societal Mattering subscale and the Greater Good Motivations Work Meaning subscale because both measure the degree to which the individual has a sense of connection to the broader world (Steger, Dik, & Duffy, 2012).

Second, job satisfaction and life satisfaction were expected to be positively associated with work mattering, as previous studies have found positive relationships among interpersonal mattering and job satisfaction and wellness (e.g., Connolly & Myers, 2003; Rayle, 2006). Similar to Steger, Dik, and Duffy’s (2012) findings regarding work meaning, correlations with job satisfaction were expected to be higher than ones with the broader construct of life satisfaction, as the measurement was of the perceived sense of mattering specific to the work setting.

Third, work mattering was expected to be positively associated with organizational commitment and to be negatively correlated with withdrawal intentions from work. Positive perceptions of their work and being valued among people working with them (i.e., interpersonal mattering) were expected to result in positive affect for commitment to the organization (Rayle, 2006). In addition, negative perceptions of the value and contribution of their work may cause them to consider leaving or influence their actual behaviors at work (Jung & Heppner, 2015).

Fourth, work mattering was expected to have small or no correlation with negative affect, as the two constructs do not have any theoretical relevance to each other. Clark and Watson (1995) raised the critique that many scales do not clearly demonstrate discriminant validity, as they often fail to differentiate the construct they are attempting to measure from negative affectivity. Unlike negative affect, small to moderate positive correlations were expected between positive affect and work mattering. Individuals who feel like they matter through their work would be more likely to be people with positive affect. However, work mattering should be conceptually distinct from positive affect: Thus, similar to Clark and Watson (1995), work mattering was expected to be differentiable from positive affect.

Method

Participants

The assessment of construct validity was conducted using the samples from Phase I and II. Data screening left 589 participants in the sample for Phase IV. For organizational commitment, only 553 participants were used for the analysis (i.e., excluded self-employee).

Measures

The following inventories were used to assess construct validity of the WMS.

Work meaning was assessed with a 10-item Work and Meaning Inventory (WAMI; see Steger et al., 2012). The WAMI is composed of three subscales: Positive Meaning in Work, Meaning Making through Work, and Greater Good Motivations. Using a 5-point Likert-type scale (1 = absolutely untrue and 5 = absolutely true), an example item is “I have found a meaningful career.” In the original study, the information regarding convergent and predictive validity was provided and the internal consistency for the total scores was .93. The internal reliability of the total scores with the present sample was .95.

Job satisfaction was measured with 5 items from Judge, Locke, Durham, and Kluger (1998), adopted from the original Brayfield–Rothe job satisfaction index (Brayfield & Rothe, 1951). Example items include “I find real enjoyment in my work” and “I feel fairly well satisfied with my present job.” Concurrent validity of the scale was found with the job descriptive index (Smith, Kendall, & Hulin, 1969). Items are rated from 1 (strongly disagree) to 7 (strongly agree). The internal consistency reliability was .88. The internal reliability for the present sample was .91.

Life satisfaction was assessed with a 5-item Satisfaction with Life Scale (SWLS; see Diener, Emmons, Larsen, & Griffin, 1985) that measures global life satisfaction (e.g., “I am satisfied with my life”). Items are rated from 1 (strongly disagree) to 7 (strongly agree). In the original study, the SWLS scores were correlated to general well-being variables (e.g., self-esteem) as well as an interviewer’s rating of a person’s life satisfaction. The internal consistency was .81 for elder adults. The internal reliability in this study sample was .92.

Organizational commitment was measured with 8 items from the scale that Allen and Meyer (1990) developed. A sample item includes “I would be very happy to spend the rest of my career with this organization.” Items are rated from 1 (strongly disagree) to 7 (strongly agree). In the original study, canonical correlation analysis revealed that higher scores were correlated to employees’ comfort in their roles and level of competence in the job. The internal consistency of the scale was .87 for working adults and .90 for the present sample.

Positive and negative affectivity was assessed with International Positive and Negative Affect Schedule–Short Form (I-PANAS-SF; Thompson, 2007), a 10-item scale that is the short form of the original 20-item PANAS (Watson, Clark, & Tellegen, 1988). Using a 5-point Likert-type scale (1 = never, 5 = always) in response to trait descriptive adjectives (i.e., upset and inspired), the internal consistency for the scores was .78 and .76, respectively, and test–retest reliability was .84. Convergent, predictive, and cross-cultural validity were tested and supported with other existing scales, predictors, and samples across 16 countries. The internal reliability for the present sample was .79 for negative affect and .82 for positive affect.

Withdrawal intentions were measured by 2 items from Blau (2000), intent to leave one’s organization (i.e., “I intend to leave this job/position as soon as possible”) and intent to leave one’s profession (i.e., “I intend to leave the profession as soon as possible”). Items are rated from 1 (very unlikely) to 5 (very likely). The internal reliability with the present sample was .83.

Analysis

Convergent and discriminant validity of the WMS was assessed through Pearson’s correlations with existing scales and age as well as Spearman’s correlations with gender. Factor analysis with principal axis factoring and promax rotation was performed to examine discriminant validity of the WMS with the I-PANAS-SF.

Results

All correlations were in the conceptually expected directions, as reported in Table 2. The WMS total and subscale scores indicated moderate to strong associations with work meaning, job satisfaction, life satisfaction, organizational commitment, withdrawal intentions from work, and positive affect. As expected, the WMS total and subscale scores indicated no or weak associations with negative affect, age, and gender. The results of factor analysis demonstrated that the subscales of the WMS were clearly differentiable from positive and negative affectivity.

Table 2.

Summary Data and Bivariate Correlations for the Work Mattering, Work Meaning, Life Satisfaction, Work Satisfaction, Organizational Commitment, Withdrawal Intentions, Positive and Negative Affect, Age, and Gender.

Variable	M	SD	2	3	4	5	6	7	8	9	10	11	12	13	14	15
1. Work mattering (WM): total	43.77	9.70	.90**	.86**	.80**	.77**	.71**	.77**	.48**	.72**	.68**	−.57**	.46**	−.16**	.10*	.09*
2. WM: societal	20.91	5.93	—	.56**	.81**	.76**	.71**	.81**	.42**	.62**	.56**	−.49**	.44**	−.11*	.12**	.06
3. WM: interpersonal	22.86	5.06		—	.59**	.59**	.53**	.52**	.42**	.66**	.65**	−.52**	.37**	−.18**	.05	.11**
4. Work meaning (WM: total)	36.46	9.67			—	.96**	.92**	.91**	.55**	.77**	.67**	−.66**	.46**	−.15**	.09*	.08
5. WM: positive meaning	14.80	4.10				—	.85**	.80**	.56**	.78**	.67**	−.68**	.46**	−.18**	.13**	.07
6. WM: meaning making	10.68	3.13					—	.74**	.52**	.68**	.61**	−.58**	.40**	−.10*	.06	.07
7. WM: greater good motivations	10.98	3.15						—	.44**	.67**	.57**	−.55**	.41**	−.12**	.09*	.08*
8. Life satisfaction	22.79	7.41							—	.55**	.45**	−.42**	.36**	−.30**	.03	.04
9. Work satisfaction	4.87	1.50								—	.78**	−.73**	.49**	−.29**	.13**	.08
10. Organizational commitment^a	4.26	1.46									—	−.71**	.36**	−.19**	.05	.09*
11. Withdrawal intentions	2.25	1.16										—	−.29**	.20**	−.12**	−.08
12. Positive affect	19.10	3.11											—	−.30**	.13**	.00
13. Negative affect	10.81	3.08												—	−.10*	.18**
14. Age	39.86	13.28													—	.09*
15. Gender^b																—

Note. N = 589.

^a n = 553. ^bSpearman’s correlations used for gender analyses.

*p < .05, **p < .01.

More specifically, large positive correlations between the total score of the WMS and work meaning were found (range = .52 to .81, p < .001). Congruent with the similarity between the constructs, correlations between the work meaning scores and ones of the societal mattering subscale (range = .71 to .81, p < .001) are larger than ones with the Interpersonal Mattering subscale scores (range = .52 to 59, p < .001). As hypothesized, high correlations between the Societal Mattering subscale and the Greater Good Motivations Work Meaning subscale were found, r = .81, p < .001.

Second, job satisfaction and life satisfaction were positively associated with the work mattering scores. In particular, there were stronger correlations between work mattering and job satisfaction (range = .62 to .72, p < .001) than ones with life satisfaction (range = .42 to .48, p < .001), which supports the hypothesis that the WMS measures work-specific perceptions.

Third, work mattering scores were positively associated with organizational commitment (range = .56 to .68, p < .001) and negatively correlated with withdrawal intentions from work (range = −.49 to −.57, p < .001). It reflects the idea that individuals would be emotionally engaged in and attached to the organization where they work and less likely to leave when they feel their work is valued and that they are contributing to the society.

Finally, the work mattering scores had no or small correlations with conceptually distinct constructs such as negative affect (range = −.18 to −.11, p < .001), age (range = non-significance to .12), and gender (range = non-significance to .10), which demonstrates discriminant validity. As hypothesized, moderate positive correlations were found between positive affect and work mattering (range = .37 to .46, p < .001). In addition, the results of the factor analysis indicated that there were four conceptually distinct factors that represent societal mattering, interpersonal mattering, positive affect, and negative affect, respectively. In essence, demonstrating the separate variance accounted for by the work mattering variables and those of positive and negative affectivity (see Table 3).

Table 3.

Items and Component Loadings of the Work Mattering Scale and Positive and Negative Affect.

	Factor Loadings
Items	F1	F2	F3	F4
I think that society values the work I do	.854	−.021	−.030	.013
I feel my work meets a societal need	.783	−.037	.013	−.088
I am connected to society through my work	.776	.044	−.016	.037
People say that my work influenced their life	.748	.001	.040	.047
My work influences people’s lives	.747	.070	.015	−.015
My coworkers/colleagues would be disappointed if they knew that I may leave my job	−.031	.867	−.065	.006
I feel like I matter to my colleagues/coworkers	.001	.843	.028	−.015
My coworkers/colleagues value my ideas and suggestions	.085	.731	.005	−.020
My boss/supervisor would be disappointed if they knew that I may leave my job	.009	.713	.066	.066
My coworkers/colleagues appreciate my support and help	−.003	.655	−.037	−.036
Attentive	−.046	−.048	.810	−.026
Determined	−.044	.035	.781	−.010
Alert	.033	−.043	.663	.067
Active	.024	−.021	.641	−.051
Inspired	.094	.104	.565	.006
Afraid	.037	−.056	.085	.801
Nervous	−.057	.030	.114	.703
Upset	−.043	.048	−.084	.636
Ashamed	.054	−.002	−.050	.624
Hostile	.007	−.020	−.136	.539

Note. h ² = communality estimates.

Phase V—Test–Retest Reliability Estimates

Method

Participants and Procedure

Test–retest reliability was assessed using the sample from a religious affiliation, Craigslist, and a Midwestern university. Participants were asked about their willingness to complete the test twice at the end of their first-time survey. Four weeks after the first-time completion, individuals who signed the consent form completed the scale for the second time. Four weeks was chosen both for practical reasons of avoiding attrition and as it seemed a potentially useful time frame should researchers seek to use the instrument in a pre-intervention-post type study. The e-mail was sent to 249 participants and there were 169 respondents indicating a response rate of 68%.

Results

To obtain the test–retest reliability, Pearson correlation coefficient was calculated. Table 4 provides the means and standard deviations for the test–retest data. The test–retest coefficient of reliability for the total WMS was .87 (p < .001), suggesting acceptable medium-run temporal stability. The test–retest coefficient of reliability for both the Societal Mattering and Interpersonal Mattering subscales turned out .88 and .85, respectively (p < .001), suggesting acceptable medium-run temporal stability.

Table 4.

Test–Retest Means and Standard Deviations for Work Mattering Subscales and Total.

Measure	M	SD
Test
Societal mattering	21.65	5.35
Interpersonal mattering	23.26	5.36
Work Mattering Scale total	44.92	8.96
Retest
Societal mattering	21.60	5.22
Interpersonal mattering	22.98	5.20
Work Mattering Scale total	44.58	8.76

Note. N = 169. Retest data were collected approximately 4 weeks after the initial testing.

Discussion

The purpose of this study was to develop a scale to quantitatively and effectively measure the construct of work mattering. On the basis of the literature review, work mattering was conceptualized with two main domains, that is, societal mattering and interpersonal mattering. This two-factor model of work mattering was supported by the results of the exploratory and CFA with a 10-item two-factor scale. The factor loadings for the two factors ranged from .68 to .86 (see Table 1). Each factor was well explained by the loaded items with more than 50% of the variance accounted for. In addition, α coefficients for the two factors were above .85. Construct validity was also supported by the correlations between the WMS and the other psychometrically sound scales that measure psychological well-being and work experience/behaviors. Moreover, the results of correlations and factor analysis between the WMS and Positive and Negative Affect Scale clearly demonstrate that work mattering is not mere positive affectivity and is distinctive from both positive and negative affects. Finally, results from the test–retest reliability investigation indicated acceptable temporal stability of the WMS. In sum, the initial results demonstrate a psychometrically sound WMS.

The two factors of the WMS closely replicate the theoretical constructs of mattering that was originally proposed by Rosenberg (1985). Rosenberg is the first who suggested that the concept of mattering consists of two components: societal mattering and interpersonal mattering. Although recent vocational studies have progressively highlighted and discussed the significance of relationships and societal contexts in work experiences (Blustein, 2011; Cook et al., 2002; Schultheiss, 2007), work mattering with two domains has not been frequently studied, despite its potential to address both relationships and societal contexts very effectively. The two factors revealed in this study not only support the existence of the societal aspect of work mattering but also elaborate on the construct of societal mattering that has not been conceptualized or studied in the existing literature. Consistent to what Rosenberg (1985) depicted societal mattering as “feeling that one’s thoughts and actions have an impact, create ripples” (p.215), the societal aspect of work mattering in the scale encompasses one’s perceived value of work as well as a sense of contribution through work.

The WMS is the first empirical measure to provide support for the theoretical construct of work mattering with two dimensions of work mattering. Influenced by Rosenberg’s emphasis on interpersonal mattering and its usefulness in explaining psychological well-being, societal mattering has rarely been studied, while in contrast, several assessments have been designed to measure the construct of interpersonal mattering (e.g., Elliott et al., 2004; Marshall, 2001; Marshall & Lambert, 2006). In vocational studies investigating work mattering, researchers created scales designed to measure interpersonal mattering only (e.g., Connolly & Myers, 2003; Rayle, 2006). The existing studies that only investigated interpersonal mattering indicated that job satisfaction was not explained by interpersonal mattering. However, recent research on the care work of full-time mothers (Jung & Heppner, 2015; Schultheiss, 2009) indicated that perceived value and recognition of work in the society has strong impact on one’s work experience and ultimately one’s job satisfaction. Thus, being able to measure both of these dimensions of mattering with one short scale is a unique contribution to vocational psychology. The WMS also provides a versatile measure for interpersonal mattering at work that is not specific to a particular field (e.g., guidance counselors). Several limitations should be noted. Although the other three fit indices indicated good fits, the RMSEA of the two-factor model indicated a borderline fit. Kline (2011) argues that RMSEA may “impose a harsher penalty for complexity on smaller models with relatively few variables or factors” (p. 207), which may account for the lower than desirable fit in the current study. In addition, although there was diversity in the sample geographically, occupationally, and in age and gender, the homogeneity of the sample in terms of race/ethnicity and education limits the generalizability of the factor structure. Work mattering may vary, as individuals from any minority status and/or with less resources may receive more societal oppression and discrimination while performing their work.

Implications for Research and Practice

This study will provide researchers and counselors with a useful tool for investigating the concept of mattering and consequently to facilitate expansion of this body of literature. Measuring work mattering is particularly important, as many studies have shown the strong impact of social, cultural, and relational contexts on an individual’s work experience (Ali et al., 2013; Richardson, 2012). A psychometrically sound measure of this construct will allow researchers and counselors to assess relational and societal aspects of the work experience.

The WMS will be helpful when designing studies, as exploration of the construct has been hindered by the lack of an adequate measurement tool. It is hoped that the WMS will propel further research and theoretical conceptualization on work mattering in vocational studies. Particularly, this 10-item WMS will be a convenient and practical tool with other measures in future vocational studies. As discussed in the limitations of the study, further research should study the factor structure and psychometric properties of the WMS with more diverse samples in terms of racial/ethnic diversity, education, and social class. In addition, the scale could be examined in relation to other measures of work behaviors or affect. For example, investigating relationships between work meaning and mattering, as they predict work satisfaction is greatly needed. Previous qualitative studies of work mattering argued that persons with a strong sense of meaningful work do not necessarily feel a strong sense of mattering from their work, which also influences their work satisfaction (e.g., Jung & Heppner, 2015). The current line of research could be expanded by investigating relationships between work mattering and stereotypes and biases toward a certain type of job or occupation. It is hypothesized that an individual whose job is undervalued or stereotyped is less likely to report a strong sense of social mattering, as mattering is a self-perception within the context not a self-creating belief like work meaning. In addition, although previous research has not found a direct link between interpersonal mattering and job satisfaction, our study indicated a .66 correlation between the two. This is a conceptually important finding and one that will need further study to confirm with future samples.

In clinical settings, the WMS can provide useful information to clients. The scale could help identify feelings or struggles that the client might be experiencing at work. Results from this study with the associated scales measuring work and psychological well-being can also be useful to counselors who investigate relevant work experiences to work mattering. As previous studies of work mattering argued (Jung & Heppner, 2015; Connolly & Myers, 2003; Rayle, 2006; Schultheiss, 2009), exploring a client’s work experience with regard to their level of perceived meaning, interpersonal mattering, and societal mattering will deepen the understanding of how the client is experiencing the work setting.

For example, using the results of the WMS, the counselor can also facilitate the clients’ processing of their work experience to reflect on how their scores influence their feelings about themselves and their work. For example, if a client scores low on the Interpersonal Mattering subscale, they are indicating that their boss or coworkers do not value them at work. They are endorsing items that indicate their coworkers would not be disappointed if they left their job, that their ideas are not valued at work, and that the support and help they provide to coworkers is not appreciated. Endorsement of these items depicts a person who may be feeling isolated, disaffected, or lonely at work. They may need to find other ways of feeling supported or may need to look for a different work setting. Being treated as though one does not matter may also be the result of bias and discrimination due to age, race, gender, and sexual orientation and thus is a signal of an unhealthy work environment.

In addition, if a client scores low on societal mattering, they are endorsing items that indicate they feel that the society does not value their work, that their work does not meet a societal need, that they do not feel connected to the society through their work, and that their work really does not impact people’s lives. Depending on how salient it is for the person to feel a sense of societal mattering, low scores in this area can again indicate an alienation or estrangement from their work, a feeling that they are not making a difference in the world, or a contribution to the lives of others. If these are salient issues to the client, it may signal a need to find societal mattering from volunteer or other vocational pursuits, a need for cognitive reframing of their work in order to highlight its societal value or even a career change.

The WMS can also be used in other venues outside of clinical or research setting. On the organizational level, the scale can be used for diagnostic or descriptive purposes to assess the overall sense of mattering at work among employees. To enhance employee’s job satisfaction and their retention rate, employers or organizations may utilize the scale to evaluate the current status of mattering among employees and to develop interventions to increase mattering. Especially, interpersonal mattering can provide useful and specific information with which to develop goals for strategic planning. The societal mattering subscale can also help managers articulate the ways their organization contributes to the society as a whole and to communicate this broader view of the importance of work to their employees. In addition, the scale can be utilized for evaluation of a program or project and testing any changes over time after implementing work mattering interventions.

Footnotes

Authors’ Note

Ae-Kyung Jung is currently affiliated with Gyeongin National University of Education, Gyeyang-gu, Incheon, South Korea.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was part of the first author’s doctoral dissertation and was supported by Dr. Mary Jane Lang Doctoral Scholarship Fund.

References

Ali

S. R.

Fall

Hoffman

(2013). Life without work: Understanding social class changes and unemployment through theoretical integration. Journal of Career Assessment, 21, 111–126.

Allen

N. J.

Meyer

J. P.

(1990). The measurement and antecedents of affective, continuance and normative commitment to the organization. Journal of Occupational Psychology, 63, 1–18.

Amundson

N. E.

(1993). Mattering: A foundation for employment counseling and training. Journal of Employment Counseling, 30, 146–152.

Arbuckle

J. L.

(2006). Amos (version 7.0) [computer program]. Chicago, IL: SPSS.

Barnette

J. J.

(2000). Effects of stem and Likert response option reversals on survey internal consistency: If you feel the need, there is a better alternative to using those negatively worded stems. Educational and Psychological Measurement, 60, 361–370.

Blau

(2000). Job, organizational, and professional context antecedents as predictors of intent for interrole work transitions. Journal of Vocational Behavior, 56, 330–345. doi:https://dx-doi-org.web.bisu.edu.cn/10.1006/jvbe.1999.1718

Blustein

D. L.

(2011). A relational theory of working. Journal of Vocational Behavior, 79, 1–17.

Brayfield

A. H.

Rothe

H. F.

(1951). An index of job satisfaction. Journal of Applied Psychology, 35, 307.

Byrne

B. M.

(2013). Structural equation modeling with AMOS: Basic concepts, applications, and programming. New York, NY: Routledge.

10.

Cattell

R. B.

(1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245–276.

11.

Clark

L. A.

Watson

(1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309.

12.

Comrey

A. L.

(1988). Factor-analytic methods of scale development in personality and clinical psychology. Journal of Consulting and Clinical Psychology, 56, 754.

13.

Comrey

Lee

(1992). A first course in factor analysis. Hillsdale, NJ: Lawrence Erlbaum Associates.

14.

Connolly

K. M.

Myers

J. E.

(2003). Wellness and mattering: The role of holistic factors in job satisfaction. Journal of Employment Counseling, 40, 152–160.

15.

Cook

E. P.

Heppner

M. J.

O’Brien

K. M.

(2002). Career development of women of color and white women: Assumptions, conceptualization, and interventions from an ecological perspective. The Career Development Quarterly, 50, 291–305.

16.

Crocker

Algina

(1986). Introduction to classical and modern test theory. New York, NY: ERIC.

17.

Diener

Emmons

R. A.

Larsen

R. J.

Griffin

(1985). The satisfaction with life scale. Journal of Personality Assessment, 49, 71–75.

18.

DiStefano

Motl

(2006). Further investigating method effects associated with negatively worded items on self-report surveys. Structural Equation Modeling, 13, 440–464.

19.

Dixon

S. K.

Kurpius

S. E. R.

(2008). Depression and college stress among university undergraduates: Do mattering and self-esteem make a difference? Journal of College Student Development, 49, 412–424.

20.

Elliott

G. C.

Kao

Grant

A. M.

(2004). Mattering: Empirical validation of a social-psychological concept. Self and Identity, 3, 339–354.

21.

France

M. K.

Finney

S. J.

(2009). What matters in the measurement of mattering? A construct validity study. Measurement and Evaluation in Counseling and Development, 42, 104–120. doi:10.1177/0748175609336863

22.

Hair

J. F.

Black

W. C.

Babin

B. J.

Anderson

R. E.

(2010). Multivariate data analysis: A global perspective (7th ed.). Upper Saddle River, NJ: Prentice Hall.

23.

Heppner

Jung

(2013). Gender and social class: Powerful predictors of a life journey. In Walsh

W. B.

Savickas

Hartung

P. J.

(Eds.), Handbook of vocational psychology (4th ed., pp. 81–102). New York, NY: Routledge.

24.

Holland

J. L.

(1987). Current status of Holland’s theory of careers: Another perspective. The Career Development Quarterly, 36, 24–30.

25.

Horn

J. L.

(1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179–185.

26.

Bentler

P. M.

(1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424.

27.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55.

28.

Judge

T. A.

Locke

E. A.

Durham

C. C.

Kluger

A. N.

(1998). Dispositional effects on job and life satisfaction: The role of core evaluations. Journal of Applied Psychology, 83, 17.

29.

Jung

(2015). Interpersonal and societal mattering in work: A review and critique. Career Development Quarterly, 63, 194–208.

30.

Jung

Heppner

M. J.

(2015). Work of fulltime mothers: Putting voice to a relational theory of work. Career Development Quarterly, 63, 253–267.

31.

Klehe

Zikic

Van Vianen

De Pater

(2011). Career adaptability, turnover and loyalty during organizational downsizing. Journal of Vocational Behavior, 79, 217–229.

32.

Kline

R. B.

(2011). Principles and practice of structural equation modeling. New York, NY: Guilford Press.

33.

Lent

R. W.

Brown

S. D.

Hackett

(1994). Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of Vocational Behavior, 45, 79–122. doi:10.1006/jvbe.1994.1027

34.

Marcus

F. M.

(1991). Mattering: Its measurement and theoretical significance (Unpublished manuscript).

35.

Marfleet

Blustein

D. L.

(2011). ‘Needed not wanted’: An interdisciplinary examination of the work-related challenges faced by irregular migrants. Journal of Vocational Behavior, 78, 381–389.

36.

Marshall

S. K.

(2001). Do I matter? Construct validation of adolescents’ perceived mattering to parents and friends. Journal of Adolescence, 24, 473–490.

37.

Marshall

S. K.

Lambert

J. D.

(2006). Parental mattering: A qualitative inquiry into the tendency to evaluate the self as significant to one’s children. Journal of Family Issues, 27, 1561–1582. doi:10.1177/0192513x06290039

38.

Motl

R. W.

DiStefano

(2002). Longitudinal invariance of self-esteem and method effects associated with negatively worded items. Structural Equation Modeling, 9, 562–578.

39.

O’Connor

B. P.

(2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32, 396–402.

40.

Rayle

A. D.

(2006). Do school counselors matter? Mattering as a moderator between job stress and job satisfaction. Professional School Counseling, 9, 206–215.

41.

Richardson

M. S.

(2012). Counseling for work and relationship. The Counseling Psychologist, 40, 190–242.

42.

Rosenberg

(1985). Self-concept and psychological well-being in adolescence. In Leahy

(Ed.), The development of the self (pp. 205–246). New York, NY: Academic Press.

43.

Rosenberg

McCullough

B. C.

(1981). Mattering: Inferred significance and mental health among adolescents. Research in Community & Mental Health, 2, 163–182.

44.

Roszkowski

M. J.

Soven

(2010). Shifting gears: Consequences of including two negatively worded items in the middle of a positively worded questionnaire. Assessment & Evaluation in Higher Education, 35, 113–130.

45.

Schlomer

G. L.

Bauman

Card

N. A.

(2010). Best practices for missing data management in counseling psychology. Journal of Counseling Psychology, 57, 1–10.

46.

Schlossberg

N. K.

(1989). Marginality and mattering: Key issues in building community. New Directions for Student Services, 1989, 5–15.

47.

Schultheiss

D. E. P.

(2007). The emergence of a relational cultural paradigm for vocational psychology. International Journal for Educational and Vocational Guidance, 7, 191–201.

48.

Schultheiss

D. E. P.

(2009). To mother or matter can women do both? Journal of Career Development, 36, 25–48.

49.

Smith

P. C.

Kendall

Hulin

C. L.

(1969). The measurement of satisfaction in work and retirement. Chicago, IL: Rand McNally.

50.

Steger

M. F.

Dik

B. J.

Duffy

R. D.

(2012). Measuring meaningful work the Work and Meaning Inventory (WAMI). Journal of Career Assessment, 20, 322–337.

51.

Super

D. E.

(1990). A life-span, life-space approach to career development. In Brown

Brooks

(Eds.), Career choice and development: Applying contemporary theories to practice (2nd ed., pp. 197–261). San Francisco, CA: Jossey-Bass.

52.

Tabachnick

Fidell

(2001). Principal components and factor analysis. In Tabachnick

Fidell

(Eds.), Using multivariate statistics (pp. 582–652). Boston, MA: Allyn & Bacon.

53.

Thompson

E. R.

(2007). Development and validation of an internationally reliable short-form of the positive and negative affect schedule (PANAS). Journal of Cross-Cultural Psychology, 38, 227–242. doi:10.1177/0022022106297301

54.

Tovar

Simon

M. A.

Lee

H. B.

(2009). Development and validation of the college mattering inventory with diverse Urban college students. Measurement and Evaluation in Counseling and Development, 42, 154–178.

55.

Watson

Clark

L. A.

Tellegen

(1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063–1070. doi:10.1037/0022-3514.54.6.1063

56.

Weems

G. H.

Onwuegbuzie

A. J.

(2001). The impact of midpoint responses and reverse coding on survey data. Measurement and Evaluation in Counseling and Development, 34, 166–176.

57.

Worthington

R. L.

Whittaker

T. A.

(2006). Scale development research a content analysis and recommendations for best practices. The Counseling Psychologist, 34, 806–838.