Abstract
Purpose:
The need for psychometrically sound measurement approaches to social work educational outcomes assessment is increasing.
Method:
The research reported here describes an original and two replication studies of a new scale (N = 550) designed to assess an individual’s self-efficacy regarding social work competencies specified by the Council on Social Work Education as part of the accreditation of social work programs.
Results:
This new measure, the Self-Efficacy Regarding Social Work Competencies Scale (SERSWCS), generally performed in line with our expectations.
Discussion:
The SERSWCS is a measure that is based on substantial theoretical and empirical work, has preliminary evidence regarding the psychometric properties of the data it produces, can be used with large numbers of students in an efficient manner, is neither expensive or subject to user restrictions, and provides views of outcomes that have utility for pedagogical considerations at multiple curricular levels.
Keywords
Demands for greater accountability at all levels of education echo across the landscape. They follow nearly a century of calls for reform in primary and secondary education (Ravitch, 2000). The “foundation of the accreditation or professional judgment approach to evaluation can be traced directly to the establishment of the North Central Association of Colleges and Secondary Schools in the late 1800s. The accreditation movement did not, however, gain great stature until the 1930s” (Madaus & Stufflebeam, 2000, p. 6). Currently, accreditation by the Council on Social Work Education (CSWE) is a dominant feature in higher education in social work in the United States. The purpose of this article is to explore an approach that could be used to produce social work higher education outcome assessments related to current accreditation-related accountability demands.
Competencies
An advocate of competency-based education noted that various definitions of competencies exist but they are essentially “the result of integrative learning experiences in which skills, abilities, and knowledge interact to form learning bundles that have currency in relation to the task for which they are assembled” (Voorhees, 2001, p. 9). In this definition, performance is at the top of a hierarchy that is the result of the application of competencies. Underlying competencies are various skills, abilities, and knowledge (e.g., Jones, Voorhees, & Paulson, 2002).
The CSWE has adopted elements of competency-based education in framing social work educational outcomes as competencies. CSWE states in their current Educational Policy and Accreditation Standards: Competency-based education is an outcome performance approach to curriculum design. Competencies are measurable practice behaviors that are comprised of knowledge, values, and skills. The goal of the outcome approach is to demonstrate the integration and application of the competencies in practice with individuals, families, groups, organizations, and communities. The ten core competencies are listed below … followed by a description of characteristic knowledge, values, skills,
What Is Needed?
How, in an educational outcomes sense, can the performances of groups of faculty (e.g., a curricular area) or of an entire school of social work be fairly assessed over a period of time in the manner that social work accreditation requires? A long-term goal of our research team has been to develop measurement approaches that allow programs to utilize an effective method of gathering data relevant to accreditation requirements. We view these efforts as worthwhile because few measures of social work educational outcomes exist that: are based on prior theoretical and empirical work, have evidence regarding the psychometric properties of the data they produce, are able to be used with large numbers of students in an efficient manner, are not expensive or subject to user restrictions, and are able to provide views of outcomes that have utility for pedagogical considerations at many levels (e.g., school, curricular area, course, and course elements).
In addition, CSWE Accreditation Standard 4.0.1 (2013) states that each program needs an assessment plan that uses multiple measures to assess competencies. We concur with this perspective and have always noted that we view our measures as one component of an overall multicomponent assessment of social work educational outcomes (cf. DeLong-Hamilton et al., 2011).
Self-Efficacy
The scales we have developed are based on Bandura’s construct of self-efficacy, which is derived from his Social Cognitive Theory (1977, 1982, 1986, 1997, 2004). Although sometimes considered at the collective level, self-efficacy is usually defined as an individual’s confidence that they will be able to successfully perform a particular sequence of behaviors in the future. Note that the individual’s actual competency is not being measured, but rather their belief in their ability to perform. A selective review reveals that self-efficacy is a truly versatile construct. Not only has it been employed across many other fields, it has been used in a variety of situations in social work education and practice. These include the following: discussions of the social work educational process in general and outcomes assessment specifically (e.g., Calderon, 2013; Carpenter, 2011; Green, 2003; Montcalm, 1999; Petrovich, 2004; Spitzer et al., 2001); as a predictor of: intentions to remain employed in child welfare (Ellett, 2009), research activity of social workers (Lynch, Zhang, & Korr, 2009), and for domestic violence screening by social workers (Tower, 2003); as an outcome in a conceptual model of practice with battered women (Danis, 2004), an HIV risk reduction intervention study (Icard, Schilling, & El Bassel, 1995), a comparison of services for battered women (Mancoske, Standifer, & Cauley, 1994), and a study of a wilderness adventure therapy intervention (Clem, Smith, & Richards, 2012); as a factor in a model that portrayed bachelor of social work (BSW) and master of social work (MSW) research instructors perceptions of their students (Maschi, Wells, Slater, MacMillan, & Ristow, 2013); and one that examined BSW field instructor attitudes toward evidence-based practice (Wiechelt & Ting, 2012); in the development of the Substance Abuse Treatment Self-Efficacy Scale (Kranz, 2003; Kranz & O’Hare, 2006, 2011), and as a component of two versions of the Evidence-Based Practice Process Assessment Scale (Parrish & Rubin, 2011; Rubin & Parrish, 2011); as an outcome measure for an intervention by BSW students with foster youths (Bruster & Coccoma, 2013), training in parental mental illness/child protection for social workers (Carpenter, Patsios, Szilassy, & Hackett, 2011), a developmental program for newly qualified child and family social workers (Newly Qualified Social Workers; Carpenter, Shardlow, Patsios, & Wood, 3), an intervention designed to enhance analytic abilities related to the assessment process for children and families (Platt, 2011), and an intervention designed to disseminate an evidence-supported intervention to community practitioners (Woody, Anderson, & D’Souza, 2015); as an outcome measure for a meta-analysis of the impact of youth empowerment programs (Morton & Montgomery, 2013), an evaluation of a training course for child protection workers (Maxwell, Scourfield, Holland, Featherstone, & Lee, 2012; Scourfield et al., 2012), and the evaluation of an intervention focused on child safeguarding (Scourfield, Smail, & Butler, 2014); as educational outcome measures in studies of field instruction (e.g., Cuzzi, Holden, Chernack, Rutter, & Rosenberg, 1997; Cuzzi, Holden, Rutter, Rosenberg, & Chernack, 1996; Holden, Cuzzi, Rutter, Chernack, & Rosenberg, 1997; Holden, Cuzzi, Rutter, Rosenberg, & Chernack, 1996; Holden, Cuzzi, Rutter, Chernack, Spitzer et al., 1997; Parker, 2006), interactive multimedia training for undergraduate social work students (Cauble & Thurston, 2000), an undergraduate writing course (Woody et al., 2014), research/evaluation courses at undergraduate and graduate levels (Holden, Barker, Meenaghan, & Rosenberg, 1999; Holden, Barker, Rosenberg, & Onghena, 2007, 2008; Unrau & Beck, 2004; Unrau & Grinnell, 2005), a comparison of web-based versus face-to-face MSW classes (Lawrence & Abel, 2013), “shadowing” (Parker, Hughes, & Rutter, 2006–2007), a service learning course (Williams, King, & Koob, 2002), a trauma informed evidence-based practice graduate course (Strand, Abramovitz, Layne, Robinson, & Way, 2014), a problem-based learning project (Westhues, Barsen, Freyond, & Train, 2014), and a social work administration course (York, 2008); as an educational outcome measure for the foundation level of an MSW program (Holden, Anastas, & Meenaghan, 2003, 2005; c.f. Ahn, Boykin, Hebert, & Kulkin, 2012, for an alternative application of this measure); for assessment of student change in the advanced concentration (Rishel & Majewski, 2009); and as an educational outcome measure for a complete MSW program (Holden, Meenaghan, Anastas, & Metrey, 2002; Meyer-Adams, Dorsey, Potts, Resales, & Koob, 2011; Rawlings, Townsend, & Gingerich, 2005).
Clearly, the construct has been appealing to a variety of researchers covering a range of social work topics for over 20 years. That said, not all studies involving self-efficacy have produced results that appear consistent with Social Cognitive Theory (the theory within which self-efficacy is embedded). We address this issue next.
Measurement issues and definitional concerns
In some instances, social work researchers have not found theoretically predicted relationships between self-efficacy and educational outcomes (e.g., Fortune, Lee, & Cavazos, 2005; Rawlings, 2012). On the other hand, we have contradictory evidence supporting the ability of self-efficacy ratings to predict future competence in teachers (van Dinther, Dochy, Segers, & Baeken, 2014). From our reading of this literature over the years, a number of factors might be contributing to contradictory results.
First, some of the studies in social work using self-efficacy have employed outcome measures of unknown reliability and validity (e.g., field instructors’ ratings of student performance, ratings of a brief videotaped simulation with a “standardized” client). While superficially appealing as measures, no rigorous evidence regarding psychometrics is typically presented.
Second, is the definition and operationalization of self-efficacy in research studies consistent with the theory? Some authors (including some cited earlier) appear to misunderstand the construct of self-efficacy. For instance, self-efficacy is not exclusively a “belief in one’s own ability to learn” (Drisko, 2014, p. 421). Our definition, based on Bandura’s theory, is that “self-efficacy is usually defined as an individual’s confidence that they will be able to successfully perform a particular sequence of behaviors in the future.” While an individual may (A) have varying levels of belief in their ability to learn, they are also likely to have (B) varying levels of belief in their ability to carry out a near endless array of other performances. A does not subsume B as Drisko is interpreted to imply. Self-efficacy is not restricted to a belief in one’s ability to learn. Furthermore, and in contrast to what Drisko implied, we did not argue in our 2002 article that self-efficacy measures indicated in situ competence. We stated that “[s]elf-efficacy is more than a self-perception of competency. It is an individual’s assessment of his or her confidence in their ability to execute specific skills in a particular set of circumstances and thereby achieve a successful outcome” (Holden et al., 2002, p. 116). The more explicit name of the Self-Efficacy Regarding Social Work Competencies Scale (SERSWCS) is in part an attempt to rectify misunderstandings of the construct that we have encountered in the past.
Third, in our own studies of self-efficacy, we have always used totally anonymous survey administrations in an effort to obtain more honest responses from students concerning their confidence (cf. Krumpal, 2013). There are a variety of social desirability pressures that may introduce measurement error when self-efficacy or other outcome variables are measured nonanonymously in situations where students may be sensitive to how they are being perceived (e.g., self-reports by students concerning their skills, ratings of students by field instructors, etc.).
Predictive validity
An important consideration of self-efficacy as a social work educational outcome measure is the issue of predictive validity. Bandura has noted, Nine large-scale meta-analyses have been conducted on findings from studies with diverse experimental and analytic methodologies applied across diverse spheres of functioning in diverse milieus with diverse populations … The evidence from these varied lines of research verify the explanatory and predictive generality of the theory. (Bandura, 2004, p. 622)
Research Questions and Hypotheses
Given the prior work of others and ourselves, we believe that the construct of self-efficacy continues to have promise as an educational outcome measure. Therefore, we developed and tested a new instrument, the SERSWCS. This article reports the findings regarding the following research questions and associated hypotheses.
Method
This article describes three concurrent studies (I, II, and III) within one school (New York University: Silver School of Social Work) that employed a pretest–posttest design using the same measures, with three different groups of master’s level students, where the pretest–posttest assessments covered different intervals. Data administrations were not done in the first or last classes of the semesters to minimize disruption. These were voluntary and anonymous surveys, administered by one of the investigators at both pre- and posttest. As soon as the process was explained to students, the administrator left the room to preserve the sense of anonymity. In this series of self-efficacy scale development studies, we have argued that it is important that respondents believe they are anonymous in order to reduce the press for socially desirable responding (Krumpal, 2013; Paulhus, 1991). Therefore, we have used a form of respondent generated identification numbers (aka “Hogben numbers”) to allow matching of pretest and posttest responses (Honig, 1995).
Samples
The overall sample across studies was comprised of 550 students who completed both the pretest and posttest. This overall sample contained three studies with different pre–post time intervals.
Study I
This school has a 16-month accelerated program where students with prior experience in the field complete a full 2-year MSW program in four consecutive semesters (spring, summer, fall, and spring). Two cohorts of these students were assessed near the beginning of their first and near the end of their second semester (6 months later). The sample consisted of 140 students who completed the measures at pretest. Of this sample, 108 students also completed the measure at posttest.
Study II
The majority of students in this school enroll in a typical 2-year sequence (fall, spring, fall, and spring). This cohort was assessed near the beginning of their program and near the end of their second semester (7 months later). In total, 298 students completed the measures at pretest and 238 students at posttest.
Study III
The third cohort of students was also from the typical 2-year sequence but had begun the Study 1 year ahead of the students in Study II. They were assessed near the beginning of their program and near the end of their fourth semester (19 months later). The total sample consisted of 330 students. At posttest, 204 students also completed the measure.
Measures
SERSWCS
The SERSWCS is a 41-item scale, developed in line with Bandura’s (2006) self-efficacy scale development guide, which was administered at pretest and posttest. The 41 items are comprised of the 41 practice behaviors that CSWE indicates are representative of the 10 competencies. Respondents indicated their level of confidence by choosing any of the 11 responses from 0 (cannot do) to 100 (highly certain can do). Total scores were computed as the mean of the 41 items. The instructions and scale are written at a Flesch-Kincaid Grade Level of 11.8/Flesch Reading Ease of 33.9. The SERSWCS is available to anyone to use at no cost by simply e-mailing the first author.
SWE Scale
The same comparison scale we had used in earlier studies was used here. Frans’ (1993) SWE was used at pretest in this study to assess the convergent (construct) validity of the Evaluation Self Efficacy. The SWE was chosen because of the conceptual similarity of the self-efficacy and empowerment constructs. The SWE has performed adequately in other self-efficacy scale development studies we have conducted. It is an easy-to-read (Flesch-Kincaid grade-level readability estimate = 8.2), 34-item, self-report measure with support for the reliability (e.g., α values of .88 and .89) and validity (e.g., correlation of .53 with the Torre Empowerment Scale) of the data collected with it (Frans, 1993). Respondents indicate their level of empowerment via a Likert scale from 1 (strongly agree) to 5 (strongly disagree). Total scores were computed as the mean of the 34 reversed-scored items. In this study, SWE yielded good internal consistency with Cronbach α values of α = .89 (Study I), .88 (Study II), and .85 (Study III).
Data Analysis Strategy
As a measure of internal consistency, Cronbach’s α and item–rest correlations were computed. The lowest acceptable values for Cronbach’s α and item–rest correlations were set at .70 and .20, respectively (Nunnally & Bernstein, 1994). In addition, principal component analysis was used to explore whether the SERSWCS items could be reduced to a single dimension. Pearson correlations were used to examine convergent validity with criteria for a small, medium, and large correlation set at .10, .30, and .50, respectively (Cohen, 1988). The Fisher r-to-Z transformation (Cohen & Cohen, 1983) was applied to compare the resulting correlations to those found in previous studies (r = .57 mean of prior studies) with the magnitude of the difference between both correlations computed as q = |Zr – Zr0 | which can be interpreted using the same conventions as reported earlier. A repeated-measures analysis of variance (ANOVA) specifying a within-subjects factor (pre- vs. posttest) and a between-subjects factor (study) was used to test the main effects of time point and study as well as the interaction between both. Effect sizes in the repeated-measures ANOVA were computed as ηp 2 with values of .01, .06, and .13, indicating a small, medium, and large effect (Cohen, 1988). Follow-up paired sample t-tests were used to gain insight into the nature of a significant interaction effect, with effect sizes computed as Cohen’s d = mean of the difference/SD mean. Values of .30, .50, and .80 were considered to reflect a small, medium, and large difference (Cohen, 1988). The significance level was set at p < .05, unless otherwise stated.
Results
Individual Item-Level Data
We continue to agree with Cronbach’s (1963) view that inspection of individual scale item data is useful for curricular improvement. Tables 1 through 3 present the pre- and posttest means (and SD) for each of the 41 SERSWCS items for all three studies. Across studies, increases in students’ self-reported self-efficacy were observed for all 41 items, with pre-to-posttest differences ranging from 8.41 to 33.33. For Study I (Table 1), the pre–post difference was the smallest for Item 4 and the largest for item 40. For Study II (Table 2), the pre–post difference was the smallest for Item 30 and the largest for Item 1. For Study III (Table 3), Item 5 had the smallest pre–post change, whereas Item 40 had the largest.
Item-Level Statistics for SERSWCS at Both Time Points for Study I.
Note. SERSWCS = Self-Efficacy Regarding Social Work Competencies Scale; PCA = principal component analysis. N = 140. Values in boldface indicate items with lowest and highest pre–post difference.
Item-Level Statistics for SERSWCS at Both Time Points for Study II.
Note. SERSWCS = Self-Efficacy Regarding Social Work Competencies Scale; PCA = principal component analysis. N = 298. Values in boldface indicate items with lowest and highest pre–post difference.
Item-Level Statistics for SERSWCS at Both Time Points for Study III.
Note. SERSWCS = Self-Efficacy Regarding Social Work Competencies Scale; PCA = principal component analysis. N = 330. Values in boldface indicate items with lowest and highest pre–post difference.
Internal Consistency
In terms of our first research question (RQ1), a principal component analysis was conducted on the 41 items SERSWCS, separately for each study and at each measurement point. The scree plots clearly pointed toward extracting a single component. In each analysis, substantial loadings (range .43–.85) were obtained for each item which underpins the unidimensionality of the SERSWCS (Table 1 through 3). As can be seen in the same Tables, all item–rest correlations, ranging from .42 to .84, clearly exceeded the .20 threshold. Across studies, Cronbach’s αs at pre- and posttest ranged from α = .97 to .98 (Table 4), which also noticeably surpassed the .70 threshold at a consistent level. As such, the results for the three study cohorts described here point toward excellent internal consistency of the SERWCS. Although one could even argue that the very high α values may be indicative of item redundancy, an inspection of the factor loadings and item–rest correlations across the different samples and measurement points did not suggest that certain items were consistently overly redundant. Combined with the different content of each item, it seems more likely that the SERWCS items tap into closely related, but distinct, indicators of self-efficacy regarding social work.
SERSWCS Total Score Results.
Note. SERSWCS = Self-Efficacy Regarding Social Work Competencies Scale.
Validity
Content validity
RQ2 asks if there is any evidence regarding content validity. As Holloway (2013) has noted “[u]nless measurement tools
Convergent validity
RQ3 focuses on the convergent validity of the data produced by the SERSWCS. Based on two prior studies (Holden et al., 2003, 2005) using the SWE, we predicted that the SERSWCS would have a large positive relationship with the SWE (specifically, r = .57). Across the three studies reported here, a significant positive correlation of r = .58, .46, and .47 (all ps < .001) was indeed found between SERSWCS and SWE at pretest. This correlation was not statistically significantly different from the expected correlation of r = .57 for Study I (Z = 0.18, p = .861), meaning that the strength of the association was in line with our expectation. Conversely, a statistically significant small difference did emerge for Study II (Z = 2.62, p = .009, q = .15) and Study III (Z = 2.57, p = .010, q = .14), indicating that the obtained correlations were slightly smaller than the expected correlation of r = .57.
Sensitivity to Change
The descriptive statistics for the SERSWCS total scores are presented in Table 4. Across studies, about 28.39% of the cases were missing posttest data. A formal comparison of cases with and without missing data on SERSWCS posttest scores did not indicate significant differences with regard to pretest study variables, SERSWCS: t(354) = −1.05, p = .295; SWE: t(765) = −1.59, p = .113.
The repeated-measures ANOVA performed to answer RQ4, indicated a small significant interaction effect between time point and study, F(2, 547) = 3.281, p = .038, ηp 2 = .012. To enhance interpretation, we conducted follow-up paired sample t-tests, which revealed statistically significant pre-to-posttest differences for all three studies (Bonferonni adjusted p < .017). In line with expectations, the SERSWCS was thus sensitive enough to detect increases in self-efficacy over time. All three effects can be considered large, but the pre-to-posttest change was more pronounced for Study III (mean difference = 23.25, t(203) = 20.81, p < .001, d = 1.46) compared to the fairly similar effect sizes obtained in Study I (mean difference = 19.65, t(107) = 12.65, p < .001, d = 1.22) and Study II (mean difference = 19.61, t(237) = 18.81, p < .001, d = 1.22). It should be noted that normality and/or sphericity assumptions associated with these parametric tests were met.
Discussion and Applications to Social Work Education
As Brandt et al. (2014) note, “replications are therefore essential for theoretical development through confirmation and disconfirmation of results” (p. 218). The goal of the studies reported here was to construct and test a new measure of social work educational outcomes that was based on prior theoretical and empirical work and would be suitable for programs to use for accreditation purposes, as well as program evaluation/curriculum development. The three studies reported here investigate SERSWCS performance with three different samples over three different time periods (N = 550). The SERSWCS generally performed in line with our prior expectations. Differences between the three studies were detailed. More generally, this series of three studies is close to our prior self-efficacy scale development studies in terms of populations, methods, and constructs employed. They are different from our prior work in terms of the time period in which the studies were conducted, as well as the participants, scale directions, pretest–posttest time frame, and behaviors assessed in relation to self-efficacy. The SERSWCS performed in a manner similar to that of our other self-efficacy scales.
In terms of internal consistency, we have repeatedly observed Cronbach’s αs greater than .90 with other self-efficacy scales that we have developed. In the current studies, the six αs all exceeded .96. Moreover, the results of six principal components analyses and six sets of item–rest correlations all provide strong support for the unidimensionality of the SERSWCS. Despite the substantial homogeneity, we argued that items were not overly redundant, but future research could examine this issue more closely.
As noted previously, one critique of self-efficacy (i.e., restricting it to one’s ability to “learn”) is mistaken. In this study, as in prior ones, we asked individuals about their confidence in their ability to successfully perform a task in a particular context. The SERSWCS scale instructions read, in part:
Agreeing with Cronbach (1963), we reported results at the individual item level. The wide range of pre–post differences across SERSWCS items provides data that may inform curricular adjustments. Moreover, this level of detail may be more informative than dichotomous decisions whether or not a result exceeds a preset benchmark. Consider SERSWCS Item 21 (use practice experience to inform scientific inquiry). For instance, suppose we had set a benchmark that students average 80.00 or greater on SERSWCS items at the posttest. In Study 1, the students averaged 72.15 at posttest. Contrast this with Item 4 (demonstrate professional demeanor in behavior, appearance, and communication) for which students averaged 91.02. Students were more confident on Item 4 than they were on Item 21. The finding that the benchmark was exceeded for Item 4 but not for Item 21 has value. Yet, in terms of “value added,” there was an increase in self-efficacy of 8.41 on Item 4 versus an increase of 23.87 for Item 21. That finding has value as well. Of course, there are additional nuances to consider (e.g., ceiling effects, time devoted to the topic in the curriculum, etc.), but our point here is that restricting examination of the data to total scale scores or to dichotomous attainment of benchmarks without recognizing where students began would curtail our understanding of the diversity of changes that are occurring in our students. Information that could suggest curricular changes might be missed. That is not to insinuate that total scale scores are unimportant. As we have consistently found over the years with other self-efficacy scales, the total scale scores for the SERSWCS increased significantly from pre- to posttest across all three studies. In Studies I and II, this significant change was over the first two semesters. Study III had a significant pre–post change (across four semesters) that was greater than the pre–post change in Studies I and II. That said, although the change in Study III was more pronounced, the statistically significant difference in pre–post change between studies should not be overstated since it was small in size.
One of our reviewers remarked on the high levels of student self-efficacy at pretest (i.e., total SERSWCS scale scores > 62). From the perspective of a seasoned social worker, this could be seen as overconfidence relative to the knowledge and skills these respondents possess. In fact, our prior research on response shift bias demonstrated that newer social work students rated themselves significantly higher on pretest surveys than they subsequently thought they should have (after they better understood the behavioral performances involved). If we were claiming that self-efficacy was a perfect equivalent of actual competency, those high pretest ratings would bother us as well. Our view is that
As always there are some caveats, readers should keep in mind in thinking about these results. Despite three studies with consistent findings, these were one group pretest–posttest designs, with nonrandom samples and attrition, carried out in a single school, by a single group of investigators. The number of complete pretest–posttest pairs is typically less than the number of pretests completed and this problem is worse over the four semester interval. Our last five self-efficacy scale development studies (covering different pre–post period lengths) had attrition rates ranging from 21% to 45%, compared to the 28% for this current set of studies. This current rate is somewhat larger than the rates reported by Hanson, Tobler, and Graham (1990) in their meta-analysis of attrition in substance abuse prevention research. They report that 21.7% of subjects (on average across 85 cohorts) were missing 6 months after the pretest, 26.6% were missing after 12 months, and 28.2% were missing after 24 months. There are a variety of explanatory possibilities here (e.g., refusals to participate, late for class or absent at the point of administration, leave of absence, shift into a different program track or to a different campus, dropped out, confused by the identification system, lack of incentives for participation, no threat of being identified as a dropout, participation discouraged by classroom faculty, etc.). Comparison of cases with and without missing data on SERSWCS posttest scores did not reveal significant differences with regard to pretest study variables. Nonetheless, more research on the psychometric performance of the SERSWCS, using more representative samples and taking additional measures to reduce attrition, is needed in order increase generalizability. The larger question of which outcome is more important (i.e., smaller attrition rates or reduced press for socially desirable responding) remains.
One of our reviewers was concerned that some of the behaviors specified by CSWE resulted in double-barreled items in our adaptation. We argue that in least some of those instances, the items were complex as opposed to simply asking about two attitudinal objects (“double-barreled”). In a classic text on asking questions, Payne (1951) referred to double-barreled items as deserving to be split but that there were “special cases where two issues necessarily have to be asked about together” (p. 233). In certain instances, when factors are interrelated, they may be asked about together (Payne, 1951). In select instances, we would have preferred to change the wording of items but decided not to in order to preserve the congruence between the results obtained and the specified behaviors that appear interrelated. It was a considered trade-off that necessitated a psychometric choice between keeping the items as written or substantially increasing the number of items, the latter choice avoiding the double (or more)-barreled problem but also potentially inducing response sets and fatigue that would create new problems. On the whole, there might be utility in CSWE considering this issue in future modifications of the EPAS.
These studies raise a number of questions and do not answer the global question: What is the optimal approach to measuring social work educational outcomes? Until the field has measures that produce reliable and valid data regarding student’s actual competence (in a broadly applicable, efficient manner), other indicators of outcome will remain important. Potential changes in the specific competencies (CSWE, 2014) may require revisions to current measures as well as changes in scale development plans. It makes sense for the field to continue working on these problems from multiple perspectives. For instance, the ongoing work of the multi-university Social Work Education Assessment Project (e.g., Hamilton et al., 2011), the University of Toronto Building Professional Competence group (e.g., Bogo et al., 2013; Logie, Bogo, Regehr, & Regehr, 2013), and the Indiana University’s Social Work Office of Education Assessment (e.g., Pike, Bennett, & Chang, 2004).
The curriculum (very broadly defined) is the responsibility of the faculty. Accreditation-related efforts can be extremely costly to faculty and the schools where they are located. Accreditation outcomes are not necessarily benign or valid indicators of quality education. While we believe in faculty engaging in self-assessment, we also believe that faculty have a right and responsibility to thoughtfully question neoliberal calls to accountability that may not be solely focused on improving (or even maintaining) the education our students receive.
Conclusion
There was empirical support for the use of self-efficacy in social work prior to this study and that body of support is now larger. The SERSWCS is a measure that is based on substantial theoretical and empirical work, has preliminary evidence regarding the psychometric properties of the data it produces, can be used with large numbers of students in an efficient manner, is neither expensive or subject to user restrictions, and provides views of outcomes that have utility for pedagogical considerations at multiple curricular levels. Although more work is needed, we believe this is a promising start.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
