Confirmatory Factor Analysis on the Professional Suitability Scale for Social Work Practice

Abstract

Objective:

This article presents a validation study to examine the factor structure of an instrument designed to measure professional suitability for social work practice.

Method:

Data were collected from registered social workers in a provincial mailed survey. The response rate was 23.2%. After eliminating five cases with multivariate outliers, confirmatory factor analysis using maximum likelihood estimation was performed on 285 cases.

Results:

A 22-item four-factor model achieved an acceptable good fit. Reliability testing results indicate an overall Cronbach’s α valued at .90 and subscale αs ranged between .75 and .89.

Conclusion:

Findings affirm good-to-excellent internal consistency of the Professional Suitability scale with two previous studies and provide acceptable results on construct validity.

Keywords

professional suitability competency social work confirmatory factor analysis

Similar to other professions like medicine, dentistry, psychology, and nursing, social work is expected to fulfill its professional accountability and responsibility in order to assure quality service and protection of the public (Bogo, 2010; Bogo, Regehr, Katz, Logie, & Mylopoulos, 2011; Canadian Association for Social Work Education [CASWE], 2011; Carraccio, Wolfsthal, Englander, Ferentz, & Martin, 2002; Epstein & Hundert, 2002; Rubin et al., 2007; Stoesz, Karger, & Carrilio, 2010). Ensuring that individuals are professionally suited to such practice is regarded by many as an essential mutual responsibility of schools of social work and professional regulatory bodies (Brear, Dorrian, & Luscri, 2008; Gibbs, 2000; Moore & Jenkins, 2000; Tam & Kwok, 2007a, 2007b). The terms professional suitability and professional competence have been used interchangeably in the literature (Barlow & Coleman, 2003; Eraut, 1994; O’Hagan, 1996). In this article, professional suitability is defined as the possession of a comprehensive understanding of social work knowledge, skills, and values, combined with the competent performance of professional behaviors in specific practice situations (Kimberly & Osmond, 2009; Lyons, 1999). However, the lack of well-developed measures assessing professional suitability has impeded the ability of social work educators and regulators to carry out the function of assuring quality practice and protecting the public (Barlow & Coleman, 2003; Carpenter, 2011; Gibbs, 2000; Koerin & Miller, 1995; Lafrance, Gray, & Herbert, 2004; Mackenzie, 2002; Ryan, Habibis, & Craft, 1997; Tam & Kwok, 2007a). The purpose of this study was to validate the factor structure of an instrument developed to measure professional suitability for social work practice. Expectedly, the Professional Suitability scale, if validated with sound psychometric properties, could be used with practicing social workers as one indicator for hiring decisions and/or performance evaluation tool.

Rooted in functionalist theory (Durkheim, 1933 in Halls, Trans., 1997), professions are entrusted with providing regulatory functions for modern society and are expected to fulfill the rising demand for professional accountability. Each profession shares a “collective orientation” that constitutes a systematic body of knowledge, the Code of Ethics, a subculture reflecting unique values and norms, and a self-regulatory professional body (Holosko & Leslie, 2001; MacDonald, 1995; McCauley, 2010; Reeser & Epstein, 1990; Toren, 1972). Such sets of professional characteristics differentiate one profession from another. Under the influence of the functionalist perspective, competence-based education and training (CBET) gained momentum in the late 1980s. The concept of CBET refers to professional training, based on certain predefined knowledge, skills, and values for practice (Hager, 2004; Horder, 1998; O’Hagan, 1996). For instance, the Council on Social Work Education and the Canadian Association for Social Work Education have integrated outcome-based learning goals and measures of competency into their Educational Policy and Accreditation Standards (Council on Social Work Education [CSWE], 2010) and Standards for Accreditation (CASWE, 2011). Moreover, the National Association of Social Workers (2008) and the Canadian Association of Social Workers (2005) have clearly defined professional competence as one of the core values of social work in their Codes of Ethics. Graduates of accredited social work programs are expected to demonstrate knowledge, skills, and values at entry-level competence that meet the requirements for granting a professional degree or a license to practice.

However, the notion of professional competence is not without detractors. Some opponents critique the assessment of competencies for its attempt to reduce professional work to compartmentalized interventions and mechanistic evaluations (Malone & Supri, 2012; Rossiter & Heron, 2011), its inability to address diverse practice (Martinez-Brawley & Zorita, 2007; Sultana, 2009), and its intent to achieve a monopoly of expertise in the market and gain support from the state to attain political, economic, and social privilege (Carniol, 2005; Dickens, 2011; Lymbery, 2011). Responding to these criticisms, some established professions have moved beyond the traditional narrow and mechanistic definitions of professional competency to a more holistic approach on competence-based education and assessment (CSWE, 2010; Epstein & Hundert, 2002; Falender & Shafranske, 2012; Kaslow et al., 2007; Leigh et al., 2007; Mulder, Guliker, Biemans, & Wesselink, 2009; Sultana, 2009) through the use of multiple evaluation methods and multiple sources for performance assessment (Carpenter, 2011; Cuyvers, 2009; Roberts, Borden, Christiansen, & Lopez, 2005; Rubin et al., 2007). However, the lack of reliable and valid assessment tools has been one of the challenges that social work educators and field supervisors face in carrying out their role in ensuring quality practice and protection of the public (Barlow & Coleman, 2003; Bogo et al., 2006; Gibbs, 2000; Koerin & Miller, 1995; Lafrance et al., 2004; Regehr, Bogo, & Regehr, 2011; Ryan et al., 1997; Tam & Kwok, 2007a; Tam, Twigg, & Margrett, 2009).

To facilitate reliable and valid assessment of professional performance, researchers have identified some criteria for professional suitability in social work (Barlow & Coleman, 2003; Bogo, Regehr, Power, Globerman, & Hughes, 2002; Bogo et al., 2011; Gibbons, Bore, Munru, & Powis, 2007; Gibbs, 1994; Gibbs & Blakely, 2000; GlenMaye & Oakes, 2002; Koerin & Miller, 1995; Lafrance et al., 2004; Miller & Koerin, 1998; Ryan et al., 1997; Tam, 2004; Tam & Coleman, 2009b). Examples of these criteria include belief in the values and goals of the profession (Barlow & Coleman, 2003); ability to organize and present data and well-written assessments (Bogo et al., 2002); ability to advocate appropriately for clients (Bogo et al., 2006); demonstration of self-awareness (Gibbons et al., 2007); commitment to social justice (Gibbs & Blakely, 2000); ability to recognize how personal feelings and biased perceptions (GlenMaye & Oakes, 2002); capacity for social work relationships (Lafrance et al., 2004); ability to manage personal problems (Koerin & Miller, 1995); openness to learning (Miller & Koerin, 1998); ability to transfer learning from field instruction to clinical practice (Regehr et al., 2011); and commitment to professional growth (Tam & Coleman, 2009b).

Even though some suitability criteria have been identified in previous studies, the transformation of these identified criteria into reliable and valid measures is far from completion (Alperin, 1996; Bogo, 2010; Brear et al., 2008; Tam, 2003; Tam & Coleman, 2009b; Tam, Coleman, & Boey, 2012). Most of the reviewed studies are largely exploratory and descriptive and few studies have moved beyond reliability testing to examining criterion-related and construct validity. Moreover, studies examining professional suitability have had limited sampling frames, low participation rates, and lacked inputs from field instructors, faculty field liaisons, and faculty members (Bogo et al., 2004, 2006; Gibbs, 1994; Lafrance et al., 2004; Miller & Koerin, 1998; Ryan et al., 1997) or relied heavily on commentary from senior administrators in educational programs (Barlow & Coleman, 2003; Gibbs, 1994; Ryan et al., 1997). All these constitute a research gap in uncovering reliable and valid indicators of professional suitability.

To address some of these gaps, Bogo, Regehr, and colleagues in the Faculty of Social Work at the University of Toronto have developed multiyear research projects aimed at developing innovative and sound approaches to evaluation of student learning and competence (Bogo et al., 2002, 2004, 2006, 2011; Regehr, Bogo, Donovan, Lim, & Regehr, 2012; Regehr et al., 2011). Results of these studies have provided valuable information on the conceptualization and measures of professional suitability; however, the works from Bogo, Regehr, and colleagues were primarily focused at master’s level social work, and data were limited to field supervisors, students, and social work educators affiliated with the University of Toronto.

To strengthen the sampling and psychometric issues, two different provincial studies were conducted in Alberta and Ontario to examine issues related to social work field education and professional suitability in 2003 and 2007, respectively (Tam, 2003, 2004; Tam & Coleman, 2009a, 2009b; Tam et al., 2012; Tam & Kwok, 2007a, 2007b, 2007c). In the 2003 study, a mailed questionnaire was used to survey the entire population of field supervisors associated with social work program in Alberta, Canada. Based on this study, a Professional Suitability scale was developed and tested in terms of reliability and validity. Detailed description on the scale development is reported in Tam (2004) and Tam and Coleman (2009a). Using the exploratory factor analysis method, five factors were identified, namely, overall, analytical, practice, personal, and ethical suitability. While the Professional Suitability scale achieved excellent internal consistency (α = .93); its factor solution only accounted for 52% of variance (Tam & Coleman, 2009a). Moreover, generalizability of the findings in this study was limited due to sampling with only field instructors in the Bachelor's programs that excluded other practicing social workers who were not field instructors at the time of the study (Tam, 2004; Tam & Coleman, 2009a).

To address the limited generalizability of the Alberta study, another study was conducted in the province of Ontario in the summer of 2007. The Professional Suitability scale was revised based on the results from the Alberta study (Tam & Coleman, 2009a, 2009b) and tested again. Detailed description on the scale modification is reported on Tam, Coleman, and Boey (2012). This questionnaire survey was responded by 341 (69%) randomly selected registered social workers in the Ontario College of Social Workers and Social Service Workers. Results of the Ontario study confirmed strong internal consistency (α = .89), but questions regarding the certainty of the stability or predictive reliability, and construct validity of the scale remained unanswered (Kane, 2006; Tam & Coleman, 2009a; Tam et al., 2012). In addition, testing on the scale was conducted by asking about respondents’ attitudes toward professional suitability. This means the scale in the Ontario study was not worded as self-assessment or peer assessment tool (Tam et al., 2012). Schmeiser and Welch (2006), accordingly, highlight the importance of specifying test purpose and intended examinee population in the test development process. With a goal to develop a scale to assess one’s professional suitability for hiring decision or job performance, the Saskatchewan study was developed to test with practicing social workers.

Method

Participants

Aimed to better understanding the factor structure of the Professional Suitability scale built upon previous studies, another provincial study was conducted with social workers registered with the Saskatchewan Association of Social Workers in November 2007. The Saskatchewan study was completed by inviting the entire population of 1,251 registered social workers to participate. The province of Saskachewan has mandatory registration for individuals who want to practice under the title of “Registered Social Worker.” The sample frame in this study represents the population of social workers in this province. Excluding three returned questionnaires, with missed responses on several back pages, 290 (23.2%) valid questionnaires were received.

Instrument

The instrument used in this study was a survey questionnaire, which consisted a section on the “Professional Suitability scale,” a section on demographic information, and three open-ended questions asking for participants’ opinions about criteria on professional suitability for social work practice, challenges faced by those who have been a social work field supervisor, and action taken to address those challenges. Following Schmeiser and Welch’s (2006) recommendation to specify the test purpose, the Professional Suitability scale used was modified from the Ontario study by rewording the stem on each item to “I consider myself …” from “A person who is professionally suitable for practicing social work …” The objective of this change was to better examine the instrument’s applicability for hiring decisions or performance evaluations. The revised scale retained the 40 items on a 7-point Likert-type scale as the Ontario study, with higher scores representing higher levels of professional suitability. The scale was administered with the intended users, practicing social workers. The revised instrument was reviewed by two experienced social workers to affirm its face validity.

Research Design

The Professional Suitability scale was administered through a provincial questionnaire survey in November 2007. Following procedures approved by University Ethics Review Committee, a prior notice of the upcoming survey was mailed to the potential participants along with a newsletter of the Saskatchewan Association of Social Work. The full survey package with stamped reply envelope was mailed 1 week after this notice and data collection was completed within a month. No follow-up mailing was sent due to limited resources.

Data Cleaning

Preliminary analyses were conducted to ascertain whether the data set met the requirements for confirmatory factor analysis (CFA) (Brown, 2006; Gorsuch, 1983; Harrington, 2009; Jackson, Gillaspy, & Purc-Stephenson, 2009; Kline, 2005; Meyers, Gamst, & Guarino, 2006; Nunnally & Bernstein, 1994; Stevens, 2002; Tabachnick & Fidell, 2007). Of the 40 items on the Professional Suitability scale, there were some missing values on the items of the scale. However, all missing values were random and none of the items had a missing value greater than 2.5%. As suggested by Tabachnick and Fidell (2007), missing data less that 5% are less serious and could use any procedures to handle the missing values. In this study, mean replacement was used to handle missing values. The data set was also checked for normality. Despite the fact that most responses on the Professional Suitability scale were negatively skewed, the highest skewness value was at −1.96 and the highest kurtosis valued at 6.83. According to Harrington (2009), if none of the skewed items had a value greater than 3.0 or a value of kurtosis greater than 10.0, the violations of normalcy are considered mild, and the original nontransformed data were hence used for CFA. A check for outliers was performed and 15 items were identified with a z score greater than 4.0. These items were entered for multivariate outlier analysis. Results of Mahalanobis distance (D ²) indicated that five cases had a significant D ² (chi-square critical valued at 37.70, df = 15, p < .001). These five cases were eliminated from final analysis and the sample size for the final analysis ended up at 285, which was slightly below the requirement of 50, more than 8 times the number of variables in the model or a minimal with 298 participants or more in this case (Meyers et al., 2006). Given the required sample size for CFA using maximum likelihood procedure varies in the literature (Gorsuch, 1983; Nunnally & Bernstein, 1994; Tabachnick & Fidell, 2007), the CFA was used with caution.

Data Analysis

The data were analyzed using AMOS 20 for Windows. CFA was used to test two models on professional suitability. Model One was a 36-item five-factor solution with the domains of social consciousness suitability, ethical suitability, practice suitability, personal suitability, and distractors (items developed to prevent response set) identified through an exploratory factor analysis in the Ontario study (Tam et al., 2012). Excluding the distractors in Model One, each suitability factor was allowed to covary and each factor had one indicator items fixed at value 1 as its unstandardized regression weight. There were also 4 items that had shared variances on two factors. These 4 items were Item 2 (shared variances on ethical and personal suitability), Item 35 (shared between practical and personal suitability), and Items 36 and 40 (shared between ethical and practical). Table 1 summarizes the list of items on Model One and expected dimensions, means, and standard deviations.

Table 1.

Five-Factor Model One of the Professional Suitability Scale With Item Numbers in Parentheses, Means, and Standard Deviations (n = 290).

Factor	Statements Started With “consider myself to …”	Statistics
Factor	Statements Started With “consider myself to …”	M	SD
Social consciousness	Be committed to promote social justice (a27)	6.00	1.12
	Be committed to changing social inequality (a37)	5.93	1.06
	Have demonstrated a commitment to social change (a5)	5.95	1.03
	Have advocated against oppression (a9)	5.91	1.20
	Be able to critically examine bureaucratic systems (a8)	5.44	1.12
	Be able to critically examine political issues (a7)	5.30	1.22
	Be aware of the impact of the global economy (a15)	5.24	1.15
Ethical suitability	Have demonstrated competence in professional practice (a18)	6.49	.66
	Have demonstrated integrity (a19)	6.64	.54
	Have maintained clear boundaries with clients (a30)	6.56	.63
	Be respectful of human diversity (a10)	6.66	.59
	Have used my professional power responsibly (a23)	6.53	.62
	Have believed in the value & dignity of each individual (a33)	6.76	.48
	Be able to examine my personal biases (a36)	6.34	.64
	Be able to make reasoned decisions (a2)	6.51	.59
	Be able to distinguish between professional/personal relationship (a11)	6.64	.59
Practice suitability	Be aware of personal strengths (a31)	6.33	.64
	Be able to transfer my learning from one context to another (a21)	6.32	.66
	Be aware of my personal limitations (a34)	6.30	.71
	Have developed a personal practice style (a20)	6.26	.83
	Be open to feedback to improve my professional practice (a40)	6.42	.70
	Have good interpersonal skills (a38)	6.42	.68
	Be friendly (a32)	6.45	.73
	Have demonstrated effective nonverbal communication skills (a22)	6.19	.82
	Be willing to receive feedback in a positive manner (a26)	6.25	.74
	Be aware of my personal values and beliefs (a16)	6.61	.57
Personal suitability	Be able to manage negative life experiences (a28)	6.14	.78
	Be able to manage personal life issues (a4)	6.26	.75
	Have taken responsibility for handling my personal problems (a35)	6.46	.62
	Be aware reasons of why I have chosen a career in SW (a12)	6.46	.70
	Have demonstrated impartiality in my practice (a13)	6.18	.88
Distractors	Have demonstrated excellent research skills (a3)	4.74	1.25
	Be able to speak a second language (a6)	2.20	1.78
	Be bilingual (a14)	2.21	1.75
	Be committed to participate in street protests (a25)	3.49	1.51
	Be knowledgeable on statistical analysis (a39)	4.38	1.49

As the distractors are not theoretically grounded in the construct of professional suitability, an alternative Model Two with 31-item four-factor model without the distractors or shared variance on indicator items was also be tested. Trial runs of the CFAs identified that Model One was underidentified and could not be processed meaningfully in the analysis. Model One was dropped; whereas Model Two was tested in the final analysis. Maximum likelihood procedure was used to estimate the values of the parameters. Model evaluation included the examination of nonstandardized and standardized regression weights, significance level was set at .001, squared multiple correlations (SMCs), modification indexes (MI), and model fit indices. Because the aim of this study was to confirm the factor structure of the scale and to predict a close fit, a nonsignificant chi-square was desired. As the most common model fit index, chi-square, is very sensitive to large sample size, chosen fit indices to evaluate the overall fit of the model were normed chi-square value (χ²/df < 2.5), root mean square residual (RMSR or RMR used in AMOS) <.05, root mean square error of approximation (RMSEA) <.08, goodness of fit (GFI) ≥.90, and comparative fit index (CFI) >.90 (Kline, 2005; Meyers et al., 2006). The analysis was to test the hypothesis that the four-factor model with simple structure that each variable loads on one factor fits the data.

Results

Descriptive Statistics

Participants included 243 (83.8%) female and 46 (15.9%) male. A total of 176 (60.4%) held a diploma or bachelor’s degree, 108 (37.9%) had a master’s or doctoral degree, and the remaining 1.7% included people who did not respond to this item or people with other qualifications. Among the participants, the modal age was between 50 and 59 (n = 98, 33.8%), followed by respondents between 40 and 49 (n = 70, 24.1%), those between 30 and 39 (n = 56, 19.3%), and those aged 60 or above (n = 37, 12.8%). The youngest group fell between the ages of 20 and 29 (n = 28, 9.7%). The mean years of work experience as a social worker was 16.4 years (SD = 11.38), with a mode of years of experience between 1 and 5 years (n = 55, 19.0%). Among the respondents, 134 (46.2%) had been a field supervisor for between less than 1 year and 39 years (M = 7.14, SD = 7.94). Slightly over half (n = 158, 54.5%) worked in government services, 51 (17.6%) participants worked in nongovernment nonprofit agencies, 29 (10.0%) in individual/group/partnership private practice or for profit organizations, 45 (15.5%) in unspecified other practice sectors, and the remaining 2.4% did not respond to this question. Slightly over two thirds (n = 196, 67.7%) of participants worked in urban areas, 56 (19.3%) worked in rural/remote rural areas, 24 (8.3%) worked in suburban areas, and the remaining 14 (4.8%) reported unspecified other practice community or did not respond.

Tests on Group Differences

Total score on the 40-item with distractors were computed using Hudson’s (1982) scoring formula: S = (∑X − N)(100)/[(N)(6)], where S was the total score; X was the denoted item response; N was the number of items on the scale; 6 was used to exclude any items that may have been scored outside the range from 1 to 7. This scoring formula produces a range of values from 0 to 100, facilitating easier interpretation of the total scores and cross-study comparison. After computation, total scores on the 40-item Professional Suitability scale ranged between 53.8 and 98.8 (M = 81.4, SD = 7.50). Results of independent t-tests using the total score as a dependent variable revealed no group differences between female and male participants, between social workers with and without field supervision experience, or between participants with a diploma or bachelor’s degree and those with a master’s or doctoral degree. Independent t-tests comparing group differences between those below and above the mean score of years of social work experience and the mean score of years of work experience as field supervisor found no difference. Furthermore, results of analysis of variance (ANOVA) using the total score as a dependent variable indicated no group difference among participants from different primary practice sectors (government, nongovernment nonprofit, individual/group private practice, and others) nor different primary practice communities (urban, suburban, and rural/remote). To examine any difference among groups of individuals with more work experience versus less, years of experience as social worker was recoded at interval level (1–5, 6–10, 11–15, 16–20, 21–25, 26–30, and 31 years or above), and so did recode on years of experience as field supervisor (1–5, 6–10, or 11 years or above). Results of Levene’s test on all of these group comparisons were insignificant and affirmed homogeneity of the sample. Table 2 summaries results of t-tests and ANOVA testing.

Table 2.

Results of t-Tests and Analysis of Variance (ANOVA) Testing for Group Comparisons (n = 290).

Groupings	t _(df)	ANOVA_(df)	p	Levene’s Statistic	p
Female versus male	.14₍₂₈₇₎	—	.89	.63	.43
With field supervision experience versus without	1.60₍₂₈₆₎	—	.11	1.17	.28
Diploma/bachelor’s versus master’s/doctoral	1.48₍₂₈₃₎	—	.14	1.51	.22
Years of work experiences below mean versus above mean	1.05₍₂₈₄₎	—	.30	1.09	.30
Primary practice sectors	—	1.56_{(3, 279)}	.20	1.37	.29
Primary practice communities	—	.69_{(3, 282)}	.56	1.39	.25
Recoded years of experience as social worker	—	1.18_{(6, 278)}	.32	.47	.83
Recoded years of experience as field supervisor	—	.15_{(2, 131)}	.87	.91	.41

Model Fit

Model Two

The 31-item four-factor model without distractors or shared variance was analyzed for model fit. On each factor, one indicator item was fixed with an unstandardized regression weight equal to 1. Moreover, all four factors were allowed to covary. Figure 1 presents the initial model, consisting of four factors, 31 indicators, and, respectively, errors. Initial run results on model fit indices (see Table 3) showed that Model Two met the following model fit criteria: normed chi-square value was 2.39, RMSR was .04, and RMSEA was .07. However, the indices of GFI (.84) and CFI (.87) were not meeting the minimal criteria of .90.

Figure 1.

Initial structure of Model Two with associated indicators and errors (n = 285).

Table 3.

Professional Suitability Scale: Goodness of Fit Indices (n = 285).

	Goodness of Fit Indices
Model Two	χ² _(df)	χ²/df	RMSR	RMSEA	GFI	CFI
31 Items without shared variance (initial model)	701.30₍₂₉₃₎	2.39	.04	.07	.84	.87
31 Items without shared variance (initial model)	p = .000	2.39	.04	.07	.84	.87
22 Items with one error covariance (final model)	453.38₍₂₀₂₎	2.24	.03	.06	.87	.91
22 Items with one error covariance (final model)	p = .000	2.24	.03	.06	.87	.91

Note. RMSR = root mean square residual; RMSEA = root mean square error of approximation; GFI = goodness of fit; CFI = comparative fit index. Recommended cutoffs: p > .001; <2.5 (χ²/df); <.05 (RMSR); <.08 (RMSEA); ≥.90 (GFI and CFI).

Model Respecification

Modification was made to delete 9 items with low SMC (SMC < .30). These items were Items 7, 8, 10, 12, 13, 15, 18, 20, and 32. Moreover, examination on the MI suggested one error covariance with the highest MI and par change value added. This error covariance was between Item 26 and Item 40 and was considered plausible because these 2 items belonged to the same latent variable of practice suitability on which some shared covariance was possible.

Results of the model respecification showed a much better fit. Table 3 shows the model fit indices on final model, which has retained 22 items and 1 error covariance. The final model met four of the five model fit indices, namely the normed chi-square (2.24), RMSR (.03), RMSEA (.06), and CFI (.91); whereas the GFI was slightly increased to .87 from .84. There is acceptable evidence to support the hypothesized four-factor model fit the data. Results from another run to fix regression weight with different indicator item per factor equal to 1 showed that all regression weights on this four-factor model were significant (p < .001), standardized regression weights ranged between .55 and .89, and SMCs ranged between .31 and .79. Results from the covariance matrix between factors show that the latent variable of social consciousness discriminated well with personal, practice, and ethical suitability in that the covariance coefficients were .28, .33, and .35 respectively. However, the covariance coefficients among ethical, practice, and personal suitability were excessively high ranged between .80 and .97. Figure 2 shows the final model and its variances and covariances. The following reliability estimation is based on the final factor structure of Model Two.

Figure 2.

Four-factor final model of professional suitability and retained indictors (n = 285).

Reliability Estimation

Reliability testing on the final model showed that this 22-item four-factor final factor structure achieved excellent overall internal consistency Cronbach’s α valued at .90, and the subscales α on social consciousness, practice suitability, ethical suitability, and personal suitability valued at .89, .84, .80, and .75, respectively. Table 4 shows the maximum likelihood parameter estimations on unstandardized coefficients (Un-S), standard errors (SEs), standardized coefficients (S), and Cronbach’s αs (αs).

Table 4.

Maximum Likelihood Parameter Estimates on the 22-Item Final Factor Structure of the Professional Suitability Scale (n = 285).

Factor (α)	Item starts with “I consider myself …”	Un-S	SE	S	SMC
SC (.89)	Be committed to promote social justice (a27)	1.10	.06	.89	.79
	Be committed to changing social inequality (a37)	1.02	.07	.86	.75
	Have demonstrated a commitment to social change (a5)	.89	.06	.78	.61
	Have advocated against oppression (a9)	.98	.07	.75	.56
ES (.80)	Have demonstrated integrity (a19)	1.15	.13	.71	.51
	Have maintained clear boundaries with clients (a30)	.96	.11	.57	.33
	Have used my professional power responsibly (a23)	.95	.10	.58	.33
	Have believed in the value & dignity of each individual (a33)	.70	.08	.56	.32
	Be able to examine my personal biases (a36)	1.10	.11	.66	.44
	Be able to make reasoned decisions (a2)	.86	.10	.55	.31
	Be able to distinguish between professional/personal relationship (a11)	.87	.10	.56	.32
PrS (.84)	Be aware of personal strengths (a31)	1.27	.14	.66	.43
	Be able to transfer my learning from one context to another (a21)	1.00	.10	.64	.41
	Be aware of my personal limitations (a34)	1.09	.11	.64	.41
	Be open to feedback to improve my professional practice (a40)	.97	.11	.58	.34
	Have good interpersonal skills (a38)	1.04	.11	.70	.50
	Have demonstrated effective nonverbal communication skills (a22)	1.30	.13	.67	.45
	Be willing to receive feedback in a positive manner (a26)	1.05	.12	.59	.35
	Be aware of my personal values and beliefs (a16)	.79	.09	.58	.34
PeS (.75)	Be able to manage negative life experiences (a28)	1.29	.12	.75	.56
	Be able to manage personal life issues (a4)	.78	.08	.61	.37
	Have taken responsibility for handling my personal problems (a35)	.78	.07	.74	.55

Note. SC = social consciousness; ES = ethical suitability; PrS = practice suitability; PeS = personal suitability; α = alpha; Un-S = unstandardized coefficients; SE = standard errors (SE); S = standardized coefficients; SMCs = squared multiple correlations (SMCs).

Discussion and Applications to Social Work

This article presents the results of a CFA examining the factor structure of the Professional Suitability scale. The scale was developed to assess professional suitability for social work practice and was intended to be used for hiring decision or job performance evaluation. The two models tested for CFA were developed from the Ontario study (Tam et al., 2012). Maximum likelihood procedures were used in the analysis. The model fit was evaluated by normed chi-square value (χ²/df), RMSR, RMSEA, GFI, and CFI. Initial model fit results indicated that Model One was underidentified and was dropped from final analysis. Results of initial run on Model Two suggested dropping 9 items and adding one error covariance between Items 26 and 40. Except for GFI, which was approximating the cutoff of .90, results of this CFA provide acceptable evidence on the hypothesis that the four-factor model with a simple structure that each variable loaded on one factor without shared variance fits the data better.

Moreover, the final 22-item four-factor solution achieved excellent overall internal consistency (α = .90) and good to excellent subscales internal consistency ranging between .75 and .89 (Harrington, 2009; Kline, 2005). Compared with the results with the Ontario study on professional suitability (Tam et al., 2012), reliability testing results on this 22-item four-factor final model in the Saskatchewan study slightly increased the overall internal consistence measured by Cronbach’s α to .90 from 89, and on the subscales of practice suitability was enhanced to .84 from .82, and on the subscales of personal suitability was improved to .75 from .72; whereas the internal consistence on social consciousness remained at .89. Despite the Cronbach’s α on the ethical suitability subscale dropped slightly to .80 from .82, its internal consistency remained excellent. Another comparison with the Alberta study (Tam & Coleman, 2009a) showed that the entire scale has maintained excellent internal consistency ranged between .89 and .93 across this and the Ontario and Alberta studies, especially the social consciousness subscale showed excellent internal consistency across the three studies.

Importantly, results of CFA in this study have provided acceptable evidence to affirm the factor structure of professional suitability, namely social consciousness suitability, ethical suitability, practice suitability, and personal suitability. Throughout the scale development process, these constructs were first identified through an exploratory factor analysis in the Alberta study (Tam & Coleman, 2009a) and then refined in another exploratory factor analysis in the Ontario study (Tam & Coleman, 2009b, Tam et al., 2012). Using CFA in this study, four of the five models fit indices support the construct validity of the Professional Suitability scale. Furthermore, results from this study showed that the social consciousness subscale was able to discriminate well between the ethical, practice, and personal suitability subscales with covariances ranged between .28 and .35 (p < .001). This is important because the Professional Suitability scale has been tested with two provincial samples and one convenience sample, and the findings on the social consciousness subscale has always been consistent and theoretically sound.

From the perspective of group comparison, one interesting finding in this study was that there was no group difference on the professional suitability scores between social workers with a diploma or bachelor’s degree and those with a master’s or doctoral degree. However, results in the Ontario Study (Tam et al., 2012) showed that participants with a diploma or a bachelor’s degree had higher professional suitability scores than participants with a master’s or doctoral degree. One of the possible questions both the Ontario and Saskatchewan studies did not ask is that whether those individuals with a master’s or a doctoral degree have an undergraduate degree in social work or not. This is because there are five schools of social work in Ontario, which are composed of 49.6% of masters’ students (CASWE, 2012) and deliver a 2-year master’s in social work program to individuals without an undergraduate degree in social work. Would people with both undergraduate and graduate degrees in social work have higher professional suitability scores than those individuals with only one graduate degree in social work? Another question that remains to be answered is whether more education increases or decreases the level of professional suitability. Further study is encouraged to examine the above two questions related to the variable of academic qualification on one’s professional suitability.

Although the CFA provides evidence on the construct validity of the Professional Suitability scale, readers should be cautioned on several limitations of this study. Among others, the reader should also be cautioned that results of this study were drawn from one province and had low response rate, suggesting limited generalizability of specifically external validity (Rubin & Babbie, 2011). Moreover, the sample size was slightly below the requirement for CFA (Meyers et al., 2006; Tabachnick & Fidell, 2007). Multiple mailings (Dillman, 2007) to increase response rate and sample size is recommended for future validation studies. Furthermore, demographic characteristic of the population in this study was unavailable, and it remains unclear whether the sample is representative of the population, further limiting generalizability of results. In addition, the data of this study were collected from a cross-sectional survey with no criterion variable included to be tested for convergent validity (Kane, 2006; Nunnally & Bernstein, 1994). This is to say that this Professional Suitability scale has yet been tested on its stability reliability and convergent validity. Further to that, there is an error covariance between Item 26 and Item 40. Examining these 2 items through item analysis to determine their inclusion, deletion, or rewording is suggested for future study (Harrington, 2009; Meyers et al., 2006).

Other directions for future research is that results of this study suggest that the latent variables of ethical, practice and personal suitability were strongly correlated with covariances ranged between .80 and .97. These results suggested that these three subscales fail to adequately discriminate from each other and are possibly measuring similar construct. One possible option is to combine these subscales and eliminate redundant items in future studies. In addition, the maximum likelihood estimation is very robust and does not take into account of ordinal and nonnormal data. Use of other statistical procedures (i.e., asymptotic distribution free estimation) is suggested for future studies. Moreover, the model fit index of GFI valued at .87, which was … instead of the excellence of .80. These two marginal figures suggest further work to strengthen the scale is warranted such as expanding the items on the Personal Suitability subscale. Furthermore, the final 22-item four-factor model is based on the deletion of 9 items with low SMC and MI. This final model requires further validation. Finally, the Professional Suitability scale has not been tested with student populations to determine the scale’s reliability and validity for decision making on student admission, retention, or graduation.

In summary, scale development requires multiple validation studies with large and representative samples to affirm its reliability and validity. This is the first of its kind to examine construct validity of the Professional Suitability scale with the entire population of registered social workers in a province. There are valuable experiences learned from this study. Among others, the scale in this study is built upon previous studies and has moved beyond asking individuals’ opinion on professional suitability to testing its applicability for self-assessment purpose by using a stem stating “I consider myself …” Despite limitations discussed above, results of this study affirm consistent internal consistency of the Professional Suitability scale with previous studies in Alberta and Ontario. Results of CFA testing show acceptable evidence on the four constructs of professional suitability. These results of good–excellent internal consistency and acceptable construct validity support the potential use of this scale as a self-assessment instrument. For instance, the scale could be used as one of the indicators for hiring decision and/or job performance evaluation in order to determine whether one has achieved the required knowledge, skills, and values for professional social work practice (Bogo, 2010; Leigh et al., 2007; Rubin et al., 2007). Benchmark setting at different stages of one’s employment could also be used to promote one’s strengths and set career goals for future professional development. Moreover, the identified factor structure and indicators could be used as a cross-reference for the Canadian Council of Social Work Regulators, which mandates to develop some national mechanisms to measure professional competence.

Footnotes

Acknowledgments

The authors would like to thank the assistance from the Saskatchewan Association of Social Workers and the diligent research assistance work of Xiao Mei and Ze Hong Zhuang. Many thanks, also, to Heather Coleman for her critical review on the early drafts of this article.

Authors’ Note

Dora M. Y. Tam is a visiting professor at the Department of Applied Social Sciences, Hong Kong Polytechnic University, China. Robert C. Twigg and Kam-Wing Boey have retired from their affiliated institutions.

Declaration of Conflicting Interests

The authors declared no conflicts of interests with respect to the authorship and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Internal Research Grant of the University of Regina.

References

Alperin

D. E.

(1996). Empirical research on student assessment in field education: What have we learned? The Clinical Supervisor, 14, 149–160.

Barlow

Coleman

(2003). Suitability for practice guidelines for students: A survey of Canadian social work programs. Social Work Education, 22, 151–165.

Bogo

(2010). Achieving competence in social work: Through field education. Toronto, ON: University of Toronto Press.

Bogo

Regehr

Katz

Logie

Mylopoulos

(2011). Developing a tool for assessing students’ reflections on their practice. Social Work Education, 30, 186–194.

Bogo

Regehr

Power

Globerman

Hughes

(2002). Evaluating a measure of student field performance in direct service: Testing reliability and validity of explicit criteria. Journal of Social Work Education, 38, 385–401.

Bogo

Regehr

Power

Hughes

Woodford

Regehr

(2004). Towards new approaches for evaluating student field performance: Taping the implicit criteria used by experienced field instructors. Journal of Social Work Education, 40, 417–426.

Bogo

Regehr

Woodford

Hughes

Power

Regehr

(2006). Beyond competencies: Field instructors; descriptions of student performance. Journal of Social Work Education, 42, 579–593.

Brear

Dorrian

Luscri

(2008). Preparing our future counseling professionals: Gatekeeping and the implications for research. Counselling and Psychotherapy Research, 8, 93–101.

Brown

T. A.

(2006). Confirmatory factor analysis for applied research. New York, NY: The Guilford Press.

10.

Canadian Association for Social Work Education. (2011). Revised standards for accreditation. Ottawa, ON: Author.

11.

Canadian Association for Social Work Education. (2012). List of students and faculty numbers 2011–2012. Ottawa, ON: Author.

12.

Canadian Association of Social Workers. (2005). Code of ethics. Ottawa, ON: Author.

13.

Carniol

(2005). Case critical” Social services and social justice in Canada (5th ed.). Toronto, Canada: Between the Lines.

14.

Carpenter

(2011). Evaluating social work education: A review of outcomes, measure, research designs and practicalities. Social Work Education, 30, 122–140.

15.

Carraccio

Wolfsthal

S. D.

Englander

Ferentz

Martin

(2002). Shifting paradigms: From Flexner to competencies. Academic Medicine, 77, 361–367.

16.

Council on Social Work Education. (2010). Educational policy and accreditation standards. Washington, DC: Author.

17.

Cuyvers

(2009). Evolving to competence based social work training. Alternatives. Cuadernos de Trabajo Social, 16, 29–42.

18.

Dickens

(2011). Social work in England at a watershed-As always: From the Seebohm report to the social work task force. British Journal of Social Work, 41, 22–39.

19.

Dillman

D. A.

(2007). Mail and internet surveys: The tailored design method (2nd ed.). New York, NY: John Wiley.

20.

Durkheim

(1933). De la division du travail social [The division of labor in society] (W. D. Halls, Trans., 1997). New York: Free Press.

21.

Epstein

R. M.

Hundert

E. M.

(2002). Defining and assessing professional competence. Journal of the American Medical Association, 287, 226–235.

22.

Eraut

(1994). Concepts of competence and their implications. In Eruat

(Ed.), Developing professional knowledge and competence (pp. 163–181). London, England: The Falmer Press.

23.

Falender

C. A.

Shafranske

E. P.

(2012). The importance of competency-based clinical supervision and training in the twenty-first century: Why bother? Journal of Contemporary Psychotherapy, 42, 129–137.

24.

Gibbons

Bore

Munro

Powis

(2007). Using personal quality assessment of selection of social work students. Australian Social Work, 60, 210–221.

25.

Gibbs

(1994). Screening mechanisms in BSW programs. Journal of Social Work Education, 30, 63–74.

26.

Gibbs

(2000). Screening out of BSW programs: Responding to professional obligations and institutional challenges. In Gibbs

Blakely

E. H.

(Eds.), Gatekeeping in BSW programs (pp. 149–168). New York, NY: Columbia University Press.

27.

Gibbs

Blakely

E. H.

(2000). Academic standards for admission and retention. In Gibbs

Blakely

E. H.

(Eds.), Gatekeeping in BSW programs (pp. 259–278). New York, NY: Columbia University Press.

28.

GlenMaye

Oakes

(2002). Assessing suitability of MSW applicants through objective scoring of personal statements. Journal of Social Work Education, 38, 67–82.

29.

Gorsuch

R. L.

(1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

30.

Hager

(2004). The competence affair: Or, why vocational education and training urgently need a new understanding of learning. Journal of Vocational Education and Training, 56, 409–433.

31.

Harrington

(2009). Confirmatory factor analysis. New York, NY: Oxford University Press.

32.

Holosko

Leslie

D. R.

(2001). Is social work a profession? The Canadian response. Research on Social Work Practices, 11, 201–209.

33.

Horder

(1998). Comptence(s) without tears? Social Work Education 17, 117–120.

34.

Hudson

W. W.

(1982). The clinical measurement package: A field manual. Chicago, IL: The Dorsey Press.

35.

Jackson

D. L.

Gillaspy

J. A.

Jr Purc-Stephenson

(2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14, 6–23.

36.

Kane

M. T.

(2006). Validation. In Brennan

R. L.

(Ed.), Educational measurement (4th ed., pp.17–64). Westport, CT: American Council on Education and Praeger.

37.

Kaslow

N. J.

Rubin

N. J.

Bebeau

M. J.

Leigh

I. W.

Lichenberg

J. W.

Nelson

P. D.

… Smith

I. L.

(2007). Guiding principles and recommendations for the assessment of competence. Professional Psychology: Research and Practice, 38, 441–451.

38.

Kimberly

Osmond

(2009). Social workers. In Turner

J. C.

Turner

F. J.

(Eds.), Canadian social welfare (6th ed., pp. 353–374). Toronto, Canada: Pearson Education Canada.

39.

Kline

R. B.

(2005). Principles and practice of structural equation modeling (2nd ed.). New York, NY: The Guilford Press.

40.

Koerin

Miller

L. S.

(1995). Gatekeeping policies: Terminating students for non-academic reasons. Journal of Social Work Education, 31, 247–260.

41.

Lafrance

Gray

Herbert

(2004). Gatekeeping for professional social work practice. The Journal of International Social Work Education, 23, 325–340.

42.

Leigh

I. W.

Smith

I. L.

Bebeau

M. J.

Lichetnberg

J. W.

Nelson

P. D.

Portnoy

… Kaslow

N. J.

(2007). Competency assessment models. Professional Psychology: Research and Practice, 38, 463–473.

43.

Lymbery

(2011). Building a safe and confident future: One year on-reflections from the world of higher education in England. Social Work Education, 30, 465–471.

44.

Lyons

(1999). Social work in higher education: Demise or development. Aldershot, England: Ashgate Publishing.

45.

MacDonald

K. M.

(1995). The sociology of the professions. London, England: Sage.

46.

MacKenzie

(2002). Historical overview and current issues in social work regulation. In Turner

F. J.

(Ed.), Social work practice: A Canadian perspective (2nd ed., pp. 491–503). Toronto, Canada: Pearson Education Canada.

47.

Malone

Supri

(2012). A critical time for medical education: The perils of competence-based reform of the curriculum. Advance in Health Science Education, 17, 241–246.

48.

Martinez-Brawley

E. E.

Zorita

P. B.-M.

(2007). Tacit and codified knowledge in social work: A critique of standardization in education and practice. Families in Society, 88, 534–542.

49.

McCauley

T. P.

(2010). A structural functional approach to the study of work and professions. In Bosanac

S. E.

Jacobs

M. A.

(Eds.), Work, occupations and professionalism (pp.53–72). Whitby, ON: de Sitter.

50.

Meyers

L. S.

Gamst

Guarino

A. J.

(2006). Applied multivariate research: Design and Interpretation. Thousand Oaks, CA: Sage.

51.

Miller

Koerin

(1998). Can we assess suitability at admission? A review of MSW application procedures. Journal of Social Work Education, 34, 437–453.

52.

Moore

L. S.

Jenkins

D. A.

(2000). The history of gatekeeping. In Gibbs

Blakely

E. H.

(Eds.), Gatekeeping in BSW programs (pp. 45–59). New York, NY: Columbia University Press.

53.

Mulder

Gulikers

Biemans

Wesselink

(2009). The new competence concept in higher education: error or enrichment? Journal of European Industrial Training, 33, 755–770.

54.

National Association of Social Workers. (2008). Code of ethics. Washington, DC: Author.

55.

Nunnally

Bernstein

I. H.

(1994). Psychometric theory (2nd ed.). New York, NY: McGraw-Hill.

56.

O’Hagan

(Ed.). (1996). Competence in social work practice: A practical guide for professionals. London, England: Jessica Kingsley.

57.

Reeser

L. C.

Epstein

(1990). Professionalization and activism in social work: The sixties, the eighties, and the future. New York, NY: The Columbia University Press.

58.

Regehr

Bogo

Donovan

Lim

Regehr

(2012). Evaluating a scale to measure student competencies in macro social work practice. Journal of Social Service Research, 38, 100–109.

59.

Regehr

Bogo

Regehr

(2011). The development of an online practice-based evaluation tool for social work. Research on Social Work Practice, 21, 469–475.

60.

Roberts

M. C.

Borden

K. A.

Christiansen

M. D.

Lopez

S. J.

(2005). Fostering a culture shift: Assessment of competence in the education and careers of professional psychologists. Professional Psychology: Research and Practice, 36, 355–361.

61.

Rossiter

Heron

(2011). Neoliberalism, competencies, and the devaluing of social work practice. Canadian Social Work Review, 28, 305–309.

62.

Rubin

Babbie

(2011). Research methods for social work (7th ed.). Belmont, CA: Brooks/Cole.

63.

Rubin

N. J.

Bebeau

Leigh

I. W.

Lichtenberg

J. W.

Nelson

P. D.

Portnoy

… Kaslow

N. J.

(2007). The competency movement within psychology: An historical perspective. Professional Psychology: Research and Practice, 38, 452–462.

64.

Ryan

Habibis

Craft

(1997). Guarding the gates of the profession: Findings of a survey of gatekeeping mechanisms in Australian bachelor of social work programs. Australian Social Work, 50, 5–12.

65.

Schmeiser

C. B.

Welch

C. J.

(2006). Test development. In Brennan

R. L.

(Ed.), Educational measurement (4th ed., pp. 307–353). Westport, CT: American Council on Education and Praeger.

66.

Stevens

(2002). Applied multivariate statistics of the social sciences (4th ed.). Mahwah, NJ: Lawrence Erlbaum.

67.

Stoesz

Karger

H. J.

Carrilio

(2010). A dream deferred: How social work education lost its way and what can be done. New Brunswick, NJ: Aldine Transaction.

68.

Sultana

R. G.

(2009). Competence and competence framework in career guidance: Complex and contested concepts. International Journal Education on Vocational Guidance, 9, 15–30.

69.

Tabachnick

Fidell

(2007). Using multivariate statistics (5th ed.). New York, NY: Harper Collins.

70.

Tam

D. M. Y.

(2003). Gatekeeping in BSW field education. Journal of Women in Welfare Education, 6, 51–62.

71.

Tam

D. M. Y.

(2004). Gatekeeping in baccalaureate of social work (BSW) field education (Unpublished doctoral dissertations). University of Calgary, Alberta, Canada.

72.

Tam

D. M. Y.

Coleman

(2009a). Defining criteria on professional suitability for social work practice. Journal of Baccalaureate Social Work, 14, 105–121.

73.

Tam

D. M. Y.

Coleman

(2009b). Construction and validation of the professional suitability for social work practice scale. Journal of Social Work Education, 45, 47–63.

74.

Tam

D. M. Y.

Coleman

Boey

K.-W.

(2012). Professional suitability for social work practice: A factor analysis. Research on Social Work Practice, 22, 227–239.

75.

Tam

D. M. Y.

Kwok

S. M.

(2007a). Challenges in undergraduate social work field supervision. International Journal of Interdisciplinary Social Sciences, 2, 345–353.

76.

Tam

D. M. Y.

Kwok

S. M.

(2007b). Controversy debate on gatekeeping in social work education. International Journal of Learning, 14, 195–203.

77.

Tam

D. M. Y.

Kwok

S. M.

(2007c). The premise of an educational continuum in social work education. The International Journal of Learning, 14, 47–56.

78.

Tam

D. M. Y.

Twigg

R. C.

Margrett

(2009). Re-think the pedagogy of social work field education. Unpublished paper presented at the Annual Conference of Canadian Association for Social Work Education, Ottawa, Canada.

79.

Toren

(1972). Social work: The case of semi-profession. Belverly Hills, CA: Sage.