Abuses and Mysteries at the Association of Social Work Boards

Money

ASWB’s social work exams are not cheap, costing roughly US$250 per person per exam. As a monopoly, ASWB has considerable latitude to charge excessive fees. When asked why the exams had become so expensive, ASWB’s then-Executive Director Donna DeAngelis blamed a recent price hike on “increases in examination development, security, and administration costs” (Marson, 2010, p. 24). That was an interesting way to phrase the fact that, according to ASWB’s (2011, p. 11) IRS Form 990 filing, its net assets would increase by US$1 million in 2011.

That was a lot of money, but apparently not enough. It seems this allegedly not-for-profit organization wished to return to the US$1.5 million annual increase in its net assets that it had achieved the previous year. The price increase did the trick and then some. Net assets rose more than US$1.6 million in 2012, leaving ASWB (2012, p. 11) with more than US$10 million in, essentially, cash. Note that ASWB is a small organization with a narrow purpose. The Form 990 (ASWB, 2012, Schedule D, p. 3) indicates that an unspecified amount of that US$10 million was invested in, of all things, financial derivatives. ASWB’s net assets, furnished overwhelmingly by exam fees, rose 62% in the 4 years between year-end 2008 and year-end 2012 (compare ASWB, 2009, p. 11 with ASWB, 2012, p. 11).

This article does not attempt a thorough review of ASWB’s Form 990 filings and other financial documentation. Readers who peruse the sources cited above, and subsequent updates, may find a variety of additional insights supplementing the foregoing brief introduction.

Validity

In an interesting study, Albright and Thyer (2010) gave first-year master of social work (MSW) students an ASWB Clinical practice test. That practice test was apparently offered by ASWB as a fair indicator of the kinds of questions that test takers would encounter in the Clinical exam, consisting of questions that had actually been used on prior ASWB exams. In their study, Albright and Thyer removed the questions from the examination, so that the students could see only the four multiple-choice answers. Students had to guess which might be the right answer to a question that was not available to them. Randomly guessing would have resulted in a correct-guess rate of 25% (i.e., one out of four). But these students did not get 25% correct. They did much better than that: they averaged 52% correct responses. They were able to guess which answer was the right one, more than half of the time, just from the way the answers were phrased. In other words, the test did not accurately measure test takers’ knowledge. On this basis, Albright and Thyer (p. 229) suggested that the ASWB Clinical exam “cannot justifiably be claimed to be a valid assessment of competence to practice social work.”

That was embarrassing for ASWB because, at about the same time, DeAngelis was coauthoring an article (Marson, DeAngelis, & Mittal, 2010, pp. 98–99) in which she claimed this:

The central focus of this article has been to illustrate reliability and validity processes, absolutely essential when the successful completion of such an evaluation instrument determines whether or not someone can work in a profession. These very high stakes require careful, responsible, and ethical construction of such instruments. ASWB has consistently shown over the years that it executes its responsibility regarding test construction and administration according to established psychometric theory and practice, and with adherence to social work ethics.

This was hardly the first time that ASWB had been warned of deficiencies in its exam development and testing processes. For example, Collins, Coleman, and Miller (2002, p. 209, citations omitted) had said this:

Concern has been raised about the validity and reliability of licensing exams. Although the Association of Social Work Boards reports that the exams are valid and reliable, the psychometric properties of these exams are not widely available. Criticism of licensing exams is disturbing. For example, several authors have challenged their content and discriminative validity. Others also argue that written tests for social work licensing fall short in measuring knowledge unique to the profession. For example, one study suggests that passing licensing exams is correlated more with the candidate’s personal therapy experience rather than fieldwork, work experience, graduate or undergraduate education, or clinical training. Many also claim that the exams fail to predict practice competence. Moreover, [one study] found that social work experience, education, and number of credit hours in social work have no significant correlation to examination scores. A further disturbing issue is whether exams discriminate on the basis of race and ethnicity.

High-Stakes Testing at ASWB

As emphasized in the foregoing remarks by Marson, DeAngelis, and Mittal (2010), ASWB’s social work exams are high-stakes in a double-edged sense: they are all-or-nothing for the test taker, and they also embody enormous claims, by ASWB, regarding its expertise in deciding who qualifies to join the next generation of social workers. The use of high-stakes testing within the social work profession is problematic in itself. As detailed by Woodcock (2014b), such testing has drawn criticism for its risks of mismeasurement, cheating, social injustice, and stigmatization. It has also seemed odd that MSWs would be screened by a high-stakes licensing test after completing MSW programs that, in many cases, do not require the high-stakes Graduate Record Examination (GRE) for admission. Given ASWB’s assertions that its exams sought to test only a fraction of the many topics covered in social work degree programs, Woodcock suggested that the stakes might be appropriately lowered by testing applicants before they devote years to such programs. Such testing might demonstrate that many are already prepared to enter licensed practice. For those passing a licensure exam at the outset, there might be no need to invest years, and great expense, in university programs covering additional subjects that, according to ASWB, are not essential for such practice.

In addition, there were grounds for concern with the process by which ASWB developed its licensing exam questions. Based on an extensive analysis of that process, Woodcock (2014c) concluded that ASWB’s occasional surveys of social workers were unlikely to provide accurate impressions of what social work practice is; that the results of those surveys did not appear to have much impact on exam questions; that the output of ASWB’s cumbersome question-generation process appeared approximately equal to the output of a single social work PhD working part time; and that Pearson VUE, on contract with ASWB, appeared to be providing the bulk of the infrastructure and expertise needed for development and administration of ASWB exams, at a fraction of the price per exam that ASWB charged its examinees.

Concealment of Testing Data

In the research cited above, Albright and Thyer (2010, p. 233) concluded, “The professional stakes are too high for anything less than the scientific transparency provided by both internal and independently conducted and published psychometric evaluations of ASWB examinations.” They called upon ASWB to disclose the data that independent examiners would need, in order to verify that ASWB exams really measured what they were supposed to measure. Among other things, Albright and Thyer may have wanted researchers to be able to look at the ASWB data reporting what happened during trial uses of specific questions. It appeared that the average test taker would have tended to guess right when answers were worded one way but might not do so with other wordings. One would want to know whether ASWB tended to select questions and answers on which examinees would achieve a relatively high success rate.

Of course, ASWB did not need Albright and Thyer to provide basic instruction in the proper construction and testing of standardized exams. ASWB had been at it for a long time. As noted above, ASWB had its own journal; moreover, it employed a well-paid director of examinations (ASWB, 2010a, p. 7). In case of doubt, ASWB could also have consulted people at Pearson VUE as well as any number of statistically trained social work professors. So the problem was not that ASWB may have made a technical error on an esoteric detail. It was that ASWB was charging thousands of people millions of dollars, each year, to subject them to an exam that the majority might be able to pass before they had even finished the first year of their MSW programs. That problem raised issues at the heart of ASWB’s business—and as noted by Collins et al. (2002) and others (above), that problem had been ignored for a long time.

An Explanation of Test Data Concealment

It would have been easy enough to dispel such concerns by releasing relevant exam data. But in 2014, as in 2002, ASWB was unwilling to do that. Needless to say, confidence in ASWB’s pricey exams was hardly enhanced by a persistent refusal to let researchers verify that exam questions were being properly formulated and tested. The refusal to release exam data raised the question of whether ASWB was providing a genuine service of statistical testing and analysis, as distinct from a profitable hoax. Given the latitude to decide how many correct answers are needed to pass, experts in testing would be able to devise an exam yielding a predetermined pass rate (of, say, 80%) even if the questions presented to MSWs had nothing to do with professional practice.

In other words, there was potential embarrassment in ASWB’s claim that it provided a legitimate testing service when that claim was not backed up by public data. And yet ASWB was not embarrassed in fact: nobody forced the issue. For many years, the profession of social work, the well-being of clients nationwide, and those millions of dollars, have all depended upon a general willingness to assume that ASWB knows what it is doing – or in some cases, perhaps, upon tacit if not complicit awareness that ASWB is not necessarily doing what it should be doing.

Why would ASWB not simply give researchers access to its test data, so as to put the matter to rest? That question became more interesting with the discovery (above) that ASWB paid large sums annually to the testing experts at Pearson (2013), which describes itself as the world’s largest commercial testing company. Presumably Pearson’s statisticians were working hard to deliver a sophisticated product conforming to ASWB’s specifications. But what were those specifications—what kind of exam was ASWB telling Pearson to produce?

An explanation for ASWB’s secrecy arose in Woodcock’s (2014c) analysis of ASWB’s question-development process. Woodcock argued that, consistent with sweeping claims about the breadth of social work practice (e.g., National Association of Social Workers[NASW], 2008), ASWB had to assert that it was testing a vast spectrum of topics—indicating, for example, that persons taking ASWB’s Masters exam would be examined on 191 areas of knowledge, skill, and ability (see ASWB, 2014b). Given that the exam consisted of only 150 scored questions and that many of those 191 areas constituted entire fields of study within themselves (e.g., “pharmacology”; “human genetics”), ASWB appeared to be obliquely acknowledging that the general practice of social work did not seem to draw upon a reasonably bounded body of knowledge.

In short, Woodcock (2014c) suggested, ASWB was postulating an impossible exam. Even if it had been theoretically feasible to develop a 4-hr exam that could test those many areas in any meaningful sense, the ASWB people who actually prepared exam questions were visibly lacking in the extreme breadth of knowledge that would have been needed to write questions covering those areas. And if those people had been able to develop such an exam, nobody would have passed it. MSW programs do not even offer course work in many of the areas supposedly being tested. Hence, thousands of MSW program graduates would have failed, and their state licensing boards would have deemed them incompetent to practice. That would have drawn public attention to the quality of education in social work programs. ASWB evidently decided to avoid any such scandal. Woodcock argued, in other words, that by its own admission (above) ASWB did not try to facilitate a rational progression from (a) a focused program of study in a social work program through (b) a licensing exam targeted on the subjects studied in such a program to (c) a professional practice of social work centered on what had thus been studied and tested.

In Woodcock’s (2014c) argument, then, it appeared that ASWB found itself presented with an opportunity and a simple if bizarre problem. In order to reap a fortune in testing fees, ASWB had to come up with an exam development process that would not be patently absurd. Concealment and obfuscation would be helpful tools to that end. There would be a genuine need for Pearson’s statistical analysis: the process would have to function in a steady and predictable manner, yielding a predetermined pass rate—not too high, lest the exam appear a joke; not too low, lest potential future applicants to schools of social work lose hope and go elsewhere.

This hypothesis suggested that the findings of Albright and Thyer (2010) were due to deliberate effort, not ineptitude, at ASWB and Pearson. It seemed that ASWB wanted a test that approximately 80% of MSW graduates could pass, and with Pearson’s help ASWB achieved that objective. It was sufficient, for this purpose, to confront exam takers with questions about rather arbitrarily selected aspects of social work practice, as long as answers were worded so that people would tend to guess the preferred answers at the desired rate. Consistent with the foregoing remarks by Collins et al. (2002), Woodcock (2014d) illustrated that ASWB’s preferred answers could be arbitrary as well—that, within those questions, a well-educated social work student might detect issues to which the question writers appeared oblivious.

Such a scenario suggested that ASWB’s selections of exam questions and design of answers were steered primarily by a determination to maintain a profitable facade. Given years of refusal to facilitate independent verification of testing data, it appeared that ASWB lacked confidence that its testing could distinguish competent from incompetent practitioners. As noted below, the refusal to cooperate with researchers may be seen in itself as unethical. That refusal positioned ASWB as a barrier to the improvement of social work practice and client service.

Concealment of Ordinary Information

The preceding sections have discussed ASWB’s withholding of test statistical data from professional researchers. Some might be tempted to view that as a relatively arcane issue, notwithstanding its implications for social work practice. Such a view might have superficial plausibility, were it not for a more obtrusive deficit in ASWB’s face to the public.

That deficit may be summarized as follows. A reputable nonprofit organization, possessing substantial amounts of money, a gold mine of data on social work licensing examinations, and a desire to serve the public, would be pouring out reams of information intended to inform the profession and to improve the licensing of practitioners. There would definitely be a place for advanced researchers, but that sort of expertise would not be absolutely necessary: anybody who had that data and who knew how to use a spreadsheet could provide very helpful guidance on a plethora of issues.

Casual visitors would know immediately if they were encountering a website like that. Among other things, there would be a visible commitment to make sure that members of the public, social work clients, and social work practitioners were not being inappropriately harmed by inadvertent (never mind deliberate) manipulation or withholding of important information.

Unfortunately, as of this writing, there appeared to be no such passion or commitment in ASWB’s (2014a) website. It would have been impressive and reassuring to find, for example, that ASWB was indicating how many people took its graduate-level exams across a number of recent years; how many passed in each year; which states and schools they were from; what their undergraduate majors and GRE scores had been; whether people from different regions, sexes, ethnicities, or income levels had scored differently; what kinds of questions seemed to be best and worst suited for relevant purposes; how ASWB’s own exams had changed from year to year; what had prompted such changes; and so forth. There would be data tables to download, reports to read, forums, and discussions. It would be a public service.

Nothing could be farther from what the user encountered on ASWB’s website as of July 2014. There was virtually no information of that sort. What was provided appeared to constitute the bare minimum. If ASWB was not actually trying to hide something, it certainly seemed to be doing everything it could to convey that impression. The withholding of information was rather extreme. For instance, ASWB’s annual reports did not even state the numbers of people who took its different exams (e.g., Masters, Clinical): the 2012 annual report provided only percentages, and the 2010 annual report did not even do that. As of this writing, ASWB’s website did not report a combined total pass rate for any year other than 2013. Older links to ASWB’s reports of pass rates in previous years—including links found in old issues of ASWB’s own Association News, such as the March/April 2010 issue (ASWB, 2010b), referring to 2009 pass rates—now produced an error message, “Sorry, there is no page or article at this location.”

It might have been possible to look up previous years’ pass rates in old copies of ASWB’s annual reports. Unfortunately, all annual reports other than the most recent year’s had also been stripped from ASWB’s website. The error message quoted above appeared, again, in response to an attempt to reach the web address where the 2010 annual report had previously been located (ASWB, 2010c). Searches using (both Google and the ASWB website’s own search box) confirmed that previous years’ annual reports were gone.

This was unusual. Similar searches would still pull up prior years’ annual reports at the websites of other major social work organizations (e.g., NASW, Council for Social Work Education [CSWE]). Removal of annual reports, when Form 990 filings would continue to be available, raised a question of whether ASWB had made a deliberate effort to remove historical information about itself from the practical reach of students, members of the public, or otherwise novice or desultory researchers.

Yet another example of providing no more than the bare minimum amount of information: consider the results of a search of ASWB’s website, seeking statements that would distinguish the purposes of its several exams. That search yielded no clear distinction of content between, for example, the Masters and the Advanced Generalist exams. Remarkably, ASWB’s (2013a) “About the Exams” webpage said virtually nothing about the exams. Its Exam Content Outlines page (ASWB, 2013b) did lead to an Advanced Generalist Content Outline—which launched immediately into another long list of topics covered, with no substantive explanation of the exam’s purpose or of concepts that could have linked such disparate topics.

ASWB did not explain its decision to withhold practical, useful information from examinees and the public. It seemed there could be several reasons to minimize disclosures. Doing so might mitigate legal exposure for ASWB and its key executives. It might exacerbate test takers’ uncertainties, stimulating the purchase of expensive ASWB exam prep materials that, according to DeAngelis, could help to “ease test anxiety” (Marson, 2010, p. 25). Open engagement with the public might encourage unwanted questions about ASWB’s work. For example, at some point people might begin to challenge the notion that the so-called “Advanced Generalist” exam could have a specific purpose. It might appear that ASWB had dreamt up the Advanced Generalist exam merely to inaugurate a new income stream.

ASWB’s withholding of useful, available information did not appear consistent with an orientation toward helpfulness and public service that one might expect of a key organization in this helping profession. From the most generally useful public information to the most refined statistical issues, ASWB appeared to behave in ways that would be beneficial to its own insiders but detrimental to the social work profession.

Concealment of Schools’ Pass Rates

One aspect of the foregoing discussion deserves additional attention. Among the kinds of information that ASWB withholds, few are of greater interest than the exam pass rates achieved by graduates of specific schools of social work. It may be, for example, that more competitive universities attract students who are better at standardized exams, or that less expensive schools disproportionately attract disadvantaged students whose perspectives are underrepresented in ASWB exam questions. Exam takers sharing the worldviews of ASWB question writers might enjoy particular advantages, for purposes of guessing which answers those writers might prefer.

In this regard, Thyer (2011) offered a contrast between social work and law. In the legal profession, he pointed out, the bar exam pass rates achieved by graduates of various schools are openly compared against one another. Schools with poor performance records are at risk of closure. In social work, by contrast, Thyer observed that ASWB provides information on a specific school’s ASWB exam pass rate only to officers at that school, and only at a steep price (see ASWB, 2013c). Hence, broad comparisons across multiple schools will typically remain unavailable until all schools purchase and share pass rates obtained from ASWB.

Fortunately, Thyer (2011) found an exception. Under the sunshine laws of his state of Florida, it was possible to request LCSW pass rates from all Florida MSW programs for the previous several years. That request resulted in some dramatic insights. In his data from 2007, for example, pass rates varied from 42% at Barry University to 78% at the University of South Florida (USF). That contrast would obviously steer students toward USF. Then again, rates at USF dropped from that high of 78% in 2007 to 67% in 2008. If that decline continued, there could be a question of whether USF had made a change for the worse in some aspect of its MSW program after 2007. Thyer also observed that Florida programs as a whole did not compare well against the average national pass rate. These sorts of facts and comparisons could be valuable in identifying and rectifying problems—but only if the data are made widely available.

It goes without saying that data can be misused and misconstrued. For instance, it could be that Barry University tended to attract students who would not do as well on a standardized exam. Barry might then be unfairly stigmatized for taking a chance on students who would not even have gotten into USF. To quote CSWE (2013, p. 1), “ASWB exams…should not be used as the sole outcome measure for an educational program.”

At the same time, to quote Thyer (2011, p. 300), “[T]hese issues do not obscure the simple fact that licensure is an exceedingly important influence in the professional practice of our field. One might even characterize it as a driving influence.” The persistently low scores achieved by graduates of the Barry University social work program, in Thyer’s data, do not seem appropriate for suppression by ASWB, when graduates of other programs scored persistently higher. If Barry University has something to explain, it should explain it, or at least contribute to discussion on relevant considerations—not leave potential applicants in the dark.

ASWB’s website does not deign to discuss such matters. It simply continues to assert that its reports on pass rates will be made available, not to the public, but “only to the individual school of social work” (ASWB, 2013d).

Ethical Issues

This article has identified a number of regards in which ASWB appears to promote and profit from unethical behavior. First, there are several large-scale and continuing violations of the NASW (2008) Code of Ethics:

In addition, ASWB’s behavior in these regards appears to violate widely understood principles of research ethics. Those principles, summarized by Resnik (2011), include the following:

(See also e.g., American Psychological Association, 2010; Scientists for Global Responsibility, n.d.)

Of course, ethical concerns are not limited to ASWB. The NASW Code makes clear that those who facilitate or condone unethical behavior are themselves culpable. Among other things, the Code says, social workers elevate service to others above self-interest; they promote ethical practices on the part of the organizations with which they are affiliated; and they take adequate measures to prevent, expose, and correct unethical behavior by other social workers.

Conclusion

At this writing, the situation appeared to be as follows: ASWB managed to establish itself as the monopolist of social work professional testing; it used that position to charge exorbitant prices of people who, in many cases, can ill afford to pay them; and for many years, it ignored reasonable calls for change and improvement. Improvement seemed to be especially needed in ASWB’s withholding of valuable information and data, at the expense of clients, exam takers, and other stakeholders. That blatantly unethical and yet broadly tolerated withholding of information illustrated the sharp contrast between the corruption of today’s social work profession and the alternative of genuine social work (see Woodcock, 2011).