Abstract
Program accreditation is a process based on a set of professional expectations and standards meant to signal competency and credibility. Although accreditation has played an important role in shaping educational leadership preparation programs, recent revisions to accreditation processes and standards have highlighted attention to the purposes, designs, and impacts of accreditation. This article examines programmatic accreditation in other professional disciplines to identify themes and trends that may have implications for the design of accreditation for educational leadership programs.
Programmatic accreditation in education is in a period of dynamic transformation. In 2013, the Teacher Education Accreditation Council (TEAC) and the National Council on the Accreditation of Teacher Education (NCATE) merged to create the Council for the Accreditation of Educator Preparation (CAEP). Educational leadership preparation programs are reviewed for accreditation in conjunction with CAEP (previously by NCATE) by a specialized professional association, the Educational Leadership Constituent Council (ELCC). Along with the merger, the educational leadership professional standards that provide the foundation for accreditation review, the Interstate School Leadership Licensure Consortium (ISLLC) standards, are also being revised. This provides a unique historical opportunity to consider changes to accreditation practice in educational leadership preparation programs.
To help inform current discussions about accreditation of educational leadership education, we follow the lead of other researchers who have examined programs outside of educational leadership for models that could help to inform leadership preparation and practice (see, for example, Tucker & Codding, 2002). In this chapter, we examine the context and evidence regarding program accreditation in several professional fields. By looking at the issues and evidence across professional fields and accrediting agencies, we hope to identify critical contextual or design features that could inform accreditation for educational leadership preparation programs. Specifically, we address the following research question:
To examine this question, we begin with an overview of the broader context of institutional and programmatic accreditation in the United States, including recent controversies and debates about the role of accreditation and its relationship to government regulation and support.
Institutional and Programmatic Accreditation
Accreditation serves as a form of self-regulation or peer review of U.S. higher education institutions and programs, designed to assure or improve academic quality (Eaton, 2012b). The Council on Higher Education Accreditation (CHEA) and the U.S. Department of Education (USDE) recognize 80 institutional and programmatic accrediting organizations. Specifically, seven regional accrediting organizations accredit public and private 2- and 4-year higher education institutions, four national faith-related accrediting organizations focus on religious-based institutions, seven national career-related organizations accredit for-profit career-based institutions, and 62 programmatic accrediting organizations accredit specific academic and professional programs. In 2010-2011, these accrediting organizations reported collecting US$114 million from member institutions to conduct institutional and programmatic reviews of 4,600 programs in over 3,000 institutions (CHEA, 2011; Eaton, 2012a).
According to the president of the CHEA (Eaton, 2012a), in this model,
Accreditation is both built on and reflective of the core values of the academy: peer review, the centrality of mission, institutional autonomy, and academic freedom. Peers judge institutional quality. . . . Review of quality is collegial, primarily qualitative, formative, and focused on improvement. Accreditation both requires and supports institutional autonomy and self-determination in making academic judgments concerning curriculum, faculty, and academic standards. It is committed to academic freedom—to assuring that faculty have appropriate discretion with regard to what is taught, who is taught, who teaches, and what standards are applied. (p. 9)
Accreditation typically involves a self-study and peer review, including a site visit from a study team. It aims to serve as an assurance of institutional or programmatic quality, to qualify institutions for eligibility for federal and state funds, to assist the private sector in making decisions about job applicants or private giving, and to ease the transfer of credits for students who transfer or take courses at more than one institution (Eaton, 2012b).
Because of its relationship to student aid eligibility, the accrediting decision has a significant impact on institutional viability. Since the 1950s, the U.S. government has recognized the decisions of specific private accrediting agencies to certify institutional quality for eligibility for federal student aid dollars. 1 The 1992 reauthorization of the Higher Education Act strengthened the regulatory role for accrediting agencies. The Act also created the National Advisory Committee on Institutional Quality and Integrity (NACIQI), the federal regulatory body responsible for certifying accrediting organizations (Ewell, 2015).
Over time, growth in federal investment in student financial aid dollars, high student loan default rates, concerns about low graduation and placement rates in some institutions, and a growing federal involvement in education accountability have reinforced federal interest in the regulatory role for institutional accreditation (Schray, 2006). Critics of this shift note conflicts in serving a regulatory role while respecting academic values of peer review, institutional mission, autonomy, and academic freedom (Eaton, 2012a; Leaf & Burris, 2002).
In 2006, the Spellings Commission
2
promoted a more active federal role in shaping accreditation activities, including recommending that as a condition of approval, accrediting organization reviews be required to assess and publicly report student outcome data (Lederman, 2007). According to the Spellings Commission,
Accreditation agencies should make performance outcomes, including completion rates and student learning, the core of their assessment as a priority over inputs or processes. . . . In addition, this framework should require that the accreditation process be more open and accessible by making the findings of final reviews easily accessible to the public and increasing public and private sector representation in the governance of accrediting organizations and on review teams (United States Department of Education, 2006, p. 25).
The 2008 reauthorization of the Higher Education Opportunity Act further strengthened the regulatory connection between the federal government and accreditation by specifying requirements of accreditation review. In particular, accrediting agencies had to monitor institutional enrollments, require institutional student credit transfer policies, monitor distance learning enrollments, and strengthen public reporting. This increased regulatory role for accreditation has been a cause of concern for higher education accreditation agencies and advocates (American Council on Education [ACE], 2012; Eaton, 2008), and debates about changes to the public–private accreditation and regulatory partnership continue in discussions regarding the next reauthorization of the Higher Education Act (see American Council of Trustees and Alumni [ACTA], 2013; Association of Public and Land-Grant Universities [APLU], 2015; Greenberg, 2014; Hartle, 2011; Higher Education Act discussion by the ACE, at http://www.acenet.edu).
Thus far, we have examined some of the current controversies surrounding institutional accreditation, which provide important context for our examination of programmatic accreditation. Our specific interest is in informing accreditation for educational leadership preparation programs by examining the landscape of accreditation in other professional fields. This discussion of institutional accreditation provides a backdrop for understanding some of the controversies that may have implications for programmatic accreditation. The literature highlights tensions in defining the purposes of accreditation; the high-financial stakes for institutions to receive a positive accreditation review; accrediting agency incentives to maintain institutional membership; concerns about a lack of rigor; the potential chilling effect on innovation, cost, and burden of accreditation; and compliance response to accreditation standards (Ewell, 2015; United States Department of Education, 2006).
However, the literature also highlights positive aspects of accreditation. The current accreditation system has been in effect since the early 1900s and has arisen with the internationally respected system of higher education in the United States. The system has a strong relationship to academic values of peer review, academic freedom, and institutional autonomy, which are hallmarks of the U.S. higher education system (Eaton, 2012b). Accreditation provides an important external assurance of quality, and there are examples of instances where research suggests that accreditation has helped to modernize professional preparation, including in the field of educational leadership (Darling-Hammond, LaPointe, Meyerson, Orr, & Cohen, 2007).
Method
To examine this question, we look to the research on accreditation across several professional fields. Recognizing that accreditation processes do not develop in a vacuum, we draw from a historical perspective in choosing our cases—focusing on medicine, psychology, engineering, and law. We chose these specific fields due to their significant history with accreditation and the evidence base available to examine the impact of accreditation design and outcomes.
Medicine has a relatively long history with accreditation, and accreditation in medicine has been used to inform the structure of accreditation in psychology. Engineering (which is accredited at the undergraduate level) underwent a dramatic change in accreditation while law has experienced much slower reform. The accrediting body in law accredits schools while those in medicine, engineering, and psychology examine programs. As such, we were able to examine programmatic accreditation across a wide spectrum of models and levels to draw a broad array of evidence regarding the relationship of accreditation to context, design features, and impacts.
We located scholarship for the review by searching for peer-reviewed literature in relevant discipline-oriented databases (ERIC, Education Full Text, Health Source, Index to Legal Periodicals, MEDLINE, Health Source for medicine, PsycArticles, PsycCritiques, PsyInfo, Academic Search Premier, and Education Research Complete), using search terms such as accredit* and medic*, psych*, educ*, engin*, legal*, and law*. To not only supplement our search of discipline-oriented databases but to also inform our understanding of the broader debates around accreditation in higher education, we also examined Google and Google Scholar and used snowball sampling to identify sources such as conference papers, policy reports, and seminal works. We excluded literature that was not situated in the United States unless it addressed issues of globalization with implications for U.S. accreditation design.
Findings
Drawing from an historical perspective, we present the evidence as case studies of accreditation in each professional field. We use case study because the historical and political context of each profession played a role in shaping the design and impact of contemporary accreditation programs. While there was variability, a number of common themes and trends emerged across the diverse fields. We begin our review with an examination of accreditation in medicine.
Medicine
The American Medical Association (AMA) was created in 1847 and began categorizing medical schools in 1905 (Eaton, 2012a). At the turn of the 20th century, in response to early concerns about the quality of U.S. medical education, the AMA sought the help of the newly formed Carnegie Foundation for the Advancement of Teaching to conduct a study of medical schools and medical education. The resulting Flexner Report (Flexner, 1910) led to substantial changes in medical education and the closure of substandard medical schools (Cooke, Irby, O’Brien, & Shulman, 2010; Duffy, 2011). Accreditation, certification, and licensure systems were established within 10 years following Flexner (Cook, Irby, & O’Brien, 2010). Currently, the Association of American Medical Colleges and the AMA sponsor the Liaison Committee on Medical Education (LCME), which is the accrediting body for programs that lead to a doctorate of medicine (MD). The Accreditation Council for Graduate Medical Education (ACGME) accredits the vast majority of graduate medical training programs (such as internships, residencies, and fellowships) (ACGME, 2012; Boulet & Zanten, 2014).
LCME aims to spur quality improvement through standards that facilitate innovation and flexibility and support a continuous improvement model (Simon & Aschenbrener, 2005). The 12 standards include (a) mission, planning, organization, and integrity; (b) leadership and admission; (c) academic and key environment; (d) faculty preparation, productivity, participation, and policies; (e) educational resources and infrastructure; (f) competencies, curricular objectives, and curricular design; (g) curricular content; (h) curricula management, evaluation, and enhancement; (i) teaching, supervision, assets, and student and patient safety; (j) medical student selectivity, assignments, and progress; (k) medical student academic support, career advising, and educational resources; and (l) medical student health services, personal counseling, and financial aid services (LCME, 2015).
The process of accreditation review begins about 1½ year before a scheduled on-site review when an institution collects comprehensive data linked to the accreditation standards and provides evidence of compliance with each standard. Next, the institution “conducts a self-study, critically reviewing the educational program, describing its responses to past citations, analyzing its current level of compliance with the standards, and identifying areas of particular strengths as well as areas undergoing change” (Simon & Aschenbrener, 2005, p. 158). At the same time, medical students “prepare an independent analysis of the educational program, student services, and educational resources” (Simon & Aschenbrener, 2005, p. 158). Uniquely, before the survey visit, the institution is urged to address any weaknesses and document its efforts. Next, survey teams, which usually include a representative from LCME and four other members who have expertise in medical education, medical practice, research, and administration, conduct on-site reviews. The survey team is charged with ensuring that institutional resources address mission, goals, and learning objectives and can assess institutional effectiveness (Simon & Aschenbrener, 2005). The LCME also works with the ACGME to ensure that students are properly prepared for the next stage of their careers (LCME, 2015; Simon & Aschenbrener, 2005).
Graduate medical education, the period of clinical training in a specialty after graduation from medical school, is required to be licensed to practice (ACGME, 2012; Boulet & Zanten, 2014). The ACGME transitioned toward the use of educational outcomes in 1997 (Leach, 2004). Previously, accreditation required that programs show proof of satisfying a long list of requirements, but there was no real incentive to promote continuous improvement. This led some directors to question the value of accreditation (Goroll et al., 2004). Yet, identifying what outcomes to measure was a difficult task because it required isolating the commonalities in medical training across specialties (Lurie, Mooney, & Lyness, 2009). ACGME used a consensus approach, identifying 13 competencies through a review of the literature, which were clustered and ranked by the broader medical community. An advisory group reviewed the rankings and condensed the list to six competencies (Leach, 2004): patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice (Lurie et al., 2009). Approved in 1999, these six competencies formed the backbone of ACGME’s Outcomes Project (Lurie et al., 2009). Implementation of the Outcomes Project occurred in phases and was invitational. The development of general competencies, which span the specialties, has been emulated by other professional fields such as psychology.
Integrating these six competencies into training programs was a slow process that required a significant amount of time and developmental work (Batalden, Leach, Swing, Dreyfus, & Dreyfus, 2002; Goroll et al., 2004). A central challenge was measuring the competencies in a valid and reliable way. Epstein and Hundert (2002) examined the research on the reliability or validity of measures of competence in medicine and argued that common assessments may reliably test basic knowledge but may underemphasize key domains such as professionalism and interpersonal skills. Considering the six areas, the authors propose a definition of competence as “the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individual and community being served” (p. 224). Epstein and Hundert argued for reliable and valid “multidimensional” assessments to capture these marginalized domains. Likewise, Leach (2004) suggested that the assessment tools identified by Swing (2002) could be particularly helpful and applicable to other disciplines. In addition to cognitive tests, these tools also include direct observations of resident skills; evaluations by patients, peers, and professional colleagues; and portfolios of clinical and educational experiences. Yet, Lurie et al. (2009) conducted a meta-analysis of 56 studies and found little evidence that current measurement tools could validly assess each of the competencies independently.
In a study of medical education by the Carnegie Foundation for the Advancement of Teaching, Cooke et al. (2010) noted that
. . . over the past ten years, the ACGME has become increasingly activist. A transformation in the approach to assessment of residents, shifting the metrics from so-called time-and-process measures, in which satisfactory completion of residency is determined by counting the number of months a resident spent on various rotations, to a competency orientation is having profound effects on assessment programs and even on the structure and organization of residency training. (p. 117)
Recent reforms by the ACGME include the Milestones Project, which calls on specialties to isolate the milestones, or observable stages that reflect the development of the competencies (Caverzagie et al., 2013). In July 2013, assessment and reporting of these milestones became a required part of ACGME’s Next Accreditation System (Caverzagie et al., 2013).
Evidence of impact
Evidence of impact in accreditation is limited. However, there is some evidence of the link between accreditation of medical schools and quality improvement. Kassebaum, Cutler, and Eaglen (1997) reviewed site visit reports of 90 medical schools in the United States that had LCME accreditation surveys between July 1992 and June 1997. Although they were not always able to isolate the impact of the LCME from the role of foundations as the main driver of education reforms, the authors concluded that the LCME was “on the leading edge of improved education and evaluation in the nation’s medical schools” (Kassebaum et al., 1997, p. 1133). More recent evidence also implies a relationship between accreditation and program quality: A report by the Carnegie Foundation found that currently, the accreditation and licensing systems in medicine were ensuring baseline quality (Cooke et al., 2010). This evidence of impact created ripple effects outside of the field—influencing accreditation reforms in psychology.
Psychology
The American Psychological Association (APA), founded in 1892, began regulating professional psychology in the mid-1940s. In 1947, APA began accrediting clinical psychology after the Veterans Administration and the U.S. Public Health Service requested the identification of doctoral training programs that properly prepared practitioners to serve veterans (Fagan & Wells, 2000; Prus & Strein, 2011). By the early 1970s, APA had common criteria for professional psychology program accreditation regardless of specialty (Prus & Strein, 2011).
Borrowing from medicine, APA moved toward a competencies approach that applied across specialties. As Kaslow and colleagues (2004) highlighted, “There was an increasing realization that the identification, training, and assessment of competencies that define a professional psychologist enable the profession to communicate to the public and legislatures about services psychologists provide” (p. 701). According to the APA Task Force (2006), competencies “involve the whole person, are teachable, observable, measurable, containable, practical, derived by experts, flexible and transferable across settings, and continually re-evaluated and redefined” (quoted in Daly, Doll, Schulte, & Fenning, 2011, p. 873). Doctoral, internship, and postdoctoral programs need to show how their programs connect to the core psychology competencies and are linked to student outcomes in each domain (Rubin et al., 2007).
Assessment of competencies around skills and attitudes (rather than focusing on knowledge) necessitated novel and multidimensional methods (Kaslow et al., 2004). In 2009, APA published a benchmarks document and a toolkit with formative and summative assessments that could be used to determine progress toward the competencies. The benchmarks document identified 15 competencies and 56 essential components that apply across specialties. The resulting Competency Assessment Toolkit is based on a similar document prepared by the ACGME and American Board of Medical Specialists for physician training. It outlines 15 types of measures that may be helpful in assessing competence, such as exams, surveys, portfolios, and simulations. The strengths and weaknesses of each tool are also described (Daly et al., 2011).
Daly and colleagues (2011) and Fagan and Wells (2000) highlighted some of the challenges of applying and measuring competencies across different accreditation agencies. Accreditation in school psychology, for instance, is particularly complex because psychologists are nested within schools. In this case, accrediting agencies in psychology and education are key players (Fagan & Wells, 2000). From the psychology side, accreditation policies are created by psychology graduate departments, professional schools, professional practice, the broader public, and consumers, reflecting APA’s perspectives. From the education side, school psychology is influenced by the positions of NCATE (now CAEP) and the National Association of School Psychologists (NASP; Fagan & Wells, 2000). Daly and colleagues (2011) suggested that published benchmark documents and toolkits from APA and NASP may not always perfectly align, which can hinder their usefulness. Likewise, Miller and Short (2011) noted, “The bright line between the dual paths to practice in school psychology, licensure by state boards of psychology and certification by state boards of education adds another layer of complexity that needs to be considered” (p. 869).
Evidence of impact
There are currently no known studies assessing the impact of accreditation on psychology programs or graduates (Fagan & Wells, 2000; Prus & Strein, 2011). This lack of empirical evidence is attributed to the methodological selection bias inherent in the fact that accredited or approved programs receive higher quality students.
Some scholars have argued that this lack of research is problematic, particularly given the tight relationship between accreditation and licensure in psychology. In most states, graduating from an APA-accredited program can satisfy the educational requirement for licensure for a school psychologist (Prus & Strein, 2011). In essence, given this strong relationship, accreditation has become less voluntary (Fagan & Wells, 2000).
Engineering
Accreditation has served as a form of quality control for engineering education in the United States for more than 70 years (Prados, Peterson, & Lattuca, 2005). In engineering, the first professional degree is at the undergraduate level (Sheppard, Macatangay, Colby, & Sullivan, 2008), and ABET (formerly, the Accreditation Board for Engineering and Technology) oversees accreditation (Prados et al., 2005).
Many of the current debates around accreditation in engineering reflect the historical development of engineering education (Prados et al., 2005). In 19th-century Europe, there was a formal mathematics-focused French system as well as an apprenticeship system that was common in England. Engineering schools in the United States were influenced by these different systems—and to this day, the tension between maintaining their place in the academy and meeting the needs of industry continues to plague the professional field (Sheppard et al., 2008). U.S. engineering schools veered closer to their French counterparts in the 1920s with “the arrival of several great ‘engineering theorists’ from Europe” (Prados et al., 2005, p. 167). Yet, until 1934, an individual in the United States could still get an engineering license without graduating from an engineering school. After World War II, however, the federal government began substantially investing in research at universities through competitive grant awards that spurred engineering schools to focus on basic research over application to industry. By the end of the 20th century, the engineering curriculum largely mirrored the applied sciences and “only ABET’s insistence on a curricular design component maintained a tenuous link to engineering practice” (Prados et al., 2005, p. 167). However, by the late 1980s, the globalization of the field and drastic changes in information technology suggested a need for reform as industry called for graduates who not only had technical skills but also had a number of non-cognitive skills or characteristics such as the ability to work well on a team or communicate with others (Prados et al., 2005; Sheppard et al., 2008).
However, ABET stood in the way of change, with accreditation criteria that were seen as too prescriptive, disconnected with the needs of industry, and heavily focused on inputs such as student seat time. In the early 1990s, ABET reacted by working with its stakeholders to create new standards known as the Engineering Criteria 2000 (EC2000), which reflected a fundamental shift in accreditation in engineering. EC2000 focused on continuous improvement and evaluation of student outcomes (Prados et al., 2005). However, measuring student proficiency proved challenging—especially because critical thinking skills and being a good team member are difficult skills to assess. Furthermore, the process itself created concerns; inconsistencies between site visits by accreditation teams (composed of engineering faculty and professional engineers) suggested a need for continuous professional development (Maranville, O’Neill, & Plumb, 2012).
In spite of ABET’s changes, the historical tension underpinning engineering education still influences debate today: “the question ‘what should engineering students learn for being successful engineers?’ is and always was a driver for intense discussion about curriculum development in engineering education” (May & Terkowsky, 2014, p. 1046). Sheppard and colleagues (2008) explored 40 schools of engineering, analyzed 100 self-study accreditation reports, and visited 11 electrical and mechanical engineering programs at six U.S. colleges and universities and found that “not only are programs packed solid with technical courses, but there are also limited conceptual openings for issues of [ethics and] professionalism” (p. 6).
Despite improvements in engineering education, a 2009 Carnegie Foundation study of engineering education similarly found “pockets of innovation” in a field that continues to “[hold] on to an approach to problem solving and knowledge acquisition that is consistent with practice that the profession has left behind” (Sheppard et al., 2008, p. xxi).
Evidence of impact
Although the literature on the value of engineering accreditation is sparse, one empirical study has rigorously explored the relationship between accreditation and program quality. In 2002, ABET commissioned the Center for the Study of Higher Education at Pennsylvania State University to examine the effects of the EC2000 standards through a 3½-year study (Lattuca, Terenzini, & Volkwein, 2006; Volkwein, Lattuca, Harper, & Domingo, 2006; Volkwein, Lattuca, Terenzini, Strauss, & Sukhbaatar, 2004). The study asked, “Are engineers who graduated from programs since implementation of the EC2000 standards better prepared for careers in engineering than their counterparts who graduated before introduction of the criteria?” (Prados et al., 2005, p. 170). 3 Volkwein and colleagues (2006) examined the impact of the change in accreditation on a representative national sample of 203 engineering programs at 40 institutions. The research team collected data from 147 program chairs, 1,200 faculty, 4,300 graduates in 2004, and 5,500 graduates of 1994 (before the accreditation changes). Although there was great variety across most of the measures and groups in 1994, the evidence from 2004 was much more uniform, which suggested that EC2000 was succeeding in its quality assurance goals (Volkwein et al., 2006; Volkwein, Lattuca, Harper, & Domingo, 2007). For instance, following implementation of EC2000, half to two thirds of faculty reported increasing use of active learning methods in a course they teach regularly. In addition, 2004 graduates had different educational experiences (such as more collaborative engagement in their learning and more interaction with faculty) than graduates from 1994 even after taking account of graduates’ traits and institutional characteristics (Volkwein et al., 2006).
Through MANCOVA, Volkwein and colleagues (2006) examined differences between the 1994 and 2004 graduates on each of nine scales, which reflected EC2000 learning outcomes. Results suggested significant gains between 1994 and 2004 in graduates’ abilities to appropriately use knowledge of mathematics, science, and engineering; use modern tools needed in engineering; apply experimental skills to analyze and interpret data; address engineering problems by designing solutions; work well in teams; understand professional and ethical obligations; understand the societal and global context; and value and participate in lifelong learning (Volkwein et al., 2006). These findings also quelled historical concerns that a focus on practical learning would crowd out content-based knowledge in mathematics and science. However, the changes cannot be attributed purely to EC2000; other factors such as faculty initiatives, external funding aimed at improving teaching and learning, and employer feedback are also likely to be relevant (Volkwein et al., 2006). Although not solely attributable to EC2000, other changes to engineering education during this period included more innovation in engineering programs, more coherence in the individual program curricula, and increased understanding among engineering faculty about assessment, instruction, and educational theory (Maranville et al., 2012). A number of engineering colleges now offer engineering education graduate degrees, and in the last 20 years, there has been an increase in rigorous educational research studies in engineering (Maranville et al., 2012).
Law
In the 1920s and 1930s, the bar, courts, and state governments called for an accreditation system in legal education to ensure “quality” and ease competition for White lawyers. These “goals” were accomplished through the exclusion of Blacks and other minorities from the profession (Areen, 2011; Portinga, 1995; Shepherd, 2003). Related to this history, accreditation in legal education has undergone reform at a slower pace than it has in many other professional fields.
In 1995, an antitrust suit was filed against the American Bar Association (ABA) by the Department of Justice. The suit claimed that the accreditation process reduced competition from schools that were not ABA accredited and fixed compensation and working conditions at schools with ABA accreditation (Areen, 2011). According to the United States of America v. American Bar Association’s (1995) Competitive Impact Statement, “rather than setting minimum standards for law school quality and thus providing valuable information to consumers, the legitimate purpose of accreditation, the ABA at times acted as a guild that protected the interests of professional law school personnel” (pp. 1-2). The suit was immediately settled and prohibited the ABA from fixing salaries and not accrediting any for-profit schools. It also reduced the percentage of law school deans or faculty on accreditation inspection teams and led to a review of accreditation standards (Portinga, 1996).
Unlike accreditation processes in other professional fields, until recently, there was little to no emphasis on whether students have acquired the necessary knowledge, skills, and professional legal values (Newton, 2012). The ABA still largely emphasized input measures such as the number of volumes in the law library or the content of the law curriculum. Portinga (1996) argued that many standards did not serve an educational purpose but rather created barriers to competition. The library facilities standards, for instance, were viewed as prescriptive and inflexible—dictating librarian salaries, the percentage of the student body the library could accommodate, and the particular sources that needed to be available (Portinga, 1996). Output was solely measured by the percentage of graduates that pass the bar exam. According to Maranville et al. (2012), “Legal education is ten to twenty-five years behind engineering and other professions in adopting outcomes-based education” (p. 1027). Noteworthy pressures to reform stemmed from the Carnegie Foundation for the Advancement of Teaching, an articulation of best practices in legal education by the academy, and a report from the Outcome Measures Committee of the ABA Section of Legal Education and Admissions to the Bar. These reports encouraged law schools to rework the provision of legal education by moving toward an outcomes-based system (Grose, 2012).
In 2010, the ABA recommended replacing inputs with outputs for accreditation purposes (Grose, 2012). The proposed standards not only identified general outcomes that would be required of all schools but also aimed to support individual schools in developing additional mission-related student outcomes (Duncan, 2010). Changes are currently underway; in August 2014, the ABA approved standards that focused more on learning outcomes and assessment and placed an increased emphasis on distance learning (ABA, 2014; Hamilton, 2014). As Hamilton (2014) noted, new standards mention “competency” that is consistent with competency-based models in assessing professional lawyers. For instance, Standard 302 requires competency in
knowledge and understanding of substantive and procedural law;
legal analysis and reasoning, legal research, problem-solving, and written and oral communication in the legal context;
exercise of proper professional and ethical responsibilities to clients and the legal system; and
other professional skills needed for competent and ethical participation as a member of the legal profession (ABA, 2014, Standard 302, p. 14).
Furthermore, Standard 206 addresses diversity and inclusion with respect to race, ethnicity, and gender among faculty and staff (ABA, 2014). By fall 2016, all the new adopted standards will apply to incoming law students (Thomson, 2014).
Evidence of impact
Although it is too early to consider the impact of recent reforms, much of the literature suggests that, previously, there was a disconnect between the accreditation process in law and quality enhancement, improvement, or innovation. According to Areen 4 (2011), compared with other programmatic accreditors, the ABA process did not reflect the same level of commitment to continuous improvement. The standards, for instance, were designed to reflect only minimum requirements. In 2010, the Council even eliminated Standard 104, which encouraged schools to go beyond the basic requirements. As such, there is an “absence of commitment to enhancing as well as assessing, quality” (Areen, 2011, p. 1490). Barnhizer (2010) had argued that law schools looked very similar to one another in terms of the curriculum and quality of teaching faculty—instead, the differences lie in prestige, academic research productivity, and job placements.
Others have emphasized how the composition of the accrediting council has been an impediment to reform (Garon, 2007; Newton, 2012). The Council of the ABA Section of Legal Education and Admissions to the Bar has 21 voting members of which 10 are full-time law professors or deans, three are members of the public, one is a law student, and seven are judges and practitioners. Although Areen (2011) called for the removal of “arbitrary limits placed on the participation of legal educators” (p. 1494), others argued that legal educators retain a strong voice (Newton, 2012) and that the involvement of judges, practitioners, and non-lawyers was minimal (Garon, 2007).
In a number of ways, scholars have suggested that accreditation has been harmful to the profession—serving the law faculty at the expense of students (Joy, 2014; Nussbaumer, 2006a, 2006b; Shepherd & Shepherd, 1998). According to Shepherd and Shepherd (1998), the ABA has had a monopoly over the market for legal training, law faculty hires, and legal services. Even in the aftermath of the antitrust suit, scholars and critics have claimed ABA accreditation continued to limit alternative forms of legal education, kept costs high for students, and deterred entry into the profession (Barnhizer, 2010; Garon, 2007; Portinga, 1996), particularly for low-income students (Shepherd, 2003).
Although the ABA has symbolically attempted to break with its past and address the racial gap in the number of lawyers, Shepherd (2003) and Nussbaumer (2006a, 2006b) had argued that the ABA continued to exclude Blacks. For instance, the ABA standards typically accredit schools that serve mainly White students but do not accredit schools where the majority of students are Black. Although the ABA does not have an official Law School Admission Test (LSAT) cutoff score, some argue that it has a de facto score of 141, and that schools which admit too many students below this score may potentially lose accreditation (Nussbaumer, 2006a). The average LSAT score is 142 for Blacks and 152 for Whites. In addition, accreditation is denied to schools where students have lower undergraduate grade point averages or pass the bar exam at lower rates, even though research suggests the bar exam is not a strong predictor of lawyering ability (Shepherd, 2003). The Law School Transparency (a nonprofit advocacy organization), however, has argued that the LSAT is a good predictor of bar passing rates and that a current trend toward accepting students with lower LSAT scores is problematic (Law School Transparency, 2015; Olson, 2015). In their recent report, Law School Transparency (2015) labeled about a third of accredited law schools as “at risk” because at least 25% of their incoming class had LSAT scores below 150.
Other scholars have focused on the relationship between ABA accreditation and the quality of graduates. In 2006, the National Bar Association, which is composed of mostly African American lawyers and judges and was founded in 1925 in response to exclusion from the ABA, released a resolution calling for evidence which links ABA’s accreditation practices with professional competency (Nussbaumer, 2006a). Often, advocates of the ABA have responded by highlighting the large number of highly qualified practicing lawyers. Yet, according to Shepherd (2003), accreditation may actually lead the public to falsely believe that any lawyer who graduated from an accredited school and passed the bar exam is capable. Echoing this, Barnhizer (2010) pointed out that while there are good lawyers, there are also those who provide low-quality legal representation—suggesting that accreditation in itself does not ensure quality of graduates.
Summary of Key Findings
In reflecting on lessons from these case examples of accreditation in medicine, psychology, engineering, and law, it is worth noting that many of the studies of accreditation and perspectives on the history of accreditation in each of the four fields were written or sponsored by the accreditation organizations themselves. The resulting narratives in a way represent the accreditors’ response to questions raised in the broader accreditation debate described in the introduction: Does accreditation enhance or detract from program quality and innovation? Is the financial and time investment in accreditation worth the costs? What are the implications of more tightly coupling accreditation with government regulation and licensure? Does the structure of accreditation itself create a conflict of interest that precludes rigor and impedes change?
The four cases suggest that these questions could be answered differently depending on the history and characteristics of the field, the design of accreditation practices, and the intervention of external influences to motivate change. Accreditation can be a conservative influence on programmatic change, but there is also evidence that it can raise the quality of preparation for professional practice. Because accreditation provides an important signal of program quality, consumers naturally rely on that signal to support student application decisions, employment recruiting and hiring, professional licensure, and government and private investment. However, in cases where the coupling between program accreditation and external stakeholders becomes too tight, accreditation may limit market competition and innovation.
As medical and engineering programs were challenged to respond to concerns about program innovation and quality, the focus of their accreditation processes shifted toward models that promoted continuous improvement. An accreditation innovation developed in medicine provided a process in which programs were expected to identify and address their own deficiencies prior to the accreditation review and report that progress to the accreditation team. In engineering, the process of accreditation review focuses on recommendations for continuous improvement, rather than summative judgment. This continuous improvement focus appears to have promoted significant changes in program quality and affect in a relatively short time.
The move to a continuous improvement model in both cases appears to have been a response to external pressure and concerns about prior program quality and inertia. Currently, accreditation in law is undergoing a similar transformation. Compared with the other cases where accreditation is occurring at the program level, in law, the ABA accredits schools. This may help account for some of the challenges in reform. Accreditation at this broader level may foster the tendency to focus on inputs rather than performance outcomes, and historically, the inputs-oriented accreditation model served to limit new entrants, stifle innovation in program design, and hinder diversity in program admissions. The tight coupling between accreditation and licensure in most states and the predominance of current law school deans driving accreditation gave existing programs a monopolistic hold on the field, creating incentives for accreditors to limit competition and maintain historical hallmarks of quality rather than promoting innovation and program improvement. Critics of the input-based system have noted its chilling effect on diversity, and its inability to accurately predict professional performance of graduates. The external pressures on law schools and criticisms of existing accreditation processes and outcomes have only recently motivated accreditation in law to begin the shift toward measures of performance outcomes.
The case examples also highlight the importance of coming to consensus about the critical competencies that should be the focus of professional preparation in a specific field. Engineering modeled its academic preparation around two different approaches—one focused richly on theoretical knowledge, and the other more responsive to application in the field. A challenge in school psychology occurred where accreditation for school psychologists fell between accreditation for psychology, education, and school psychology. The nature of the field and its breadth and complexity created challenges for accreditation in terms of the burden on programs to respond to multiple accreditation audiences and program design demands. In engineering, the challenges of theoretical and applied programs were addressed as the field moved forward and developed some consensus about the nature of theoretical and applied knowledge in engineering curricula. In school psychology, these tensions have not been resolved, and programs continue to struggle to serve multiple accreditation demands. The field of medicine posed a related challenge in that as the field developed, a plethora of specialties emerged. Medical accreditation ultimately responded by designing accreditation models that identified core competencies across specialties, which served to simplify accreditation and unify the profession.
A key ongoing challenge to accreditation is its ability to define and measure competencies in a valid and reliable way. The most progress in validation has occurred in medicine and engineering, fields that tend to have a predisposition to evidence-based practice and significant resources to support studies of accreditation design and impact. Despite progress, an ongoing challenge is the measurement of the softer skills, such as professional dispositions and ethics.
The convergence in policies and standards of accreditation across fields may stem in part from the increasingly important role of globalization and technology. For example, in 2005, the World Health Organization (WHO)/World Federation of Medical Education (WFME) Strategic Partnership released guidelines for basic medical education accreditation and has also worked on capacity building, reform processes, and evaluations of medical education on a global scale (Alwan, 2012; Karle, 2006). Likewise, Prados et al. (2005) highlighted how the shift in accreditation’s focus on learning outcomes (instead of inputs) assists in the evaluation of engineering graduates from international educational systems and facilitates a global recognition of engineering credentials. Professional fields such as engineering are also facing globalization-related pressures to handle more complex technical problems in shorter amounts of time and to work on international teams that are culturally diverse (Sheppard et al., 2009). Such pressures increase the importance of “global competence” and, to some extent, are already reflected in some accreditation policies such as ABET’s emphasis on understanding “the impact of engineering solutions in a global, economic, environmental, and societal context” (May & Terkowsky, 2014, p. 1048).
Alongside globalization, technology is also likely to be a double-edged sword for programmatic accreditation—not only creating obstacles but also fostering new opportunities. Recently, accreditation in psychology began focusing on technology as both a professional skill and as a tool in gathering outcome and program data. This promotes opportunities for programs to consider creative new uses of technology (Prus & Strein, 2011), but, particularly in the short term, may complicate accreditation. Scholars such as Fagan and Wells (2000) had noted that when accreditation becomes too overwhelming or complex, it runs the risk of hindering its main goal of quality improvement.
Across the cases, evidence (albeit often anecdotal) suggests that there can be a heavy faculty burden that accompanies the accreditation process. Scholars in law note critiques by faculty that the ABA process created work for them that was not necessarily appreciated, supported, or compensated by the academy (Nahstoll, 1982; Portinga, 1996). In psychology, Prus and Strein (2011) suggested that the shift toward competencies can not only increase faculty workload but can also allow for more process-based flexibility and incentivize programs and faculty to collaborate to develop and validate performance assessments. Anecdotal evidence suggests that reform in engineering also created a burden for faculty. For instance, there was a need to train 1,000 ABET volunteers and secure support from faculty. Some faculty felt that they were forced to collect large amounts of raw data without a specific focus (Prados et al., 2005) and that the assessment requirements were onerous (Maranville et al., 2012). According to Maranville and colleagues (2012), many programs also had to create a new assessment position to deal with the burden.
With these themes in mind, we turn to consider implications for the accreditation of educational leadership preparation programs.
Implications for Accreditation in Educational Leadership
To examine implications for educational leadership, we return to the features of the case examples that highlight interesting points of comparison between educational leadership and the other professions. First, an important context factor that has driven innovation in accreditation design in other professions is external pressure to change, including attention to the quality and impact of professional preparation. In educational leadership, we face many of these same external pressures to reform. Educational leaders, education as a profession, and schools have been criticized and pressured to improve the quality of graduates and the performance outcomes of schools. These external pressures have already had some impact on the structure and focus of educational leadership accreditation, and they signal a possible window of opportunity for change.
A second theme in the accreditation cases is the importance of coming to consensus about the core competencies needed to be successful in professional practice. With the development of the ISLLC and ELCC standards and their revision and codification in state program approval processes and principal evaluation systems, educational leadership as a field has come to some consensus about guidelines for effective leadership preparation and professional practice.
Because educational leadership programs are accredited by a unified education accrediting body (CAEP), there may be fertile ground in education accreditation to consider the kind of unifying professional standards characteristic of accreditation in the medical field. Furthermore, external pressures in education provide the same landscape in educational leadership as existed in medicine, psychology, engineering, and law to encourage accreditors to rethink accreditation design and focus it on promoting innovation and continuous program improvement. For instance, the ELCC review process focuses on how the use of meaningful data can direct continuous improvement efforts (ISLLC Refresh Project, 2014). Likewise, these same pressures have encouraged attention to assessment and performance outcomes. The medical accreditation model, where programs self-identify problems 1 to 2 years prior to formal review, provides an interesting design that allows programs to correct and document improvement prior to the formal accreditation review.
One cautionary tale from our case studies comes from analysis of the experience of accreditation in law, which has tight coupling between programmatic accreditation and state program approval for licensure. The literature suggests that this tight coupling enabled the law school deans involved in peer review to use the accreditation process to exercise significant power in maintaining their monopoly on preparation for the legal profession, such as by fostering costly infrastructure requirements that would impede new entrants from entering the field. It suggests that maintaining some separation between regulation and accreditation may reduce these monopolistic incentives and thereby enhance the value of peer feedback in program reviews.
In educational leadership, most states recognize accreditation as a signal of program quality, but they also maintain their own review procedures. CAEP addresses the accreditation of educator preparation programs—including teacher preparation, leader preparation, and the preparation of all other educational staff—and is the main organization through which the ELCC standards are used for leadership program reviews. 5 In the accreditation of educator preparation programs, CAEP encourages a focus on outcomes and the use of valid evidence. Providers undergo a program review process (as evidence of meeting CAEP standards) followed by a self-study and an on-site visit. Five standards serve as the foundation of the accreditation process: content and pedagogical knowledge; clinical partnership and practice; candidate quality, recruitment, and selection; program impact; and provider quality assurance and continuous improvement (CAEP, 2015a, 2015b).
In a few states, accreditation is a requirement for institutional certification of educational leaders for state licensure. The CAEP model also provides for state agreements in which state officials serve as members of the CAEP review team. In others states, there is a looser connection between licensure, program approval, and accreditation. Some high-quality programs, such as the University of Wisconsin–Madison, choose not to participate in programmatic accreditation in educational leadership. We believe the evidence suggests that this diversity in coupling is healthy and may promote competition that enhances innovation in both accredited programs and programs that choose not to seek accreditation.
There is clearly a lack of research evidence about the impact of accreditation on program quality, innovation, and impact, and accrediting organizations themselves often sponsor the research that does exist. However, looking across the history of accreditation in these four professions, there is at least circumstantial evidence to suggest that accreditation can have a positive impact on program innovation and improvement, diversity and impact, changes in technology, and the unification of professional fields. Conversely, our case examples also show that accreditation can impede innovation and diversity, dilute program impact, and divide professions. Whether accreditation primarily produces positive or negative results depends in part on pressures external to the program/accreditor relationship for improvements in preparation or professional practice; the diversity of academic and professional leadership governing the accreditation process; consensus around the body of knowledge needed to succeed in the field; and to some degree, the separation between accreditation and government regulation of the field.
The impact of accreditation on professional practice and the identification of the features of accreditation that leverage positive outcomes can be difficult to measure because of the tight coupling between state policy and standards that influence both accreditation and state program approval. However, through mixed-methods designs, we believe it would be possible to build an evidence base for assessing the impact of accreditation on programs and professional outcomes.
In theory, accreditation processes are designed to reflect and leverage the academic culture (Eaton, 2012a). These processes respect norms of peer review, academic freedom, and institutional autonomy. When carried out in an environment that is responsive to external stakeholder demands as well as institutional and professional contexts, it has the potential to foster creativity, innovation, quality, and impact. Accreditation is influenced by the history, politics, and ideologies inherent to a discipline as well as the accreditation trends that occur in other professional fields. Accreditation is neither a hero nor a villain, but it has the potential to provide an impetus for important and ongoing self-reflection and peer feedback to strengthen and improve professional preparation programs.
Footnotes
Authors’ Note
A previous version of this article was originally prepared for the University Council for Educational Administration and the Wallace Foundation Accreditation Project.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a grant from the Wallace Foundation. Any opinions, findings, or conclusions expressed in this paper are those of the authors and do not necessarily reflect the views of the Wallace Foundation, the University of Wisconsin-Madison, or Southern Methodist University.
Notes
Author Biographies
), a multi-source formative assessment that measures distributed leadership practices in schools.