Meeting the Core Competencies for Public Health Professionals Using Backward Design

Abstract

The mission of public health programs is to prepare competent public health professionals for a dynamic workforce. One way to accomplish this mission is to design curricula that support learners’ competency attainment in today’s public health practice. The purpose of this article is to demonstrate the actual steps taken to ensure that graduates meet public health core competencies. The major stages of this process are (a) backward design of the curriculum, (b) front-end analysis of the curriculum, and (c) back-end analysis of the curriculum. The backward design stage aligns a course curriculum with the core competencies. Front-end analysis provides evidence that the overall program curriculum aligns with the core competencies. The back-end analysis provides evidence of learners’ competency attainment. All three phases embed principles of continuous improvement to benefit the curriculum through a feedback-loop mechanism. Challenges, opportunities, and future directions for academic leaders of curriculum program development are discussed.

Keywords

competency curriculum alignment backward design public health

It is crucial for the public health degree programs to share best practices focusing on meeting core competencies for public health professionals. In the literature, there is a lack of detailed explanations of strategies focusing on the development of the competency-based public health degree programs (Ervin, Carter, & Robinson, 2013). The majority of best practices focus on the alignment of the intended curriculum to the competencies (e.g., Harden, 2001; Mazurat & Schönwetter, 2008; Sherborne, 2008) that generally fail to provide evidence on learner’s competency attainment. Curriculum alignment focusing solely on intended curriculum often results in a visual depiction of the alignment of certain curricular components using a spreadsheet format (e.g., Kies, 2010; Mazurat & Schönwetter, 2008; McNutt, Furner, Moser, & Weist, 2008; Perlin, 2011; Plaza, Draugalis, Slack, Skrepnek, & Sauer, 2007) or a visual concept map (e.g., Britton, Letassy, Medina, & Er, 2008; Ferns, McMahon, & Yorke, 2009; Watson et al., 2007). To compensate for the lack of evidence on competency attainment as a result of the alignment of intended curriculum, degree programs subsequently conduct additional assessments such as a student self-assessment of competency attainment (e.g., Bradley et al., 2008) or a comprehensive exam (e.g., Perlin, 2011). Curriculum mapping process needs to focus on the alignment of the intended, enacted, and assessed curriculum (Gjerde, 1981) rather than the intended curriculum alone. Therefore, the actual learning assessments included in the program curriculum could provide more meaningful data on learners’ competency attainment and the need for additional assessments could be minimized.

In this article, designing a competency-based curriculum is a prerequisite to assist the learners in the program to meet the core competencies. However, implementing this competency-based curriculum, analyzing the learners’ competency attainment, and performing necessary improvements are also critical factors. The authors demonstrate a three-step process that ensured that the learners met the core competencies identified by the degree program. In the first step, the intended, enacted, and assessed curriculum were aligned using backward design at the course level. In the second stage, front-end analysis was conducted to analyze if the overall program curriculum was aligned with the core competencies as a result of every individual backward design of the course curriculum. In the last step, back-end analysis was conducted using student learning outcome assessment data to analyze if certain competencies were in fact attained by the students.

The proposed strategy in this article addressed the standards set forth by the Council on Education for Public Health (CEPH), the accreditation agency for public health programs. CEPH is in the process of standardizing the list of competencies for all public health degree programs and expecting the programs, regardless of degree granted, to demonstrate their assessment strategies as well as alignment with these proposed competencies (CEPH, 2016). This is a significant step for CEPH, which creates the expectation that program curriculum will focus more strictly on assessing public health competencies.

The strategy presented in this article was successfully applied into the development of three different competency-based degree programs (Master of Health Care Administration, Master of Public Health (MPH), and Master of Physician Assistant Studies) with various corresponding competency models in the authors’ institution. In the MPH program, the competencies were adopted from the 2014 Core Competencies for Public Health Professionals model (The Council on Linkages Between Academia and Public Health Practice, 2014). A particular learning analytics system (Desire2Learn Insights™) along with a particular learning management system (LMS; Desire2Learn Brightspace™) was used to collect data on curriculum alignment and competency attainment for the core competencies presented in this article. However, the proposed strategy could be applied in other programs using various LMSs. Other LMSs may also provide capability for learning analytics on course objectives and competencies in their structure rather than only participation data, such as content visits, course access times, discussion posts, and other indirect measures of learning.

Background

The MPH Program at Des Moines University went through a curriculum revision between October 2013 and January 2015. This revision was triggered by a number of factors including the emphasis on curriculum mapping in the college’s and university’s strategic plans, the expectations of the program’s accreditation agency, and the transition into a new LMS and corresponding learning analytics system. The program undertook multiple curriculum mappings in the past to collect student competency attainment data. However, these reports were more topically based and lacked depth and breadth. More important, without additional steps outside the LMS or within the previous LMS itself the program faculty were not able to provide data on each and every student’s attainment of every competency. In the past, the program faculty adapted the 2010 Core Competencies for Public Health Professionals with modifications and mapped selected assessments in courses to the competency domains of this model. These assessments covered the majority of competencies under the selected competency domain. Other than the lack of robustness of the reports, aggregating and reporting of the data were very laborious and time-consuming processes for course instructors and program faculty responsible for the final product. It required at least one dedicated faculty’s complete attention to export data from the previous LMS to a Microsoft Excel™ spreadsheet and calculate means for different semesters from different assessments to present as an overall students’ competency attainment in the selected semester and competency domain. These spreadsheets were only able to present aggregate data and did not present any information regarding the individual student’s competency attainment. In addition, this manual process did not present any detailed information on each competency under each domain. This process was only able to provide high level detail information of the overall competency coverage of the entire program.

During the planning of the transition to the new LMS, the program decided to follow a more systematic approach that would provide evidence on not only the coverage of competencies in the Core Competencies for Public Health Professionals 2014 but also show the degree to which these competencies were attained by students. To do that, backward design was implemented to align the curriculum of 14 required courses. To align the whole required curriculum, all instructors went through a faculty development process as explained in detail in Ozdemir and Stebbins (2015).

Backward Design

Backward Design (Wiggins & McTighe, 2005) was selected as the design framework to align the MPH curriculum to public health competencies. The stages of backward design were (a) “identify desired results,” (b) “determine acceptable evidence,” and (c) “plan learning experiences and instruction” (Wiggins & McTighe, 2005, p. 18). As a competency-based degree program, identification of the desired results for the entire program was relatively straightforward. The challenge was the identification of the desired results at the course level. Based on a preliminary analysis of program, college, and university expectations, core competencies for public health professionals and course objectives were determined to be the desired outcome of backward design for each course. The specific backward design process was established on the premise that the accomplishment of measurable course objectives would indicate the attainment of certain core competencies. The second stage, to determine acceptable evidence, was identified through revising the summative student learning assessments. Summative student learning assessments were the graded assessments at the end of each major unit in each course. Then, each course objective was directly aligned with a part of or an entire summative learning assessment. Therefore, it was possible to assess more than one course objective in a particular summative student learning assessment.¹ These summative learning assessments were also associated with assessment rubrics, whenever necessary, and thresholds of acceptable performance as established by the program (e.g., 70% of the total score) were assigned to determine the accomplishment of individual course objectives. In the last stage, the instructors planned and integrated learning experiences and instruction that aligned with the course objectives. To assist students to accomplish the course objectives, the instructors integrated sufficient learning experiences and instruction.

Identification of Desired Results

The authors believe that it would be helpful for the audience to understand how backward design is accomplished when presented in one course. For this purpose, Overview of the U.S. Health Care System, an introductory-level course in the program, was selected as an example of how the three stages of backward design were followed. Considering that the MPH program prepares students for the mid-careerist positions in public health, the competencies for all courses were selected from Tier II of the Core Competencies for Public Health Professionals model (The Council on Linkages Between Academia and Public Health Practice, 2014).

In the first step, course objectives served as a central anchor for aligning the core competencies with summative assessments. Establishing the course objectives was accomplished by reviewing how the course was situated in relation to other courses in the curriculum, and how this set of learning experiences and activities were intended to prepare students for the coursework level, culminating experiences, and workforce expectations. Overview of the U.S. Health Care System is often the first graduate course for a student’s program of study. Other courses, such as health care financial management or community health planning, build upon this prerequisite content. One of the course objectives reads, “Upon successful completion of this course, the student will be able to compare and contrast components of the U.S. health care system with those of other countries, including the functions of financing, insurance, delivery, and payment.” This objective was selected because learners must have an operational knowledge base by which to examine the intersection of public health and the health care system. Because this objective relies on important foundational information, it was aligned with the first assessment in the 12-week semester term. The instructor chose to place this objective early in the course so students would later be able to comprehend and apply this content to the evolution of health care delivery since the adoption of the Patient Protection and Affordable Care Act (2010), also known as the Affordable Care Act. By being able to compare and contrast components of the U.S. health care system with those of other countries, students learn how patients experience health care in other countries, and they are then able to make informed judgments about the U.S. system and potential points of integration with the public health system.

Course objectives are then aligned with the core competencies. Associated with accomplishment of the particular course objective is the core competency from the Analytic/Assessment Skills Domain (Tier II), “1B1. Describe factors affecting the health of a community (e.g., equity, income, education, and environment)” (The Council on Linkages Between Academia and Public Health Practice, 2014, p. 5). The rationale behind this alignment is that program graduates must be able to distinguish between traditional health care delivery for one patient at a time versus nonmedical interventions that influence health outcomes of patients and communities typified by population health approaches. Therefore, the course objective directly aligns with this particular core competency. As a result, the establishment of the course objectives and alignment of those course objectives to the particular competencies allowed the instructors to identify the desired results both at the course (e.g., course objectives) and programmatic (e.g., core competencies) level.

Determining Acceptable Evidence

The attainment of the aforementioned course objective was assessed through a writing assignment. The grading rubric for the written paper informs students about the acceptable evidence of achievement. The four criteria of the rubric are knowledge, critical thinking, graduate-level writing, and evidence. To achieve 100%, students must demonstrate knowledge of the components of the U.S. health care system and articulate how this system is similar and different from another system. Not only must they produce factual descriptions required in the assignment, but they must also demonstrate the ability to analyze benefits and limitations of the system and the impact the system has on those receiving care. By producing a written assignment articulating their knowledge and understanding, providing critical thinking skills through analysis, and graduate-level writing with appropriate credible evidence, students’ performance can be evaluated on the established criteria and levels of performance. The rubric scoring corresponds to the program’s grading scale. Therefore, if a student’s composite score was below 70% for the entire paper, it did not meet the achievement of the course objective, and the student subsequently failed the assignment and did not demonstrate the attainment of the aligned core competency.

Planning Learning Experiences and Instruction

This particular summative student learning assessment was supported by the required reading chapters from the textbook and the purposeful selection of learning resources in the prior modules such as videos on health care around the world, white papers and publications from the National Academies Press on integration of public health and primary care, and student-led discussions on the assignment questions. The instructor held weekly online instant messaging chat sessions through the LMS to answer questions about the assignment and discuss key concepts such as social versus market justice, public health delivery, sources of external references, and explanation of the grading rubric criteria. The instructor also provided weekly recorded audio lectures for students to access asynchronously. To prepare students for this course objective and the competencies, the course learning experiences and instruction had to be updated frequently throughout the year to take advantage of new research and developments with the Affordable Care Act and public health resources.

Front-End and Back-End Analyses

Kopera-Frye, Mahaffy, and Svare (2008) described front-end analysis as an examination of the face-validity of the curriculum. During the front-end analysis, the curriculum was examined to find out if it covered what it intended to cover regarding core competencies, course content, teaching and learning activities, and summative learning assessments. In this article, front-end analysis particularly focused on the core competencies in the program curriculum. As a result of backward design, the learning analytics system provided detailed front-end analysis on which competency domains and competencies were covered in specific courses based on the connection between the competencies and their corresponding course objectives in various courses in the curriculum. By Fall 2015 academic semester, our results showed 6 to 12 courses mapped all 8 competency domains in the Core Competencies for Public Health Professionals competency model (The Council on Linkages Between Academia and Public Health Practice, 2014). As the MPH degree at the authors’ institution was defined as a Generalist Concentration according to CEPH, this result was expected.

After the first front-end analysis, the program administration, upon the recommendations of the program curriculum committee, contacted several faculty to consider focusing on certain competency domains in their courses. The MPH curriculum committee had an important role in the integration of the front-end analysis process with curriculum evaluation. After the mapping and alignment process, the curriculum committee reviewed all syllabi to ascertain that the course objectives were appropriate for the course description, the assessments were properly designed to address course objectives, and that the selected competencies had been aligned with university and program competencies. As questions, discrepancies, or potential mapping errors occurred, the curriculum committee documented their review, and the chair contacted the respective instructor with the committee feedback to initiate dialogue to resolve the issue prior to publication of the syllabus. The curriculum committee occasionally recommended modifications of assessments such as written papers that required students to demonstrate synthesis and critical analysis of content over memorization and recall of information.

For instance, as a result of the front-end analysis, the instructor of the Occupational and Environmental Health course was requested to focus on four additional competencies in the Policy Development/Program Planning Skills domain that had either been removed by the instructor or that the curriculum committee deemed important for this course in the context of the entire curriculum. This resulted in the instructor modifying the course objectives and assessments and providing relevant learning opportunities for students to attain those competencies. Another result of the front-end analysis conducted by the curriculum committee at the programmatic level was the determination that the 14 core courses primarily covered lower to midlevel competencies in each competency domain of the second tier of the Core Competencies of Public Health Professionals competency model (The Council on Linkages Between Academia and Public Health Practice, 2014). The program also expected this competency coverage considering the nature of the program. The program’s advisory group confirmed this result was appropriate and satisfactory based on workforce assessment and overall program goals.

The program subsequently determined that higher level competencies within the Tier II level of the model (The Council on Linkages Between Academia and Public Health Practice, 2014) could potentially be accomplished through the practicum courses, as well as a variety of elective courses, and actual work experience. Instructors of the field-based internship and culminating capstone experience expanded their syllabi, course instruction, and selection of higher level competencies. Therefore, the most important question, “Has the curriculum met the core competencies for public health professionals?” is answered. Using backward design, each and every core course met the intended core competencies after conducting the front-end analysis and taking the necessary steps for improvement. In addition, the curriculum met the core competencies by providing the students with sufficient learning experiences, instruction, and relevant assessment of their competency attainment as shown in the achievement of the aligned course objectives. For instance, Competency 1B1 (Describes factors affecting health of a community) was aligned to 29 course objectives in 8 required courses and 3 elective courses. Due to the backward design, those 29 course objectives had the aligned summative student learning assessments and learning opportunities. Front-end analysis results triggered improvements by identifying the gaps in competency coverage. This feedback loop and closely monitoring the course objective mapping became a more integral part of the program curriculum committee’s regular curriculum evaluation to assure coverage of the competencies in the program’s model.

According to Kopera-Frye et al. (2008), back-end analysis focuses on the student accomplishment of learning outcomes. In this particular case, back-end analysis determined if the target core competencies were accomplished by the students. Backward design allowed the learning analytics system to provide comprehensive information on student attainment of competencies. This way, the program was able to collect data on competency attainment and to identify the areas for improvement. In cases where the overall competency attainment was poor, it became possible to identify the course, the course objective, and those students who were performing poorly on certain competencies, and then take the necessary steps by contacting them or by providing additional assistance. The employment of these analyses allowed the MPH program administration to assure that the curriculum and students met the core competencies for public health professionals and provide specific feedback to instructors for faculty development.

Summary

During the curriculum alignment process using backward design, challenges are often related to the maintenance of the alignment. For instance, by the time the project started, the Council on Linkages between Academia and Public Health Practice revised their competency model adopted in 2010 and published the new core competencies by June 2014. Although a crosswalk existed between the old and new competencies, several new competencies were added as well as a number of old competencies were removed. This required the MPH program faculty to revisit the alignment between the competencies and course objectives in each and every course. Every major change in the competency model required a realignment and follow-up front-end analysis.

Another challenge was the new required mind-set for faculty during their continuous course improvements. In the past, the faculty made improvements on different components of their courses in isolation (e.g., content, assessments, course objectives, etc.). However, with the aligned course curriculum, they needed to consider the connections between core competencies, course objectives, teaching and learning activities, assessment strategies, and rubrics. After the first year of the aligned curriculum, faculty gained more awareness of the alignment and were enthusiastic about learning about how data from this process could be used for course improvement.

The next challenge could also be considered an opportunity. The backward design approach along with the utilization of a learning analytics system provided an almost overwhelming amount of data that required additional time to analyze and to reflect upon how it could be used to improve student competency attainment. Many MPH programs, however, may not currently be prepared for handling such “big data.” At the same time, however, these data sets potentially enable program stakeholders to efficiently and accurately identify best practices focusing on specific competencies. MPH programs may also be in need of a framework to evaluate their competency-based curriculum. Fortunately, such frameworks have begun to appear in the professional literature (e.g., Ozdemir & Stebbins, n.d.). Comprehensive curriculum data could also allow the programs to delve deep into the curriculum and identify best practices focusing on identified competencies without requiring much effort to modify existing or create different assessments. Leathwood and Phillips (2000) called the efforts focusing on the evidence-based curriculum improvement as “curriculum evaluation research.”

Backward design is a dynamic process when instructors, course coordinators, curriculum committees, and program directors incorporate student feedback to make improvements. Course objectives must be reviewed no less than annually in conjunction with an environmental scan related to the major tenets of the course, and with input from community and professional experts. Modification of course objectives should occur as either the program itself or workforce and practice needs evolve. Such efforts could assure optimally prepared public health professionals for the 21st century. For meaningful data on student learning outcomes, instructors must discern levels of performance through accurate application of the grading rubric to summative assessments. Learning outcome data corresponding to respective competencies and course objectives need to be analyzed to inform quality improvement efforts. Tutorials and training from the course coordinator on backward design, as well as experience in its use prior to each semester, may increase transparency and meaningful instruction for students, and they may lead to increased sensitivity of the grading instrument and interrater reliability among instructors.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Bradley

E. H.

Cherlin

Busch

S. H.

Epstein

Helfand

White

W. D.

(2008). Adopting a competency-based model: Mapping curricula and assessing student progress. Journal of Health Administration Education, 25, 37-51.

Britton

Letassy

Medina

M. S.

(2008). A curriculum review and mapping process supported by an electronic database system. American Journal of Pharmaceutical Education, 72(5), 99. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2630156/

Council on Education for Public Health. (2016). CEPH draft accreditation criteria. Retrieved from http://www.aspph.org/app/uploads/2016/02/Draft_criteria_3_released_02.01.2016.pdf

The Council on Linkages Between Academia and Public Health Practice. (2014). Core competencies for public health professionals. Retrieved from http://www.phf.org/resourcestools/Documents/Core_Competencies_for_Public_Health_Professionals_2014June.pdf

Ervin

Carter

Robinson

(2013). Curriculum mapping: Not as straightforward as it sounds. Journal of Vocational Education & Training, 65, 309-318.

Ferns

McMahon

Yorke

(2009, November). Course assessment profiling for enhanced educational and employment outcomes. Paper presented at Australasian Association for Institutional Research Annual Forum, Adelaide, Australia. Retrieved from http://www.aair.org.au/app/webroot/media/pdf/AAIR%20Fora/Forum2009/Papers/12-3_Ferns.pdf

Gjerde

C. L.

(1981). “Curriculum mapping”: Objectives, instruction, and evaluation. Academic Medicine, 56, 316-323.

Harden

R. M.

(2001). AMEE Guide No. 21: Curriculum mapping: A tool for transparent and authentic teaching and learning. Medical Teacher, 23, 123-137.

Kies

S. M.

(2010). Curriculum mapping: Knowing where you are going and how you are going to get there. Medical Science Educator, 20(2). Retrieved from http://www.iamse.org/mse-article/curriculum-mapping-knowing-where-you-are-going-and-how-you-are-going-to-get-there/

10.

Kopera-Frye

Mahaffy

Svare

G. M.

(2008). The map to curriculum alignment and improvement. Collected Essays on Learning and Teaching, 1. Retrieved from http://ojs.uwindsor.ca/ojs/leddy/index.php/CELT/article/view/3171

11.

Leathwood

Phillips

(2000). Developing curriculum evaluation research in higher education: Process, politics and practicalities. Higher Education, 40, 313-330.

12.

Mazurat

Schönwetter

D. J.

(2008). Electronic curriculum mapping: Supporting competency-based dental education. Journal (Canadian Dental Assoiciation), 74, 886-889.

13.

McNutt

L.-A.

Furner

S. E.

Moser

Weist

E. M.

(2008). Applied epidemiology competencies for governmental public health agencies: Mapping current curriculum and the development of new curriculum. Public Health Reports, 123(Suppl. 1), 13.

14.

Ozdemir

Stebbins

(n.d.). A framework for the evaluation of competency-based curriculum. In Rasmussen

Northrup

Colson

(Eds.), Handbook of research on competency-based education in university settings. Hershey, PA: IGI Global.

15.

Ozdemir

Stebbins

(2015). Curriculum mapping for the utilization of an online learning analytics system in a competency-based Master of Health Care Administration program: A case study. Journal of Health Administration Education, 32, 543-562.

16.

Perlin

M. S.

(2011). Curriculum mapping for program evaluation and CAHME accreditation. Journal of Health Administration Education, 28, 33-53.

17.

Plaza

C. M.

Draugalis

J. R.

Slack

M. K.

Skrepnek

G. H.

Sauer

K. A.

(2007). Curriculum mapping in program assessment and evaluation. American Journal of Pharmaceutical Education, 71(2), 20. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/pmc1858603/

18.

Sherborne

(2008). Mapping the curriculum: How concept maps can improve the effectiveness of course development. In Okada

Shum

S. B.

Sherborne

(Eds.), Knowledge cartography (pp. 183-198). London, England: Springer. Retrieved from https://link-springer-com.web.bisu.edu.cn/chapter/10.1007/978-1-84800-149-7_9

19.

Watson

E. G.

Moloney

P. J.

Toohey

S. M.

Hughes

C. S.

Mobbs

S. L.

Leeper

J. B.

McNeil

H. P.

(2007). Development of eMed: A comprehensive, modular curriculum-management system. Academic Medicine, 82, 351-360.

20.

Wiggins

G. P.

McTighe

(2005). Understanding by design. Alexandria, VA: Association for Supervision and Curriculum Development.