Meeting the Core Competencies for Public Health Professionals Using Backward Design

Background

The MPH Program at Des Moines University went through a curriculum revision between October 2013 and January 2015. This revision was triggered by a number of factors including the emphasis on curriculum mapping in the college’s and university’s strategic plans, the expectations of the program’s accreditation agency, and the transition into a new LMS and corresponding learning analytics system. The program undertook multiple curriculum mappings in the past to collect student competency attainment data. However, these reports were more topically based and lacked depth and breadth. More important, without additional steps outside the LMS or within the previous LMS itself the program faculty were not able to provide data on each and every student’s attainment of every competency. In the past, the program faculty adapted the 2010 Core Competencies for Public Health Professionals with modifications and mapped selected assessments in courses to the competency domains of this model. These assessments covered the majority of competencies under the selected competency domain. Other than the lack of robustness of the reports, aggregating and reporting of the data were very laborious and time-consuming processes for course instructors and program faculty responsible for the final product. It required at least one dedicated faculty’s complete attention to export data from the previous LMS to a Microsoft Excel™ spreadsheet and calculate means for different semesters from different assessments to present as an overall students’ competency attainment in the selected semester and competency domain. These spreadsheets were only able to present aggregate data and did not present any information regarding the individual student’s competency attainment. In addition, this manual process did not present any detailed information on each competency under each domain. This process was only able to provide high level detail information of the overall competency coverage of the entire program.

During the planning of the transition to the new LMS, the program decided to follow a more systematic approach that would provide evidence on not only the coverage of competencies in the Core Competencies for Public Health Professionals 2014 but also show the degree to which these competencies were attained by students. To do that, backward design was implemented to align the curriculum of 14 required courses. To align the whole required curriculum, all instructors went through a faculty development process as explained in detail in Ozdemir and Stebbins (2015).

Backward Design

Backward Design (Wiggins & McTighe, 2005) was selected as the design framework to align the MPH curriculum to public health competencies. The stages of backward design were (a) “identify desired results,” (b) “determine acceptable evidence,” and (c) “plan learning experiences and instruction” (Wiggins & McTighe, 2005, p. 18). As a competency-based degree program, identification of the desired results for the entire program was relatively straightforward. The challenge was the identification of the desired results at the course level. Based on a preliminary analysis of program, college, and university expectations, core competencies for public health professionals and course objectives were determined to be the desired outcome of backward design for each course. The specific backward design process was established on the premise that the accomplishment of measurable course objectives would indicate the attainment of certain core competencies. The second stage, to determine acceptable evidence, was identified through revising the summative student learning assessments. Summative student learning assessments were the graded assessments at the end of each major unit in each course. Then, each course objective was directly aligned with a part of or an entire summative learning assessment. Therefore, it was possible to assess more than one course objective in a particular summative student learning assessment. ¹ These summative learning assessments were also associated with assessment rubrics, whenever necessary, and thresholds of acceptable performance as established by the program (e.g., 70% of the total score) were assigned to determine the accomplishment of individual course objectives. In the last stage, the instructors planned and integrated learning experiences and instruction that aligned with the course objectives. To assist students to accomplish the course objectives, the instructors integrated sufficient learning experiences and instruction.

Front-End and Back-End Analyses

Kopera-Frye, Mahaffy, and Svare (2008) described front-end analysis as an examination of the face-validity of the curriculum. During the front-end analysis, the curriculum was examined to find out if it covered what it intended to cover regarding core competencies, course content, teaching and learning activities, and summative learning assessments. In this article, front-end analysis particularly focused on the core competencies in the program curriculum. As a result of backward design, the learning analytics system provided detailed front-end analysis on which competency domains and competencies were covered in specific courses based on the connection between the competencies and their corresponding course objectives in various courses in the curriculum. By Fall 2015 academic semester, our results showed 6 to 12 courses mapped all 8 competency domains in the Core Competencies for Public Health Professionals competency model (The Council on Linkages Between Academia and Public Health Practice, 2014). As the MPH degree at the authors’ institution was defined as a Generalist Concentration according to CEPH, this result was expected.

After the first front-end analysis, the program administration, upon the recommendations of the program curriculum committee, contacted several faculty to consider focusing on certain competency domains in their courses. The MPH curriculum committee had an important role in the integration of the front-end analysis process with curriculum evaluation. After the mapping and alignment process, the curriculum committee reviewed all syllabi to ascertain that the course objectives were appropriate for the course description, the assessments were properly designed to address course objectives, and that the selected competencies had been aligned with university and program competencies. As questions, discrepancies, or potential mapping errors occurred, the curriculum committee documented their review, and the chair contacted the respective instructor with the committee feedback to initiate dialogue to resolve the issue prior to publication of the syllabus. The curriculum committee occasionally recommended modifications of assessments such as written papers that required students to demonstrate synthesis and critical analysis of content over memorization and recall of information.

For instance, as a result of the front-end analysis, the instructor of the Occupational and Environmental Health course was requested to focus on four additional competencies in the Policy Development/Program Planning Skills domain that had either been removed by the instructor or that the curriculum committee deemed important for this course in the context of the entire curriculum. This resulted in the instructor modifying the course objectives and assessments and providing relevant learning opportunities for students to attain those competencies. Another result of the front-end analysis conducted by the curriculum committee at the programmatic level was the determination that the 14 core courses primarily covered lower to midlevel competencies in each competency domain of the second tier of the Core Competencies of Public Health Professionals competency model (The Council on Linkages Between Academia and Public Health Practice, 2014). The program also expected this competency coverage considering the nature of the program. The program’s advisory group confirmed this result was appropriate and satisfactory based on workforce assessment and overall program goals.

The program subsequently determined that higher level competencies within the Tier II level of the model (The Council on Linkages Between Academia and Public Health Practice, 2014) could potentially be accomplished through the practicum courses, as well as a variety of elective courses, and actual work experience. Instructors of the field-based internship and culminating capstone experience expanded their syllabi, course instruction, and selection of higher level competencies. Therefore, the most important question, “Has the curriculum met the core competencies for public health professionals?” is answered. Using backward design, each and every core course met the intended core competencies after conducting the front-end analysis and taking the necessary steps for improvement. In addition, the curriculum met the core competencies by providing the students with sufficient learning experiences, instruction, and relevant assessment of their competency attainment as shown in the achievement of the aligned course objectives. For instance, Competency 1B1 (Describes factors affecting health of a community) was aligned to 29 course objectives in 8 required courses and 3 elective courses. Due to the backward design, those 29 course objectives had the aligned summative student learning assessments and learning opportunities. Front-end analysis results triggered improvements by identifying the gaps in competency coverage. This feedback loop and closely monitoring the course objective mapping became a more integral part of the program curriculum committee’s regular curriculum evaluation to assure coverage of the competencies in the program’s model.

According to Kopera-Frye et al. (2008), back-end analysis focuses on the student accomplishment of learning outcomes. In this particular case, back-end analysis determined if the target core competencies were accomplished by the students. Backward design allowed the learning analytics system to provide comprehensive information on student attainment of competencies. This way, the program was able to collect data on competency attainment and to identify the areas for improvement. In cases where the overall competency attainment was poor, it became possible to identify the course, the course objective, and those students who were performing poorly on certain competencies, and then take the necessary steps by contacting them or by providing additional assistance. The employment of these analyses allowed the MPH program administration to assure that the curriculum and students met the core competencies for public health professionals and provide specific feedback to instructors for faculty development.

Summary

During the curriculum alignment process using backward design, challenges are often related to the maintenance of the alignment. For instance, by the time the project started, the Council on Linkages between Academia and Public Health Practice revised their competency model adopted in 2010 and published the new core competencies by June 2014. Although a crosswalk existed between the old and new competencies, several new competencies were added as well as a number of old competencies were removed. This required the MPH program faculty to revisit the alignment between the competencies and course objectives in each and every course. Every major change in the competency model required a realignment and follow-up front-end analysis.

Another challenge was the new required mind-set for faculty during their continuous course improvements. In the past, the faculty made improvements on different components of their courses in isolation (e.g., content, assessments, course objectives, etc.). However, with the aligned course curriculum, they needed to consider the connections between core competencies, course objectives, teaching and learning activities, assessment strategies, and rubrics. After the first year of the aligned curriculum, faculty gained more awareness of the alignment and were enthusiastic about learning about how data from this process could be used for course improvement.

The next challenge could also be considered an opportunity. The backward design approach along with the utilization of a learning analytics system provided an almost overwhelming amount of data that required additional time to analyze and to reflect upon how it could be used to improve student competency attainment. Many MPH programs, however, may not currently be prepared for handling such “big data.” At the same time, however, these data sets potentially enable program stakeholders to efficiently and accurately identify best practices focusing on specific competencies. MPH programs may also be in need of a framework to evaluate their competency-based curriculum. Fortunately, such frameworks have begun to appear in the professional literature (e.g., Ozdemir & Stebbins, n.d.). Comprehensive curriculum data could also allow the programs to delve deep into the curriculum and identify best practices focusing on identified competencies without requiring much effort to modify existing or create different assessments. Leathwood and Phillips (2000) called the efforts focusing on the evidence-based curriculum improvement as “curriculum evaluation research.”

Backward design is a dynamic process when instructors, course coordinators, curriculum committees, and program directors incorporate student feedback to make improvements. Course objectives must be reviewed no less than annually in conjunction with an environmental scan related to the major tenets of the course, and with input from community and professional experts. Modification of course objectives should occur as either the program itself or workforce and practice needs evolve. Such efforts could assure optimally prepared public health professionals for the 21st century. For meaningful data on student learning outcomes, instructors must discern levels of performance through accurate application of the grading rubric to summative assessments. Learning outcome data corresponding to respective competencies and course objectives need to be analyzed to inform quality improvement efforts. Tutorials and training from the course coordinator on backward design, as well as experience in its use prior to each semester, may increase transparency and meaningful instruction for students, and they may lead to increased sensitivity of the grading instrument and interrater reliability among instructors.