Educational Needs Assessment,Development of Learning Objectives,and Choosing a Teaching Approach

Abstract

Background:

How physicians are evaluated throughout their careers and how training programs are accredited has shifted from demonstrating what we teach and what we are learning to demonstrating the achievement of competence.

Discussion:

This article discusses some of the components involved with determining and providing necessary educational experiences. These steps apply to various educational needs and settings such as development of curriculum for training programs, clinical staff development, courses in continuing medical education, and identifying learning needs throughout practice. Covered in this article are three components to this process: needs assessment, development of learning objectives, and choosing a teaching approach. A needs assessment is a first step in this process and can be applied to determining revisions in curriculum, identifying needs in clinical staff development, and to self-identifying individual learner status. Once conducted, the identified curriculum needs are linked to the development of learning objectives and outcome statements that identify the knowledge, skills, and attitudes that learners are expected to demonstrate. Finally, understanding of teaching methods is essential to selecting the ones that best fit the identified needs and outcomes of the educational experience.

Conclusion:

Understanding the separate components involved with learning and curriculum development can ultimately lead to improvement and enrichment of the experiences of learners and educators.

Introduction

Learning is the life-blood of our profession. Learning—the acquisition of knowledge and skill by instruction or study—is a continuous process in which new experiences integrate into those previously acquired. A learning need can be considered a gap between an individual's present level of ability and a higher level of performance as determined by the learner, the organization or society.¹ It may be a specific knowledge, skill, or attitude that can be addressed by an educational experience toward a defined outcome and may not be recognized by the learner.² Educational experiences that close these gaps are part of a life-long process of learning.

Shifts in medical education were occurring at the same time that two reports, To Err is Human: Building a Safer Health System (1999) and Crossing the Quality Chasm: A New Health System for the 21st Century (2001), highlighted the need for changes in health care.^3,4 To meet these identified needs, how physicians are evaluated throughout their careers and how programs are accredited has shifted to demonstrating the achievement of competence as defined through competency-based outcomes. Competence in our profession has been defined as “the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice.”⁵ This shift in medical education reflects a call for closing the “quality chasm,” including “the importance of adequately preparing the workforce to make a smooth transition into a thoroughly revamped health care system.”⁴

This article focuses on key components in the process of determining and providing necessary educational experiences. The steps in the process are similar whether we are developing curriculum (such as for medical school or residency, clinical staff development, professional development course, or a Continuing Medical Education course) or identifying learning needs throughout practice. Steps identified by the Accreditation Council for Graduate Medical Education (ACGME) for developing competency-based curriculum are⁶:

Conduct needs assessment.

Identify competencies addressed by this rotation or experience (what does the learner need to know and do to be a competent physician?).

Write goals and objectives (what do you want the learner to be able to do?).

Determine teaching methods (what activity will facilitate the learning?).

Determine assessment methods.

Determine program improvement methods.

Similar steps are outlined in tips for effective short course design⁷ (such as needs assessment, outcomes based objectives, and speaker preparation) and for palliative care curriculum development.^8,9

Needs Assessment

Educational needs assessment is conducted to identify deficiencies in the current teaching practices, or to anticipate deficiencies based on expected changes in health care needs.¹⁰ It is a critical stage in the development or review of educational curriculum. Educational needs assessment can be used at various levels of the educational system: at the program level it can focus on revisions of curriculum offerings, within an health care organization it can identify needs in clinical staff development, and at the individual level it can help to determine current learner status in regard to desired learner outcome.

Needs assessment is a systematic process of collecting and analyzing information by which educational needs are identified and ranked in order of priority. It identifies the gaps to be addressed educationally and measures the discrepancy between current and desired competence.¹¹ Needs assessment is not the same as the assessment of a learner, although such information feeds back to determine ongoing learning needs of individuals and curriculum changes in programs.

Needs assessment process

Suggested steps in a needs assessment plan include the following^2,11:

Purpose: Why is the needs assessment being done? Who will be involved in the process? Are all necessary individuals on board in the beginning?

Audience: Who will use the results of the needs assessment? Is it for top management, the course provider, or an individual activity?

Issues: What strategies will be used to ensure an effective needs assessment? What issues should the needs assessment address—organizational, accreditation, or program design? What techniques will be used to collect the data? How will data be analyzed? What are the priorities?

Resources: What resources are available to design, implement, and analyze the assessment? Do instruments already exist or do they need to be developed? What financial requirements, personnel, time, and expertise are required?

Data collection: What types of data should be collected? Who will collect the data? What sources of information will be used? What is the timeline?

Analyzing the data and prioritizing a need: What is the problem the continuing education activity should resolve? Is this content being provided elsewhere? Is the intended learner aware of the need? How significant will it be if the need is not resolved?

Types of information for needs assessment

Sources of information to utilize in a needs assessment include^2,12 objective information (test scores), subjective information (the expressed needs of individuals), normative needs as defined by experts (specialty boards, accrediting and professional organizations), and comparative needs as determined by group comparison (comparing palliative care education for final year medical students in Britain and the United States¹³). Individual needs will relate to professional interests and performance whereas organizational needs will tie into organizational goals and outside requirements. Group learning outcomes will produce an average picture for the organization but will fail to identify needs of individual members. Each approach has its use and must be used for the intended purpose.¹⁴

Needs assessment methods

Needs assessment methods^2,12 range from individual learner self-assessments to complex surveys and interviews. The information obtained from a needs assessment can then be used as baseline data to determine the effectiveness of educational interventions. In addition to the information obtained, the process can assist with “buy-in.” Methods of needs assessment include the following.

Surveys offer an efficient and low-cost way to obtain quantitative data anonymously from large numbers of individuals, including those otherwise unavailable by distance. It is important to note that the quality of the data is limited to self-identified needs and is dependant on the design and the quality of the questions identified for the survey.^15,16 It is beneficial to have differences noted between ‘interests and real learning needs' and between “nice-to-know” versus “have-to-know.”²

Focus groups are a useful qualitative needs assessment strategy.^17,18 They allow small groups such as 8–10 participants to convene for structured and informal discussion. The method is beneficial for addressing major issues or obtaining reactions to new ideas. A skilled facilitator will maximize listening to other participants while exploring differences in opinion. Planning, developing core questions, facilitating or moderating the session, and analysis of data are the essential steps in conducting focus group interviews.¹⁷ Focus groups provide a broad range of qualitative data in a timely, cost-effective manner and can help to clarify quantitative data. They are very useful in areas of health care where physician attitudes or behaviors are targeted as modification in such areas can only occur when the concerns of the individuals and group are addressed.¹²

Interviews can provide in-depth insight into someone's perspective and allow clarification of information.¹⁹ Interviews can be conducted face-to-face or by phone. They require more time and effort including the analysis of the descriptive data obtained.

Key informants are leaders in the organization or profession who have valuable opinions and insights into the educational needs of a specific group. These individuals may not be part of the target audience but are knowledgeable about the needs. They can participate in questionnaires or provide information in one-on-one interviews.

Brainstorming is a method for a group to provide as many solutions as possible to identified problems. The process can be facilitated with a flip chart, with the group asked to select and rank the best ideas gathered.

Chart audits are reviews of patients' medical records that can be used as a needs assessment tool in a variety of situations, such as to determine learning needs for continuing professional education, to assess change after education, to assess variations in professional practice, and to evaluate competence.²⁰ Random or targeted chart audits can identify areas of weakness in patient care that can be addressed through educational interventions.

Several examples highlight potential needs assessment steps and methods used in addressing needs in palliative care.^8,9,21 Four steps were described in the process of developing a palliative care curriculum at two institutions: needs assessment, curricular design, implementation, and program evaluation.^8,9 Components of the needs assessment included literature review, survey (residents), ranking of topics by various groups, interviews (patients, family members, residents, nurses, attending physicians, interviews with organizational leaders (administrators, educators), interviews with key informants (palliative care and educational design experts), and focus groups (residents, nurses, attending physicians). Fewer steps in the needs assessment were needed at the institution where greater receptiveness to palliative care was identified.^8,9

Another example describes the process of developing a pediatric palliative program.²¹ Methods used included focus groups and chart audits. Ten focus groups were convened with 6–10 members per group, utilizing structured interviews with five questions posed to each group. The chart audits consisted of medical record reviews of 145 patient records to identify end-of-life characteristics (cause and place of death, whether cardiopulmonary resuscitation was initiated, whether do-not-resuscitate orders were present in the medical record before death, length of end-of-life care before death, whether sibling counseling and bereavement counseling were offered to the family after death, and the stated preferences of the patient and family regarding the death experience). With this information, a vision statement and strategic plan was drafted, new research protocols were implemented, consensus building was facilitated and institutional resources were obtained.

The mandate to implement palliative care curriculum and the lack of resources can make this process feel overwhelming. Yet the process of a needs assessment helps prioritize where to focus resources and identify existing resources within an institution. Table 1 summarizes an example of some practical outcomes that can occur. As an example, identifying the need to improve resident knowledge in interdisciplinary care and communication between team members was met by inviting the resident and primary nurse caring for the patient prior to discussing each patient at weekly palliative care rounds. These rounds were referred to as “virtual” team rounds in recognition of the essential team members beyond the palliative care team.

Table 1.

Linking Needs Assessment to Implementation, Learning Objectives, and Teaching Methods (Pediatric Residents and Fellows in Critical Care and Oncology)

Needs assessment
Process
• Survey: residents, fellows, faculty, nurses, social workers
• Chart audits: patient deaths in the hospital and patients discharged to hospice
• Focus groups: nurses, chief residents, fellows, faculty, parent advisory committee
• Interviews: hospital administration, fellowship directors, head of nursing and social work
• Pre-test: residents and fellows
Focus of information
• Gaps in knowledge, gaps in patient care delivery
• Institutional resources: faculty with expertise, available time for teaching

Implementation
• Faculty identified (those with skills and interest in participating)
• Curriculum incorporated into monthly fellowship curriculum: symptom management and communication (attended by all fellows)
• Monthly curriculum incorporated into resident oncology rotation: pain management (along with monthly debriefing rounds previously incorporated into rotation)
• Primary bedside nurse and resident caring for the patient invited to attend weekly palliative care rounds by calling to join rounds 5 minutes before specific patient up for discussion
• Chief resident member of monthly advisory committee meetings to continue a feedback loop
• Pre and post test

Learning objectives ³⁶	Teaching/Learning methods
Patient Care
Describes the use of opioids in pain and non-pain symptom management	Direct patient care activities, attending rounds, bedside teaching rounds, patient presentations, learning prescription, program curriculum

Interpersonal and Communication Skills
Assesses patient's and family members' decision-making capacity, and other strengths and limitations of understanding and communication	Direct patient care activities, attending rounds, rolemodeling, self-assessment (fellows), formative assessment (fellows), interdisciplinary meetings

Personal needs assessment methods

Competence is a habit of life-long learning, not just an achievment.²² Assessment ties back to helping physicians identify and respond to their own learning needs.²³ In addition to external assessment, self-assessment and reflection are critical skills toward the goal of life-long learning. There is some evidence that the accuracy of self-assessment can be enhanced by feedback, particularly video and verbal, and by providing explicit assessment criteria and benchmarking guidance.²⁴ The accuracy of self-assessment in clinical training may be improved by increasing the learner's awareness of the standard to be achieved.

Portfolios are tools intended to enhance self-reflective and self-directed longitudinal learning. A portfolio is a collection of material gathered by a learner to record and reflect on learning and personal development. It can be used as an assessment tool, as a record of achievement, and as evidence for continuous professional development. There is evidence that if well implemented, portfolios are effective and practical in a number of ways including increasing personal responsibility for learning and supporting professional development.²⁵ Factors for success include clearly communicated goals and procedures, integration with curriculum and assessment, flexible structure, and close mentoring.²⁶

Other tools that enhance personal needs assessment, especially at capturing areas of unperceived needs, include chart audit and chart-stimulated recall.²⁰ Chart-stimulated recall, although requiring more time effort, is a case-based interview that allows the learner to indicate the rationale for management decisions. Interviewers should be health professionals trained to ensure a consistent and reliable approach. A framework for rating and scoring is required.

Personal needs assessment has also been incorporated into the American Board of Medical Specialty’ recertification programs, supporting a goal of continuous professional development (ABMS Maintenance of Certification [ABMS MOC®]).²⁷ ABMS MOC assures that the physician is committed to lifelong learning and competency in a specialty and/or subspecialty by requiring ongoing measurement of six core competencies adopted by ABMS and ACGME in 1999.

Learning Objectives and Outcomes

Needs assessment is a link to determining curriculum, which can be considered “the expression of educational ideas in practice.”²⁸ Competency-based education focuses on learner performance (learning outcomes) as defined by specific objectives (goals and objectives of the curriculum).⁶

A learning objective is an outcome statement that identifies the knowledge, skills, and attitudes learners are expected to demonstrate. Objectives are used in a range of settings: what the learner should be able to exhibit following a course, a rotation, or a longitudinal experience. Learning objectives in medical education can be considered the building blocks toward the demonstration of performance accomplished over time in various learning experiences (learning outcomes). In residency education, objectives “reflect the knowledge, skills and attitudes residents should acquire for each rotation or educational experience and each level of the residency program.”⁶

Once defined, the outcomes and objectives are then used to determine other elements: the content of curriculum; teaching and learning strategies; assessment and evaluation of learner, teacher, and effectiveness of curriculum.^28,29 Learning outcomes and objectives give learners a clear picture of what to expect and what is expected of them. In developing objectives, one must be mindful to not restrict the curriculum to skills and knowledge that can be readily expressed in behavioral terms. Higher order thinking, problem solving, and processes for acquiring values should not be excluded.³⁰

The process of writing learning objectives utilizes Bloom's Taxonomy of Educational Objectives, expanded on by others such as Fink.^31,32 Desirable characteristics of objectives include that they are specific, measurable, clear to the learner and educator, use discipline-specific competencies and standards, are targeted to the level of learning, and use a variety of taxonomy levels. An objective includes a verb or action phrase to indicate what the learners need to do. Objectives are also considered in the context of the desired level or dimension of learning, such as outlined by Fink: knowledge (identify, describe), application (interpret, create), integration (compare, correlate), human dimension (collaborate, lead), caring (explore, value), or learning how to learn (self-assess, inquire). The objectives can be tied to assessment by indicating how the objective will be measured. The ACGME outline an objective to specify the year of the resident, who will do what (action verb) as measured by (type of assessment).⁶ Resources are available to provide guidance and examples.^33–35

Palliative care learning objectives and outcomes

The Competencies Work Group of the American Board of Hospice and Palliative Medicine developed a set of initial competency-based outcomes for fellowship programs in hospice and palliative medicine.³⁶ Table 1 indicates several examples, linking these to educational and learning methods. The outcomes developed can facilitate development of learning objectives, reflecting unique aspects of each program and its learning environment.

Teaching Approach

Teaching is the explicit use of educational methods. Various teaching approaches are used to facilitate the acquisition of knowledge, skills, and attitudes as defined through learning objectives. Methods of teaching in palliative care as well as other areas of medicine include: clinical experiences, structured educational exercises (post-clinical rounds, conferences, journal club), case based discussions, role playing, standardized patient exercises, lectures, Balint-group type meetings to explore challenging cases and explore their emotional reactions to the work of caring for dying patients, longitudinal experiences, interdisciplinary meetings, and home visits.³⁷

Other methods include role modeling, mentoring, simulation, self directed learning modules, individual or group projects, research projects, chart audits and chart stimulated recall.⁶

The manner of teaching selected will also vary by the size of the group and the setting, such as: large groups, supervision of teams and individuals, one-on-one role modeling, and mentoring.³⁸

Teaching individuals is enhanced by an informal review of the learners' abilities and needs, prioritizing learning, focusing learning to themes or 1–2 teaching points, and incorporating learning into opportunities.³⁸ Other benefits include reflective modeling from the educator/preceptor, abstraction of an experience to a general concept (how to utilize in a novel circumstance), and reflective questioning to assess that learning has occurred. Specific models, such as the One Minute Preceptor, can facilitate efficient clinical teaching and be taught to residents.^39,40

Learner-centered teaching includes active engagement of the learner. Teaching methods that are inherently passive can incorporate methods that enhancing active learning, such as building into lectures a pause of 3–4 minutes at regular intervals (suggested as less than every 18 minutes) to allow groups of 2 or 3 to make collaborative notes on the major issues presented.⁴¹ Successful models that develop positive partnerships between faculty and learners recognize that the adult learner brings experience (such as problem solving skills, self awareness, knowledge of other cultures, management experience), is self-directed, values opportunities to make a positive contribution to the learning of the teacher, and values flexible approaches from an educator.⁴²

This is complemented by extrinsic motivation, understanding of why the knowledge or skill is needed and reflective practice.³⁷ Some educational approaches naturally tie into formative and summative assessment as well as facilitate continuous improvement of resident performance, educational experiences, and residency program outcomes.

Effective education also benefits from faculty development.^43,44 Programs in faculty development identified that participants reported increased knowledge of educational principles and gains in teaching skills, changes in teaching behavior that were also detected by students, greater involvement in education, and establishment of colleague networks. Key features of effective faculty development included the use of experiential learning, provision of feedback, effective peer and colleague relationships, well-designed interventions following principles of teaching and learning, and the use of a diversity of educational methods within single interventions.

Each step is a necessary part of developing and revising medical education programs, working toward closing the gap in the quality of the care that we deliver. Effective educational programs utilize the process of needs assessment along with knowledge of creating learning objectives that tie to learning outcomes, methods of learning, and assessment of the learner and program.

Success in education will be realized as we improve translating this process into physicians prepared for the evolving health care environment.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

References

Knowles

. The Modern Practice of Adult Education: Andragogy versus Pedagogy. New York: Association Press, 1970; 85.

DeSilets

. Needs assessments: An array of possibilities. J Contin Educ Nurs, 2007; 38:107–112quiz 113–114.

Institute of Medicine, Committee on Quality of Health Care in America. Kohn

, Corrigan

, Donaldson

. To Err is Human: Building a Safer Health System. Washington, D.C.: National Academies Press, 1999.

Institute of Medicine, Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, D.C.: National Academies Press, 2001.

Epstein

, Hundert

. Defining and assessing professional competence. JAMA, 2002; 287:226–235.

Accreditation Council for Graduate Medical Education (ACGME). www.acgme.org/outcome/. 2009 November 24.

Lockyer

, Ward

, Toews

. Twelve tips for effective short course design. Med Teach, 2005; 27:392–395.

Ury

, Arnold

, Tulsky

. Palliative care curriculum development: A model for a content and process-based approach. J Palliat Med, 2002; 5:539–548.

Ury

, Reznich

, Weber

. A needs assessment for a palliative care curriculum. J Pain Symptom Manage, 2000; 20:408–416.

10.

Donald

EMJ

, Donald

. Needs assessment. Continuing Medical Education: A Primer. Westport, CT: Praeger, 1992; 42–51.

11.

Adelson

, Manolakas

, Moore

. Assessing educational needs. Adelson

, Watkins

, Caplan

. Continuing Education for the Health Professional: Educational and Administrative Methods. Rockville, MD: Aspen Systems Corporation, 1985; 15–28.

12.

Ratnapalan

, Hilliard

. Needs Assessment in Postgraduate Medical Education: A Review. Med Educ Online, 2002; 7:1–8.

13.

Hammel

, Sullivan

, Block

, Twycross

. End-of-life and palliative care education for final-year medical students: A comparison of Britain and the United States. J Palliat Med, 2007; 10:1356–1366.

14.

Grant

. Learning needs assessment: Assessing the need. BMJ, 2002; 324:156–159.

15.

Mann

. Not another survey! Using questionnaire effectively in needs assessment. J Contin Educ Health Prof, 1998; 18:142–149.

16.

Lockyer

. Getting started with needs assessment: Part 1—The questionnaire. J Contin Educ Health Prof, 1998; 18:58–61.

17.

Tipping

. Focus groups: A method of needs assessment. J Contin Educ Health Prof, 1998; 18:150–154.

18.

Barbour

. Making sense of focus groups Med Educ, 2005; 39:742–750.

19.

Crandall

SJS

. Using interviews as a needs assessment tool. J Contin Educ Health Prof, 1998; 18:155–162.

20.

Jennett

, Affleck

. Chart audit and chart stimulated recall as methods of needs assessment in continuing professional health education. J Contin Educ Health Prof, 1998; 18:163–171.

21.

Harper

, Hinds

, Baker

, Hicks

, Spunt

, Razzouk

. Creating a palliative and end-of-life program in a cure-oriented pediatric setting: The zig-zag method. J Pediatr Oncol Nurs, 2007; 24:246–254.

22.

Leach

. Competence is a habit. JAMA, 2002; 287:243–244.

23.

Epstein

. Assessment in medical education. N Engl J Med, 2007; 356:387–396.

24.

Colthart

, Bagnall

, Evans

, Allbutt

, Haig

, Illing

, McKinstry

. The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no. 10. Med Teach, 2008; 30:124–145.

25.

Tochel

, Haig

, Hesketh

, Cadzow

, Beggs

, Colthart

, Peacock

. The effectiveness of portfolios for post-graduate assessment and education: BEME Guide No 12. Med Teach, 2009; 31:299–318.

26.

Driessen

, van Tartwijk

, van der Vleuten

, Wass

. Portfolios in medical education: why do they meet with mixed success? A systematic review. Med Educ, 2007; 41:1224–1233.

27.

American Board of Medical Specialties, Maintenance of Certification. www.abms.org/Maintenance_of_Certification/. 2009 December 9.

28.

Prideaux

. ABC of learning and teaching in medicine. Curriculum design. BMJ, 2003; 326:268–270.

29.

Harden

. Outcome-based education–the ostrich, the peacock and the beaver. Med Teach, 2007; 29:666–671.

30.

Prideaux

. The emperor's new clothes: From objectives to outcomes. Med Educ, 2000; 34:168–169.

31.

Bloom

. Taxonomy Of Educational Objectives, The Classification of Educational Goals—Handbook I: Cognitive Domain. New York: McKay, 1956.

32.

Fink

. Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses. San Francisco: Jossey-Bass, 2003.

33.

The University of New Mexico School of Medicine. Effective use of performance objectives for learning and assessment. http://ccoe.umdnj.edu/forms/EffectiveUseofLearningObjectives.pdf. 2010 January 11.

34.

Florida State University, College of Medicine. Information About Behavioral Objectives and How to Write Them. http://med.fsu.edu/education/FacultyDevelopment/objectives.asp. 2010 January 11.

35.

University College Dublin: Guide to taxonomies of learning outcomes. www.ucd.ie/t4cms/taxonomies3.pdf. 2010 January 11.

36.

Morrison

, Scott

, Block

. American Board of Hospice and Palliative Medicine Competencies Work Group. Developing initial competency-based outcomes for the hospice and palliative medicine subspecialist: Phase I of the hospice and palliative medicine competencies project. J Palliat Med, 2007; 10:313–330.

37.

Block

, Bernier

, Crawley

, Farber

, Kuhl

, Nelson

, O'Donnell

, Sandy

, Ury

. Incorporating palliative care into primary care education. National Consensus Conference on Medical Education for Care Near the End of Life. J Gen Intern Med, 1998; 13:768–773.

38.

Abela

. Adult learning theories and medical education: A review. Malta Med J, 2009; 21:11–18.

39.

Neher

, Gordon

, Meyer

, Stevens

. A five-step "microskills" model of clinical teaching. J Am Board Fam Pract, 1992; 5:419–424.

40.

Furney

, Orsini

, Orsetti

, Stern

, Gruppen

, Irby

. Teaching the one-minute preceptor. A randomized controlled trial. J Gen Intern Med, 2001; 16:620–624.

41.

Graffam

. Active learning in medical education: strategies for beginning implementation. Med Teach, 2007; 29:38–42.

42.

Newman

, Peile

. Valuing learners' experience and supporting further growth: Educational models to help experienced adult learners in medicine. BMJ, 2002; 325:200–202.

43.

Steinert

, Mann

, Centeno

, Dolmans

, Spencer

, Gelula

, Prideaux

. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Med Teach, 2006; 28:497–526.

44.

Sullivan

, Lakoma

, Billings

, Peters

, Block

. PCEP Core Faculty: Teaching and learning end-of-life care: Evaluation of a faculty development program in palliative care. Acad Med, 2005; 80:657–668.