A Systematic Review of the Outcome Measures Used to Evaluate Interprofessional Learning by Health Care Professional Students During Clinical Experiences

Abstract

Interprofessional education (IPE) occurs when members of more than one health or social care profession learn interactively together to improve interprofessional collaboration and health care delivery. Interprofessional experiences provide students with the skills and knowledge needed to work in a collaborative manner; however, there is no review on the outcome measures used to assess the effectiveness of IPE learning. The current systematic review examined the outcome measures used to assess interprofessional learning during student clinical experiences. An electronic search of databases retrieved trials of health professional students who completed an IPE intervention during a student clinical experience. Methodological quality of twenty-five studies meeting the inclusion criteria published between 1997 and 2018 was scored independently by two raters using the Physiotherapy Evidence Database and the Confidence in the Evidence from Reviews of Qualitative Research tool. The Interdisciplinary Education Perception Scale was used most frequently to assess interprofessional learning during a student clinical experience. This review provides a summary of outcome measures for educators to consider for evaluation of interprofessional activities during student clinical placements and serves to inform future conversations regarding the use and development of outcome measures to provide evidence for student achievement of IPE objectives and competencies.

Keywords

clinical placement interprofessional education students systematic review

Interprofessional Education (IPE) has been the subject of several World Health Organization (WHO) reports because interprofessional collaboration is critical to the provision of effective and efficient health care (Reeves et al., 2013; Wei et al., 2020). IPE is associated with improvements in connecting patients with primary care, patient safety and case management, optimal use of skills in a healthcare team, and the provision of better health services for patients (Pinto et al., 2020; Reeves et al., 2010). Canada has embraced the WHO recommendation to embed interprofessional collaboration in academic programs (Canadian Interprofessional Health Collaborative [CIHC], 2010), and as an essential component of healthcare innovation—the “Triple aim for Canadians: Better health, better care, and better value” (Government of Canada, 2015). Regulated health professional programs in Canada have been requested to demonstrate the foundation of interprofessional collaboration in academic curriculum through IPE activities in an accreditation self-study report for more than a decade (Health Professions Regulatory Advisory Council, 2008). Additionally, health professional Colleges promote discipline-specific core competencies as basic building blocks for health professional trainee education and professional development (CIHC, 2010). IPE-related curriculum activities and core competencies relating to interprofessional collaboration are being implemented as evidence by accreditation of health professional programs and graduation of health professional students. However, less is known about how these IPE-related activities and core competencies are being evaluated across health professional programs. A systematic review of studies employing evaluation tools or outcome measures to assess student learning from IPE-related academic and/or clinical curriculum was not found. Therefore, this systematic review is unique and addresses a gap in the literature about what outcome measures could be used to assess health professional student learning of IPE-related skills, knowledge and behaviors in academic and clinical learning settings.

Evidence shows that, regardless of the interprofessional care model adopted, there are increasingly positive patient, provider and system-level outcomes as a result of interprofessional engagement. In a 2013 Cochrane review, seven studies indicated that interprofessional teams produced more positive outcomes in: diabetes care; emergency department culture and patient satisfaction; collaborative team behavior in operating rooms; management of care delivered in cases of domestic violence; and mental health practitioner competencies related to the delivery of patient care (Reeves et al., 2013). Authors also found a significant reduction in clinical error rates for emergency department teams (Reeves et al., 2013). In summary, the findings suggest that interprofessional health care teams perform better than health professionals do in isolation.

For health professional students, the clinical training environment circumvents previous challenges to IPE and training including common training program(s) timetabling, student level(s) of education/preparation, professional culture(s), and additional clinical preceptor involvement. Embedding interprofessional clinical health education and training of health care professional students within in-patient and out-patient settings augments academic curricula preparation during lectures, laboratories, tutorials/seminars, simulation suites, patient partners and objective structured clinical examinations (OSCEs). Evidence also suggests that students enjoy participating in programs that aim to provide IPE. In research conducted by Sheu et al. (2011), working in interprofessional clinics allowed medical (n = 42) and physiotherapy (n = 28) students to better understand each other’s disciplines. The experience improved the students’ teamwork and communication skills and increased mutual respect while reducing the disciplinary perceptual divide. Of the students participating in this study, 92% agreed that interprofessional learning helped them become a more effective member of the health care team (Sheu et al., 2011). In another student-led clinic, physiotherapy students (n = 18) developed leadership skills, competency in hands-on clinical and administrative experience, and pride from working in a student-led clinical setting (Black et al., 2013).

There are multiple published systematic reviews addressing the effects of IPE interventions on collaboration and patient care (Brewer et al., 2013); however, no published systematic review of studies employing evaluation tools or outcome measures to assess the effectiveness of IPE activities and learning during student clinical placements was found. The aim of this review was to: (1) collate the evaluation tools used to assess the effectiveness of interprofessional activities during student clinical experiences, placements and/or residencies, and (2) to determine the quality of evidence of studies assessing interprofessional learning during student clinical experiences. These aims align with Health Force Ontario’s (2010) goal of enabling system-wide implementation by providing systems and tools that will allow IPE evaluation to be organized. This review supports the concept of creating and using a standard measurement for the assessment of the effectiveness of IPE activities in health professional students. Understanding the available outcome measures used to assess IPE activities and learning is a necessary step toward supporting an evidence-based approach to future IPE implementation and evaluation (Brewer & Stewart-Wynne, 2013).

Method

Data Sources

Computer-aided searches were conducted in the following databases: PubMed (MEDLINE), CINAHL, EMBASE, and SCOPUS (Proquest Nursing and Allied Health). These databases were selected to ensure literature saturation based on the recommendation of a Research and Instructional University Librarian at xx University. A manual search of bibliographies from review and original articles was also performed to ensure literature saturation, and no additional articles were found. The International Prospective Register of Systematic Reviews (PROSPERO) was searched for completed systematic reviews on this topic. The current systematic review protocol was registered with PROSPERO on July 31, 2018 (registration number: CRD42018103397). The protocol was written in accordance with the PRISMA-P and PRISMA E&E transparent reporting of systematic reviews recommendations.

The words used in the search of computerized databases included interprofessional education OR IPE OR IPEP OR Collaborat* practice* OR interdisciplinary education AND clinical placement OR placement OR clinic* OR student placement OR interdisciplinary placement OR clinical education AND evaluation tool* OR outcome measure.* All search terms were searched as keywords in addition to each database-specific subject heading, which varied between databases (e.g., student placement in CINAHL and clinical education in EMBASE). The search strategy was developed in conjunction with a Research and Instructional University Librarian on October 22, 2018 and was conducted on July 4, 2018. The search was replicated in July 2019 and no new additional articles were found.

Selection Process

The inclusion criteria were: (1) design: randomized controlled trials (RCTs), including cluster RCTs, controlled (non-randomized) clinical trials (CCTs), cluster trials, interrupted time series (ITS) studies with at least three data points before and after the intervention, controlled before-after (CBA) studies, prospective and retrospective comparative cohort studies, case-control or nested case-control studies, cross-sectional studies, case series, case reports and qualitative research studies; (2) sample population: any student training population with participants in any health professional program (e.g., Physiotherapy/Physical Therapy, Occupational Therapy, Speech-Language Pathology, Audiology, Medicine, Nursing, Dentistry, Dietetics, Social Work, Podiatry, Pharmacy, Exercise Physiology, Clinical Psychology, Paramedics); and (3) intervention: any intervention that included evaluation tools/outcome measures assessing the effectiveness of interprofessional activities and learning during student clinical placements. Studies were selected for inclusion based on any length of follow-up and there were no restrictions by type of setting. Studies were excluded from the systematic review if they were: (1) theoretical articles; (2) descriptions of treatment approaches or methodological protocols; (3) review articles; or (4) non-English language articles. No limit was applied to year of publication.

For the purpose of this review, an IPE intervention is defined as members of more than one health profession learning interactively together for the purpose of improving interprofessional collaboration or the health/wellbeing of patients/clients. Interactive learning requires active learner participation, and active exchange between learners from different professions (Reeves et al., 2013).

Data Extraction

Both review authors independently examined the titles and abstracts yielded by the search against the inclusion criteria. Both reviewers then screened the full text reports and confirmed that the reports met the inclusion criteria. The two reviewers sought additional information from study authors where necessary to resolve questions about eligibility. Disagreements between the reviewers were resolved through discussion to achieve consensus. Reasons for excluding trials from the review were recorded. Neither of the reviewers were blind to the journal titles, study authors, or institutions.

Quality Assessment

The methodological quality of each study with quantitative or mixed methods research designs (i.e., a research design that used both quantitative and qualitative methodologies) was assessed independently by the two review authors using the Physiotherapy Evidence Database (PEDro) Scale and a Cochrane informed assessment of bias. The PEDro Scale, designed to be used for therapy/intervention studies, includes assessment of randomization, blinding, attrition, design, and statistics. Each of the ten criteria were independently graded “1” for “yes” and “0” for “no” or “unclear” with a maximum score of 10 as per the PEDro Scale scoring protocol. A judgment as to the possible risk of bias was made from the extracted information, rated as “high” or “low” risk (below or above 5/10, respectively; Verhagen et al., 1998). If there was insufficient detail reported in the study, we judged the risk of bias as “unclear” and the original study investigators were contacted for more information. When authors did not respond with clarifications within a period of seven business days, the score given on the item was “0.”

The Cochrane informed assessment of bias judged studies as having “high” or “low” risk of bias in five categories: selection, performance, detection, attrition and reporting bias. Selection bias assessments considered systematic differences between baseline characteristics of the groups being compared and allocation concealment. Performance bias assessments considered systematic differences between groups in the care that was provided and possible blinding of study participants and personnel. Detection bias assessments considered differences between groups in how outcomes were determined. Attrition bias assessments considered differences between groups in withdrawals from the study, and reporting bias assessments considered differences between reported and unreported findings (Higgins & Green, 2011).

The methodological quality of each study with qualitative or mixed methods research designs was assessed independently by the two review authors using the Confidence in the Evidence from Reviews of Qualitative Research (CERQual) tool for assessing confidence in qualitative research findings. The tool seeks one of four overall judgments (i.e., high, moderate, low, or very low confidence in results) by assessing methodological limitations, relevance, coherence and adequacy. Studies with high overall confidence are very likely to be reasonable representations of their phenomenon of interest. The assessment of methodological limitation seeks to analyze the extent to which there are problems in the design or conduct of the study. The assessment of relevance asks if the evidence is applicable to the context specified within the study. Assessments of coherence seek to describe the fit between the data and the findings. Lastly, assessments of adequacy seek to determine the richness and quantity of data supporting a finding (Lewin et al., 2018).

Data Analysis

The first aim of this review was to examine the evaluation tools used to assess the effectiveness of interprofessional activities during student clinical placements. The evaluation tools used to assess the interprofessional activities were chosen a priori as variables that may influence the primary study outcome. The authors also intended a priori to complete a meta-analysis. However, the heterogeneity across the retrieved studies did not allow meta-analysis of the results for different study characteristics (i.e., the length and types of interventions, and outcome measures used assess their effectiveness).

Results/Findings

Study Selection and Characteristics

A summary of the selection process used in this review is illustrated in Figure 1. The original search yielded 857 articles. There was, however, duplication of 142 articles among the databases. After review of the titles and abstracts of 631 articles, 84 full text articles were assessed on the basis of the inclusion criteria established for this systematic review. Studies were excluded because: studies did not include student participants (n = 4); studies did not include an interprofessional student clinical experience, placement or residency (n = 45); studies did not include an IPE effectiveness outcome measure (n = 5); and studies were not intervention studies (i.e., they were reviews, methodological protocols, or theoretical articles; n = 5). Twenty-five articles published between 1997 and 2018 were included in the review. Six of the papers used quantitative research methods, eight used qualitative research methodologies, and 11 used a mixed methods approach to data collection and analysis.

Figure 1.

Flowchart of the process of the systematic review identification, screening, and eligibility.

Participants and Settings

The studies included in this review contained a total of 1,675 participants ranging from nine (Lumague et al., 2006) to 388 (McGettigan & McKendree, 2015) participants per study (see Table 1 for study characteristics). The mean age of the study participants ranged from 23.11, with unreported variation (McNair et al., 2005), to 27.1 ± 6.8 years (Seamen et al., 2018). Only five of the 25 included studies included information about age, and only seven studies included information about the sex of their participants, with a mean of 65% female participants. Eleven studies were conducted in Australia, six in Canada, four in the United Kingdom, and one in each of the United States of America, Scotland, Denmark and New Zealand.

Table 1.

Summary of Study Details for Papers Included in the Systematic Review.

Study	Study Setting	Sample	Study Design	Duration of Study	Intervention	Frequency of Exposure	Outcome(s)	Measure(s)	Variable Measured	Methodological Quality
Anderson et al. (2014) Australia	Southern Clinical School at Monash University Clinic	N = 38 Age = NA % female = NA Students in final year Medicine and Nursing (mainly), and students in Pharmacy and “Allied Health” were involved when they were available	Quantitative	6-months	All students participated in ward-based interprofessional clinical placements in emergency or rehabilitation settings	20 rounds of placement, each 2-weeks in length	Students rated their learning environment highly (mean = 83%). The question-set was rated to have a Content-Validity Index = 0.93 and a Cronbach α = 0.80	The Interprofessional Clinical Placement Learning Environment Inventory (ICPLEI)	Measures student’s perceptions of four key variables in an interprofessional clinical learning environment: orientation, supervision, roles, learning and autonomy	PEDro score (x/10) = 4 High risk of bias
Annear et al. (2016) Australia	Two separate Residential Aged Care Facilities (RACF’s) Community	N = 127 Age = NA % female = NA Students in Undergraduate Medicine, Nursing and Paramedics	Quantitative	5-days	The Interprofessional Education Initiative was based on a seven-step protocol that revolves around collaborative assessment of adults with complex care needs: (1) preparatory meetings with academic and RACF stakeholders, (2) student introductions to IPE, (3) collaborative biopsychosocial assessments with older-adult volunteers, (4) RACF mentor review of student assessments and care recommendations, (5) Grand-round presentation of student recommendations, (6) Organizational implementation of interprofessional team recommendations, (7) Protocol review	Daily	On The UWEIPQ, significant improvements on communication, teamwork and interprofessional relationships (p < 0.001) On the DKAT 2, significant improvement in knowledge of dementia (p < 0.001)	University of West England Interprofessional Questionnaire (UWEIPQ) Dementia Knowledge Assessment Tool Version 2 (DKAT 2)	UWEIPQ -Measures students’ self-assessments of their communication skills, their attitudes toward interprofessional learning, and their perceptions of different professions and interprofessional interactions DKAT 2—Measures family and care workers’ foundation-level knowledge of dementia	PEDro score (x/10) = 4 High risk of bias
Brewer et al. (2013) Australia	Royal Perth Hospital and Curtin University Acute Care	N = 79 Age = 24 % female = 54% Students in Medicine, Nursing, Physiotherapy, Occupational Therapy, Social Work, Pharmacy, Dietetics and Medical Imaging	Quantitative	6-weeks	During clinical placement students were instructed to include the client and their families in care planning in student teams. The placement consisted of profession specific tasks (e.g., dispensing medications, showering and wound dressing), facilitated group learning sessions and reflective sessions	Three rotations of 2- or 3-week clinical placements (75 hrs total)	Improvements in ISVS score (p < 0.05) indicated an improvement in attitudes towards interprofessional collaboration	Curtin University’s Interprofessional Capability Assessment Tool (IAT) Interprofessional Socialization and Valuing Scale (ISVS) Quantitative tool* designed to measure attitudes and values towards interprofessional collaboration (34-items)	IAT—Self- and facilitator assessments of Client-Centred Service/Care, Client Safety & Quality Collaborative Practice ISVS—Measures self-report interprofessional socialization among students and health practitioners and their readiness to function in interprofessional teams	PEDro score (x/10) = 4 High risk of bias
Craig et al. (2014) Australia	Rural Southeast New South Wales Clinic	N = 79 Age = NA % female = NA Students in Pharmacy, Nursing, “Allied Health”	Quantitative	2-years	Students collaborated with interprofessional team members on a project, responded to patient and community needs, and demonstrated an awareness of how a rural location affects community care	6-weeks	IEPS scores increased with participation (p < 0.001) and the TPS differed between teams (p < 0.05)	Interprofessional Education Perception Scale (IEPS) Team Performance Scales (TPS) Feedback Questionnaire	IEPS—Measures professionally oriented perceptions and related affective domains for participants in interdisciplinary education programs through four subscales: competency and autonomy, perceived need for cooperation and perception of actual cooperation and understanding others’ roles. TPS—measures the self-report quality of your team interactions and performance	PEDro score (x/10) = 4 High risk of bias
Dando et al. (2011) United Kingdom	Princess Alice Hospice Rehabilitation Ward Rehabilitation	N = 59 Age = NA % female = NA Students in Nursing, Occupational Therapy, Physical Therapy and Medicine	Mixed Methods	9-month period of three-week placements	Multidisciplinary groups n = 12 students provided hands-on care for a selected group of patients under supervision	3-weeks (plus a 2-day induction program developed to help prepare students for the hospice environment)	Qualitative: The placement was positively evaluated, and students reported an increased understanding of both their own role and that of other professionals on the team. An in-patient unit of a hospice caring for patients with life-limiting illness provides a suitable environment to demonstrate and learn about interprofessional practice	Quantitative: Quantitative data assessments were not provided Qualitative: A generic written evaluation of student experience of the placement (questionnaire with responses measured along Likert scales and free-text comments), and a specific evaluation for students on placement using open-ended questions to explore experience in a palliative care setting	NA	PEDro score (x/10) = 2 High risk of bias CERQual risk of bias = Moderate confidence
Frakes et al. (2014) Australia	Capricornia Allied Health Partnership (CAHP) Clinic	N = 47 Age = NA % female = >50% Students in Exercise Physiology, Nutrition and Dietetics, Occupational Therapy, Pharmacy, Podiatry and Social Work	Qualitative	≥ 3-weeks	Placements in an interprofessional chronic disease ambulatory clinic focusing on early intervention for clients with multiple chronic disease	30% of students time was spent in an interprofessional team completing intake interviews, participating in conferences and completing professional communication responsibilities, and 70% was spent performing profession specific activities and consultations	In interviews, the largest improvements were reported in discipline-specific knowledge (77%), understanding of interprofessional practice (72%), and communication skills (60%)	Interviews with thematic analysis	NA	CERQual risk of bias = Moderate confidence
Freeth et al. (2001) United Kingdom	London Royal Hospital Training Ward Acute Care	N = 18 Age = NA % female = NA Students in Nursing, Medicine, Occupational Therapy, and Physical Therapy	Mixed Methods	4-weeks	Students planned and delivered care for a group of orthopedic and rheumatology patients in an interprofessional team during clinical placement. The students participated in reflective sessions	2-week placement	Quantitative: Students developed both profession-specific skills in a real-world setting and the quality of their IPE teamwork Qualitative: The findings were grouped into a number of themes, which offer insight into the varying perspectives of the IPE training: the ward was a success, gaining real life experience, nurse facilitation, learning and teaching, team duties, equality and patients	Quantitative: Questionnaires* (on learning outcomes and patient satisfaction) were used Qualitative: Semi-structured interviews (exploring staff views and expectation of the pilot ward with training ward facilitators and clinical staff)	NA	PEDro score (x/10) = 3 High risk of bias CERQual risk of bias = High confidence
Hood et al. (2014) Australia	Clinical School Rehabilitation Ward Rehabilitation	N = 23 Age = NA % female = NA Students in Medicine, Nursing, Occupational Therapy and Physical Therapy	Qualitative	2-weeks	Participation in a ward-based interprofessional education placement	10-days	Qualitative: Five themes emerged from the focus group interviews: describing their engagement in trying on a professional role; experiencing independence and autonomy; seeing clearly what nursing’s all about, altered images of other professions; ways of communicating and collaborating and becoming a functional team member	Qualitative: Focus Group Interviews	NA	CERQual risk of bias = High confidence
Hunter et al. (2015) Canada	Metropolitan Pediatric Academic Tertiary Care Centre Acute Care	N = 21 Age = NA % female = 86% Students in Child Life Studies, Medicine, Nursing, Occupational Therapy, Pharmacy, Physical Therapy and Radiation Sciences	Mixed Methods	1-year	Multidisciplinary groups (n = 6–9) participated in an introductory tutorial followed by four weekly facilitated tutorials. During the fifth tutorial, trainees delivered a joint presentation and reflected on their experiences	Weekly for 5 weeks	Quantitative: Significant changes were reported in pain knowledge outcomes on the PNKAS (p < 0.01), on the IEPS (p < 0.05), PAC (p < 0.01) and PNCS (p < 0.05) Qualitative: Three themes emerged: knowledge of other people’s roles and scopes of practice, knowledge base of resources and referrals, and greater understanding of pain and pain management	Quantitative: The Pediatric Nurses Knowledge and Attitudes Survey (PNKAS) Interdisciplinary Education Perception Scale (IEPS) Perception of Actual Cooperation (PAC) Perceived Need for Cooperation Subscale of IEPS Interprofessional Care Core Competencies Global Rating Scales (IPC-GRS) Qualitative: Focus group interviews conducted by one experienced individual were analyzed using content analysis that coded data according to the study objectives and were organized into categories that reflected the emerging themes	PNKAS—measures pediatric nurses’ current attitudes and knowledge regarding pain IEPS—Measures professionally oriented perceptions and related affective domains for participants in interdisciplinary education programs through four subscales: competency and autonomy, perceived need for cooperation and perception of actual cooperation and understanding others’ roles. PAC –measures self-reported perceptions of team cooperation IPC-GRS—Measures perceived learning of the IPE core competencies based on the Framework for the Development of IPE Values and Core Competencies	PEDro score (x/10) = 5 Low risk of bias CERQual risk of bias = Very low confidence
Jakobsen et al. (2014) Denmark	Danish Interprofessional Training Unit Acute Care	N = 16 Age = NA % female = NA Students in Nursing, Physiotherapy, and Occupational Therapy	Qualitative	2-weeks	Students worked with patients admitted for planned arthroplasty or hip fracture with experienced Interprofessional Clinical Tutors	7 hrs/day on weekdays limited to 5-days	A 1-week interprofessional clinical placement can contribute to students learning about interprofessional collaboration and their development of professional identity	Focus Group semi-interviews with questions about interprofessionalism, unprofessionalism, professional identity, supervision and information. Three themes emerged: student experience, clinical tutor experience, and managerial perspective	NA	CERQual risk of bias = Low confidence
Lawlis et al. (2016) Australia	Residential Aged Care Facilities Community	N = 12 Age = NA % female = NA Students in Nursing, Occupational Therapy and Aged Care	Mixed Methods	3 weeks	Students were allocated to one of two interprofessional groups in pairs, and in consultation with facilitators and nurses, with the goal to improve residents who were living with dementia’s food intake via “dementia friendly eating tools” or “food memories”	3hrs/week	Quantitative Positive changes were found in students’ attitudes towards other health professionals (p < 0.05). Understanding of the importance of working in interprofessional teams was reported with significant differences between two statements: “learning with healthcare students before qualifications would improve relationships after qualifications,” and “I learned a lot from the students from the other disciplines” (p < 0.05) Qualitative Students reported greater awareness of self and situations, improved communication and teamwork and understanding of their environment	Quantitative Modified Readiness for Interprofessional Learning Scale (RIPLS) Qualitative: 2 open-ended questions on a questionnaire and three formal group debriefing sessions	RIPLS—Measures the attitudes and perceptions of students and professionals to determine their readiness for interprofessional learning and change through three subscales: teamwork and collaboration, negative and positive professional identity, and roles and responsibilities	PEDro score (x/10) = 4 Low risk of bias CERQual risk of bias = Moderate confidence
Lumague et al. (2006) Canada	Toronto Rehab Stroke Inpatient Unit Rehabilitation	N = 9 Age = NA % female = NA Students in Medicine, Nursing, Occupational Therapy, Pharmacy, Physical Therapy, Social Work, and Speech-Language Pathology	Qualitative	5-weeks	Interprofessional Group Sessions and weekly tutorials.	Weekly	Increased respect and knowledge of roles, contributions and expertise of various health care professional in the delivery of health care services to patients/clients Increase. Knowledge and understanding of the importance of interprofessional collaboration and communication, and improved skills in building interprofessional relationships to maximize team function	Group interview	NA	CERQual risk of bias = Moderate confidence
McGettigan & McKendree, 2015 The United Kingdom	Goole and District Hospital and Hull York Medical School (HYSM) Acute Care	N = 388 Age = NA % female = NA Students in Medicine, Nursing, Physiotherapy and Occupational Therapy	Mixed Methods	3-years	Students participated in all aspects of care, not just their own profession-specific activities. They negotiated with colleagues “who does what.” All students contributed as needed to assisting patients with activities of daily living. Multidisciplinary team meetings were held daily	Daily for 7 consecutive days within a 2-week rotating period over 7-months	Quantitative: Students showed significant improvements in the RIPLS measures of Teamwork (p < 0.0001), Professional Identity (p = 0.0001), and Patient-Centered Care (p = 0.0001)	Quantitative: Questionnaire for Psychological and Social factors at work (QPSNordic) Readiness for Interprofessional Learning Survey (RIPLS) Qualitative: Focus Groups	QPSNordic—self-report measure of how workers perceive their work RIPLS—Measures the attitudes and perceptions of students and professionals to determine their readiness for interprofessional learning and change through three subscales: teamwork and collaboration, negative and positive professional identity, and roles and responsibilities	PEDro score (x/10) = 5 Low risk of bias CERQual risk of bias = Moderate confidence
McNair et al. (2005) Australia	14 Departments across eight Victorian Universities Clinic	N = 91 Age = 23.11 % female = 71% Students in Undergraduate Medicine, Nursing, Physical Therapy and Pharmacy	Quantitative	2-weeks	Activities involved sharing of views (exchange-based), working collaboratively on a project (action-based), shadowing different professions followed by discussion (observation-based), and assigning two or more students from different professions to the same placement (practice-based)	Daily	Quantitative: The rural IP educational experience improved IP abilities. Several significant changes to pre and post placement included knowledge and understanding of roles and how tasks are shared in a primary care team, attitudes towards nurses, and agreement that IPE should be a core part of pre-service training was found (p < 0.01)	Quantitative: Quasi-experimental pre-posttest design with self-report measures related to student learning objectives	NA	PEDro score (x/10) = 3 High risk of bias
Nagelkerk et al. (2018) Canada	Midwest Interprofessional Practice, Education and Research Center Clinic	N = 22 Age (SD) = 27 (3.8) % female = 50% Students in Medicine, Pharmacy and Physician Assistant Programs	Mixed Methods	3-years	Program included educational modules on Interprofessional collaborative practice and daily huddles, focus patient visits, phone calls, team-based case presentations, medication reconciliation, and student-led diabetes educational classes	Medical Students = 4-weeks Pharmacy Students = 6-weeks Physician Assistant Students = 8-weeks	Increased practice efficiency measured by the average number of patients seen/provider/hour Significant knowledge gains in Team Dynamics and Tips for Behavioral Changes knowledge tests (p < 0.05) Significant decreases in HgbA1c (p < 0.05) and glucose (p < 0.01)	Interdisciplinary Education Perception Scale (IEPS) Entry-level Interprofessional Questionnaire (ELIQ)	IEPS—Measures professionally oriented perceptions and related affective domains for participants in interdisciplinary education programs through four subscales: competency and autonomy, perceived need for cooperation and perception of actual cooperation and understanding others’ roles. ELIQ—measures student self-assessment of communication, team work skills, and attitudes toward collaborative learning and collaborative working	PEDro score (x/10) = 3 High risk of bias CERQual risk of bias = Moderate confidence
O’Brien et al., 2013 New Zealand	Akoranga Integrated Health of the School of Interprofessional Health Studies at AUT University Clinic	N = 37 Age = NA % female = NA Students in Physiotherapy, Podiatry, Oral Health, Nursing, Occupational Therapy and Counselling Psychology	Mixed Methods	1-2 days to 1-year	Students in the clinic attended weekly meetings, participated in interprofessional tutorials, and combined treatment sessions with patients requiring input from two or more professions	NA	Quantitative: The ISVS subscale scores were high; 4.92, 4.70, 4.47, (p < 0.01) Qualitative: On the Clinical Placement Short Responses Questionnaire 83% of students had a good experience, 91% stated that it changes how they related to other health professionals and 78% gained a better understanding of what other health professionals did	Quantitative: Interprofessional Socialization and Valuing Scale (ISVS) Qualitative: Clinical Placement Short Responses Questionnaire* (contains short open-and-closed-ended answer questions about student placement perceptions)	ISVS—Measures self-report interprofessional socialization among students and health practitioners and their readiness to function in interprofessional teams	PEDro score (x/10) = 5 Low risk of bias CERQual risk of bias = Moderate confidence
O’Carroll et al. (2012) Scotland	National Health Service (NHS), Fife Clinic	N = 44 Age = NA % female = NA Students in Medicine, Nursing and “Allied Health Professions”	Qualitative	6-months	Workshops with learning activities that explored comparing professional identities, discussing roles and responsibilities within a healthcare team and practicing nontechnical clinical skills via case scenarios	Four workshops lasting 2-3 hours	Students who participated in the workshops reported that they developed a better understanding of each other’s roles and responsibilities and also identified that this would be transferable knowledge to their future practice	Questionnaire* based on Kirkpatrick’s model (Barr et al., 1999) was used with open-ended questions that explored the students view of the learning experience and workshops that evaluated their acquisition of skills, knowledge and perceptions of learning with students	NA	CERQual risk of bias = Moderate confidence
Pinto et al. (2012) Canada	Toronto Hospitals Acute Care	N = 64 Experimental Group n = 22 Age = 24 Range = 10 (21-31) % female = 95% Control Group n = 22 Age = 26 Range = 11 (22-33) % female = 90% Note: demographics were only collected for 44/64 participants Students from Creative Arts Therapies, Clinical Dietetics, Nuclear Medicine, Nursing, Occupational Therapy, Physical Therapy, Physical Therapy Assistants, Pharmacy, Radiation Technology, Radiological Technology, Speech-Language Pathology, Social Work, Therapeutic Recreation and “other” (n = 2)	Mixed Methods	5-weeks	Experimental Group: facilitated program consisting of one introductory tutorial and four weekly patient-themed tutorials and an inter-professional student presentation. Students learned about one another, explored common professional interested, knowledge, experiences and individual learning styles and needs. In weeks 2-5 participants met for case studies Control Group: traditional placement—no effort was made to promote collaboration, but interactions may have occurred because of the nature of their working environment	Week 1: 3-hour tutorial Weeks 2-5 once/week for 1.5-hour tutorials	Quantitative: There were no statistically significant differences between groups after the structured IPE clinical placement, but the intervention group showed a greater positive trend in total IEPS scores from baseline to follow-up (p < 0.05) Qualitative: Students valued the knowledge and skills gained through the structured IPE clinical placement	Quantitative: Interdisciplinary Education Perception Scale (IEPS) Qualitative: Focus Group Interviews (n = 7) included students from Pharmacy, Creative Arts Therapy, Physical Therapy Assistant and Speech-Language Pathology	IEPS—Measures professionally oriented perceptions and related affective domains for participants in interdisciplinary education programs through four subscales: competency and autonomy, perceived need for cooperation and perception of actual cooperation and understanding others’ roles.	PEDro score (x/10) = 5 Low risk of bias CERQual risk of bias = Low confidence
Reeves et al. (2002) United Kingdom	Royal London Hospital Acute Care	N = 36 Age = NA % female = NA Students in Medicine, Nursing, Occupational Therapy and Physiotherapy	Mixed Methods	2-weeks	Student teams provided patient care in a musculoskeletal training ward with both team-oriented and profession-specific supervision	2-week placement	Quantitative: Questionnaire data was analyzed by generating frequencies and cross-tabs Qualitative: Group interviews held with all students at end of the placement found that students valued the experiential learning they received and felt it prepared them for future practice; however, many students encountered difficulties adopting an autonomous learning style	Quantitative: Questionnaire* Qualitative: Group interview data was analyzed by searching for recurrent themes with a search for “counter instances”	NA	PEDro score (x/10) = 3 High risk of bias CERQual risk of bias = High confidence
Richardson et al. (1997) Canada	St. Peter’s Geriatric Day Hospital at McMaster Hospital Clinic	N = 10 Age = NA % female = NA Students in Nursing, Social Work, Occupational Therapy, Physiotherapy, and Therapeutic Recreation	Qualitative	2-years	Multidisciplinary groups (n = 5) students placed in student teams the context of the larger clinical Geriatric Day Hospital team	6-weeks (with a weekly 2 hr tutorial)	Qualitative: Four themes emerged during the team conference; Outside Resources and Outcome Measures, Goal Setting with Cognitively Impaired Clients, Communication/Conflict Style, and Client- and Family-Centered Practice	Qualitative: Feedback Form (FF) (specific questions and answers available in authors Table 2), Exit Interviews (semi-structured) and Team Conferences	NA	CERQual risk of bias = High confidence
Richardson et al. (1999) Canada	St. Peter’s Geriatric Day Hospital at McMaster Hospital Clinic	N = 30 Age = NA % female = NA Experimental Group: n = 15 Control Group: n = 15 Students in Nursing, Physical Therapy, Occupational Therapy, Social Work, and Therapeutic Recreation	Mixed Methods	3-years	Experimental Group: Multidisciplinary groups (n = 5) with 1 student from each discipline placed in student teams in the context of the larger clinical geriatric day hospital team. With weekly 2 hr problem-based tutorials. Control Group (n = 5): traditional placement—no effort was made to promote collaboration, but interactions may have occurred because of the nature of their working environment. Not offered the problem-based tutorials Experimental and control groups were matched for gender, educational program and level within program	6-weeks (with a weekly 2 hr problem-based tutorial)	Quantitative: Students’ scores on the FAQ increased (p < 0.05) Qualitative: Two themes emerged from the analysis of the exit interviews; “preceptor roles and interplay between student team,” “tutorial experience”, and “functioning on larger team.” Four themes emerged from the Team Meetings; “outcome measures,” “client-centered practice,” “professional interpretation,” and “student collaboratively operating as a team”	Quantitative: The Facts on Aging Quiz (FAQ) The Aging Opinion Survey (AOS) Feedback Form (specific questions and answers available in authors Table 2) Interprofessional Education Perception Scale (IEPS) Team Observation Protocol (TOP) Control group did not complete the feedback form or participate in the qualitative evaluation Qualitative: Exit interviews (semi-structured) and team meetings	FAQ—measures knowledge about basic facts and frequent misconceptions about aging AOS—measures personal opinions with regard to the process of aging IEPS—Measures professionally oriented perceptions and related affective domains for participants in interdisciplinary education programs through four subscales: competency and autonomy, perceived need for cooperation and perception of actual cooperation and understanding others’ roles. TOP—measures conversations in a multidisciplinary team context through seven categories of statement: client, team, question, information, interpretation, alternative, and decision.	PEDro score (x/10) = 6 Low risk of bias CERQual risk of bias = High Confidence
Saunders et al. (2016) Australia	Ambulatory Care Setting at the University of Western Australia Clinic	N = 19 Age = NA % female = NA Students in the final semester of graduate entry to the Master of Nursing Science program	Qualitative	2-years	Case-based discussions as part of reflection on holistic, patient-centered care in respiratory, diabetes and chronic pain clinics	1 discussion/week for a 2-week placement	A deductive analysis of nursing student reflective journal entries, the analysis found that all students were able to reflect on their interprofessional clinical experience, but the ability to reflect varied between students	Written reflections after the placement	NA	CERQual risk of bias = Moderate confidence
Saunders et al. (2018) Australia	Outpatient Clinics (specializing in Respiratory Medicine/ Endocrine/ Pain) Clinic	N = 92 Age = NA % female = NA Students in Medicine and Nursing	Qualitative	2-weeks	Preplacement preparation included completing an online self-directed learning activities. In pairs, students provided interprofessional care to patients in clinic and during home visits	2-week placement with weekly student led cased based discussions	Quantitative: No statistically significant differences in median overall scores across all questions after the placement Qualitative: Students strongly supported the model of clinical supervision with the nurse educator and teaching registrar working collaboratively to facilitate the placement; however, some students indicated they already had been well prepared for IPE	Quantitative: 14 fixed-response questions (available in the authors Table 3 and 5) rated on a 5-point Likert-scale (“strongly disagree” to “strongly agree”) Qualitative: Five open-ended questions	NA	CERQual risk of bias = Low confidence
Seamen et al. (2018) Australia	Clinical Outpatient Clinics in Ambulatory Care Clinic	N = 62 Age = 27.1 (SD = 6.8) % female = 67.7% Students in Nursing and Medicine	Mixed Methods	4-years	Students were placed in two of four clinical outpatient areas in interprofessional pairs. Students took part in collaborative patient reviews and assisted in the delivery of outpatient care under supervision while supporting the care of patients with chronic illnesses in hospital outpatient clinics and home visits	2-week placement	Quantitative: There was statistically significant increase in scores on the ISVS following placement (p < 0.001) with a mean improvement = 6.76 on the overall ISVS score. Group differences were found between nursing and medical students with nursing students showing larger change in overall score and subscales Qualitative: Themes emerged to indicate positive experiences and change in attitudes regarding interprofessional activities (48.5% rated as “good: or “very good”; however, students expressed concern about the timing of placements and lack of opportunity to participate in profession-specific clinical activities	Quantitative: Interprofessional Socialisation and Valuing Scale (ISVS) Qualitative: Open-ended question survey (regarding the placement experience, perspective on the impact of placements, learning experiences, and future planning)	ISVS—Measures self-report interprofessional socialization among students and health practitioners and their readiness to function in interprofessional teams	PEDro score (x/10) = 5 Low risk of bias CERQual risk of bias =Low confidence
Zaudke et al. (2016) The United States of America	University of Kansas Medical Centre Department of Family Medicine Acute Care	N = 252 Age = NA % female = NA Students in Medicine, Physical Therapy, Occupational Therapy, Health Information Management, Clinical Psychology and Medical-Legal Partnership	Quantitative	2-years	Integrated Experiential Model during clinical placements integrating nine curricular activities associated different topics that pair with the IPEC core competency domains; conflict resolution interprofessional simulation, error disclosure simulation, agents of change, social determinants of health simulation, journal club, quality improvement, teaching, and home visits	The number of clinic days varied between students, e.g. Physical Therapy and Occupational Therapy students worked at least one half-day per week, and Medicine students worked six half-days per week	On the RIPLS, students’ attitudes toward teamwork (p < 0.05) and patient-centeredness (p < 0.01) were significantly more favorable for all respondents after exposure to the interprofessional practice experience	Readiness for Interprofessional Learning Scale (RIPLS)	RIPLS—Measures the attitudes and perceptions of students and professionals to determine their readiness for interprofessional learning and change through three subscales: teamwork and collaboration, negative and positive professional identity, and roles and responsibilities	PEDro score (x/10) = 4 High risk of bias

Note. BMI = body mass index, BP = blood pressure, NA = not available, IP = Interprofessional, CERQual: Confidence in the Evidence from Reviews of Qualitative Research, PEDro: Physiotherapy Evidence Database, hrs: hours, *indicates when questionnaires or feedback forms were not provided or available by study authors. Studies are listed alphabetically by first authors last name.

Group Comparison

One study (Pinto et al., 2012) used two groups of participants—one control group (with a traditional placement where no effort was made to promote collaboration—although these types of interactions may have occurred because of the nature of the working environment), and one experimental group (with a facilitated program consisting of one introductory tutorial, four weekly patient-themed tutorials, and an interprofessional student presentation). The other 24 studies used a single group study design.

Intervention Programs

Table 1 presents the characteristics of the IPE interventions. The total duration of the student interprofessional placements varied from daily (Annear et al., 2016; McGettigan & McKendree, 2015; McNair et al., 2005) to weekly (Hunter et al., 2015), while the intervention duration ranged from 5-days (Annear et al., 2016) to 4-years (Seaman et al., 2018).

Outcome Measures and Intervention Effects

Table 1 also presents the IPE outcomes and significance values (where applicable) of the 25 included studies, with 13 of the quantitative/mixed-methods studies reporting statistically significant improvements in at least one measure of IPE (Annear et al., 2016; Brewer et al., 2013; Craig et al., 2014; Hunter et al., 2015; Lawlis et al., 2016; McNair et al., 2005; McGettigan & McKendree, 2015; Nagelkerk et al., 2018; O’Brien et al., 2013; Pinto et al., 2012; Richardson et al., 1999; Seamen et al., 2018; Zaudke et al., 2016).

Studies included one (Anderson et al., 2014; Frakes et al., 2014; Jakobsen & Hansen, 2014; Lumague et al., 2006; McNair et al., 2005; O’Carroll et al., 2012; Saunders et al., 2016; Zaudke et al., 2016) to six (Richardson et al., 1999) IPE outcome measures. The 12 outcome measures used across the 25 studies were: (1) The Readiness for Interprofessional Learning Scale (RIPLS); (2) University of West England Interprofessional Questionnaire (UWEIPQ); (3) Interdisciplinary Education Perception Scale (IEPS); (4) Interprofessional Clinical Placement Learning Environment Inventory (ICPLEI); (5) Perception of Actual Cooperation (PAC); (6) Perceived Need for Cooperation Subscale; (7) Interprofessional Care Core Competencies Global Rating Scales (IPC-GRS); (8) Interprofessional Socialization and Valuing Scale (ISVS); (9) Entry-level Interprofessional Questionnaire (ELIQ); (10) Questionnaire for Psychological and Social factors at Work (QPSNordic); (11) Interprofessional Clinical Placement Learning Environment Inventory (ICPLEI); and (12) Curtin University’s Interprofessional Capability Assessment Tool (IAT).

Among the 13 studies which found significant changes in their IPE outcome measure, the RIPLS (Lawlis et al., 2016; McGettigan & McKendree, 2015; Zaudke et al., 2016), UWEIPQ (Annear et al., 2016), IEPS (Craig et al., 2014; Hunter et al., 2015; Nagelkerk et al., 2018; Pinto et al., 2012; Richardson et al., 1999), and ISVS (Brewer et al., 2013; O’Brien et al., 2013; Seamen et al., 2018) were used as outcome measures. These significant quantitative studies had interventions that ranged from 5-days (Zaudke et al., 2016) to 3-years (Richardson et al., 1999).

The 12-item IEPS was the most frequently used outcome measure in studies with quantitative data reporting—appearing in seven of the included studies (see Table 2). IEPS measures attitudes and perceptions of the interprofessional experience using a 5-point Likert scale and many versions of the IEPS exist (Vaughan et al., 2014). The scale has four subscales: competency and autonomy, perceived need for cooperation, perception of actual cooperation and understanding other’s roles (Cameron et al., 2009). The IEPS has good psychometric properties with a Cronbach α consistently reported to be > 0.80 (McFayden et al., 2007) and as high as 0.91 (Lie et al., 2013). Test-retest weighted kappa values for items on the IEPS are consistently rated as “fair to moderate” (McFayden et al., 2007).

Table 2.

IPE Outcome Measures Used Among the Quantitative and Mixed Methods Studies (N = 17) in This Review.

		Type of Setting
Outcome Measure	Frequency of Use (/17)	Rehabilitation	Community	Home Care	Acute Care (Hospital)	Clinic (Outpatient/Ambulatory Care)
Interdisciplinary Education Perception Scale (IEPS)*	6	1	2		2	1
Interprofessional Socialization and Valuing Scale (ISVS)*	2				1	1
The Readiness for Interprofessional Learning Scale (RIPLS)*	2	1			1
Curtin University’s Interprofessional Capability Assessment Tool (IAT)	1				1
Entry-level Interprofessional Questionnaire (ELIQ)	1				1
Interprofessional Care Core Competencies Global Rating Scales (IPC-GRS)	1	1
Interprofessional Clinical Placement Learning Environment Inventory (ICPLEI)	1				1
Interprofessional Clinical Placement Learning Environment Inventory (ICPLEI)	1	1
Perceived Need for Cooperation Subscale	1	1
Perception of Actual Cooperation (PAC)	1	1
Questionnaire for Psychological and Social factors at Work (QPSNordic)	1				1
University of West England Interprofessional Questionnaire (UWEIPQ)*	1			1

Note. *indicates outcomes with statistically significant changes as a result of the IPE intervention. Outcome measures are listed first by frequency of use, then alphabetically.

The 19-item RIPLS and the 24-item ISVS were each used in two of the studies in this review, both of which found significant changes in these measures after an IPE intervention. The RIPLS also uses a 5-point Likert scale to assess perceptions of healthcare students’ knowledge, skills and attitudes regarding their readiness to learn with other healthcare professionals, and overall demonstrates appropriate reliability and validity measures (Binienda, 2015). The RIPLS has been found to have a Cronbach α of 0.85 (Lie et al., 2013). The ISVS uses a 7-point Likert scale to measure shifts in beliefs, behaviors, and attitudes that underlie interprofessional socialization. It was designed to measure transformative learning (King et al., 2010) and has a Cronbach α of 0.90 (Schmitz, 2016).

Methodological Quality of the Included Studies

The results of the methodological quality assessment of studies with quantitative reporting are presented in Table 1 (PEDro Scores) and Figure 2 (Cochrane informed assessment of types of bias). On the PEDro Scale, the studies with quantitative components (n = 17) in this review fulfilled 2 to 6 quality criteria out of the maximum of 10. Initial inter-rater reliability of PEDro scores before discussion to reach consensus was ρ = 0.67. Thirteen of the 17 studies scored < 5 on the PEDro Scale, and therefore were considered to have risk of high bias (Pedroso et al., 2012). On average, the 17 studies were of high bias (PEDro = 4, SD = 0.88). Therefore, only four of the included studies were considered to have low risk of bias (see Table 1). All but three studies met the following standard criteria of the PEDro: specified eligibility criteria and obtained measures of a least one of their key outcomes of interest (e.g., an IPE outcome measure) from more than 85% of the participants (O’Brien et al., 2013; Seaman et al., 2018; Zaudke et al., 2016). All but one study provided the treatment or control condition as allocated (McNair et al., 2005), and all but four studies provided both point measures and measures of variability for at least one key outcome (Dando et al., 2011; Freth et al., 2001; McGettigan & McKendree, 2005; Nagelkerk et al., 2018). However, no investigators were able to blind participants, investigators, or assessors to the IPE intervention and as a result, allocation was not concealed. The lack of blinding and allocation is not indicative of low study quality since it was not possible to blind participants to their IPE intervention. A lack of blinding resulted in a deduction of 3 points on the PEDro score for each of these studies. The two most frequent methodological shortcomings or biases identified included lack of specifying eligibility criteria and lack of statistical comparisons being reported for at least one key outcome. In addition, the Cochrane risk of bias assessment (see Figure 2) shows that reporting bias was relatively low across studies (i.e., >75% were rated as having “low bias”), while performance bias was high (i.e., >75% were rated as having “high bias”). Attrition, detection and selection biases were rated between 50-75% as highly biased.

Figure 2.

Percentage of studies containing quantitative reports with low and high risk of different types of bias.

On the CERQual assessment (n = 19) of confidence in qualitative study findings, the number of studies associated with each overall judgement of study confidence was as follows: high confidence, n = 5; moderate confidence, n = 9; low confidence, n = 4; and very low confidence, n = 1 (see Figure 3). Methodological limitation ratings of high confidence were present in nine studies, moderate confidence in four studies, low confidence in five studies, and very low confidence in one. Relevance ratings of high confidence were present in 17 studies, moderate confidence in one study, and low confidence in one study. Coherence ratings of high confidence were present in eleven studies, moderate confidence in four studies, low confidence in three studies, and very low confidence in one. Adequacy ratings of high confidence were present in six studies, moderate confidence in four studies, and low confidence in nine studies.

Figure 3.

Breakdown of CERQual assessments of confidence in study findings for studies containing qualitative reports.

Discussion

The current systematic review was designed to examine studies employing evaluation tools to assess the effectiveness of interprofessional education (IPE) activities for learning during student clinical placements. Understanding what tools are used to measure IPE activities and learning is a necessary step toward a system-wide approach to IPE implementation (Health Force Ontario, 2010). The results of 13 studies with quantitative reporting demonstrated statistically significant improvement in IPE outcome measure scores. The IEPS was the most commonly used outcome measure to assess interprofessional learning by health care professional students during clinical experiences; however, the fact that so few studies used the same outcome measures to assess their IPE intervention demonstrates that further research is needed before generalization of reliable evidence can be made. These findings offer a summary for future researchers aiming to implement outcome measures in their research to assess IPE activities in clinical training of health professional students. Consistency in measures used in research would enable future comparison and potentially meta-analyses of scores on outcome measures used to evaluate interprofessional learning.

The results of the qualitative reports suggest that IPE results in: increased respect and knowledge of roles, contributions and expertise of various health care professionals in the delivery of health care services to patients/clients (Dando et al., 2011; Hunter et al., 2015; Lumague et al., 2006; McNair et al., 2005; O’Brien et al., 2013; O’Carroll et al., 2012); increased knowledge and understanding of the importance of interprofessional collaboration and communication (Frakes et al., 204; Jakobsen & Hansen, 2014; Pinto et al., 2012; Richardson et al., 1999; Saunders et al., 2018); improved skills in building interprofessional relationships to maximize team functions (Freeth et al., 2001; Hood et al., 2014); and shared student IPE experiences that included acknowledgment of the benefits gained (Lawlis et al., 2016; Lumague et al., 2006; Seamen et al., 2018).

Limitations

The results of this systematic review were limited by the fact that there was no standard outcome measure for the assessment of IPE interventions. In this review a variety of measures for assessing student learning with IPE interventions were used across the included studies. Six of the studies used only quantitative assessment measures, eight used qualitative assessments measures, and eleven employed mixed-methods assessments. Importantly, the different tools assess different components of IPE. The results of this review offer an opportunity to circumvent this limitation in future data synthesis and meta-analyses by suggesting commonly used outcome measures that researchers could use in future studies.

We are unable to determine, based on the results of this review, whether the effectiveness of an interprofessional intervention for students in health disciplines was due to the volume of time spent in an interprofessional environment, properties of the outcome measures used to access change in interprofessional learning, or to quality of the interprofessional activity experiences. A definitive statement about which outcome measure was employed by health discipline cannot be made from this systematic review because the study participants were not homogenous in terms of health discipline.

Future Research

Enhanced IPE learning and practice for health professional students and improved patient outcomes are expected to facilitate and expand interprofessional clinical training within in-patient and out-patient rehabilitation clinical programs. This integrated interprofessional clinical learning approach may be attractive to potential health professional students and, with demonstrated outcomes, may be a recruitment opportunity for health care programs and provide a high-quality student education experience. Despite this, a previously published systematic review of studies summarizing the evaluation tools/outcome measures used to assess the benefit of IPE activities during clinical placements was not found.

Many students of health care programs, supervised by preceptors of their respective discipline, are on-site at health care centers in which intrinsic opportunities exist to foster a culture of IPE and practice with a focus on excellence in client-centered health care. Members of the patient health care team regularly supervise students in clinical experience placements. These patient health care team members work together to provide interprofessional client-centered care. Students of these health care team members observe elements of interprofessional health care practices and may contribute discipline-specific care within the context of the health care team. An opportunity exists to deepen the student contribution by fostering a culture for “students working with students” under the supervision of their discipline-specific preceptors, if we can better understand how to evaluate the benefit of IPE activities for student learning in clinical placements.

Conclusions

Interprofessional student placements contribute to developing healthcare providers with the skills and knowledge needed to work in a collaborative manner (CIHC, 2010). However, guidance about which measures are useful in what settings, and how students demonstrate learning from IPE activities during clinical placements was limited. This systematic review provides a summary of outcome measures for educators to consider for evaluation of interprofessional activities during student clinical placements in order to allow future comparisons between interventions. The IEPS was the most frequently used outcome measure to assess interprofessional learning during a student clinical experience and may therefore serve as the standard outcome measure for researchers to use in their studies in order to allow for future direct comparison between IPE interventions. Further, the findings of this review may serve to inform future conversations regarding the use of existing measures of IP learning and the development of new outcome measures to evaluate achievement of IPE objectives and competencies.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Nicole A. Guitar

References

Anderson

Cant

Hood

(2014). Measuring student’s perceptions of interprofessional clinical placements: Development of the Interprofessional Clinical Placement Learning Environment Inventory. Nurse Education in Practice, 14(5), 518–524. https://doi.org/10.1016/j.nepr.2014.05.009

Annear

M. J.

Goldberg

L. R.

Robinson

(2016). Interprofessional curriculum development achieves results: Initial evidence from a dementia-care protocol. Journal of Interprofessional Care, 30(3), 391–393. https://doi.org/10.3109/13561820.2015.1117061

Binienda

(2015). Critical synthesis package: Readiness for interprofessional learning scale (RIPLS). MedEdPORTAL, (11). https://doi.org/10.15766/mep_2374-8265.10274

Black

J. D.

Palombari

K. M.

Dole

R. L.

(2013). Student experiences in creating and launching a student-led physical therapy pro bono clinic: A qualitative investigation. Physical Therapy, 93(5), 637–648. https://doi.org/10.2522/ptj.20110430

Brewer

M. L.

Stewart-Wynne

E. G.

(2013). An Australian hospital-based student training ward delivering safe, client-centred care while developing students’ interprofessional practice capabilities. Journal of Interprofessional Care, 27(6), 482–488. https://doi.org/10.3109/13561820.2013.811639

Cameron

Ignjatovic

Langlois

Dematteo

DiProspero

Wagner

Reeves

(2009). An introduction to interprofessional education for first-year health science students: Perspectives of pharmacy students and faculty. American Journal of Pharmaceutical Education, 73(4), 1–7. https://doi.org/10.5688/aj730462

Canadian Interprofessional Health Collaborative Competencies Working Group. (2010). A national interprofessional competency framework. Canadian Interprofessional Health Collaborative. http://www.cihc.ca/files/CIHC_IPCompetencies_Feb1210.pdf

Craig

P. L.

Barnard

Glasgow

May

(2014). Evaluating the health “Hubs and Spokes” interprofessional placements in rural New South Wales, Australia. Journal of Allied Health, 43(3), 176–183. https://www.ingentaconnect.com/content/asahp/jah/2014/00000043/00000003/art00011

Dando

d’Avray

Colman

Hoy

Todd

(2011). Evaluation of an interprofessional practice placement in a UK in-patient palliative care unit. Palliative Medicine, 26(2), 178–184. https://doi.org/10.1177/0269216311400479

10.

Frakes

K. A.

Brownie

Davies

Thomas

Miller

M. E.

Tyack

(2014). Experiences from an interprofessional student-assisted chronic disease clinic. Journal of Interprofessional Care, 28(6), 573–575. https://doi.org/10.3109/13561820.2014.917404

11.

Freeth

Reeves

Goreham

Parker

Haynes

Pearson

(2001). “Real life” clinical learning on an interprofessional training ward. Nurse Education Today, 21(5), 366–372. https://doi.org/10.1054/nedt.2001.0567

12.

Government of Canada. (2015). Unleashing innovation: Excellent healthcare for Canada. Report of the advisory panel on healthcare innovation (Naylor Report). https://healthycanadians.gc.ca/publications/health-system-systeme-sante/report-healthcare-innovation-rapport-soins/alt/report-healthcare-innovation-rapport-soins-eng.pdf

13.

Health Force Ontario. (2010). Implementing interprofessional care in Ontario: Final report of the interprofessional care strategic implementation committee. http://www.ipe.uwo.ca/Administration/teachings/HFO%20IPC%20Final%20Report%202010.pdf

14.

Higgins

J. P. T.

Green

(2011). Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration. www.handbook.cochrane.org.

15.

Hood

Cant

Leech

Baulch

Gilbee

(2014). Trying on the professional self: nursing students’ perceptions of learning about roles, identity and teamwork in an interprofessional clinical placement. Applied Nursing Research, 27(2), 109–114. https://doi.org/10.1016/j.apnr.2013.07.003

16.

Health Professions Regulatory Advisory Council. (2008). Interim report to the Minister of Health and Long-term Care on mechanisms to facilitate and support interprofessional collaboration among health colleges and regulated health professionals. Retrieved September 17, 2020, from https://www.hprac.org/en/reports/resources/HPRAC-EnglishInterprofessionalCollaborationInterimReportMarch08.pdf

17.

Hunter

J. P.

Stinson

Campbell

Stevens

Wagner

S. J.

Simmons

van Wyk

(2015). A novel pain interprofessional education strategy for trainees: Assessing impact on interprofessional competencies and pediatric pain knowledge. Pain Research and Management, 20, e12–e20. https://doi.org/10.1155/2015/159580

18.

Jakobsen

Hansen

(2014). Spreading the concept: An attempt to translate an interprofessional clinical placement across a Danish hospital. Journal of Interprofessional Care, 28(5), 407–412. https://doi.org/10.3109/13561820.2014.900479

19.

King

Shaw

Orchard

C. A.

Miller

(2010). The interprofessional socialization and valuing scale: A tool for evaluating the shift toward collaborative care approaches in health care settings. Work, 35(1), 77–85. https://doi.org/10.3233/WOR-2010-0959

20.

Lawlis

Wicks

Jamieson

Haughey

Grealish

(2016). Interprofessional education in practice: Evaluation of a work integrated aged care program. Nurse Education in Practice, 17, 161–166. https://doi.org/10.1016/j.nepr.2015.11.010

21.

Lewin

Booth

Glenton

Munthe-Kaas

Rashidian

Wainwright

Carlsen

(2018). Applying GRADE-CERQual to qualitative evidence synthesis findings: Introduction to the series. Implementation Science, 13(2), 1–70. https://doi.org/10.1186/s13012-017-0688-3

22.

Lie

D. A.

Fung

C. C.

Trial

Lohenry

(2013). A comparison of two scales for assessing health professional students’ attitude toward interprofessional learning. Medical Education Online, 18, 21885. https://doi.org/10.3402/meo.v18i0.21885

23.

Lumague

Morgan

Mak

Hanna

Kwong

Cameron

Sinclair

(2006). Interprofessional education: The student perspective. Journal of Interprofessional Care, 20(3), 246–253. https://doi.org/10.1080/13561820600717891

24.

McFadyen

A. K.

Maclaren

W. M.

Webster

V. S.

(2007). The Interdisciplinary Education Perception Scale (IEPS): An alternative remodelled sub-scale structure and its reliability. Journal of Interprofessional Care, 21(4), 433–443. https://doi.org/10.1080/13561820701352531

25.

McGettigan

McKendree

(2015). Interprofessional training for final year healthcare students: A mixed methods evaluation of the impact on ward staff and students of a two-week placement and of factors affecting sustainability. BMC Medical Education, 15, 185. https://doi.org/10.1186/s12909-015-0436-9

26.

McNair

Stone

Sims

Curtis

(2005). Australian evidence for interprofessional education contributing to effective teamwork preparation and interest in rural practice. Journal of Interprofessional Care, 19(6), 579–594. https://doi.org/10.1080/13561820500412452

27.

Nagelkerk

Thompson

M. E.

Bouthillier

Tompkins

Baer

L. J.

Trytko

Groeneveld

(2018). Improving outcomes in adults with diabetes through an interprofessional collaborative practice program. Journal of Interprofessional Care, 32(1), 4–13. https://doi.org/10.1080/13561820.2017.1372395

28.

O’Brien

McCallin

Bassett

(2013). Student perceptions of an interprofessional clinical experience at a university clinic. New Zealand Journal of Physiotherapy, 41(3), 81–87. https://search.proquest.com/docview/1468441975?pq-origsite=gscholar&fromopenview=true

29.

O’Carroll

Braid

Ker

Jackson

(2012). How can student experience enhance the development of a model of interprofessional clinical skills education in the practice placement setting? Journal of Interprofessional Care, 26(6), 508–510. https://doi.org/10.3109/13561820.2012.709202

30.

Pedroso

R. V.

de Melo Coelho

F. G.

Santos-Galduróz

R. F.

Costa

J. L. R.

Gobbi

Stella

(2012). Balance, executive functions and falls in elderly with Alzheimer's disease (AD): a longitudinal study. Archives of Gerontology and Geriatrics, 54(2), 348–351.

31.

Pinto

Lee

Lombardo

Salama

Ellis

Kay

Landry

M. D.

(2012). The impact of structured inter-professional education on health care professional students’ perceptions of collaboration in a clinical setting. Physiotherapy Canada, 64(2), 145–156.

32.

Pinto

R. M.

Kay

E. S.

Choi

C. J.

Wall

M. M.

(2020). Interprofessional collaboration improves linkages to primary care: A longitudinal analysis. AIDS care, 32(8), 970–978.

33.

Reeves

Freeth

McCrorie

Perry

(2002). It teaches you what to expect in future: Interprofessional learning on a training ward for medical, nursing, occupational therapy and physiotherapy students. Medical Education, 36(4), 337–344. https://doi.org/10.1046/j.1365-2923.2002.01169.x

34.

Reeves

Perrier

Goldman

Freeth

Zwarenstein

(2013). Interprofessional education: Effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, (3). https://doi.org/10.1002/14651858.CD002213.pub3

35.

Reeves

Zwarenstein

Goldman

Barr

Freeth

Koppel

Hammick

(2010). The effectiveness of interprofessional education: Key findings from a new systematic review. Journal of Interprofessional Care, 24(3), 230–241. https://doi.org/10.3109/13561820903163405

36.

Richardson

Maureen

Cripps

Mohide

E. A.

Macpherson

A. S.

(1997). Educating students for interprofessional teamwork in the clinical placement setting. Educational Gerontology, 23(7), 669–693. https://doi.org/10.1080/0360127970230705

37.

Richardson

Montemuro

Mohide

E. A.

Cripps

Macpherson

A. S.

(1999). Training for interprofessional teamwork-evaluation of an undergraduate experience. Educational Gerontology, 25(5), 411–434. https://doi.org/10.1080/036012799267684

38.

Saunders

Dugmore

Seaman

Singer

Lake

(2018). Interprofessional learning in ambulatory care. The Clinical Teacher, 16, 41–46. https://doi.org/10.1111/tct.12764

39.

Saunders

Singer

Dugmore

Seaman

Lake

(2016). Nursing students’ reflections on an interprofessional placement in ambulatory care. Reflective Practice, 17(4), 393–402. https://doi.org/10.1080/14623943.2016.1164686

40.

Schmitz

C. C.

(2016). Interprofessional Socialization and Valuing Scale (ISVS-21). Assessment and evaluation at the National Center for Interprofessional Practice and Education. Retrieved October 19, 2019, from https://nexusipe.org/advancing/assessment-evaluation/interprofessional-socialization-and-valuing-scale-isvs-21

41.

Seaman

Saunders

Dugmore

Tobin

Singer

Lake

(2018). Shifts in nursing and medical students’ attitudes, beliefs and behaviours about interprofessional work: An interprofessional placement in ambulatory care. Journal of Clinical Nursing, 27(15–16), 3123–3130. https://doi.org/10.1111/jocn.14506

42.

Sheu

L. C.

Zheng

Coelho

A. D.

O’Suliva

P. S.

O’Brien

B. C.

A. Y.

Lai

C. J.

(2011). Learning through service: Student perceptions on volunteering at interprofessional Hepatitis B Student-run clinics. Journal of Cancer Education, 26(2), 228–233. https://doi.org/10.1007/s13187-010-0142-6

43.

Vaughan

Macfarlane

Dentry

Mendoza

(2014). The interdisciplinary education perception scale (IEPS): Which factor structure?. Education in Medicine Journal, 6(3).

44.

Verhagen

A. P.

De Vet

H. C.

De Bie

R. A.

Kessels

A. G.

Boers

Bouter

L. M.

Knipschild

P. G.

(1998). The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. Journal of Clinical Epidemiology, 51(12), 1235–1241.

45.

Wei

Corbett

R. W.

Ray

Wei

T. L.

(2020). A culture of caring: the essence of healthcare interprofessional collaboration. Journal of Interprofessional Care, 34(3), 324–331.

46.

Zaudke

J. K.

Paolo

Kleoppel

Phillips

Shrader

(2016). The impact of an interprofessional practice experience on readiness for interprofessional learning. Family Medicine, 48(5), 371–376. https://europepmc.org/article/med/27159096