Professional practice behaviour: Identification and validation of key indicators

Abstract

Introduction

Professional behaviour is regarded as an important competency for occupational therapy practice, yet little guidance exists for indicators underpinning development or remediation in the educational or practice settings. This study sought to confirm the content validity of observable professional behaviour indicators from an existing evaluation framework for representativeness and relevance for occupational therapy practice.

Methods

A modified Delphi approach was conducted with expert panellists (n = 30) consisting of regulators, administrators, faculty members, practitioners, and students for professional behaviour indicator consensus, together with a cross-sectional survey of practitioners (n = 119). Fleiss’ κ and χ2 contingency tables were completed for agreement across panellists, and between panellist and survey groups. Cross-case qualitative analyses identified facilitators and barriers for professional behaviour practice.

Results

Content validity of 17 professional behaviour indicators was achieved, with >85% agreement from the expert panellists and the cross-sectional survey group. Main professional behaviour reporting issues in practice included fear of reprisal, lack of formal policies, and an unsupportive culture. Support from others, documented workplace policies, and self-regulation/duty to monitor were the critical facilitators for supporting professional behaviour in practice.

Conclusion

The professional behaviour indicators in this study offer observable behaviours from which assessment rubrics or tools may be developed. Further study is warranted.

Keywords

Professional behaviour professional competence professionalism occupational therapy education

Introduction

Professional behaviour is an expected competency in occupational therapy practice standards (see for example Association of Canadian Occupational Therapy Regulatory Organizations (ACOTRO), 2011; Royal College of Occupational Therapists, 2017), yet there is no singular definition, nor expression of how educational programmes or practice contexts should train or remediate this important competency. Value-laden language for licensed professionals often does not offer clarity of the specific behaviours or indicators necessary to shape, develop, and maintain the competency. For example, the World Federation of Occupational Therapists (2016) states that occupational therapists are responsible for demonstrating ‘professionalism’ when working with organizations and establishing working relationships, yet the term ‘professionalism’ is not defined and specific behaviours are not articulated. While educational and practice settings may have codes of conduct and policies for use of facilities/equipment, these policies are not necessarily linked to the assessment, development, or maintenance of professional behaviour. Despite professional practice, ethical, and regulatory standards, many organizations have observed alarming incidence rates of disrespectful and unprofessional behaviours in healthcare settings, and have since conducted reviews related to the need to define, address, and promote professional and respectful behaviours (see for example Health and Care Professions Council, 2014; Health Quality Council of Alberta, 2013; The Joint Commission, 2008). The aim of this study was to distil expert opinion to define what contributes to professional behaviour, validate behavioural indicators that may be reliable expressions of issues commonly related to professional behaviour, and explore the facilitators and barriers for supporting professional behaviour in practice.

Literature review

Similar to other regulated health professionals, occupational therapists and occupational therapy students are expected to demonstrate professional behaviours and respond appropriately to observed unprofessional behaviours (ACOTRO, 2011; Mak-van der Vossen et al., 2017). While health profession educational programmes have been designed to meet these demands, there is evidence that even though students can demonstrate competence related to content knowledge and hands-on skills within a structured educational environment, they often encounter challenges upon entering a professional practice setting (Evenson et al., 2015). The discrepancy between a student’s in-class knowledge and skill demonstration, and performance within clinical settings, has been linked to challenges with communication and professional behaviour – such as rigidity of thinking, discomfort with ambiguity, lack of psychological insight, difficulty interpreting and responding to feedback, externalization of responsibility, and difficulty learning from mistakes (James and Musselman, 2006).

Similarly, research in medicine has shown a strong association between unprofessional behaviour and complaints, discipline, and litigation against physicians. For example, medical students with the identified behaviours of severe irresponsibility and diminished capacity for self-improvement were more likely to encounter future disciplinary actions (Bahaziq and Crosby, 2011). Furthermore, while a student’s poor academic performance within the first 2 years of the medical programme was a predictor of disciplinary action, incidents of unprofessional behaviour were the strongest predictors of future disciplinary action by a medical licensing board (Papadakis et al., 2005). Additionally, unprofessional behaviour has been linked to poorer health outcomes of patients, patient dissatisfaction, lawsuits, and decreased job satisfaction/retention among healthcare providers (Bahaziq and Crosby, 2011; Hayes et al., 2010).

It has been argued that professional behaviour is a learned skill that should be explicitly taught, modelled, and evaluated within classroom and practice settings (Bahaziq and Crosby, 2011; Cruess and Cruess, 1997). However, a key barrier to the development of effective and reliable teaching and evaluation strategies has been the long-standing challenge of adequately defining professional behaviour (Adam et al., 2013). Professional behaviour has often been documented in terms of unprofessional behaviour, yet there was no common definition constituted in the unprofessional behaviour terms. Recently Mak-van der Vossen and colleagues’ systematic review (2017) found four themes of unprofessional behaviour with 30 descriptors within medical student studies. While instructive for a common unprofessional language, it still defines professional behaviour in terms of what it is not, rather than what it should be.

Learning profession-specific skills and knowledge is not enough: the evidence indicates that health profession students must also acquire requisite behaviours and attitudes if they are to become competent healthcare providers. While the need to cultivate and evaluate professional behaviour has been established, the subsequent development of educational and professional development strategies and evaluative tools in response has been a challenge among many health profession educators and providers (Evenson et al., 2015; Health and Care Professions Council, 2014). Within the profession of occupational therapy, academic and practice-based evaluations inconsistently address the professional behaviours that are observed, and students report a desire for more specific feedback regarding their professional behaviour development (Scheerer, 2003; Steward, 2001). Yuen and colleagues (2016) developed a self-assessment of professional behaviour tool for occupational therapy students. However, while self-assessment is an important aspect of reflective learning and practice, it is often difficult for learners to accurately assess their own behaviour due to limited awareness; thus, feedback from multiple perspectives is a crucial part of developing professional behaviours (Rees and Shepherd, 2005; Regehr and Eva, 2006).

Professional behaviour rubric (PBR)

To address the need for professional behaviour development and in response to professional behaviour concerns within an entry to practice occupational therapy degree programme, a professional behaviour rubric (PBR) was developed (MacKenzie et al., 2004). The original PBR was developed within an educational setting where the term ‘rubric’ is often used to identify a scoring guide or tool that clearly articulates the performance expectations of students (Dawson, 2017). At inception, a working group held structured feedback sessions with students, practice educators, and faculty/staff to gather the range and the scope of the voiced issues. The group also reviewed occupational therapy practice documents, regulatory competencies, and professional standards and codes of ethics from several countries. The PBR was designed to provide consistent and structured formative and/or summative feedback on observed behaviours, raise awareness of unprofessional behaviour, and explore the underlying causal issues to effectively address and advance abilities. The original PBR contained 12 observable behaviour indicators as well as sections for a narrative, descriptive learning plan to address or remediate identified behavioural issues, and student feedback for a learner perspective and response to the documented observations.

The PBR was originally intended to be used as a professional behaviour observational tool by faculty, staff, or preceptors, or as a structured self-reflective evaluation wherein learners had the opportunity to identify their own strengths and/or challenges with the behaviour indicators. However, because abstract ideals and principles do not map easily onto behaviours in the absence of context (Ginsburg et al., 2004), the rubric did not link causal factors to behaviour, but rather identified if the behaviour met or did not meet the levels of performance expected within a given context. This simplicity allowed for the expected behaviour to progress or change across varying practice contexts and learners’ acquisition of occupational therapy knowledge, skills, and attitudes. Finally, to underscore the importance of professional behaviour, the PBR was linked to a school programme progression policy as well as a faculty professional unsuitability policy.

The purpose of this research study was to explore what constitutes a definition of professional behaviour, examine the content validity of the PBR indicators for use within the broader occupational therapy community (for example students and occupational therapists across various settings), and identify barriers and facilitators for professional behaviour development across practice settings.

Methods

A modified Delphi approach (Hasson and Keeney, 2011) was selected for exploring the definition of professional behaviour and PBR indicator validation. To enhance the confidence that our modified Delphi process would produce indicators that were consistent beyond our expert participants, we also employed a cross-sectional online survey with a broader licensed occupational therapy audience (across four Canadian provinces). This follow-up survey was conducted to confirm and verify the panellist group qualitative and quantitative findings from the modified Delphi process (Hasson and Keeney, 2011; Polit and Beck, 2006). This research study was reviewed and approved (REB#2013-3066) by Dalhousie University’s office of Human Research Ethics Administration and all participants provided written or digitally recorded informed consent prior to participation.

Modified Delphi

The modified Delphi design was selected given its ability to: solicit expert opinion in an area of literature insufficiency (Fletcher and Marchildon, 2014); evaluate the representativeness and relevance of each indicator (Polit and Beck, 2006); validate content (van der Schaaf and Stokking, 2011); develop rubric applicability (Parratt et al., 2016); allow for variation in the number of consensus rounds (Hasson and Keeney, 2011); and allow flexibility in mode of participation (Vernon, 2009). Because we had pre-existing PBR indicators from our previous work, we used a more structured process to guide the modified Delphi rounds.

Round 1

The authors started with a review of the original PBR indicators and completed a literature search to explore healthcare definitions of professionalism, professional behaviours, observable traits, evaluation tools, key findings, and recommendations.

Round 2

Expert panellists

Panellists were purposively invited from a range of key stakeholders within the occupational therapy community: student representation from both years of an MSc(OT) programme; clinicians/preceptors from diverse practice sites (for example practice profile and geographic location); occupational therapists with administrative/supervisory responsibilities; faculty members/tutors from Canadian universities known to be involved with professional behaviour education and research; and regulators from the Association of Canadian Occupational Therapy Regulatory Organization. Panellist groups were purposefully arranged according to their stakeholder category to decrease potential power dynamics (Krueger and Casey, 2015), and to prevent the hesitancy to speak openly in the discussion if there was fear of retribution from the hierarchical nature of the various panellists’ positions (for example from students through to professional regulators) (Morrow et al., 2016). Each panellist group attended their respective stakeholder group meeting and completed an online survey.

Prior to the respective stakeholder panellist group meetings in person and/or via teleconference, all participants were provided with the semi-structured interview guide that was used to gather the qualitative data. The questions probed participants’ descriptions of professional behaviour, perceived facilitators or barriers to professional behaviour development or deployment in practice, and an open-ended provision for additional comments or suggested revisions to the PBR indicators. Each facilitated panellist group discussion was audio recorded, then transcripts were transcribed verbatim, reviewed, cleaned, and coded by hand.

In planning for the study and as part of our reflexive analysis, the researchers acknowledge that having an interest in the topic together with being members of the group being studied could potentially affect the research process and outcome (Berger, 2015). To mitigate these concerns of our potential biases, we consciously put several safeguards into place during the qualitative data collection and analysis. More specifically, the initial coding structure was created by a non-occupational therapist research associate not familiar with the PBR or study content. Transcripts and coding were then reviewed by the first two authors, who assessed the structure’s comprehensiveness and accuracy in representing the data. The inductive process of open coding continued with panellist responses examined for meaning in the context of each panellist’s group. Following the individual analysis of each panellist group, cross-case analyses were conducted to determine similarities, unique features, and emerging themes for each panellist group (Thomas, 2006). A constant comparative analysis, relying on both inductive and deductive approaches, was utilized. During this process the researchers were conscious of their influential role in the handling of the qualitative data (Vernon, 2009). To ensure information accuracy during the analysis, the resultant consensus major and minor thematic analysis and PBR indicator revisions were redistributed to each panellist group for member-checking and feedback. Throughout the analysis process the author group discussed and reviewed the coding scheme looking for emerging themes, groupings, and patterns to identify relationships and insights. An audit trail was used to track the decision-making process of data analysis.

Panellists were also provided with a link to an online survey to assess the content validity that the original 12 PBR indicators adequately reflect key professional behaviours. Panellists were asked to rate their level of agreement with each PBR indicator for their respective practice site (1 to 5 Likert scale with 1 = strongly agree, 2 = agree, 3 = neutral, 4 = disagree, 5 = strongly disagree). The online survey was designed using Opinio software (Opinio 7.1.1, Copyright 1998–2019 Object Planet) hosted on the University’s server. Data from Opinio were exported into Microsoft Excel 2016 for Mac 15.23.1 (Microsoft, Redmond, WA).

Additionally, to address concerns that a modified Delphi approach has limited reliability and validity, measures of rigour were applied to both the qualitative and quantitative data. For item content validity, each professional behaviour indicator had to reach an a priori minimum of 85% consensus to be included in further analysis (Diamond et al., 2014).

Cross-sectional survey

The results from the modified Delphi were used to inform the cross-sectional survey to confirm the qualitative and quantitative findings from the consensus round (Hasson and Keeney, 2011; Polit and Beck, 2006). In addition to the PBR indicator analysis, the thematic findings from professional behaviour descriptors, concern regarding a colleague’s professional behaviours, and facilitators or barriers to addressing professional behaviour informed questions with multiple descriptor options.

The cross-sectional survey was sent to registered and licensed occupational therapists in four Canadian provinces and to year 1 and 2 MSc (OT) students at our institution. In three of the four provinces, the cross-sectional survey was distributed by the provincial regulatory body with a survey reminder sent at 2 weeks post initial contact. The fourth province advertised the study in a monthly newsletter to their registrants.

Results

Round 1

In preparation for the study, a literature review was completed to explore definitions of professional behaviour in healthcare educational settings, as well as tools used to evaluate professional behaviour. The following scientific databases were searched: PubMed, CINAHL, and PsycINFO. Additionally, the following grey literature databases were searched: BASE, CogPrints, and Open Grey. Inclusion and exclusion criteria applied included: English, humans, educational setting, and 2000–2015. Two searches were completed: (1) Professional behaviour definition search keywords included: ‘professional behaviour’, ‘define’, ‘education’, and ‘healthcare’. The search yielded 86 articles across the databases and no additional articles were found through hand searching. After review, two articles were selected; (2) Evaluation tools used to assess professional behaviour in healthcare profession educational settings were searched for using the keywords: ‘professional behaviour’, ‘measuring OR evaluating OR assessment’, ‘education’, and ‘healthcare’. The search yielded 172 articles across the databases, which were reviewed. Twelve articles were selected in total, eight from the search and an additional four were added through hand searching.

To situate our data synthesis and overall findings in the current literature, we once again searched the literature (2015–2019) using the same key words and methods as the original literature search. The professional behaviour in the educational setting search yielded eight additional articles, and the tools for professional behaviour evaluation search yielded an additional 15 articles for review. After applying the inclusion and exclusion criteria, three additional articles were selected: one for defining professional behaviours and two regarding evaluation tools.

Table 1 contains the literature synthesis of all 17 articles selected. The majority of literature reviewed on professional behaviour focused on medical student education. A variety of evaluation tools were discussed in the literature. The recommendations from the majority of articles were for additional research to address the validity and reliability of the tools used to evaluate professional behaviours (see for example Roberts et al., 2017; Stratton and Conigliaro, 2012; van Luijk et al., 2010). Although various articles addressed the components of professional behaviour required for a definition, an overarching theme was that until a definition of professionalism is developed, an applicable and singular definition of professional behaviour cannot truly be defined. However, in 2017 Mak-van der Vossen et al. proposed new language for the behavioural themes, and reiterated that common language was important for research, teaching, assessment, and remediation of professional behaviours.

Table 1.

Descriptive summary of articles reviewed.

Source (Country)	Study design	Sample	Defining versus evaluating PB	Tool used to evaluate PB	Key findings	Recommendations
Arnold, 2002(USA)	Literature review	30 years of professionalism literature	Defining and evaluating	N/A	Current assessment tools plentiful, but not strong psychometric properties.Different methods for measurement of PBs: as a part of clinical performance, by itself as a comprehensive entity, and as separate elements of professionalism.	Further research required to strengthen psychometric properties of tools used to evaluate PBs. Suggests research explore whether or not separate evaluation tools should be created for each stage of medical career.Concept of ‘professionalism’ requires clarification.
Cruess et al., 2006(Canada)	Instrument development	Medical clerks	Defining and evaluating	Professional mini evaluation exercise (P-MEX)	Preliminary results indicate content and construct validity of 24 items for the evaluation of professional behaviour that loaded on to 4 factors: doctor–patient relationship skills; reflective skills; time management; and interprofessional relationship skills. Of note: poorly rated items related to reflective skills.	P-MEX is a feasible format for evaluating professionalism in clerkship training to guide teaching and learning. Further research required to determine if certain items are predictive of future professionalism difficulty.
Koenig et al., 2003(USA)	Quantitative study	Occupational therapy clinicians	Evaluating	Philadelphia Region Fieldwork Consortium (PRFC) level 1 student evaluation.	Clinicians surveyed to assist with the development of the PRFC level 1 student evaluation. Panel of experts reviewed results. Focus group to discuss results.Psychometrically strong tool.	Further research required with a national sample.
Mak-van der Vossen et al., 2013(Netherlands)	Design not defined	Medical education programme	Defining and evaluating	‘Professional behaviour assessment scale’. Measures dealing with tasks, others, and oneself.	Definition of PB: ‘the observable aspects of practising professionalism’.Integrated PB into medical training over the six-year programme, including formal training.Developed a PB assessment, including consequences for unprofessional behaviours.	PB as an educational theme should be embedded within medical curriculum.
Randolph, 2003(USA)	Pilot study	Occupational therapy students and faculty	Evaluating	PDE form.Surveys and focus groups to revise PDE.	PDE used in classroom and laboratory setting to prepare students for fieldwork and practice goals.	Further research to assess reliability and validity of PDE, and to determine if accurate for predicting fieldwork behaviours.
Rogers and Ballantyne, 2010(Australia)	Theoretical and empirical investigation	Medical students	Defining	N/A	Five domains of PB: responsibility, relationships with and respect for patients, probity and honesty, self-awareness and capacity for reflection, collaboration and working with colleagues.	Definition of PB must be narrow and have behaviours that can be observed and recorded.Further research to test the validity of the definition.
Schonrock-Adema et al., 2007(Netherlands)	Quantitative study	Medical students	Evaluating	Peer-assessments: based on the assessment for medical schools across the Netherlands	Peer-assessments enhanced scores for the second trimester, once students were accustomed to the learning environment.	Further research to determine replicability.
Stratton and Conigliaro,2012(USA)	Quantitative study	Medical students	Evaluating	Case scenario videos, self-administered questionnaire for responses.	Regardless of gender, students rated behaviours similarly, on average (both between students and between genders of actors).PB varies based on the ‘stage of career/training, level of experience, setting, and/or situation urgency’.	More research required to determine if findings accurate across other medical fields, programmes, and regions.
van Luijk, et al., 2010(Netherlands)	Qualitative survey	Medical, dental, and veterinary schools	Evaluating	Multi-pronged approach	All schools involved with the implementation of recommendations for teaching and assessing PBs made progress (initiated in 2002).Teaching: PB more explicit/visible in curriculum.Assessment: additional committees to support examination boards/give advice.Training faculty: area requiring improvement.	Additional sessions for faculty focusing on providing feedback related to PBs should be implemented, as well as for students.
van Mook et al., 2007(Netherlands)	Quantitative study	Medical students	Evaluating	Questionnaire for students; perceptions of assessment of PB in tutorial groups.	Tutors should not avoid confrontation with students, but should engage and provide appropriate feedback.Tutors should be trained on how to provide feedback of PBs.	Longitudinal design should be used to further explore changes over time.
van Mook et al., 2009a(Netherlands)	Literature review	Medical education	Evaluating	Literature review	No ‘magic bullet’ assessment.Assessment of PB should combine multiple methods (triangulation): varied contexts, multiple observers, over a long period of time.Remediation should be included to address PB follow-up.	Most instruments have been insufficiently studied for content validity, reliability, and feasibility.
van Mook et al., 2009b(Netherlands)	Literature review	Medical education	Evaluating	Literature review	PB frames ‘professionalism’ as observable behaviours: more practical than attempting to assess professionalism, which is mainly a theoretical construct.Outlines problems with current assessments of PB: lack of gold standard, ignoring context such as social pressure, ‘faking’ PB during assessment.	PB should be assessed over multiple occasions, using subjective and objective methods.
van Mook et al., 2010(Netherlands)	Design not defined	Medical education	Evaluating	Reviewing literature related to the Maastricht medical school framework	Framework being used is costly; should decrease number of courses with PB assessment, if possible, to maintain the reliability and validity.Increased attention should be given for the longitudinal aspect of PB assessment.	Formative and summative assessments are complementary and can be used within one PB assessment framework.
Zijlstra-Shaw et al., 2005(Netherlands)	Descriptive analysis/validation study	Dentistry students	Evaluating	‘Assessment of professional behaviour’: ordinal scale assessment form developed	Initial difference between student and staff assessments of PB, decreased over time.The form can be used as a teaching aid, to initiate discussion and feedback regarding PB.	Further research to address the validity and reliability of the form, as well as the sub criteria used.
Mak-van der Vossen et al., 2017(Netherlands)	Systematic review	Medical education	Describing unprofessional behaviours	NA	A total of 30 unprofessional behaviours were identified and were classified into the following four major themes: failure to engage, dishonest behaviour, disrespectful behaviour, and poor self-awareness.	There is a need for common descriptors of unprofessional behaviour within pre-clinical and clinical undergraduate medical education. Common descriptors may support evaluating unprofessional behaviours and providing formative feedback to advance learning.
Roberts et al., 2017(Australia)	Quasi-experimental	Medical education	Evaluating	Modified version of Peer Assessment Instrument (Papinczack et al., 2007)	Peer assessment of professional behaviours is unreliable for decision-making outside of a specific problem-based learning group and cannot be utilized as a summative evaluation of a student’s progression towards curricular standards of professional behaviour.	Formative feedback from educators regarding students’ professional behaviours within PBL groups is recommended, with continued evaluation of the impact on student learning.
Tai and Adachi, 2017(Australia)	Literature review	Post-secondary education	Evaluating	NA	Peer assessment of professional behaviour is a lifelong skill; in professional practice it is peers who evaluate the professional behaviours of their colleagues.	Opportunities for peer evaluation in post-secondary education programmes should not be abandoned; rather, there is a need to advance our knowledge of how and why peer evaluation of professional behaviour supports learning.

PB: professional behaviour; PDE: professional development evaluation; PBL: problem-based learning; NA: not applicable.

Round 2

Expert panellists

In total, 30 expert panellists participated in an online survey and dedicated panellist discussion group, which reflected their stakeholder designation as student (8), faculty member (5), clinician/preceptor (10), administrator (3), and regulator (4). The regulator and faculty panellists included members from across Canada. The remaining panellists consisted of licensed occupational therapists and MSc(OT) students from a restricted geographic area in Canada.

Describing and defining professional behaviour

Cross-case analyses of the panellist qualitative data revealed major and minor themes for describing professional behaviour and identifying facilitators and barriers to addressing professional behaviour concerns. While one unifying definition of professional behaviour did not emerge from the data analysis, panellist data revealed an overarching major theme that awareness underlies a person’s overall ability to behave in a professional manner. Panellists linked awareness (in relation to professional behaviour) to self-regulation, emotional regulation, and contextual flexibility (Figure 1). The other major themes related to defining professional behaviour included being ethical, respectful, responsible, and a competent communicator. Minor themes within each of these four major themes can be found in Figure 1.

Figure 1.

Describing professional behaviour: major and minor themes.

Facilitators and barriers

Panellists identified eight facilitators and nine barriers to addressing professional behaviour concerns. Further analysis of the data revealed that each of the identified facilitators and barriers could be categorized as either an environmental, interpersonal, or intrapersonal factor. The environmental factors that the panellist group identified as facilitators to addressing professional behaviour concerns included (a) support from peers and supervisors when addressing professional behaviour concerns; (b) a workplace culture that supports and encourages peer feedback as a mechanism to enhance professional development and occupational therapy practice; (c) a workplace policy that describes professional behaviour expectations and a process for addressing challenging professional behaviours; and (d) professional practice standards that require occupational therapists to take action to ensure that their own work and the work of other occupational therapists supports professional standards (self-regulation/duty to monitor). Within the student panellist session, one group member emphasized the importance of self-regulation and being responsible for actively addressing ‘subtle disrespect’, and a preceptor panellist group member highlighted the importance of support from others and how the culture of an organization can facilitate addressing professional behaviour concerns:

The biggest facilitator is support. So support from whoever your manager or boss is, and also support from your team. Some teams have a culture or some programmes have a culture to say, you know, we’re going to be honest and open, and we’re going to address things, and kind of move on from there. So, I would say that’s the biggest thing.

Likewise, a panellist from the preceptor group described their beliefs about the importance of having a policy in place and how this can facilitate the process of addressing concerns:

If there’s a rule or a policy in place or something so that it doesn’t feel like you’re making a judgement call but you’re actually kind of following the protocol when you’re looking to address something.

The interpersonal factors reported by the panellists that facilitate professional behaviour development included having professional relationships that embody effective communication, and the ability to give and receive feedback. The intrapersonal factors that were identified as facilitators included feeling safe and supported, yet willing to be vulnerable during times of growth and new learning.

The environmental factors that panellists viewed as barriers to addressing professional behaviour concerns included limited support from others when taking action to address unprofessional behaviours; a workplace culture that does not cultivate or promote peer feedback to enhance practice; not having a policy in place to support professional behaviour accountability; and hierarchical personnel structures that restrict one’s ability and/or willingness to approach peers or superiors to address their unprofessional behaviours. The identified interpersonal factors that present barriers to addressing professional behaviour concerns included poor team communication and inadequate skills when giving and/or receiving feedback. Lastly, the intrapersonal factors that were viewed by the panellists as barriers to addressing professional behaviour concerns included fear of reprisal, feeling unsafe when giving/receiving negative feedback, and an unwillingness to feel vulnerable during times of growth and new learning. A panellist from the administrator group commented on how fear of reprisal prevents team members from addressing the challenging behaviours of colleagues:

The problem is what would happen afterwards … it’s huge for some people. They have the skill. They are able to speak up against that behaviour. It is the after effect – how your colleague relates.

PBR indicator survey

The panellist survey data was assessed for agreement with the 12 original PBR indicators. PBR indicator content validity was determined by those who rated the indicator as a 1 or 2 divided by the total sum of participants in the respective analysis (Polit and Beck, 2006). From Round 2 panellist feedback and trimming of the PBR indicators based on the a priori criterion, 17 PBR indicators were used in the cross-sectional survey for item content validity analysis (Polit and Beck, 2006). The indicators included nine from the original PBR, three revised indicators, and five new indicators.

Cross-sectional survey

Table 2 provides the demographic composition of the participants from the cross-sectional survey.

Table 2.

Cross-sectional survey respondent demographics.

	No.* (%)**
Primary occupational therapy role (n = 119)
Clinician/preceptor	93 (78)
Student	15 (13)
Y1 MScOT = 9; Y2 MScOT = 6
Administrator	8 (7)
Faculty member	2 (2)
Regulator	1 (0.8)
Years since graduation (n = 103)
0–5 years	21 (20)
6–10 years	18 (17)
11–15 years	21 (20)
>15 years	43 (42)
Practice setting (n = 101)
Acute care	11 (11)
Rehabilitation	10 (10)
Long-term care	4 (4)
Mixed practice setting	15 (15)
Outpatient / day treatment	7 (7)
Community	32 (32)
Other	22 (22)
Practice location (n = 100)
Rural	28 (28)
Urban	72 (72)
Concern about professional behaviour of a colleague (n = 115)
Prefer not to answer	5 (4)
No	47 (41)
Yes	63 (55)
Did you address your concerns? (n = 61)
No	29 (48)
Yes	32 (52)

*Survey respondents per category; **Relative percentage per question category.

PBR indicators

Table 3 reports the item content validity for the 17 PBR indicators and the panellist and survey results, respectively. The PBR indicators were converted into dichotomous variables, with 1 and 2 Likert scale values converted to ‘agree’ and the rest converted to ‘disagree’. Missing values were excluded case-wise. Two groups of participants were assessed: the panellist and survey groups. The percent agreement and Fleiss’ κ was computed for both the panellist (n = 30) and survey groups (n = 107). The panellist group was assessed for agreement with the 12 original PBR indicators, and the cross-sectional survey group was assessed with the revised 17 PBR indicators. The percent agreement and Fleiss’ κ was computed for both groups. A z statistic associated with the kappa was used to test the null hypothesis of no agreement. The two groups were compared on the basis of nine common questions (Table 3 PBR indicators 1–8 and 12) using a χ² contingency table analysis. R version 3.3.0 with the irr package (R Core Team, 2013) was used to compute the κ statistics and SPSS, version 23 (IBM, 2015) was used to compute the χ² statistics.

Table 3.

Professional behaviour indicators and item content validity.

PB indicators		Item content validity (%)
	Item source	Survey group(n=107)	Panellistgroup(n=30)
Professional interactions and responsibility
1. Gives adequate notice and explanation for absenteeism	x	93	90
2. Demonstrates respect for the time of others	x	94	93
3. Demonstrates time management skills	x	87	87
4. Demonstrates behaviours respectful to clients, colleagues, guests, instructors, and preceptors	x	96	93
5. Shows ability to balance own needs with the needs of clients, colleagues, guests, instructors, and preceptors	x	88	90
6. Contributes effectively as a team player	x	92	93
7. Self-monitors appearance suited to classroom and/or fieldwork environment	x	86	93
8. Shows respect for the dignity of others	x	94	93
9. Prepared to interact and is accountable across contexts	x*	93
10.Engages in an ethical and safe manner across all contexts	‡	94
11. Actions reflect awareness of professional boundaries	‡	93
Communication
12. Follows appropriate lines of communication	x	92	93
13. Receives constructive feedback effectively	x*	93
14. Gives constructive feedback effectively	x*	92
15. Uses language and communication that meets context and audience expectations	‡	94
16. Body language is appropriate to the context and audience	‡	91
17. Demonstrates responsible, respectful, and safe electronic communication and use of social media	‡	93

PB: professional behaviour; x: original PB indicator; *: original PB indicator with revised wording; ‡: new indicator. Respondents did not answer all questions; missing values are excluded.

Fleiss’ κ analysis shows that there is significant agreement within both the panellist (Fleiss’ κ = 0.359, z = 16, p<0.001) and survey (Fleiss’ κ = 0.521, z = 61.7, p<0.001) groups. Fleiss’ κ and the corresponding z value were substantially greater for the survey group over the panellist group. The χ² contingency table analysis, comparing nine common indicators between the two groups, found there was no significant difference in agreement between the panellist and survey groups, with an agreement rate of about 70%, χ² (6) = 1.895, p = 0.929. In the survey group, content validity of the revised version of the PBR was achieved, demonstrating that the 17 items contribute to identifying potential professional behaviour issues. Our aim was not to create a scoring instrument, but rather a rubric of observable behaviours linked to professionalism that could be applied across multiple stakeholders. As a result, our statistics are not developing cut scores but rather interrater reliability and construct validity. Survey participants were also asked if they would change the wording of the PBR indicators; 78% said no. The yes respondents’ (22%) wording suggestions were either too context-specific or were already represented in another indicator, thus no wording changes were adopted.

Merged panellist and survey data

Defining and describing professional behaviour

The major and minor themes that emerged from the panellist qualitative data were verified through examining the cross-sectional survey data. When asked to select the top three attributes of professional behaviour, 80% of the responses corresponded to the four major themes identified from the panellist qualitative data (ethical behaviour, respectful behaviour, responsible behaviour, and competent communication). The remaining 20% of the responses were attributed to value-based decision-making, awareness, engaged behaviour, authentic behaviour, and demeanour/image. Review of the data revealed that the most frequently selected items across all four questions matched the minor themes shown in Figure 1.

Facilitators and barriers

The cross-sectional survey data confirmed the panellist findings for both the facilitators and barriers to addressing professional behaviour concerns. The top three facilitators (n = 39 respondents) were having support from others, professional behaviour workplace policies, and self-regulation or duty to monitor. The top three barriers (n = 42 respondents) identified included fear of reprisal, lack of formal policies, and lack of support from others. Interestingly, while more than half of the cross-sectional survey participants reported they have had concerns about the professional behaviour of a colleague, only half (52%) of those participants reported that they had taken steps to address their concerns.

Discussion

The aim of this study was to explore the description of professional behaviour in the context of occupational therapy and examine the content validity of the PBR by identifying and evaluating the representativeness and relevance of each indicator across the broader occupational therapy community. Additionally, the study investigated facilitators and barriers for professional behaviour development and maintenance.

Within this study, the 17 PBR indicators validated by participants across various occupational therapy settings share some similarity to those developed by Cruess et al. (2006) for medical students. The indicators in our study are not intentionally or solely for new learners, and were conjointly developed with stakeholders from a broad range of practice contexts. The intent of the PBR design is that the professional behaviour indicators can be used in a competency-based framework, provided the indicator’s level of expected performance is identified (for example from novice to expert). There is a caution for only considering an evaluation category of ‘meets expectations’ (Gingerich et al., 2014; Kogan et al., 2011), as it fails to clearly define the expected benchmark, which then leads to ‘norm referencing’ with peers, who may also not meet the standard of competency (Yeates et al., 2013). While the professional behaviour indictors could be used with a Likert scale or within an analytic rubric, in either case we strongly suggest anchoring a numerical or categorical scoring to specific behavioural descriptions. Ultimately, this allows for the flexible use of the indictors where expectations can be used to support professional behaviour development for context-specific applicability.

Contextual features that facilitate or limit professional behaviour were also central to this study. The compiled results from all participants in this study revealed the most common facilitators and barriers to addressing problematic professional behaviours of colleagues. The identified barriers are similar to those identified by previous researchers, who reported fear of retaliation and workplace hierarchies as key barriers to addressing unprofessional behaviours in healthcare settings (The Joint Commission, 2008). The barriers reported by participants in this study may have resulted in not taking steps to address their observed concerns, with only half of those reporting professional behaviour challenges in the workplace stating that they had taken steps to address the problem. Others have also found that healthcare providers often tolerate and develop an indifference to disruptive and unprofessional behaviour (Porto and Lauve, 2006). Failure to address unprofessional behaviours of colleagues can lead to decreased patient safety, workplace stress, and decreased job satisfaction (see for example Martinez et al., 2017).

Hamblin and colleagues (2015) identified unprofessional behaviour as the most commonly reported behavioural (versus organizational) challenge that triggered a formal incident report of an adverse work event or mistreatment. Based on the results of this study and the work of Porto and Lauve (2006), there is a high likelihood that many unprofessional behaviours, especially those characterized as subtle disrespect, likely remain unreported and may precipitate more severe or unsafe behaviour and the subsequent filing of a formal incident report. Subtle disrespect includes behaviours perceived as rude, unpleasant, or inappropriate, such as making insensitive comments, avoiding certain individuals, publicly criticizing a colleague, and making belittling comments towards an individual (Health Quality Council of Albert, 2013; Zimmerman and Amori, 2011). These relatively common and unreported subtle behaviours can escalate to verbal aggression, bullying, and physical violence. Andersson and Pearson (1999) refer to this escalation of behaviours as the incivility spiral, where relatively minor unprofessional behaviour over time can lead to more aggressive and unsafe behaviours. Creating an academic or workplace culture that proactively identifies and addresses unprofessional behaviour early may halt this spiralling effect and result in the creation of effective, civil, and safe learning and work environments. The use of the validated PBR (Table 3) could assist educators, students, professionals, and organizational leaders in setting behavioural expectations and identifying strategies to address unprofessional behaviours.

In line with previous research (see for example The Joint Commission, 2008) and building on the results of this study, educational settings and healthcare workplaces seeking to cultivate a collaborative approach towards professional behaviour development may want to (a) develop clear policies, procedures, and expectations related to professional behaviour; (b) implement strategies to promote a culture that fosters open communication, support, safety, and professional development; (c) provide education related to effective mechanisms for giving and receiving feedback, especially feedback related to professional behaviour; and (d) provide education sessions on effective self-regulatory practices.

Building on our study’s finding that awareness underlies a person’s overall ability to behave in a professional manner, strategies to develop professional behaviour must also include mechanisms to effectively cultivate one’s awareness of their professional behaviour within varying contexts. We propose that the recently revised and validated PBR indicators can serve as a tool to inform policy development and implementation, cultivate a culture of respect and accountability, and communicate both excellent and challenging professional behaviours. Combined, these actions and strategies can form the foundation for developing self-awareness of one’s professional behaviour and support the improved health and performance of future and current healthcare providers.

Limitations and future directions

The purposive expert panel sampling, while diverse in selection of practice representation, may limit generalizability to all areas of occupational therapy practice, including educational and practice settings outside of Canada, where the study was conducted. Since the indicators on the PBR were gathered from and are relevant to a variety of contexts, including both educational and practice settings, it could provide a seamless tool for use across complex organizational settings (for example from community to hospital, from trainees to experts) and could provide a valuable mechanism for practice education. Further research is required to determine the replicability, evaluation accuracy, and predictability of items linked to professional difficulty across a variety of learning and practice settings. Additionally, the applicability of indicators beyond occupational therapy practice and toward interprofessional interactions warrants investigation.

Conclusion

The updated indicators on the PBR are linked to expected behaviour and practice standards. However, it does not attribute cause, nor do the indicators on their own explicitly set a level of achievement. The PBR serves as a structured feedback form where behaviours are identified and used as an opportunity for formative feedback to facilitate the individual’s awareness and address the behaviour or underlying issues contributing to the behaviour. This study identified key agreed-upon indicators that contribute to professional behaviour and barriers and facilitators for professional behaviour development, and confirmed the content validity of the updated PBR.

Key findings

This study confirmed the content validity of the revised Professional Behaviour Rubric (PBR) indicators.

Facilitators and barriers for professional behaviour development are identified.

Strategies to cultivate professional behaviour are suggested.

What the study has added

This study confirms the content validity of the PBR indicators; this could be a useful directive for formative feedback or to address gaps in research related to professional behaviour development or maintenance.

Footnotes

Acknowledgements

We would like to acknowledge all the participants who contributed to the different parts of this research as well as Kristie Smith for her initial qualitative coding work with the focus group transcripts.

Research ethics

Ethical approval was obtained from Dalhousie University’s office of Human Research Ethics Administration (REB#2013-3066) in 2013; study amendment approval was obtained in 2015 and initiation of the project in 2016.

Consent

All participants provided written or digitally recorded informed consent prior to study participation.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research of this article: Dalhousie Centre for Learning and Teaching Grant (CA$5,000.00).

Contributorship

Diane MacKenzie and Brenda Merritt applied for ethical approval. Diane MacKenzie, Brenda Merritt, and Rebecca Holstead researched the literature and interpreted the qualitative data. Diane MacKenzie, Brenda Merritt, and Gordon Sarty contributed to the methodology of the project. Diane MacKenzie and Gordon Sarty carried out and interpreted the statistical analysis. All authors contributed to writing, reviewing, editing, and approving the manuscript.

References

Adam

Peters

Chipase

(2013) Knowledge, skills and professional behaviours required by occupational therapist and physiotherapist beginning practitioners in work-related practice: A systematic review. Australian Occupational Therapy Journal 60(2): 76–84.

Andersson

Pearson

(1999) Tit for tat? The spiraling effect of incivility in the workplace. Academy of Management Review 24(3): 452–471.

Arnold

(2002) Assessing professional behavior: Yesterday, today, and tomorrow. Academic Medicine 77(6): 502–515.

Association of Canadian Occupational Therapy Regulatory Organizations (2011) Essential Competencies of Practice for Occupational Therapists in Canada, 3rd ed. Toronto, ON: Association of Canadian Occupational Therapy Regulatory Organizations.

Bahaziq

Crosby

(2011) Physician professional behaviour affects outcomes: A framework for teaching professionalism during anesthesia residency. Canadian Journal of Anesthesiology 58(11): 1039–1050.

Berger

(2015) Now I see it, now I don’t: Researcher’s position and reflexivity in qualitative research. Qualitative Research 15(2): 219–234.

Cruess

McIlroy

Cruess

, et al. (2006) The professionalism mini-evaluation exercise: A preliminary investigation. Academic Medicine 81(10): S74–S78.

Cruess

(1997) Professionalism must be taught. British Medical Journal 315(7123): 1674–1677.

Dawson

(2017) Assessment rubrics: Toward clearer and more replicable design, research and practice. Assessment and Evaluation in Higher Education 47: 347–360.

10.

Diamond

Grant

Feldman

, et al. (2014) Defining consensus: A systematic review recommends methodologic criteria for reporting Delphi studies. Journal of Clinical Epidemiology 67(4): 401–409.

11.

Evenson

Roberts

Kaldenberg

, et al. (2015) National survey of fieldwork educators: Implications for occupational therapy education. American Journal of Occupational Therapy 69(Suppl 2): 691235002p1--5.

12.

Fletcher

Marchildon

(2014) Using the Delphi method for qualitative, participatory action research in health leadership. International Journal of Qualitative Methods 13: 1–18.

13.

Gingerich

Kogan

Yeates

, et al. (2014) Seeing the ‘black box’ differently: Assessor cognition from three research perspectives. Medical Education 48(11): 1055–1068.

14.

Ginsburg

Regehr

Lingard

(2004) Basing the evaluation of professionalism on observable behaviors: A cautionary tale. Academic Medicine 79(Suppl 10): S1–S4.

15.

Hamblin

Essenmacher

Upfal

, et al. (2015) Catalysts of worker-to-worker violence and incivility in hospitals. Journal of Clinical Nursing 24(17–18): 2458–2467.

16.

Hasson

Keeney

(2011) Enhancing rigor in the Delphi technique research. Technological Forecasting and Social Change 78(9): 1695–1704.

17.

Hayes

Bonner

Pryor

(2010) Factors contributing to nurse job satisfaction in the acute hospital setting: A review of recent literature. Journal of Nursing Management 18(7): 804–814.

18.

Health and Care Professions Council (2014) Professionalism in Healthcare Professionals. London: Health and Care Professions Council.

19.

Health Quality Council of Albert (2013) Managing Disruptive Behaviour in the Healthcare Workplace. Calgary: Health Quality Council of Albert.

20.

IBM Corporation (2015) IBM SPSS Statistics for Windows, Version 23.0. Armonk, NY: IBM Corp.

21.

James

Musselman

(2006) Commonalities in level II fieldwork failure. Occupational Therapy in Health Care 19(4): 67–81.

22.

Koenig

Johnson

Morano

, et al. (2003) Development and validation of a professional behavior assessment. Journal of Allied Health 32(2): 116–121.

23.

Kogan

Conforti

Bernabeo

, et al. (2011) Opening the black box of clinical skills assessment via observation: A conceptual model. Medical Education 45(10): 1048–1060.

24.

Krueger

Casey

(2015) Focus Groups: A Practical Guide for Applied Research, 5th ed. Thousand Oaks CA: SAGE.

25.

MacKenzie

Landry

Beagan

, et al. (2004) Professional Behaviour Rubric (PBR). Dalhousie University Internal document: unpublished, available from the authors.

26.

Mak-van der Vossen

Peerdeman

Kleinveld

, et al. (2013) How we designed and implemented teaching training, and assessment of professional behavior at VUmc School of Medical Sciences Amsterdam. Medical Teacher 35(9): 709–714.

27.

Mak-van der Vossen

van Mook

van der Burgt

, et al. (2017) Descriptors for unprofessional behaviours of medical students: A systematic review and categorization BMC Medical Education 17(1): 164, 1--12.

28.

Martinez

Lehmann

Thomas

, et al. (2017) Speaking up about traditional and professionalism-related patient safety threats: A national survey of interns and residents. BMJ Quality & Safety 26(11): 869–880.

29.

Morrow

Gustavason

Jones

(2016) Speaking up behaviours (safety voices) of healthcare workers: A metasynthesis of qualitative research studies. International Journal of Nursing Studies 64: 42–51.

30.

Papadakis

Teherani

Banach

, et al. (2005) Disciplinary action by medical boards and prior behavior in medical school. New England Journal of Medicine 353(25): 2673–2682.

31.

Papinczak T, Young L, Groves M, Haynes M (2007) An analysis of peer, self, and tutor assessment in problem-based learning tutorials. Medical Teacher 29(5): e122--32.

32.

Parratt

Fahy

Hutchinson

, et al. (2016) Expert validation of a teamwork assessment rubric: A modified Delphi study. Nurse Education 36: 77–85.

33.

Polit

Beck

(2006) The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health 29(5): 489–497.

34.

Porto

Lauve

(2006) Disruptive clinician behavior: A persistent threat to patient safety. Patient Safety and Quality Healthcare 3: 16–24.

35.

R Core Team (2013) R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing. www.R-project.org/.

36.

Randolph

(2003) Evaluating the professional behaviours of entry-level occupational therapy students. Journal of Allied Health 32(2): 116–121.

37.

Rees

Shepherd

(2005) Students’ and assessors’ attitudes towards students’ self-assessment of their personal and professional behaviours. Medical Education 39(1): 30–39.

38.

Regehr

Eva

(2006) Self-assessment, self-direction, and the self-regulating professional. Clinical Orthopaedics and Related Research 449: 34–38.

39.

Roberts

Jorm

Gentilcore

, et al. (2017) Peer assessment of professional behaviours in problem-based learning groups. Medical Education 51: 390–400.

40.

Rogers

Ballantyne

(2010) Towards a practical definition of professional behaviour. Journal of Medical Ethics 36: 250–254.

41.

Royal College of Occupational Therapists (2017) Professional standards for occupational therapy practice. www.rcot.co.uk/practice-resources/rcot-publications/downloads/professional-standards (accessed 17 Feb 2019).

42.

Scheerer

(2003) Perceptions of effective professional behaviour feedback: Occupational therapy student voices. American Journal of Occupational Therapy 57(2): 205–214.

43.

Schonrock-Adema

Heijne-Penninga

van Duijn

MAJ

, et al. (2007) Assessment of professional behaviour in undergraduate medical education: Peer assessment enhances performance. Medical Education 41(9): 836–842.

44.

Steward

(2001) Using nominal group technique to explore competence in occupational and physiotherapy students during the first-year placements. British Journal of Occupational Therapy 64: 298–304.

45.

Stratton

Conigliaro

(2012) Does gender moderate medical students’ assessment of unprofessional behaviour? Journal of General Internal Medicine 27(12): 1643–1648.

46.

Tai

Adachi

(2017) Peer assessment and professional behaviours: What should we be assessing, how, and why? Medical Education 51(4): 342–350.

47.

The Joint Commission (2008) Behaviors that undermine a culture of safety. Sentinel Event Alert 40: 1–3.

48.

Thomas

(2006) A general inductive approach for analyzing qualitative evaluation data. American Journal of Evaluation 27(2): 237–246.

49.

van der Schaaf

Stokking

(2011) Construct validation of content standards for teaching. Scandinavian Journal of Educational Research 55(3): 273–289.

50.

Van Luijk

Gorter

van Mook

WNKA

(2010) Promoting professional behaviour in undergraduate medical, dental and veterinary curricula in the Netherlands: Evaluation of a joint effort. Medical Teacher 32(9): 733–739.

51.

Van Mook

WNKA

de Grave

Huijssen-Huisman

, et al. (2007) Factors inhibiting assessment of students’ professional behaviour in the tutorial group during problem-based learning. Medical Education 41(9): 849–856.

52.

Van Mook

WNKA

Van Luijk

Fey-Schoenmakers

MJG

, et al. (2010) Combined formative and summative professional behaviour assessment approach in the bachelor phase of medical school: A Dutch perspective Medical Teacher 32(12): e517–e531.

53.

Van Mook

WNKA

Van Luijk

O’Sullivan

, et al. (2009a) The concepts of professionalism and professional behaviour: Conflicts in both definition and learning outcomes. European Journal of Internal Medicine 20(8): e85–e89.

54.

Van Mook

WNKA

Van Luijk

O’Sullivan

, et al. (2009b) General considerations regarding assessment of professional behaviour. European Journal of Internal Medicine 20(4): e90–e95.

55.

Vernon

(2009) The Delphi technique: A review. International Journal of Therapy and Rehabilitation 16(2): 69–76.

56.

World Federation of Occupational Therapists (WFOT) (2016) Minimum Standards for the Education of Occupational Therapists Revised September 2016. World Federation of Occupational Therapists Available at: https://www.wfot.org/assets/resources/COPYRIGHTED-World-Federation-of-Occupational-Therapists-Minimum-Standards-for-the-Education-of-Occupational-Therapists-2016a.pdf (accessed 1 October 2019).

57.

Yeates

O’Neill

Mann

, et al. (2013) ‘You’re certainly relatively competent’: Assessor bias due to recent experiences. Medical Education 47(9): 910–922.

58.

Yuen

Azuero

Lackey

, et al. (2016) Construct validity test of evaluation tool for professional behaviors of entry-level occupational therapy students in the United States. Journal of Educational Evaluation for Health Professions 13: 22, 1--6. DOI: 10.3352/jeehp.2016.13.22

59.

Zijlstra-Shaw

Kropmans

TJB

Tams

(2005) Assessment of professional behaviour – A comparison of self-assessment by first year dental students and assessment by staff. British Dental Journal 198(3): 165–171.

60.

Zimmerman

Amori

(2011) The silent organizational pathology of insidious intimidation. American Society of Healthcare Risk Management 30(3): 5–15.