Abstract
Coaching's rapid growth has increased pressure to have effective coaches. The competency models currently used for training and certification are based on opinion surveys. In management, such a process for developing competency models has been shown to confuse standards of excellence with prejudicial fads. This study examined competencies demonstrated in critical incident work samples of coaching by more effective and less effective coaches as measured by client behavioral change over 2 years. Effective coaches more frequently demonstrated the competencies of achievement orientation, empathy, emotional self-awareness, and coaching and mentoring as well as the total number of emotional and social intelligence competencies shown by a coach as compared to their less effective counterparts. More effective coaches discussed more role differentiation as a coach from other related roles as compared to less effective coaches. Implications for future research and practice in coach training and certification are discussed.
Keywords
Within the constant quest for efficient use of resources to achieve maximum performance, organizations seek selection, development, and retention of human talent to best serve clients and deliver quality (Barney et al., 2021). The decisions and assessment of the situation are framed by models or theories of performance (Boyatzis, 1982). At the same time, in the current era, organizations are attempting to maximize engagement and motivation of their human resources through developmental activities (Van Iddekinge et al., 2023). Similarly, executives, managers, and leaders select developmental activities, and the providers of these activities, guided by models or theories of development and the inherent models or theories of provider performance.
Coaching has grown exponentially as a preferred developmental activity (Zhou, 2024). The training, certification and selection of coaches has, to date, been based on theory-driven or opinion-survey-guided competency models. Without adequate testing of such competency models against the actual performance of coaches, the models may be providing the wrong model of causality for research and practical guidance.
In a study of managers and executives in 12 large organizations (i.e., six from the private sector and six from the public sector), Boyatzis (1982) showed that opinion surveys of competencies perceived to be needed for effective managers at all levels were half incorrect as compared to studies of the competencies that predicted actual performance outcomes. In fact, 25% of the survey-based competencies were found to be irrelevant to performance and 25% were found to be opposite to the competencies that mattered. A review of the competencies used to certify coaches by the four largest associations worldwide derived from opinion surveys suggested that as few as 22% of the behaviors might actually be competencies that predict effective performance of coaches (Boyatzis et al., 2023).
The current study was designed to identify competencies of effective coaches, as compared to less effective ones. Effectiveness was assessed as the degree of client behavior change over 2 years. Client behavior change was measured by a composite score of others’ observations (i.e., not self- assessment) of the improvement in those behaviors of the clients considered relevant to the client's profession. Coach competencies were identified through critical incident interviews.
The contributions of this study include: (a) identifying the most effective competencies of coaches in predicting client behavior change; (b) using qualitative methods to determine behaviors (i.e., competencies) demonstrated in effective coaching experiences; (c) adding support for a theory of effective performance of coaches; (d) suggesting ideas for future research on coach development; and (e) recommending new guidance for designing and using competency models in coach training and certification programs.
Competencies of Effective Coaches
Before competencies are explored, the nature of what is meant by, or can be determined to be, effective coaching should be examined. Coaching can be a role and is also a process used to help people learn, grow, and change. The International Coaching Federation (ICF) defines coaching as, “partnering with an individual or group in a thought-provoking and creative process that inspires them to maximize their personal and professional potential” (ICF, 2019). Coaching is distinct from mentoring or advising in that it is typically shorter in duration and focuses on manifesting desired changes in a person's life, career, or their functioning in specific settings, like health or a job. In an effort to be more precise, the definition adopted in this study was “a facilitative or helping relationship with the purpose of achieving some type of change, learning, or new level of individual and organizational performance” (Boyatzis et al., 2019, p. 15).
Given these definitions, the effectiveness of coaching is determined by the degree to which the client reaches desired outcomes. There is a wide variety of possible outcomes, such as confirmation or change in a client's vision and purpose in life. Other desirable outcomes include pursuit of a new career path, an improvement in their behavior relevant to a specific job or role, or better functioning relationships with others with whom they live and work. Other desired outcomes have been identified as a greater sense of well-being (Spence & Grant, 2007), better health (Hammoud et al., 2023), and/or other possible factors like increased engagement, motivation at work, and career satisfaction (Boyatzis et al., 2022; Van Oosten et al., 2019). A major criticism of any desired outcome is the question of sustainability or duration of the change. Prior research suggests changes from management and leadership training are often fleeting lasting for a “honeymoon period” of 3 weeks to 3 months (Boyatzis et al., 2022; Campbell et al., 1970).
The most frequent desired outcome from coaching is a change in the client's behavior. Passarelli et al. (2022) showed that a year or more after the coaching ended, executives and the coaches continued to assess the effectiveness of the engagement in terms of how much the client's behavior had changed. An eventual desired outcome should also include performance improvement (Boyatzis et al., 2022), including goal attainment. Randomized assignment, comparison group studies of coaching have shown that clients feel as if the process of coaching helped them in their goal aspiration (Spence & Grant, 2007) and they felt satisfied with the coaching (De Haan et al., 2019; Jones et al., 2015). The larger question about these desired outcomes, again, is whether these changes are sustained and whether they affect the client's performance at work, life, and career satisfaction.
Effective Coaching Through Job Competency Assessment
Effective coaching can be assessed through an approach called job competency assessment (Boyatzis, 1982; McClelland, 1973). Job competency research has been based on a theory of performance for over 50 years (Boyatzis, 1982; McClelland, 1973). The quest for identification of talent and the possible key contributors, including the first mention of “competency,” began much earlier in psychology (McClelland et al., 1958). The theory predicts that performance is maximized (i.e., most effective) when there is a convergence and consistency of the integration of three major factors: (1) individual characteristics of a person in a role or job, such as a person's style, competencies, and other unique dispositions; (2) the role responsibilities, tasks, or demands of a specific job; and (3) the environment within which the job or role is performed. The latter includes the organizational structure, culture, climate, and strategy. It also includes the larger environmental context in which the organization functions, including the economic, social, and political conditions, as well as industrial trends. Therefore, to be able to determine if and which individual characteristics make a difference in performance, the research design should hold the job and environment constant.
The research design to determine characteristics of the individual that predict effectiveness would be cumbersome unless the job/role was specified and the organizational and larger environment held constant for the sample studied. Of course, the first design issue is to specify a measure of performance or effectiveness as the dependent variable. Then, the question as to which of the many individual characteristics would be studied precedes the measurement challenges.
What is a Competency?
Because of the proliferation of competency models used to train and certify coaches and the on-going criticism that these competency models are not based on evidence of effective coaching (Bachkirova & Smith, 2015; Boyatzis et al., 2022; Nadeem et al., 2021), a choice was made in the design of this study to focus on competencies. A competency is a learned capability or ability of a person, not a characteristic of a job, that results in effective performance of a role or job (Boyatzis, 1982; McClelland, 1973). It is a set of related but distinct behaviors (i.e., often a collection of skills). The variety of behaviors expressed when someone demonstrates a competency have been considered alternate manifestations organized around an underlying intent (Boyatzis, 1982). In this sense, competency development of clients can be considered a proximal outcome or mediator of the distal outcome of increased effectiveness (Bozer et al., 2013).
Competencies differ from traditional models of knowledge, skills, and abilities. Knowledge is declarative, procedural, or meta-cognitive statements, facts, or beliefs. Abilities are typically defined as personality traits, like extroversion or efficacy. They are stylized responses to generalized stimuli and lack specific intent in situations (Boyatzis, 1982). Skills are components of competencies (Boyatzis, 2018). For example, active listening is a skill. It can be used to understand another person, demonstrating a competency called empathy. Active listening can also be used to determine an argument or approach to convince another person to do something, demonstrating a competency called influence. Competencies are behaviors that a person demonstrates. They are a set of skills organized around an underlying, possibly unconscious, intent. The distinctive skills may be alternate ways to demonstrate, use, or exercise the competency. Without the underlying intent, it is not clear as to why the person is showing that behavior.
Other individual characteristics relevant in a theory of performance are styles and values. Style is the comportment of a person and is affected by cultural compatibility and conformity to a set of organizational or larger social system norms and values (Boyatzis, 2018) addressing the compatibility of individual characteristics and the organizational environment within which the job exists. Some personality traits can be described as habitual and, therefore, a style of a person. Values are opinions or beliefs that also affect interpretation of performance within an organization through perceptions of the individual-organizational compatibility.
A common criticism of competency models and research is that they are reductionistic and too narrow (Bachkirova & Smith, 2015). This confusion has occurred when competencies are viewed as proficiencies or skills or confused with tasks. Proficiencies are the minimal effort needed to accomplish a specific task (Boyatzis, 2018).
The mislabeling of job tasks as competencies creates a major deflection of effort from an individual's ability to a feature of a job. This mislabeling also drives the need to clarify what a competency is (Nadeem et al., 2021) and creates an oversimplification of what is needed to do coaching effectively (Bachkirova & Smith, 2015). For example, a coach stating the ethical boundaries of their conversations with a client is a job task. It is an important activity or task for a coach. It is something a coach should do to establish trust, when role confusion arises with a client or the client fears exposure of their challenges to their managers or relatives. It requires several competencies to perform, like empathy and understanding how to communicate with the client, systems thinking in determining the causal need for clarifying such ethical boundaries in a coaching session, and so forth. Further, as explained in a future section, development of more effective coaches is not helped by focusing more and more attention on job tasks such ethical boundaries. That is, if a coach expressed ethical boundaries every time they talked to a client, the clients might get frightened or alarmed at the possible preoccupation being portrayed by the coach.
Competency models built from opinion surveys suffer from the likely confusion between competencies, behavior, tasks, values, knowledge, style, and so forth. Each person responding to the survey may have a different meaning of the term and be responding with different types of variables. Inclusion of current values into such a model may suffer from expressing current fads or even prejudices and not be tested against actual effectiveness. Such divergent variables included in a model make Type II errors more likely. As Bachkirova and Smith (2015) said, “We may be in danger of claiming a degree of quality assurance that cannot be justified” (p. 129).
Following the observations about the inaccuracies of competencies of managers and leaders based on opinion surveys (Boyatzis, 1982), a review of the competency model used by the International Coaching Federation (ICF, 2019), revealed a collection of skills, tasks, values, knowledge, style, and components of competencies. A similar review of the competency models of the Worldwide Association of Business Coaches (WABC, 2022), the European Mentoring and Coaching Council (EMCC, 2015), and Association for Coaching (AC, 2012) showed 473 indicators of competencies listed. These models had 53‒240 indicators each subsumed into 8‒35 competencies. This review was conducted at the indicator level because of the preponderance of job tasks, style, values, and knowledge in these models. Most indicators were job tasks (i.e., 38%) followed by style (i.e., 27%) and values (15%). Likely components of competencies were 22% of the indicators with 1% claiming cognitive abilities. A few examples of job tasks and styles from the various models are: “Provides observational feedback where relevant, leaving the client free to choose to act upon it or not” (AC model); “Maintains confidentiality with client information per stakeholder agreements and pertinent laws” (ICF model); “Critically reflects on practitioner paradigms and their impact on clients and client systems” (EMCC model).
Another effort to specify the competencies of coaches of physicians used a Delphi technique with successive rounds of opinion surveys (Passarelli et al., 2024). Some of the research team had extensive, prior research experience in competency assessment, and as a result coded 36% of the indicators in their model as likely competencies. To be clear, they were attempting to define the role of coaching, including job tasks, knowledge, role differentiation, and currently desired values.
A competency as a label may need careful translation in other cultures. For example, in Italy, a competency, within the meaning of this stream of research, is not translated to “competenze” but to “capacitá.” Similarly, in early work in the UK education system, competencies and capabilities were declared synonymous (Boyatzis, 2018).
How Do Coaches Act?
Since behavior is at the heart of articulating a competency, studies seeking to understand the behavior of coaches should be reviewed. They have been conducted with a variety of methods (Calasso et al., 2024) and examined various outcomes (Athanasopoulou & Dopson, 2018). Unfortunately, some studies did not use a dependent variable separating coaches by their relative effectiveness on various outcomes (Bachkirova et al., 2015; Bickerich et al., 2018; De Haan, 2008; Will et al., 2019), further confusing interpretation of results with Type II errors. Despite that omission, these studies can be viewed as an early attempt to determine effective coaching behavior that awaits testing against effectiveness. The behavior found among coaches, regardless of effectiveness, that seemed important included mentioning relationships, boundaries and self-doubts about coaching capabilities within micro-moments during the coaching experience (De Haan, 2008). It was also documented that different behaviors were used at different phases of the coaching process (Will et al., 2019). Coaches themselves identified “best coaching behaviors” (Bachkirova et al., 2015) or their self-assessed best behaviors (Bickerich et al., 2018). A descriptive study of coaches, their supervisors, and a survey of clients revealed that coaching helped clients develop competencies, achieve desired goals, and increase their capability to manage their teams more effectively (Dolot, 2017).
Again, there was no determination of whether these coaches or the experiences studied caused or led to discernable change in clients.
When development of a quality relationship was seen as a preliminary desired outcome of coaching (Calasso et al., 2024), what might be considered a mediator or moderator variable in research design, a series of studies were conducted exploring the behavior of coaches that produced a better relationship. The Working Alliance Inventory, a measure originally developed for use in therapeutic and mental health applications (Hunik et al., 2021; Horvath, 1981), began to appear in the coaching literature (Gessnitzer & Kauffeld, 2015). A study using this measure showed that a coach's actions toward bonding with the client and achieving agreement on goals and tasks of the process seemed to help develop a stronger working alliance (Gessnitzer & Kauffeld, 2015). Another study showed that reciprocal friendliness and in particular “dominant-friendly behavior of the coach” (i.e., confident and assertive), improved the working alliance (Ianiro et al., 2015).
While the quality of the coach–client relationship does seem crucial to the durability of a coaching relationship and the likely desired outcomes, one must be careful drawing conclusions from self-report data. Any measure of client-reported satisfaction or perception of the quality of the relationship with the coach used as the desired outcome could be contaminated by social approval and cognitive dissonance reduction forces (Podsakoff et al., 2012). The desire for social approval might lead a client to attribute positive outcomes and “effective behavior” of the coach to please their coach. Once time, effort, and/or money are spent in the coaching process, the drive to reduce cognitive dissonance could result in a client justifying the use of resources by claiming the coaching was successful and attributing that to the quality of the coach (Festinger, 1957).
Another stream of research examined coach behavior through analyzing it against progress that the client made toward their espoused goals using a measure called a Goal Attainment Scale (GAS; Calasso et al., 2024; Zimmermann & Antoni, 2018). Gessnitzer et al. (2016), after coding coaching behaviors during coaching sessions, found that when coaches asked open-ended questions, were involved in proposing solutions to the presenting problems the client was facing, and provided support and made self-efficacious statements, the clients showed perceived progress on their goals for the coaching. Another study showed that when coaches used open-ended questions with a “solutions focus” (Cavanaugh & Grant, 2010) in the coaching conversation and demonstrated a positive mood, the clients reported greater progress on their goal attainment (Jordan & Kauffeld, 2020). Again, as with perception of the client relationship, goal attainment as judged by the client may be contaminated by a desire to reduce cognitive dissonance (Festinger, 1957) and self-report of one's progress from coaching (Baumeister et al., 2007) given the time, effort, and funding invested into the coaching experience.
An effort to determine the degree of congruence in coach's and client's views of empathy (i.e., empathic behavior of the coach) was conducted by interviewing coach–client dyads (Will et al., 2016). Interestingly, neither the qualitative nor quantitative data showed that clients and coaches views of the nature, quality, or occasions when empathy was occurring overlapped. A broader study examined managers’ 360° assessment after a year of coaching (Smither et al., 2003). Researchers found that the observations from direct reports and supervisors showed improvements, but the effect size was small. Compared to a comparison group, those who worked with coaches “set more specific goals and solicited ideas for improvement” more than other managers. Further studies of empathy in the coaching process showed similar results (Diller et al., 2024).
Boyatzis et al. (2023) showed that a set of coach competencies predicted client behavior change 2 years later. A review of prior literature showed the likely relevance to effectiveness studies from other related helping professions, such as psychotherapy and counseling. Findings revealed that achievement orientation was the strongest competency, as observed by others (not self-report) to effect positive client behavioral change. Other competencies that showed significant predictive effects in multiple analyses included: empathy, emotional self-control, adaptability, organizational awareness, influence, and conflict management. The impact of positive outlook, coach and mentor, and teamwork were suggested in one of the analyses in that study, showing near significant effects.
The Boyatzis et al. (2023) study used informant observations of the coach's behavior patterns with an assessment called 360°. One limitation of this study was that the informants completing their observation of the coach's behavior were not focused exclusively on the coach's behavior during coaching. The coaches were predominantly physicians who also served multiple roles in a medical school. Any 360° relies on observers being able to see and discern behavior. Another critique of the study was that not all of the competencies possibly relevant to effective coaching are easily observable by others, such as emotional self-awareness. The other difficulty with the 360° is that the observers may not have been exposed to the thoughts and feelings of the coaches, which limits possible observations to those competencies most evident in behavior.
The measurement challenges and limitations of previous studies of coach's behavior and even competencies suggest a need to study the coaches while they are coaching, or at least in the context of their coaching experiences. For this study, once measures of each coach's effectiveness were obtained in terms of their clients’ behavior change over 2 years, the coaches were interviewed using a modified critical incident interview technique (Boyatzis, 1982; Flanagan, 1954; McClelland, 1973; Spencer & Spencer, 1993). The interviewers asked each coach to describe an event or incident in which they had felt effective in that job. The common probes included: what led up to the situation, what did you say or do next, what were you thinking or feeling at that moment, and what was the outcome? The interviewers tried to adopt a journalistic style and avoid leading the coaches, as is often characteristic of a therapeutic interview. The coaches were asked to describe the event in terms of who said and did what to whom and when. It was a time compressed version of what a video of the event might reveal. The critical incident interview is discussed in more detail in the Methods section of this paper.
Methods
The design for this study was a qualitative analysis of work samples of coaching conversations. The sampling method was to identify two samples using a criterion referenced, extreme case approach (Boyatzis, 1998). This means finding two samples of coaches from the population studied. One sample is identified as “effective coaches” in which effectiveness is measured on the basis of a criterion of positive client behavior change in the 2 years subsequent to the initiation of the coaching. The other sample is identified as “less effective coaches” on the basis of a criterion of negative client behavior change in the 2 years subsequent to the initiation of the coaching. The method of data collection was critical incident interviews to capture the behavior patterns of the coach during coaching sessions, as well as their relevant thoughts and feelings at the time (Boyatzis, 1982, 1998; McClelland, 1998; Spencer & Spencer, 1993). The transcripts of the interviews were then coded by multiple researchers seeking high interrater reliability (i.e., comparable judgments). Differences in patterns of the behavior, thoughts, and feelings were then be inferred to be indicative of the criterion difference, in this case, effectiveness as a coach. This study was conducted and approved under the Ethical Use of Human Subjects Institutional Review Board of a Midwestern research University.
Sample
The sample of coachees was medical school students enrolled between 2012 and 2018 in a United States college of medicine at two different campuses. Each year, 90 medical students (i.e., called clients hereafter) seeking extra preparation in leadership were selected into a cohort. Each member of the cohort was allocated two different coaches for the entire four years of their medical education. The coaches received a 3-day training on coaching, exploring techniques, and skill practice. The training also focused on distinctions between coaching and typical professorial, mentoring, advising, preceptorship, and physician roles. The coaching sessions were predominantly one-on-one. The coaches would also convene a group coaching session which was optional to the students about once every month or two.
The population of coaches during the years 2012 and 2020 were predominantly physicians at the medical school and associated health care centers with two who were PhD professional coaches. The coaches were comprised of 19 males (31.67%) and 41 females (68.33%). The age range of the coaches was 27‒ 65 years old.
Among the 314 clients in the eligible cohort, 172 were male (55%) and 142 were female (45%). The age range of the clients at entry into the program was 21‒ 35 years old. The clients completed the Emotional Social Competencies Inventory (ESCI, Boyatzis & Goleman, 2007), a 360° survey on 12 behavioral emotional and social intelligence (ESI) competencies, both soon after entry into the program and at the beginning of the 3rd year. For purposes of this study, only assessments by others were included to establish behavioral competencies, not self-assessed competencies. While all of the emotional and social intelligence competencies are required for effective physician performance (Nowacki et al., 2015; White et al., 2022), it was assumed that improvement on the composite set of 12 would be an indicator of desired development. In addition, lack of improvement would be an indicator of less than desired development.
Each clients’ composite improvement scores (i.e., the sum of others’ observations of their behavior on the 12 ESCI scales) were calculated as their composite score at Time 2 minus their composite score at Time 1. Of the total of 70 coaches, 17 coaches were identified whose average client's improvement scores were positive. They were selected for interviews as a sample classified as “effective.” Another 17 coaches were identified whose average client's improvement scores were negative and labeled as “less effective.” After contacting all 34 coaches, 22 agreed to be involved in the study and were interviewed. The result was a sample of 12 “effective” coaches and 10 “less effective” coaches. Among them, 14 were female and eight were male. Of the 12 effective coaches, three classified themselves as male and nine as female. Of the less effective coaches, four classified themselves as male and six as female.
The Interview Process
Each coach was interviewed using a modified critical incident interview (CII) protocol (Flanagan, 1954) without knowledge of the criterion subsample of the coach (i.e., effective versus less effective). The CII has been thought to be an effective way to collect behavioral information for decades (Campbell et al., 1970; Dailey, 1971; McClelland, 1998). It has been used in thousands of competency studies by scholars and consultants on samples in many countries. Many of the scholarly studies have been published (Boyatzis et al., 1992, 2017; Boyatzis & Ratti, 2009; Camuffo et al., 2012; Dreyfus, 2008; Gutierrez et al., 2012; Ryan et al., 2009; Spencer et al., 2007; Spencer et al., 2008; Spencer & Spencer, 1993; Williams, 2008). The interviews ranged from 30 to 60 min. Three interviewers conducted approximately one third of the interviews each. The interviews were recorded and transcribed.
The critical incident interview protocol asked the respondent to recall a coaching experience in the last 2 years in which they felt effective as a coach. After one effective incident, they were asked about an ineffective incident. Then, they were asked about another effective incident. Once the respondent provided an overview of the event, they were asked about how it began and then what happened next, until they reached a type of conclusion of the event. At each point, if not offered, the respondent was asked what they specifically said or did, how they were feeling, and/or what they were thinking at the time. The interview concluded by asking if there was anything else the coach wanted to add (Boyatzis, 1982; Spencer & Spencer, 1993).
The CII is considered an effective qualitative method for collecting behavior, thoughts, and feelings in a time compressed manner as contrasted to direct observation (Boyatzis, 1998; McClelland, 1998; Motowidlo et al., 1992). Direct observation takes many hours more in both collection and coding. In addition, the presence of recording devices whether audio or video or direct human observation can interfere with the client and make them self-conscious. Direct observation does not allow the research to access information about that the coach was thinking or feeling at that time.
A limitation of the CII is possible distortion from lack of accurate memory or recall. In this study, it was attempted to be minimized by adaptation of three techniques used in the autobiographical literature to increase accuracy of recall (Rubin, 1986). Namely, inquire as to actions and behaviors not just descriptive or general perceptions. Second, inquire as to relatively more recent events, for example, within the last 2 years. Third, inquire as to events that had higher valence or saliency than other events. In this case, the question asking for two coaching events that were effective and one that was less effective is thought to cue salience.
Data Analysis: Coding
The unit of analysis was the coach. The unit of coding was each critical incident. Each incident was coded for the presence or absence of each of the codes, or themes. The three interviewers and principal investigator independently conducted a thematic analysis (i.e., coding) of all the interviews. Once coding was completed, reconciliation meetings were held and each moment in which any of the researchers identified a theme as present was discussed until all four researchers agreed. This method of thematic identification emerged from fields as different as personality psychology, cultural anthropology, industrial and organizational psychology, nursing, epidemiology, and such. It was refined as a qualitative method by numerous scholars (Atkinson, 1958; Boyatzis, 1998; Campbell & Fiske, 1959; Coffey & Atkinson, 1996; McAdams, 1985; Smith et al., 1992;).
The coding involved two reviews by each researcher. The first was done with an established codebook for qualitative observation of 12 emotional and social intelligence competencies from prior research (Boyatzis, 1982; Spencer & Spencer, 1993), as shown in Table 1. To minimize space for this paper, the codebook is available as supplemental materials and upon request from the authors.
This codebook is a compilation of the details for how to recognize a theme, or code, as present in the coach's behavior, thoughts, or feelings. It evolved from research studies conducted over 45 years in many countries from competency studies of a wide variety of professional and managerial jobs and roles (Boyatzis, 2018; Boyatzis et al., 2017; Boyatzis & Ratti, 2009; Camuffo et al., 2012; Dreyfus, 2008; Gutierrez et al., 2012; Ryan et al., 2009; Spencer et al., 2007, 2008; Spencer & Spencer, 1993; Williams, 2008).
In the first wave, three coaches were selected at random from each of their respective criterion groups (i.e., effective or less effective) by one of the researchers (Boyatzis, 1998). Each researcher looked for the 12 competencies and any emergent themes that were more prevalent in either of the subgroups using a compare-and-contrast thematic analysis of the two coach subgroups (i.e., three effective coaches vs. three less effective coaches).
The second wave of the coding was done independently by each researcher on the remaining 16 interviews without knowledge of the coach's effective or less effective criteria subgroup. As mentioned earlier, once coding was complete, each incident was discussed in depth until all four coders agreed.
Two themes emerged from comparing and contrasting the effective versus less effective coaches. They were role differentiation and commitment to continuous learning. Role differentiation was defined as statements differentiating the behavior of being a coach from various other helping roles. For example, a statement about being a coach versus sponsor, mentor, counselor, teacher, clerkship director, preceptor, academic advisor, or career advisor would be coded. It could also be coded if the coach, when describing the coaching incident, stated that they were helping a client understand their strengths or be reflective and not telling the client what to do.
Commitment to continuous learning was defined as the coach seeking new learning or developmental activities for their own learning. For example, a statement about specific additional developmental programs, experiences, courses, or getting a coach for themselves would be coded.
Each coach was coded as showing each of the 12 ESI competencies in each incident or not, and each of the two emergent themes as present or absent in the entire interview. The frequency of incidents within which the coach showed each of the 12 competencies was calculated as a frequency of demonstration. The distributions were checked for significance using SPSS chi square analysis with a Fischer's Exact Test for small sample sizes, and then t-tests and ANOVAs for frequencies with one-tailed significance levels.
Findings
The results of coding for the presence or absence and frequency of ESI competencies, as well as total ESI, are shown in Table 2. Competencies demonstrated significantly more during coaching by the effective coaches as compared to the less effective coaches include: emotional self-awareness, achievement orientation, empathy, coach and mentor, and total emotional and social intelligence competencies. Among the emergent themes, role differentiation was significant among the effective versus less effective coaches.
The Emotional and Social Intelligence Competencies (ESCI and ESCI-U).
Note: Adapted from Boyatzis and Goleman (2007).
Competencies/Emergent Themes Among Effective (n = 12) and Less Effective Coaches (n = 10).
ESI competencies and emergent themes highlighted in bold show significant differences between the performance groups on a chi square analysis using Fischer’s Exact Test for presence versus absence and t-tests and ANOVAs for frequencies with one tailed significance levels. These have been characterized as distinguishing competencies and emergent themes. +p < .10; *p < .05; **p < .01; and ***p < .001.
Emotional self-awareness is acknowledging one's own strengths and weaknesses, linking one's feeling to behavior or consequences. One example was, “I think I felt less effective when the issues got very, very personal. So, I had one student who was having trouble feeling ostracized and excluded and not getting along with some of her fellow students. And feeling very awkward socially, shy and awkward. And I did not feel as a coach, professional coach, that I had quite as much to offer, I found I sort of got stuck. And I found myself more likely to give her direct advice than give her space to figure that out. And I think it was because I felt like I really didn't know how to coach her.” Another example of a coach using emotional self-awareness was, “…being able to support them in that is really something I really enjoy. And I think it gives me joy and it gives me purpose to be able to help them work through that as far as emotions that I feel during it.”
Using the achievement orientation competency, coaches are trying to improve their own performance as a coach and trying to help guide the client with their stated priorities such as better managing stress. Two examples of how achievement orientation appeared in the work samples of the effective coaches of this study are: “I remember after a session reaching out to my co-coach and saying, ‘I don’t think today was really the best I’ve done. What could I do better next time? If we were to replay this scenario of what just happened, what could I do to be more engaging?’” Another example was, “His family lived in a different state with a huge time difference. From the beginning, he started talking about how he felt distant from his family, and the various measures that he had taken to try to strengthen those interpersonal bonds, those familial bonds, but he still didn't feel as connected. He was saying that he was so busy. I had him get out his calendar, to see where he was losing so much time, where there were blank spots in his day. And we came up with a goal to talk to the people that he wanted more often. He also figured out that whenever he had transitional periods, whenever he was driving in this car or walking from one building to another, he would go through and he would think this is someone that I haven't talked to that I want to talk to. During those transitional periods, he could call those family members. He also really enjoyed that idea because it gave him a definitive cut-off to the conversations because he says that some of the conversations could go on for hours.”
Demonstrating empathy is trying to understand the client and their experience. One example from the coaching incidents was, “I think that over video, as you know, you can kind of notice some behaviors that show that someone may be a little stressed. [He] was a little fidgety and distracted. That's how I knew he was stressed. I also know we (as coaches) get a detailed calendar of what the clients are doing at the time. We know when they have an upcoming test so can be alert to signs of the effect on them.” Another example of a coach using empathy was, “One of the clients had not had an easy route to medical school. I think she carried some assumptions that she was going to have a really hard time. She found her stride really fast and figured out how to study and do the work. Because of her caution, she asked me, “Should I apply for a conservative residency? Or should I really go for what I want to do?” I asked her to review her path to date through medical school and how she had handled it. It took her a minute to notice, “Hey, I’m actually rocking this thing. I should apply for whatever I want.” She did a match in surgery, which is a really big deal.”
Coach and mentor is a desire and taking actions to help another person develop. It may seem odd that it is the purpose of the role being studied, but nonetheless, it does not mean that the behavior is seen or experienced by others. An example was, “And last year, his professional development plan really didn't get off the ground. He really never worked on the plan and had various reasons why. But yesterday, we had another coaching session. We discussed what his plan needs to look like this year. And through coaching, he was able to determine the next steps that he needed to make, why this was important, and what the next steps he needed to do to ensure that he had a realistic and timely goal. Unlike last year, and at the end of the session, he really said, ‘Wow, this was amazing how you got me to kind of figure this out.’” Another example of a coach demonstrating the coach and mentor competency was, “She had the drive, the hard work, the work ethic. I just had to say, hey, you can do that, right. This is what I'm noticing. You did really well with this. And that's a sign of leadership. It's a sign of diligence and work ethic and pointed out it is a reflection of what she was already doing herself. I didn't have to push her.”
Of the two emergent themes, role differentiation was demonstrated significantly more often by effective than less effective coaches. Role differentiation is the explicit statement of what a coach would or should do versus what a person in any of the other typical helping roles (i.e., mentor, counselor, therapist, sponsor, teacher, clerkship director, preceptor, academic advisor, career advisor, or parent) might do. It was often mentioned with a statement about not telling the clients what to do, but getting the clients to discover it.
An example of a coach demonstrating role differentiation was, “I remember thinking to myself, because one of my prior roles was a clerkship director, so I remember thinking intentionally in my mind, okay, I'm not a director. I'm not a preceptor. I’m not, you know, a mentor role. I’m a coach right now. So, how can I be here to be supportive and help direct the student by having them really understand their own strengths, their resources, and what makes sense for them to choose as the resource and just help guiding them using those tools in particular.”
Another example of a coach using role differentiation was, “So, as a coach, I need to open up the space to allow them to explore what they're going through, and kind of shift the definition of self-directed learning. It's kind of a space for them to use me as a sounding board to grow. It's not me to, to give very much advice, I have a tendency that whenever I get into these, these sessions, I want to give advice. I want to be the advisor. And I can't be, that that's not my role. Not only am I not an academic advisor to them, I'm not a career advisor to them. I'm their coach. And, so, I help them come to where they want to be with topics that they want to talk about, as opposed to me starting up the coaching session, you know, hitting them with a bunch of questions about their experiences, what they've had throughout the month.”
Yet another example was, “I think especially in these conversations that we're talking about these one-on-one conversations are private, I actively do my best to not be in any other role. Right. Like, like I said, not be in the role of a therapist, not be in the role of a mother, of a friend or anything else. I mean, a mother is a big one. I'm a mom with children. And so like, you know, that kind of comes naturally when you're talking to someone that's younger. And they sometimes they'll literally say, ‘What do you think I should do?’ Right? They're asking you for advice. And so as much as I would like to tell them, you don't, right, because that's not, that's not helpful. At the end of the day, it really isn't. It's…they could get advice from tons of people. But I know that what's gonna be most helpful is for me to ask the questions like, you know, ‘What is important to you? What is what is really the issue here for you?’”
Discussion
The results of this study showed that the competencies of coaches matter and have an effect on the clients and the effectiveness of the coaching. The results from this study showed that the degree of ESI behavioral patterns (i.e., competencies) shown by coaches predicted how much their clients changed their behavior, as observed by others 2 years later, in the desired direction for their future effectiveness in their profession.
This is consistent with research in other helping professions. While training and certification in various helping professions, from medicine, nursing, teaching, therapy, etc. often focus on cognitive and knowledge development, the distinction of effectiveness of the helpers appears as a direct result of their development and use of what have been termed ESI competencies. For example, among dental graduate students, when peers described each other's behavior several months into their graduate program, the resulting observation predicted their grades in the third and fourth years of their program, which are entirely in the clinic working with actual patients (Victoroff & Boyatzis, 2013). In education, ESI competencies, in aggregate, predicted effectiveness of Turkish headmasters (i.e., school principals) in terms of job satisfaction of teachers (Turanlı-Camsarı, 2007) and perceived effectiveness by colleagues of US community college faculty (Babu, 2016). In clerical roles, the ESI of Catholic priests predicted multiple measures of parishioner satisfaction (Boyatzis et al., 2011). In other helping roles, the set of social intelligence competencies in aggregate were predictive of effectiveness for physician executives (Quinn, 2015), athletic coaches (Van Sickle, 2004), and Olympic coaches (Cote & Sedgwick, 2003).
The relevance of ESI to coaching effectiveness is in large part because coaching is an interpersonal process in which the coach and client have exchanges of ideas and emotions at many different levels (Boyatzis et al., 2019). Qualities of the coach–client relationship will affect the degree of engagement, learning, and sustainability of any learning and change emerging from the coaching (Boyatzis et al., 2019). It followed, therefore, that the coach's handling of their own emotions and behavior will have a substantial impact on the client and their relationship. The clients in this study were medical students. With the complexity of their roles and the range of activities demanding their attention and effort over a period of 2 years, the findings are notable in predicting effectiveness and clients’ behavior change.
In this study, within the overall effect of ESI competencies demonstrated during coaching experiences, several were distinctly important in predicting coach's effectiveness. They were specifically achievement orientation, empathy, emotional self-awareness, and/or coach and mentor.
First, an exploration of how each of these competencies are relevant to coaching effectiveness is useful. Achievement orientation, as a competency, is defined as seeking to do better often characterized by setting goals, planning, evaluating progress, and seeking better options to increase the output-to-input ratio (McClelland, 1961). The clients were in the first 2 years of a 4-year development program. During the program, there are daily requirements for them to demonstrate new knowledge and skills, ethical standards and comportment, and earn certification into the profession as a physician. They do this by passing numerous courses, clinical rotations, standardized patient assessments, and through internships and residencies in a hospital. During their program, the clients are heavily stressed, often fatigued, and always preoccupied with being seen by supervising physicians as capable and caring physicians. The coaches are spending a great deal of their time helping the clients juggle the competing demands.
As shown in the brief examples from the coaching incidents, the coaches use their achievement orientation by asking questions and offering ideas to the client to help them organize and prioritize their time. The coaches are role models of how to attend to the highest priorities and think through alternatives. The coaching conversations may help clients choose the most effective and efficient path to their desired outcomes. The coach may use their achievement orientation to not only strive to improve as a coach but also to remind the client of their purpose and consider more than just their tasks. Supporting this study's finding, achievement orientation was previously found to be the strongest competency predicting coach effectiveness in a quantitative study of coaching competencies (Boyatzis et al., 2023).
This finding, based on coaching medical students with multiple, competing demands for their time and focus, may also apply to coaches with managerial, leadership, or executive clients who also have multiple, competing demands for their time and focus. Coaching people with less demanding work or less complex home situations, achievement orientation might be a less potent differentiator.
Research briefly reviewed earlier showed that client's perception of progress made toward their espoused goals during coaching (Calasso et al., 2024; Zimmermann & Antoni, 2018) occurred when the coaches were discussing various possible solutions the client might have as options (Cavanaugh & Grant, 2010; Gessnitzer et al., 2016; Jordan & Kauffeld, 2020). Related research in other helping settings, achievement orientation predicted effectiveness of middle school and high school principals in an urban school system (Williams, 2008) and better performance for human service workers (Spencer & Spencer, 1993). Achievement orientation was also predictive of greater engagement and motivation of community college faculty (Babu, 2016).
Beyond helping the client manage their experiences and development, relating to the client seems critical to a coaching relationship as evident in the Working Alliance Inventory and research on the role of empathy in coaching cited earlier (Diller et al., 2024). A second competency revealed as important in this study was empathy. It can be defined as seeking to understand another person and possibly feel what they are feeling (Reiss & Neporent, 2018). When the coach demonstrates empathy, they are seeking to understand the client. It may appear as deep listening and being attentive in contrast to the possibility of being distracted when talking with someone. Tuning into the client's feelings, moods, and state requires empathy. When the coach shows an interest in understanding the client, even if the client is different than the coach, the coach is also modeling desirable behavior for clients. For physicians in training, empathy could be helpful in their work with patients, their families, and staff.
Although one study showed coach's and client's views of empathy demonstrated during the coaching interactions were not congruent as to the nature, quality, or moments when empathy was revealed (Will et al., 2016), several other studies show empathy to be vital to the coaching relationships (i.e., working alliance) in terms of perceived helpfulness, friendly behavior, and bonding of the coach (De Haan et al., 2009; Gessnitzer & Kauffeld, 2015; Ianiro et al., 2015; Ianiro & Kauffeld, 2014). Empathy was found in a prior quantitative study to be predictive of client behavioral change (Boyatzis et al., 2023). As in the association opinion-based competency models of coaches, empathy has been viewed as a needed capability of a therapist (Carkhuff, 1969; Rogers, 1961). Demonstrating the empathy competency predicted effectiveness in alcoholism counselors (Boyatzis, 2018) and effectiveness in a variety of human resource jobs (Spencer & Spencer, 1993). Empathy has been revealed to predict inspirational professors (Barco, 2019) and athletic coaches of Division I sports teams (Van Sickle, 2004).
While not surprising to coaches or coach instructors, this study showed that the emotional self-awareness (ESA) competency was predictive of coach effectiveness. ESA refers to a coach being aware of their feelings, thoughts, values, and how it affects their behavior and impact on others. It is often believed that self-awareness precedes use of other competencies or behavior. In fact, it is often cited as a precursor to other aspects of ESI (Goleman, 1998). Many coach training programs spend a great deal of time helping a coach or coach candidate examine and learn more about themselves and increase their sensitivity to clues about their internal states.
Of all the ESI competencies, emotional self-awareness is perhaps the most elusive one because it is more difficult to observe. It was a significant predictor of coach effectiveness in this study, but did not appear in a previous study of the same sample (Boyatzis et al., 2023). In that study, competencies of the coach were assessed through informant's observations (i.e., 360° assessment). Chon and Sitkin (2021, p. 631) observed, Given the extreme difficulty of finding a benchmark with which to empirically measure the accuracy of self-awareness, it is understandable that the use of other-ratings as a baseline is tempting, given its operational ease and clarity…[referring to 360° self-versus-other assessments]. Unfortunately, the availability of this ready-to-use indicator has made more objective measurements of accuracy less appealing and less adequately utilized in extant self-awareness models (Kassin, 1985).
Another aspect of assessing emotional self-awareness requires access to the focal person's thoughts and feelings. The critical incident interview technique used in this study allowed for the coach's thoughts and feelings to be elicited during the recollection of the coaching moments. Supporting the finding in this study, ESA was previously found to predict effectiveness in alcoholism counselors (Boyatzis, 2018) and personal, financial advisors (Goleman, 1998), using variations of critical incident interviews.
The choice to be a coach is a commitment to helping others develop. It would seem fundamental to this role or even using coaching as a process amid other roles, like physician, nurse, teacher, manager, or even parent, that the person thinks and acts on a desire to help others develop and grow. The coach and mentor competency is at the core of wanting to be a coach. This may seem self-evident, but not all people in helping roles demonstrate coach and mentor as a competency. Performance improvement demands and norms can take priority over development priorities. In such a situation, the client may feel that the coach is trying to get them to comply with their employer's wishes over the client's own development.
In this study, use of coach and mentor competency showed the client that, even when not in physician-patient interactions, developing others is relevant and useful. It reminds the client of their primary purpose of helping and healing others. It did appear as related to effectiveness in the Boyatzis et al. (2023), but not strongly. In that study, a 360° assessment was used and the observers may have taken a broader view of the coach's behavior in the context of their entire work, not just as a coach. Since most of the coaches in this study were also physicians, observers could have been noticing a focus on diagnosing or surgically repairing a patient, not developing them.
Another theme that appeared to differentiate the effective coaches was an articulation of awareness of the distinctions between the role of a coach from other forms of helping, such as advising, supervising, mentoring, and such. This is particularly relevant in this study because the coaches were predominantly physicians with multiple roles. They were treating their own patients. They were training medical doctors. They were advising and being preceptors to the medical students. During the several days of training as a coach, the role differences were discussed at length. This was clear in the statement by the effective coaches. They were aware of the role differences and continued to keep it in their consciousness.
For all the other possible competencies mentioned in various models, and those found in Boyatzis et al. (2023), there were no others evident in this study as predicting coach effectiveness. Boyatzis et al. (2023) found that ESI competencies of achievement orientation, adaptability, emotional self-control, positive outlook, empathy, organizational awareness, influence, and conflict management predicted effectiveness (Boyatzis et al., 2023). In that study, the demonstration of the competencies was assessed through a 360° using informants’ observation of the coach's behavior. Near significant findings in that study included competencies of coaching and mentoring and teamwork. It is worth noting that in that study, the coach's general mental ability did not predict degree of client behavior change.
The competencies not observed in this study should not lead to the conclusion that they are not important. As Bachkirova and Smith (2015) said, competencies manifested in behavior “require repetitive observations of them to infer their stability” (p. 130). They also emphasized that not observing a behavior pattern does not automatically mean the irrelevance or non-existence. This emphasizes the importance of using multi-methods of assessment to determine a comprehensive competency model that predicts effective role performance. Multiple methods of assessment and assessing a coach multiple times, and in multiple settings, will increase the confidence that observations of the link to effectiveness is accurate and dependable.
Limitations
As a study of coaching competencies assessed against actual client behavior change, it is a beginning. More studies are needed with direct outcomes, like client behavior change as seen by others, not only self-assessment. This study was within one cultural context and that is a limitation of its generalizability. Studies are needed with a wide range of clients and coaches, in various cultures, and among groups with individual differences like race, gender, ethnicity, and faith.
This study used one desired outcome, notably the client's behavior change. More studies are needed with a variety of desired outcomes. The ultimate study should include multiple types of outcomes within the same sample, using multi-method data collection.
Potential differences between coaches trained through different approaches and systems, as well as those certified by various organizations, were not studied. Therefore, any conclusions about the benefit of certification on the coaches’ competencies could not be ascertained. Although all coaches in this sample were formally trained in many approaches to coaching, specific certifications were not required for their participation.
Implications for Theory and Research
In particular, within helping professions, the specification of competencies for effective coaching can make a substantial contribution to an emerging theory of coaching and help to narrow the wide spectrum of individual characteristics often conjectured to be important to coaches. This study adds to the growing literature on competencies of professionals in specific helping professions. While the theory of performance has been exhaustively studied with regard to managers, executives, and leaders, individual contributor professionals have been the less frequent focus.
In addition, greater refinement of a theory of coaching can help in development of a more general theory of helping, which would apply to related helping professions including physicians, nurses, therapists, teachers, social workers, clerics, and such. Distinguishing the behavioral habits or patterns demonstrated by effective coaches can help to differentiate actual behavior from values, style, job tasks and requirements, and, most importantly, fads.
Research on the effective competencies should be conducted with other jobs and roles, examining possible effects of race, gender, age, faith, ethnicity, and socio-economic status of the coaches, as well as the clients. Other intermediary outcomes should also be tested, such as the coach's perception of the degree of client change in behaviorally anchored scales or Goal Attainment Scales, and the coach's and client's perception of the quality of the relationship.
Although it would border on program evaluation, the competencies determining effective client behavior change should be examined as to the nature of the coach's background, training, and certifications. Coaches certified by each association should be compared to each other, or to those without formal certifications, to fully appreciate the impact of such certifications.
Implications for Practice
This study along with the findings in Boyatzis et al. (2023) may be useful in several practical contexts. They provide an empirical basis to understand essential coach competencies. If these findings are replicated, the implication for coach certification is to create more accurate competency models which would be important for organizations providing coach certification. Organizations hiring coaches can apply the competency model to better identify, vet, and select coach candidates.
In the certification efforts, assessment can be streamlined from current practices by separating assessment of tasks characteristic of the job and the competencies of a person in the job. Meanwhile, assessment of values and cultural compatibility can also be made easier by observing coaching sessions. Knowledge of tasks required in a role, as well as cultural value compatibility and relevant knowledge about coaching, can be achieved through a test. Training in these topics can be accomplished through a set of brief, asynchronous videos followed by testing the candidates for their understanding and retention of the information. Similarly, testing cultural compatibility of values held and preferred styles can be appropriately done with tests, especially scenario tests. All of these characteristics should not be labeled competencies, but possibly desired threshold characteristics of a coach.
The assessment of actual competencies can be done through reliable coding of audio or video recordings of actual coaching sessions. The list of competencies emerging from effectiveness research is both smaller and more precisely defined than existing competency models. Both of these aspects would result in more reliable and accurate coding. Current practices are overly cumbersome, time consuming, and expensive. But more importantly, they deflect attention from the smaller number of competencies actually relevant to coaching effectiveness by seating them with a large array and list of characteristics. The above steps would simplify the certification process considerably from present activities and decrease the introduction of “noise” into the assessment process.
An understanding of coach competencies that lead to actual behavior change, learning, growth, and development are useful to organizations who are interested in fostering a culture of development in addition a culture of performance. Training and developing managers’ ability to coach direct reports and associates is a paramount priority. For training and retraining coaches, whether through workshops or on-going supervision, the distinction of the actual competencies helps focus the development on those behaviors that really matter, and minimize the introduction of ineffective fads. In this way, leaders of talent development are positioned to better align organizational resources to support optimal performance and engagement.
Conclusion
A set of competencies used by coaches during coaching conversations were found to predict effectiveness in terms of client behavioral change. While this is only the second study of coaching competencies assessed against actual client change, even these insights can provide training and certification programs with evidence of what is needed to be developed and screened for in coaches to increase their desired impact on clients.
Supplemental Material
sj-docx-1-jab-10.1177_00218863251316600 - Supplemental material for A Qualitative Study of Competencies of Coaches That Predict Client Behavior Change
Supplemental material, sj-docx-1-jab-10.1177_00218863251316600 for A Qualitative Study of Competencies of Coaches That Predict Client Behavior Change by Richard Boyatzis, Sharon F. Ehasz, Han Liu and Ellen Van Oosten in The Journal of Applied Behavioral Science
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
