Exploring the use of a goal-attainment,artificial intelligence (AI) chatbot coach to support first-time graduate employees

Abstract

First-time graduate employees¹ face numerous challenges in the workplace. Organizational coaching is a proven individual HR support intervention, however due to prohibitive cost, graduate employees typically don’t qualify for this type of assistance. This study investigated a cost effective, artificial intelligence (AI) chatbot coach to support graduate employees. Nine graduate employees used a goal-attainment AI chatbot coach for 4 weeks to help with their career goals. They were interviewed afterwards on their perceptions of the chatbot. Interviews were analysed using thematic analysis. Four main themes emerged. Participants liked the convenience and accessibility and the chatbot coach’s effectiveness in career development as well as the chatbot’s ability to assist their reflection and self-awareness. They missed the human touch and flexible (limitations). Findings were interpreted using technology adoption and self-determination theories. The chatbot coach satisfies the ease-of-use technology adoption factor and promoted autonomy and competence. Organisations should seriously consider the use of AI coaching to support graduate employees given the significant challenge they face and the lack of support. The use of AI coaching is a new frontier in organisational coaching. This study provides initial insights into potential role of AI coaching in niche organizational context such as graduate employee support.

Keywords

Artificial intelligence coaching AI coaching graduate employment

Introduction

First-time graduate employees¹ face significant challenges in the workplace. They have difficulty adjusting to the work environment, managing workload, balancing work and personal life, building professional relationship and they struggle with goal-setting and attainment (Bandaranaike and Willison, 2015; Matsouka and Mihail, 2016; Plant et al., 2019; Tlaiss et al., 2017). Institutions of higher education tend to focus more on developing cognitive skills, leading to these skills gaps once graduates enter the workplace (Bandaranaike and Willison, 2015). Graduates also have needs that are not easily met such as their preference for participative management styles (Tlaiss et al., 2017) and a high need for learning and development (Naim and Lenka, 2018). Graduates with better competencies in interpersonal attributes, soft skills and realistic goal-setting are more likely to survive in the work environment. It is therefore imperative that graduates are assisted upon entering the workplace, not only for their own benefit but also since employers gain a competitive advantage through the quality of their organization’s human resources (Muchete and Shayo, 2020). Organizations do investment in training and on-boarding programs to develop the graduates’ skills, but these programmes are costly and their efficacy questionable (Plant et al., 2019).

Workplace on-boarding programs aim to introduce the new entrants into the organisation and socialise them into the environment (Armstrong, 2017; Stewart and Brown, 2019). An ideal support intervention that should form part of in-boarding programs is individual coaching. Organizational coaching is a human resources (HR) intervention that has proven efficacy (Blackman et al., 2016). Coaching is an individually tailored activity guided by a goal agreed between the coach and client with the focus on achieving a professional or personal outcome valued by the client and organisation (Grant and Cavanagh, 2011). While coaching is a frequently used HR intervention, coaching seems to be available to a few privileged individuals only, with the cost of coaching and scarcity of professionally qualified coaches the main reason for this shortage (Terblanche et al., 2022a). A more cost-effective and scalable coaching solution such as AI coaching could address this challenge.

AI coaching is defined by Graßmann and Schermuly (2021: 109) as a “machine-assisted, systematic process to help clients set professional goals and construct solutions to efficiently achieve them.” While AI coaches currently do not have the same capability as a human coach (Terblanche et al., 2022c), there are promising signs of comparable performance in specific narrow use cases such as goal attainment (Terblanche et al., 2022b). The use of AI in the workplace has been met with mixed responses. While some claim that AI could increase inequality in the workplace (Holm and Lorenz, 2022), other research has shown that AI and especially chatbots can in fact enabled otherwise marginalised workers (Flanagan and Walker, 2021). Similarly, the use of AI in coaching could significantly reduce the cost of coaching and make the service accessible to previous excluded, challenged employees (Terblanche et al., 2022a) such as first-time graduate.

Graduates struggle with the transition into the workplace and could benefit from the proven support offered by coaching, if only it was available to them. The problem that this research therefore addressed was to what extent AI coaching could assist first-time graduate employees with their transition into the workplace. Insight into this novel area of AI application could help to scale and democratize the significant benefits of coaching to previously excluded employees.

The challenges faced by first-time graduate employees

The transition from a university student to a graduate employee can be challenging. The integration of graduates into the workforce is often hindered by several factors for which their tertiary education does not prepare them, including goal setting and attainment (Bandaranaike and Willison, 2015; Matsouka and Mihail, 2016; Plant et al., 2019; Tlaiss et al., 2017). To become valuable assets for the organization, graduates need purposeful training programs aimed at achieving this goal (Muchete and Shayo, 2020). However, the cost of learning and development programs is high, which often leads to limited availability of these programs (Plant et al., 2019).

Skills and graduate attributes are used holistically to describe the employability of a graduate, which is where the graduate challenge arises (Rowe and Zegwaard, 2017). Researchers argue that soft skills are more important and are in higher demand by employers (Dunbar et al., 2016); and at least equally as important as technical skills (Pang et al., 2019). For example, a study using engineering graduates found that although technical skills were learned in a formal manner in universities, graduates might have a challenge in application of their knowledge in the unstructured world of employment (Azmi et al., 2018). One explanation for this challenge is their lack of soft skills. Soft skills are interpersonal attributes relating to the way graduates relate to others in the workplace (Hannula, 2018). These soft skills are also referred to as pervasive skills and are categorized into ethical behavior, professional skills, and personal attributes (Viviers et al., 2016). Graduate affective skills have largely been neglected by institutions of higher learning, with a sharper focus on graduate cognitive skills, even though employers have emphasized the need for these skills (Bandaranaike and Willison, 2015).

The focus of the present study is on assisting first-time graduate employees with goal attainment. In this regard, research by Matsouka and Mihail (2016) investigated the views of university graduates (n = 178) and human resource managers (HRMs) (n = 29) on graduates’ employability in terms of the skills required by organizations. They found that the greatest differences in the perceived importance of skills between graduates and employers are in relation to emotional intelligence, goal setting and professionalism. For goal setting organizations rated this skill at 82.7% importance, however they rated graduates ability to perform goal setting at only 3.4% (Matsouka and Mihail, 2016). Clearly coaching, which is primarily goal focussed, could address this significant gap in organizational expectation and graduate ability, if only coaching was more affordable. Employing AI coaching could potentially overcome this coaching availability challenge by deliver the benefits of coaching to graduates at scale and at a vastly reduced cost.

The use of AI in coaching

Artificial Intelligence (AI) is a rapidly developing technology that is increasingly making inroads into various aspects of life, including the helping professions of healthcare, psychology, and coaching (Bakker et al., 2016; Fiske et al., 2019; Terblanche et al., 2022a). AI aims to mimic human cognitive functions and is an effective tool in helping people deal with mental and physical health issues. However, AI’s potential in the coaching domain is relatively new, and there is limited research on its effectiveness in coaching. One landmark study on AI coaching efficacy that followed a longitudinal randomized controlled (RCT) design found that a goal-focused AI chatbot coach was able to improve goal attainment in the experimental group at double the rate of the control group (Terblanche et al., 2022b). The Terblanche et al. (2022b) study opened the door to investigating the application of AI coaching in various contexts such as to support first-time graduate employees.

The potential benefits of AI coaching are numerous. In the related field of psychology, AI offers new modes of treatment, the ability to reach currently excluded populations, improve patient response, and free up limited resources such as highly trained psychologists (Zhou et al., 2022). Similarly, in the field of coaching, AI could provide personalized and customized coaching sessions for individuals based on their unique needs and preferences. AI coaching could also offer 24/7 support and access, which can be particularly useful for individuals with busy schedules or those in remote areas where access to traditional coaching may be limited (Graßmann and Schermuly, 2021). Another potential benefit of AI coaching is its ability to provide objective and unbiased feedback to individuals. Unlike human coaches who may have their own biases and preferences, AI coaching could provide objective feedback based on data and algorithms, which can help individuals identify their strengths and weaknesses and make more informed decisions about their personal and professional development (Graßmann and Schermuly, 2021).

However, there are also potential drawbacks and limitations to the use of AI coaching. One concern is the potential loss of the human connection and emotional support that traditional coaching can provide. While AI coaching can provide personalized feedback and guidance, it may lack the empathy and emotional support that human coaches can offer. Additionally, there may be concerns about privacy and data security when using AI coaching platforms, as personal information may be collected and stored by the AI system (Kamphorst, 2017). Another potential limitation of AI coaching is the lack of a standardized approach or regulation in the field. As AI coaching is relatively new, there are currently no standardized guidelines or regulations for AI coaching platforms, which may lead to inconsistent quality and effectiveness (Terblanche, 2020).

Despite these concerns, the potential benefits of AI coaching cannot be ignored. As technology continues to advance, it is likely that AI coaching will become an increasingly common and accessible tool for personal and professional development, especially in contexts where human coaching is expensive or unavailable such as for first-time graduate employees. As such, it is important for researchers and practitioners to continue to explore and evaluate the use and efficacy of AI coaching, as well as to develop guidelines and regulations to ensure the quality and effectiveness of coaching programs. A useful theoretical lens to study the adoption of AI coaching is the technology adoption model.

Technology adoption theory

The technology adoption model (TAM) is a well-accepted theory for explaining the intention of a user to accept and use technological innovations (Davis, 1993). The strength of TAM lies in its underpinning by the Theory of Reasoned Action (TRA) and the Theory of Planned Behaviour (TPB), both with roots in cognitive psychology. TRA was proposed by Ajzen and Fishbein (1975) and stated that specific behaviour was determined by behavioural intent (BI). TPB was proposed by Ajzen (1991) and allowed for greater accuracy and reliability in understanding one’s attitudes and predicting deliberate, planned, and resulting in actual behaviour.

The TAM model can provide awareness and insight on the attitudes, internal beliefs, and intentions of users. TAM suggests that some factors influence the decision about how and when they will use the technology when users are presented with technology (Davis et al., 1989). TAM states that the user’s intention (behavioural intention) to use a technology or system determines their actual use of the technology or systems. Behavioural intention in turn is determined by both the user’s perceived usefulness and perceived ease of use of the system (Marangunić and Granić, 2015). Perceived usefulness is defined as the degree to which a person believes that using a particular system would improve performance in an organisational context. Perceived ease of use is defined as the extent to which a prospective user expected the system to be used without effort (Davis et al., 1989). Both of these factors influence the actual usage of technology. TAM has been researched extensively in the literature and has proven to be reliable and robust in predicting user acceptance of technology and systems in the context of organisational related studies (Chismar and Wiley-Patton, 2003), including the use of AI coaching (Terblanche and Kidd, 2022). In the present study TAM will be used to interpret some of the qualitative findings of this study

Self-determination theory

To study the efficacy of AI coaching, self-determination theory (SDT) may be useful. SDT is a motivational theory that assumes individuals seek to integrate their experiences and act to create a coherent self. SDT proposes that the motivation to act and integrate one’s experience, to be one’s authentic self, is most powerfully manifested through intrinsic motivation and, secondarily, through integrated extrinsic motivation (Deci et al., 2017).

SDT proposes that a person’s level of functioning and well-being depends upon the satisfaction of three basic psychological needs: autonomy, competence and relatedness (Deci and Ryan, 1985). Autonomy refers to the need to feel in control of one’s life and decisions, competence refers to the need to feel effective in one’s actions, and relatedness refers to the need to feel connected to others. According to SDT, people do well and feel their best when the socio-cultural conditions of their lives support the satisfaction of these basic needs (Deci and Ryan, 1985).

Coaching has been linked to the facilitation of SDT by creating supportive socio-contextual conditions that promote intrinsic motivation, integrated extrinsic motivation, and the satisfaction of basic psychological needs. SDT, together with intentional change theory (ICT) has been used to explain what constitutes an effective leadership coach (Taylor et al., 2019). SDT has also been used as a theoretical basis to understand coaching as a practice on both a micro and macro level (Spence and Oades, 2011). In the present study SDT is used to interpret some of the qualitative findings of first-time graduate employees’ experience of using an AI chatbot coach.

In summary, this literature review showed that first-time graduate employees face significant challenges when they start their first jobs. Organization try to assist but many of the costly on-boarding programmes fail to deliver. Individual coaching is a proven, tailored intervention that could assist first-time graduates employees to transition into the workplace, however human coaching is expensive and typically not available to this level of employees. AI coaching has the potential to provide aspects of human coaching such as goal-setting and tracking at a much lower cost with proven efficacy and scalability, however the application of this coaching solution has not been investigated with first-time graduate employees and is therefore the focus of the present study.

Methodology

The application of AI coaching in the workplace is a recent phenomenon with very few empirical studies available. Given the novelty of using AI to assist first-time graduate employees, the present study required an emergent, exploratory approach where the in-depth, personal and first-hand experiences of these participants needed to be captures and analysed. A qualitative, interpretive design was therefore selected as it is typically used to gather in-depth insights from participants who have a first-hand understanding of a relative under-researched phenomenon (Elliott et al., 1999). This type of design also allowed the research to examine the social and organisational context by giving the researcher access to the inner worlds of the participants (Creswell et al., 2007).

Research setting

The company where the study was conducted (a bank) runs a graduate programme that aims to assist first-time employees to transition into the workplace. At the time the study was conducted the population consisted of 30 first-time employees between the ages of 18 and 26. Some of them were straight from school while others completed tertiary education. The programme included on-the-job learning which gave the graduates exposure to the work environment. The duration of this program was 24 months, and it was closely monitored by the organisation’s Learning and Development Department in partnership with a local business school.

Sample

The sample selected from the available population were nine first-time graduate employees who had completed a three- or 4-year university qualifications at an institution of higher learning. The participant were identified by asking for volunteers from the 30 people in the graduate programme. Details of the nine graduates who volunteered are shown in Table 1.

Table 1.

Participant biographical information.

Participant	Gender	Qualification	Division
P1	Male	BSC computer sciences and BSC geological science	Sales
P2	Female	BCom financial sciences	Sales
P3	Female	BCom hons business finance	Credit
P4	Female	BCom hons economics	Group risk
P5	Male	BCom hons finance	Finance
P6	Female	BCOM hons finance	Project finance
P7	Female	BCom honours finance	Treasury
P8	Female	BCom honours environmental sciences	Project finance
P9	Male	BCom honours	Corporate strategy

Data collection

The participants were given access to the AI coaching chatbot called Vici which they could access via the Telegram instant messaging platform on their smartphones. Coach Vici is a rules-based (non-generative) coaching chatbot that uses the GROW coaching model. Vici was designed based on the Designing AI Coach (DAIC) framework (Terblanche and Kidd, 2022). This framework combines the best practices of chatbot design with proven human-coaching efficacy aspects to create coaching chatbots. The primary aim of Vici is to assist users in achieving their goals through two types of text-based conversations. In the first conversation, Vici helps users to establish realistic goals by questioning them on the importance, feasibility, and impact of their objectives. Vici then helps users commit to achievable actions that would help them reach their goals. The second type of conversation involves users checking in with Vici to report on their progress and reflect on obstacles preventing them from reaching their goals. Through these conversations, users can monitor their progress and adjust their action plans as necessary. A sample conversation with Vici is illustrated in Figure 1.

Figure 1.

Example of Coach Vici’s two types of goal-setting conversations.

The second author checked in with the participants on a weekly basis in case they had technical issues with Vici. At the end of the 4 weeks, individual qualitative interviews were conducted by the second author with the sample group through Microsoft Teams. Each interview lasted approximately 45 min to an hour. Seven open-ended questions were asked about the experience of the participants in using the AI coach, whether or not they had derived any value from using it and whether using the chatbot coaching in the future was a consideration or not. The interview questions were in-depth and open-ended with probes for more information when appropriate (Belotto, 2018). Examples of questions include “To what extend did you use coach Vici to help you as a graduate in the workplace?,” “Please tell me what you liked most/least about using coach Vici.” Interviews were transcribed using the otter.ai application, however the application is not 100% accurate which meant the interviewer had to listen to each recording to correct transcription errors.

The chatbot also collected the conversation data such as the goals and goal progress and this data was analysed to obtain an indication of the types of goals participants worked on.

Data analysis

Transcripts were analysed by the first author using thematic analysis (TA) (Braun and Clarke, 2012). TA was chosen as phenomenological method of analysis because it offers simplicity and flexibility to systematically identify, organise and derive insight into patterns of meaning (themes) across data sets (Braun and Clarke, 2012). TA is not tied to a particular theoretical outlook and is well suited to social phenomenology (Joffe, 2011). Braun and Clarke’s (2012) analysis process was applied as follows.

Step 1 involved data familiarisation with the aim of identifying initial areas of interest. The suggestion of “immersion in the data” (Braun and Clarke, 2012: 60) was realised by checking the transcriptions personally. This forced the researcher to listen to the recordings carefully. After transcription the researcher read all the transcripts several times and made initial notes on the nature of the content in relation to the research question of how an AI chatbot coach could assist first-time graduate employees. In Step 2, initial codes were assigned to sections of the transcripts that showed relevance to the research question by providing succinct labels to each text section (Braun and Clarke, 2012: 61). Sections varied in size from single sentences to paragraphs depending on the density and relevance of the content. For example, the sentence “One day I went to the [hair] salon during the weekend. And then I was like, whatever, like, Okay, let me talk to her now, so I love that flexibility” was assigned the code “Always available.” In step 3 all the initial codes were listed and similar codes were grouped together. These groupings represented higher-order data elements that enabled the identification of overarching categories that captured meaningful patterns in the data relative to the research question (Braun and Clarke, 2012). For example, the initial codes “Always available and “24/7 available” were merged into the “Availability” category. Step 4 reviewed these categories across all the interviews to see if they told a coherent and convincing story about the actual data (Braun and Clarke, 2012). Step 5 grouped categories with a similar meaning by analysing the essence of each to creating higher level umbrella themes (Braun and Clarke, 2012). For example, the categories “Availability” and “User friendliness,” were grouped under a theme “Convenience and accessibility.” This process was followed until all codes and categories were grouped under four succinct themes. The four themes and eight categories are discussed next.

In terms of chatbot data, the goals recorded in the chatbot by the participants were grouped into high level themes. The nine participants worked on multiple goals during the time they used the chatbot resulting in a total of 33 goals. Most of the goals (15) were career related followed by financial goals (10) and personal/health related goals (8). Examples of goals are: Career goal - “Get prompted before the end of the year”; financial goal – “I need to start saving regularly”; personal/health goal – “Go to gym 3 times per week and be fitter for summer.”

Findings

Four main themes were identified from the nine interviews with graduate employees about their impressions of using a goal-setting coaching chatbot. The themes are Convenience and accessibility, Limitations; Effectiveness in career development; and Reflection and self-awareness. These main themes and their sub-themes are summarised in Table 2 and elaborated on in the rest of this section.

Table 2.

Main and sub-themes from graduate employee interviews.

Main theme	Sub-theme	Illustrative verbatim quote
Convenience and accessibility	Availability	One day I went to the [hair] salon during the weekend. And then I was like, whatever, like, Okay, let me talk to her now, so I love that flexibility. (P6)
Convenience and accessibility	User-friendliness	It was well structured, well conceptualized, easy to navigate. It didn’t chow a lot of my time. (P4)
Limitations	Human touch	It’s so obvious that it’s a chatbot there’s no, I don’t know, no human element to it. (P1)
Limitations	Personalization and flexibility	I would have enjoyed it more if It was personalized and make it less robotic. Making more personal, to make me feel like I am not speaking to a robot. (P2)
Effectiveness in career development	Action orientation	I liked how detailed the chatbot wanted me to be like be very practical about it. (P3)
Effectiveness in career development	Incremental progress	It really helps to see those things. I would say this is the timeline that I can have, and see if it’s going to be achievable by Jun. (P6)
Reflection and self-awareness	Accountability and progress tracking	So it really helped me to keep track of things, of my goals and I could see where I’m at. (P3)
Reflection and self-awareness	Personal development	It made me think about, about how I do things, how I think about my goals and what I need to change. (P9)

Convenience and accessibility

All the interviewees mentioned the convenience and accessibility of using the chatbot as a major advantage. They appreciated being able to use it at any time of the day and its 24/7 availability. The two sub-themes that capture these sentiments are Availability and User-friendliness.

Availability

The convenience and flexibility of being able to access a coaching chatbot 24/7 was highlighted as a major advantage by participants as expressed by P1:

And then I like the fact that it’s also 24/7. You could wake up at 3 a.m. have an epiphany about something and then once you engage with a coaching or whatever you guys call it, so I like that as well. The fact that it’s at the tip of your fingers, you could wake up, like I say wake up at 3 a.m. and be able to use it at any time that it is quite convenient for me. (P1)

Participants appreciated the ability to engage with the chatbot at any time, without having to work around a human coach’s busy schedule and availability:

[The coaching chatbot is] unlike the actual mentor you know, have to work around his schedule you have to work around, you know, his timing and if you know, his busy schedule, things are cancelled and just end up even forgetting what you were going to do. (P3)

P6 and P9 particularly appreciated the instant availability of the chatbot, allowing them to receive coaching whenever they need it.

So she’s available anytime… I love that flexibility (P6)

It’s not the way you have to wait forever, like instant. So I liked that particular part.” (P9)

Overall, the accessibility and convenience of the coaching chatbot was seen as a significant benefit by the participants.

User-friendliness

The ease of use and user-friendly interface of the coaching chatbot were appreciated by some interviewees. They found the app to be well-structured, well-conceptualized, and easy to navigate:

I find that the chatbot is very user-friendly and intuitive. Even if you’re not particularly tech-savvy, it’s easy to navigate and figure out how to use. (P3)

Participants appreciated the relevant questions and follow-up questions that helped them keep track of their goals.

I think that the questions and the follow up questions were relevant. And there’s no discrepancy or a huge dispersion between like, you know, when you have a conversation that is normally a follow up question, which is related to what you answered, so I like that element, I like the fact that it’s, it’s more like it helps you keep track of where you want to go. (P5)

P9 appreciated the fact that the chatbot “incorporate my name” and that it had a personality with “chirps.” Generally participants found the flexibility and availability of the chatbot to be convenient and stress-relieving.

Limitations

Some participants found the coaching chatbot too impersonal and preferred a human touch. Participants also felt that the coaching chatbots should focus on more flexible and personalized goals that are applicable to their specific needs. While participants appreciated the structure and user-friendliness of the chatbot, they wanted it to be less robotic and more tailored to their needs. These sentiments are captured in the sub-themes human touch, goal flexibility and personalisation.

Human touch

Some participants expressed a desire for a more personalized and human touch in their coaching experience. They found the chatbot to be too robotic and impersonal, and were quick to spot that they were communicating with a machine:

I would have enjoyed it more if it was personalized and less robotic, to make me feel like I am not speaking to a robot. (P2)

P3 found the responses to be limited due to the structured nature of the chat:

And because it’s not a person obviously that I’m speaking to, it’s all machine-like. I think you could tell that it was a machine especially with certain responses which were duplicated at times. (P3)

P8 also noted the limited conversation range of the chatbot and remarked on the lack of emotion:

I think the responses are limited. Sometimes I want us to have a conversation. It feels like conversation but not. It doesn’t have that emotional thing. (P8)

Personalization and flexibility

Participants expressed their desire for more flexibility in the types of goals that can be discussed with the chatbot. They feel that Vici tended to focus on short-term goals that are not always relevant to their specific needs:

Sometimes the milestones, it seems like it’s very focused on short term goals, really, really short term goals (P1)

Participants wanted the chatbots to be more personalized and less robotic. P3 wanted it to be less “one dimensional” with more variation and more attuned to her specific needs. This sentiment was echoed by P7:

Every time you tell her something, she goes, Oh, that’s great or we should work on this or she gets very positive, constructive feedback. But in my experience, at least, she’s very prompting and repetitive. (P7)

So while participants appreciated the user-friendly and well-structured interface, they would have liked the chatbot to be tailored to their individual needs.

Effectiveness in career development

Many participants found the chatbot helpful in achieving their career goals. They appreciated the granular level of detail the chatbot provided and its focus on measurable actions and milestones as captured in the sub-themes Action orientation and Incremental progression.

Action orientation

The chatbot was perceived as highly effective in helping participants achieve their career goals through its focus on actionable steps, measurable outcomes, and granular level of detail:

I like that [using the chatbot] because we tend to just set goals but no one is gonna ask you if it’s smart, or whatever, usually you say it’s correct. So I like that I can set goals I could elaborate on why I chose that particular goal and my target. (P8)

It helps you keep track of where you want to go and helps you with advise. (P5)

The chatbot helped participants create action plans and conceptualize their goals, which they had a clear understanding of but were not actively working towards.

I realized I had a clear understanding of what I wanted to achieve but was not actively working towards it. (P2)

It helped me realize that there are things that I had to do that I was not aware of, like an action plan that I had to put into place that I was really ignoring. (P4)

The chatbot also helped participants build discipline, work-life balance, and become more goal-oriented by setting small goals to achieve bigger ones:

It also builds in discipline in terms of my own my lifestyle and also builds in like a work life balance, which I guess are crucial to me. (P5)

Become more goal oriented, in terms of setting small goals to make big goals that makes sense. (P7)

Overall it seems that the participants appreciated the chatbot’s ability to provide clear direction and guidance towards achieving their career goals.

Incremental progress

Some participants found the chatbot’s focus on specific, measurable, and time-bound goals helpful in progressively achieving their career goals. For P2 it was useful for the chatbot to “keeps track of my progress as far as your career goals are concerned.” P3 liked the idea of the chatbot helping with steady progress:

So, for instance, in a week, I would say there is a certain process that you know, I were to dedicate two to three hours in the week for, and then on Friday, I would have to account to the Chatbot and myself to say okay, did I really do this two hour or did I not do it? (P3)

The chatbot helped participants break down their goals into smaller milestones, allowing them to see their progress and make adjustments where necessary:

It helped me with setting my goal up as long as what I need to do after the grad program. Or what I would like to see for myself. So I was able to set that goal and elaborate on why and exactly how many months range. (P8)

Overall, the chatbot was seen as a helpful tool for creating a sense of progression and motivation towards achieving their career goals.

Reflection and self-awareness

Participants noted that the chatbot helped them reflect on their personal development and weaknesses, leading to greater self-awareness. This, in turn, helped them develop action plans and clear conceptualizations of their goals. The sub-themes Accountability and progress tracking, and Personal development captures these aspects.

Accountability and progress tracking

The chatbot’s accountability and progress tracking features were found to be valuable by participants in developing self-awareness and reflection.

During that exercise made me realize that I do have an idea, but then I’m not actively trying to take it through. It’s just something that is in my head that I wanted to that I want to achieve, but I’m not working towards achieving that (P4)

Participants appreciated the bot’s ability to hold them accountable for their goals, track their progress, and remind them of commitments made in previous sessions.

It will recap the milestones. It reminds me of what I have committed to in the last sessions (P2)

Overall, the chatbot was seen as a useful accountability tool that encouraged users to be more practical and proactive in achieving their goals.

Personal development

The chatbot assisted the users in confronting their weaknesses, reflecting on their personal growth, and identifying areas for improvement. Participants noted that the chatbot helped them become more accountable for their goals, build discipline in their lifestyles, and achieve a better work-life balance.

I think in a way that yeah, it [the chatbot] also helped me build in discipline in terms of my own my lifestyle and also builds in like a work life balance, which I guess are crucial to me. Succeeding in the workplace because you know, the lines between the workplace and yourself outside the workplace are so it’s so blurry. (P5)

Moreover, the chatbot helped users express their conflicts in a professional way and jot down their thoughts in a journal. Participants also mentioned that the chatbot helped them realize that their approach to achieving their goals was not working, and it helped them come face to face with the truth.

It [using the chatbot] just reveals to me that actually, you know, what you’ve been doing, it’s actually not working, you know, your approach to how you want to navigate those goals that you set or setting for yourself, was not working. So, I think I was really coming face to face with the truth. (P4)

Overall, the chatbot provided users with “someone” they could relate to and helped them on their personal development journey

Discussion

This study explored the perceptions of first-time graduate employees on the potential of a goal-attainment AI chatbot coach to support their transition into the workplace. This discussion interprets the findings using technology adoption and self-determination theories as lenses.

Technology adoption and AI coaching

AI coaching holds significant promise to scale and democratize coaching for first-time graduate employees, typically excluded from receiving human coaching due to costs. However, a critical success factor in providing such a service is the adoption levels. There is often resistance in using new technology (Venkatesh et al., 2016). Technology adoption theory (TAM) can help us make sense of the sentiment of participants in the present study towards this new type of technology. The first theme (Convenience and accessibility) and its two sub-themes Availability and User-friendliness speaks directly to key the construct “ease of use” of the TAM. A system that is always available and user-friendly is perceived as easy to use and will therefore encourage the intention and actual use of the technology (Davis et al., 1989), in this case the coaching chatbot. This finding is in line with a similar study where final year university students also found a coaching chatbot to be easy to use (Terblanche et al., 2022c). In a world where people are overwhelmed with technology it is imperative that any obstacles to adopting a new type of technology such as AI coaching chatbots should be removed. It seems that in the present study the chatbot design philosophy prescribed by the DAIC framework (Terblanche, 2020) assisted in the adoption of the AI chatbot coach.

AI chatbot coaching and self-determination theory

Promoting the adoption of new technology among employees is a very necessary but insufficient aspect of successfully deploying an AI chatbot coach for first-time graduate employees. Ultimately there must be evidence of efficacy. The present study did not set out to prove that the AI chatbot coach is effective, however the interpretation of the qualitative findings provides a few clues to the chatbot’s efficacy. Self-determination theory is a useful lens to view AI coaching efficacy.

The first aspect of self-determination theory, autonomy, holds that people need to feel in control of their own behaviors and goals. People need to take direct action that will result in real change to help them feel self-determined (Taylor et al., 2019). Evidence of the chatbot promoting autonomy in participants are found in the theme “Effectiveness in career development” and the sub-themes “Action orientation” and “Incremental progress.” The fact that the chatbot helped participants set realistic, granular goals with actions made them feel more in control of their everyday lives at work as well as their future careers. This type of assistance addresses challenges identified by Matsouka and Mihail (2016) who found that goal setting was one of the major challenges graduate employees struggled with.

The second aspect of self-determination theory, competence, states that people need to gain mastery of tasks and learn different skills. It also holds that when people feel they have the necessary skills, they are more likely to take actions that will help them achieve their goals (Deci and Ryan, 1985). This notion of competence is captured by the theme “Reflection and self-awareness” and sub-themes “Accountability and progress tracking” and “Personal development.” The greater self-awareness generated through using the chatbot made participants feel more in touch with their realistic self. This in turn assisted them to set goals in line with their abilities. One of the challenges first-time graduate employees face is dealing with the unstructured and complex world of work (Azmi et al., 2018; Plant et al., 2019), which could lead to a feeling of incompetence. Using the coaching chatbot seem to have aided them in an increased feeling of realistic competence which helped them navigate the challenges of workplace complexity more easily.

Interestingly, and admittedly speculatively, there are also indications that some participants gained second order insights into their thinking and behaviour in terms of goal attainment by becoming aware of how they think about goals. This is reminiscent of double-loop learning or premise reflection (Mezirow, 1997) which is a requirement for deep, significant and transformative learning. It could therefore be possible that AI chatbot coaches can facilitate learning not only at a transactional level, but on a deeper transformative level too.

In terms of relatedness, the third aspect of self-determination theory, the chatbot coach fell short. This is captured by the “Limitation” theme and related sub-themes of “Human touch” and “Personalization and flexibility.” Relatedness holds that people need to experience a sense of belonging and attachment to other people (Deci et al., 2017). Participants did not experience these aspects while using the chatbot. One explanation for these sentiments is that the version of chatbot coach used in the present study is a scripted, rules-based chatbot. This type of chatbot follows pre-programmed conversation paths and generic responses to user input (Adamopoulou and Moussiades, 2020). These types of chatbots have limited ability to understand the context of the conversation and can come across as robotic, as pointed out by some of the participants. While rules-based chatbots have impressive efficacy (Terblanche et al., 2022a) they can be significantly enhanced by employing generative AI such as ChatGPT (Dwivedi et al., 2023). The insight from this finding is therefore that future generations of AI chatbot coaches should actively make use of generative AI to help promote the relatedness between users and chatbot.

Limitations and recommendations for future research

The first limitation is the relatively small sample size. Although the aim of qualitative research is not to generalise, a slightly larger sample spread across n multiple organisations would have made the study more transferrable. The application of AI coaching in organisational setting is, however very new so despite this limitation we feel that the insights gained from this study is still significant as an initial exploration of the phenomenon. Future research should address this limitation and consider quantitative experimental designs to properly test AI coaching efficacy in this particular context.

The second limitation is the type of chatbot (rules-based) and duration of usage (4 weeks). This study was conducted before the general availability of generative large language models. This study should be repeated with a chatbot that incorporates generative AI conversations to improve on the robotic nature of the current bot. A longer duration of usage would also provide a more realistic assessment of sentiment towards this type of technology. A follow-up study could also be conducted to ascertain the long-term outcomes of the goal setting facilitated by the AI chatbot coach. Another type of study worth conducting is to compare the difference in participant goal attainment between human coach or a supervisor or manager with the AI chatbot coach. This could be done as a randomised controlled trial.

Conclusion

Coaching is a powerful HR intervention with proven efficacy. Unfortunately, this form of support is not available to everyone in the organisation due to high costs. This study explored the potential of AI coaching to address this challenge, in particular relating to first-time graduate employees who face significant challenges in the workplace. The study found that although the chatbot coach had certain limitations relating to humanness, it was easy to use and promoted autonomy and competence in participants. At a vastly reduced cost compared to human coaching, even if it does not completely emulate a human coach, this type of assistance with proven efficacy in terms of goal attainment should be taken seriously by organisations.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

NHD Terblanche

Note

References

Adamopoulou

Moussiades

(2020) An overview of chatbot technology. In: Artificial intelligence applications and innovations: 16th IFIP WG 12.5 international conference, AIAI 2020, proceedings, part II, Neos Marmaras, Greece, 5–7 June 2020, 373–383. Springer International Publishing.

Ajzen

(1991) The theory of planned behavior. Organizational Behavior and Human Decision Processes 50(2): 179–211.

Ajzen

Fishbein

(1975) A Bayesian analysis of attribution processes. Psychological Bulletin 82(2): 261.

Armstrong

(2017) Reinventing Performance Management: Building a Culture of Continuous Improvement. 4th edition. London, UK: Kogan Page.

Azmi

Kamin

Noordin

(2018) Competencies of engineering graduates: what are the employer’s expectations? International Journal of Engineering and Technology 7(2.29): 519–523. DOI: 10.14419/ijet.v7i2.29.13811.

Bakker

Kazantzis

Rickwood

, et al. (2016) Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Mental Health 3(1): e4984.

Bandaranaike

Willison

(2015) Building capacity for work-readiness: bridging the cognitive and affective domains. Asia-Pacific Journal of Cooperative Education 16(3): 223–233.

Belotto

(2018) Data analysis methods for qualitative research: managing the challenges of coding, interrater reliability, and thematic analysis. Qualitative Report 23(11): 2622–2633. https://nsuworks.nova.edu/tqr/vol23/iss11/2

Blackman

Moscardo

Gray

(2016) Challenges for the theory and practice of business coaching: a systematic review of empirical evidence. Human Resource Development Review 15(4): 459–486. DOI: 10.1177/1534484316673177.

10.

Braun

Clarke

(2012) Thematic Analysis. Washington, DC: American Psychological Association.

11.

Chismar

Wiley-Patton

(2003) Does the extended technology acceptance model apply to physicians. In: Proceedings of the 36th Annual Hawaii International Conference on System Sciences, 2003, Big Island, HI, 6–9 January 2003, 8. IEEE.

12.

Creswell

Hanson

Clark Plano

, et al. (2007) Qualitative research designs: selection and implementation. The Counseling Psychologist 35(2): 236–264.

13.

Davis

(1993) User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International Journal of Man-Machine Studies 38(3): 475–487.

14.

Davis

Bagozzi

Warshaw

(1989) User acceptance of computer technology: a comparison of two theoretical models. Management Science 35(8): 982–1003.

15.

Deci

Ryan

(1985) The general causality orientations scale: self-determination in personality. Journal of Research in Personality 19(2): 109–134.

16.

Deci

Olafsen

Ryan

(2017) Self-determination theory in work organizations: the state of a science. Annual Review of Organizational Psychology and Organizational Behavior 4: 19–43.

17.

Dunbar

Laing

Wynder

(2016) A content analysis of accounting job advertisements: skill requirements for graduates. E-Journal of Business Education and Scholarship of Teaching 10(1): 58–72.

18.

Dwivedi

Kshetri

Hughes

, et al. (2023) “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management 71: 102642.

19.

Elliott

Fischer

Rennie

(1999) Evolving guidelines for publication of qualitative research studies in psychology and related fields. British Journal of Clinical Psychology 38(3): 215–229.

20.

Fiske

Henningsen

Buyx

(2019) Your robot therapist will see you now: ethical implications of embodied artificial intelligence in psychiatry, psychology, and psychotherapy. Journal of Medical Internet Research 21(5): e13216.

21.

Flanagan

Walker

(2021) How can unions use Artificial Intelligence to build power? The use of AI chatbots for labour organising in the US and Australia. New Technology, Work and Employment 36(2): 159–176.

22.

Grant

Cavanagh

(2011) Coaching and positive psychology. In: Sheldon

Kashdan

Steger

(eds) Designing Positive Psychology: Taking Stock and Moving Forward. Oxford, UK: Oxford University Press, 293–309. DOI: 10.1093/acprof:oso/9780195373585.003.0019.

23.

Graßmann

Schermuly

(2021) Coaching with artificial intelligence: concepts and capabilities. Human Resource Development Review 20(1): 106–126. DOI: 10.1177/1534484320982891.

24.

Hannula

(2018) Graduate Employment in the Era of Globalisation: Challenges, Opportunities, and what Finnish Tertiary Education Could Learn from the UK [Unpublished Report]. Helsinki, Finland: Metropolia Ammattikorkeakoulu. https://urn.fi/URN:NBN:fi:amk-2018053111671

25.

Holm

Lorenz

(2022) The impact of artificial intelligence on skills at work in Denmark. New Technology, Work and Employment 37(1): 79–101.

26.

Joffe

(2011) Thematic analysis. In: Qualitative Research Methods in Mental Health and Psychotherapy: A Guide for Students and Practitioners. Hoboken, NJ: John Wiley & Sons, 209–223.

27.

Kamphorst

(2017) E-coaching systems: what they are, and what they aren’t. Personal and Ubiquitous Computing 21(4): 625–632. DOI: 10.1007/s00779-017-1020-6.

28.

Marangunić

Granić

(2015) Technology acceptance model: a literature review from 1986 to 2013. Universal Access in the Information Society 14: 81–95.

29.

Matsouka

Mihail

(2016) Graduates’ employability: what do graduates, and employers think? Industry and Higher Education 30(5): 321–326. DOI: 10.1177/0950422216663719.

30.

Mezirow

(1997) Transformative learning: theory to practice. New Directions for Adult and Continuing Education 1997(74): 5–12.

31.

Muchete

Shayo

(2020) The role of induction training on performance of new employees at workplace: case study of the open university of Tanzania. International Journal of Business Management and Economic Review 3(1): 285–300. DOI: 10.35409/IJBMER.2020.3158.

32.

Naim

Lenka

(2018) Development and retention of generation Y employees: a conceptual framework. Employee Relations 40(2): 433–455. DOI: 10.1108/ER-09-2016-0172.

33.

Pang

Wong

Leung

, et al. (2019) Competencies for fresh graduates’ success at work: perspectives of employers. Industry and Higher Education 33(1): 55–65.

34.

Plant

Barac

Sarens

(2019) Preparing work-ready graduates–skills development lessons learnt from internal audit practice. Journal of Accounting Education 48(September): 33–47. DOI: 10.1016/j.jaccedu.2019.06.001.

35.

Rowe

Zegwaard

(2017) Developing graduate employability skills and attributes: curriculum enhancement through work-integrated learning. Asia-Pacific Journal of Cooperative Education 18(2): 87–99. https://hdl.handle.net/10289/11267

36.

Spence

Oades

(2011) Coaching with self-determination theory in mind: using theory to advance evidence-based coaching practice. International Journal of Evidence-Based Coaching and Mentoring 9(2): 37–55.

37.

Stewart

Brown

(2019) Human Resource Management. Hoboken, NJ: Wiley.

38.

Taylor

Passarelli

Van Oosten

(2019) Leadership coach effectiveness as fostering self-determined, sustained change. The Leadership Quarterly 30(6): 101313.

39.

Terblanche

(2020) A design framework to create artificial intelligence coaches. International Journal of Evidence Based Coaching and Mentoring 18(2): 152–165. DOI: 10.24384/b7gs-3h05.

40.

Terblanche

Kidd

(2022) Adoption factors and moderating effects of age and gender that influence the intention to use a non-directive reflective coaching chatbot. Sage Open 12(2): 21582440221096136.

41.

Terblanche

Molyn

De Haan

, et al. (2022a) Coaching at scale: investigating the efficacy of artificial intelligence coaching. International Journal of Evidence Based Coaching and Mentoring 20(2): 20–36.

42.

Terblanche

NHD

Molyn

De Haan

, et al. (2022b) Comparing artificial intelligence and human coaching goal attainment efficacy. PLoS One 17(6): e0270255. DOI: 10.1371/journal.pone.0270255.

43.

Terblanche

Molyn

Williams

, et al. (2022c) Performance matters: students’ perceptions of artificial intelligence coach adoption factors. Coaching: An International Journal of Theory, Research and Practice 16: 1–15.

44.

Tlaiss

Martin

Hofaidhllaoui

(2017) Talent retention: evidence from a multinational firm in France. Employee Relations 39(4): 426–445. DOI: 10.1108/ER-07-2016-0130.

45.

Venkatesh

Thong

JYL

(2016) Unified theory of acceptance and use of technology: a synthesis and the road ahead. Journal of the Association for Information Systems 17(5): 328–376. DOI: 10.17705/1jais.00428.

46.

Viviers

Fouché

Reitsma

(2016) Developing soft skills (also known as pervasive skills): usefulness of an educational game. Meditari Accountancy Research 24(3): 368–389. DOI: 10.1108/MEDAR-07-2015-0045.

47.

Zhou

Zhao

Zhang

(2022) Application of artificial intelligence on psychological interventions and diagnosis: an overview. Frontiers in Psychiatry 13: 811665.