Old dog,new tricks? Exploring the potential functionalities of ChatGPT in supporting educational methods in social psychiatry

ChatGPT in social psychiatry education

The purpose of this work was to explore conceivable applications of ChatGPT in supporting educational methods in social psychiatry. Historically, incorporating the subdiscipline of social psychiatry comprehensively into training programmes has presented inherent issues, especially when academic departments are oriented towards different schools of thought (e.g. Rosenbaum & Zwerling, 1964), like biological approaches (Serby, 2000). More broadly, the biopsychosocial model has been prevalent in the psychiatric field since the 1970s (Frances, 2014), but it has been depicted as presenting pedagogical challenges; Khatri et al. (2020) found that the biopsychosocial model was a threshold concept, which students found difficult to theoretically understand and integrate within patient dialogues. Notably, the progenitor of the biopsychosocial model in psychiatry, George Engel, cited interdisciplinary problems in teaching this framework (Engel, 1982). To that end, our results suggest that ChatGPT can be an effective and quick resource that requires less human interaction.

ChatGPT’s responses to Task 1 underline its possible use-cases in psychiatric training. These broadly align with the categories proposed by Wollny et al. (2021) about how chatbots can be utilised within educational settings: skill improvement, efficiency of education, students’ motivation and availability of education. As the tool writes in its replies, ChatGPT can initiate instantaneous exchanges with students, which could enable them to gain a broader understanding of a particular field. For example, ChatGPT might assist in simulating the role of a patient, the efficacy of which has been demonstrated by other AI technologies (Dolianiti et al., 2020). From a social psychiatric perspective, when engaging with ChatGPT in this manner, learners could practise clinical interactions, mimicking a caregiver and developing a holistic profile of a ‘patient’. For evaluations, instructors could then ask students to identify structural risk factors, psychosocial stressors, extant vulnerabilities and additional social psychiatric considerations based off these exchanges. In this regard, ChatGPT may be a valuable aid to support and assess active learning outcomes and enhance training for patient-caregiver dialogues (Doshi & Bajaj, 2023).

Similarly, in our findings, ChatGPT exemplified how it can create course materials that could be embedded into psychiatric programmes across diverse educational scenarios. This has previously been foregrounded as an advantage of AI in medical education (Kaur et al., 2021) and was identified by ChatGPT in Task 1. Specifically, in our dialogues, we asked ChatGPT to generate a fictionalised case vignette about a migrant who is experiencing mental health issues, as these at-risk populations may necessitate interventions from social psychiatry (e.g., (Smith et al., 2023)). As is evident, the chatbot presented a plausible account of this hypothetical situation. Although ChatGPT’s case report includes a variety of psychopathological symptoms that may hint towards an affective disorder, it could be used to teach differential diagnoses. Alternatively, learners could be asked to provide their opinions with the aim of illustrating the notion of inter-rater reliability. Further, the vignette does not encompass a treatment plan or explore how this could be construed amidst complex determinants of mental illness, like limited psychosocial support, language barriers or environmental risk factors, which students could delineate. For another example, instructors could ask whether this case report contains enough supporting evidence to warrant a psychopharmacological therapeutic course.

Generally, using problem-based vignettes has been productive in psychiatric training (Fidler et al., 2011). Agrawal et al. (2022) underscore the effectiveness of case-based learning to develop adaptive expertise when working with individuals with severe mental illnesses. Simmons and Wilkinson (2012) found that psychiatry students in case-based learning environments exhibited higher enjoyment and understanding than their peers in didactic lectures. Nevertheless, for teachers wishing to adopt this approach, case reports may be challenging to quickly envision, especially for complex psychopathology and comorbidities. Likewise, for real-life cases, difficulties around ethics and informed consent may generate possible barriers for dissemination in a classroom. Accordingly, chatbots like ChatGPT can circumvent impediments to implementing case-based learning, quickly conceptualising and producing hypothetical vignettes oriented around explicit conditions or pedagogical intentions; Han and colleagues demonstrated this in a different medical education context (Han et al., 2023).

Whereas our example was not overly composite for the purposes of demonstrating ChatGPT’s capabilities, the difficulty of these vignettes could be increased to incorporate clinical realities in social psychiatry, such as multiple diagnoses that warrant a tailored treatment intervention. Equally, they could also be extended in length or complexity to match the teaching mode, time constraints and education level (i.e. undergraduate, trainee, doctoral). Since ChatGPT is able to write content in various languages, case reports could be created for different geographies, which would support personalised teaching initiatives outside of the English language.

Besides content generation, another prospective function for ChatGPT in social psychiatry education is its capacity to synthesise content. Although the technology did not emphasise this in its replies to us, accurate synopses of medical text from ChatGPT have been shown elsewhere (e.g. Patel & Lam, 2023; Shen et al., 2023). This could be useful for learners who are often consuming large quantities of psychiatric research as part of their educational programme. Here, ChatGPT’s ability to offer simplified or lay summaries of complex research might prove advantageous.

Current limitations of ChatGPT in the classroom

Whilst our results indicate that ChatGPT can already offer various educational benefits, there are inherent limitations to the technology in its current form. For instance, we have several concerns about the responses from ChatGPT in Task 1, where it affirms that it can deliver students resources or answer real-time questions about social psychiatry. ChatGPT does not have the same strengths in every language, which may engender a lack of structure and grammatical errors (Rudolph et al., 2023). Another known problem with ChatGPT is that it can ‘sometimes writes plausible-sounding but incorrect or nonsensical answers’ (OpenAI, 2022) and there have been newsworthy instances of this (e.g. Bowman, 2022); searching ‘#ChatGPTfails’ on Twitter provides further examples. Moreover, Bard, a language model competitor to ChatGPT recently launched by Alphabet, gave an erroneous answer in a promotional video, causing the company’s stock prices to drop significantly (Milmo, 2023). Depicted as a ‘hallucination effect’, these pitfalls are prevalent amongst chatbots in their present versions (Shen et al., 2023). Although human instructors are not infallible, caution is required to guard against possible misinformation within the current iterations of these tools, which could hinder learning outcomes and invoke other detrimental consequences. Interestingly, Stathakarou et al. found that perceptions of knowledge expertise can impact students’ trust of chatbots in healthcare education (Stathakarou et al., 2020).

Correspondingly, ChatGPT incorporates generic language extracted from the internet, meaning that it can produce subjective content (Zhai, 2023). This has led to responses that could be considered to be biased against minorities and claims that the chatbot can perpetuate discriminatory patterns (Chowdhury, 2023). Conversely, commentators have accused the tool of being predisposed to progressive opinions (Chowdhury, 2023). Taken together, these could have adverse implications in educational frameworks. For social psychiatric purposes, it could be counterproductive if ChatGPT perpetuated ideas associated with health inequalities. Equally, Ventriglio, Gupta and Bhugra propose that the existing psychiatric evidence-base has an intrinsic emphasis on aetiology and biological factors (Ventriglio et al., 2016). Therefore, it may follow that the system propagates this epistemological predisposition in psychiatric literature. This latter contention goes beyond the scope of this current study, but if substantiated, it could limit AI knowledge about diverse psychiatric concepts, wider determinants of mental health and the biopsychosocial model as a whole.

Finally, there may be further limitations with the existing iteration of ChatGPT. The chatbot entails a lack of replicability and replies cannot be reproduced exactly. Owing to its proprietary nature, it is difficult to ascertain how ChatGPT arrives at certain conclusions, which can engender misinformation and biases, as we have highlighted. The technology has limited awareness of world events beyond 2021 (OpenAI, 2022), meaning that contemporary knowledge and latest findings could be overlooked; here ChatGPT’s claim to provide ‘up-to-date information’ in Task 1 is demonstrably incorrect. Moreover, psychiatric teaching should ideally be flexible to encompass scientific developments (Rubin & Zorumski, 2003), but there may be structural impediments to integrating AI. Whilst it is currently freely usable, access to ChatGPT may eventually involve a paywall, which could inhibit its universal availability as a teaching tool (Liebrenz et al., 2023). This could impinge upon the accessibility benefits that Wollny et al. (2021) identified in educational contexts. Notably, the premium version of ChatGPT already requires a monthly subscription (OpenAI, 2023a).

Ethical aspects could also affect its use, including privacy and data storage, as have been specified for different chatbots (Okonkwo & Ade-Ibijola, 2021). Again, these issues may be especially pertinent given the commercial interests surrounding ChatGPT (Gal, 2023). Whilst our study specifically involved a fictional vignette, engaging with ChatGPT during simulated patient dialogues may increase the possibility that sensitive or personal data is inadvertently shared, which is a recognised security and privacy issue with chatbots (Hasal et al., 2021). Hence, if these technologies are adopted more widely in the academy, there may be a need for educational institutions to implement bespoke security policies and regulations that address data privacy concerns and provide guidance around best practice.

Recommendations for use and future research directions in social psychiatry education

Based on our findings and our experiences with ChatGPT that we have described elsewhere (Liebrenz et al., 2023), we argue that the language model can be advantageous in supporting educational methods in social psychiatry, if used appropriately. This corresponds with other educational studies using the tool, where researchers have concluded similarly (e.g. Han et al., 2023; Tlili et al., 2023).

Presently, in the authors’ opinion, the chatbot offers benefits for creating hypothetical case vignettes for complex symptom presentations or distinct populations, which can be difficult to obtain or conceptualise. These may generate effective course materials and facilitate wider problem-based learning initiatives, forming the foundations for future assessments. Further, ChatGPT could be useful in simulating caregiver situations, providing active-learning opportunities for students to engage in instantaneous dialogues with a ‘patient’. For students, language models can be used to synthesise complex academic text and provide a synopsis of an abstract or medical case.

Given ChatGPT’s limitations in its extant state, instructors need to consider how best to incorporate these technologies into psychiatric curricula. Currently, we would suggest avoiding an over-reliance on chatbots, especially to provide real-time information on a specific topic or subdiscipline like social psychiatry. At the time of writing, we believe that possible misinformation and biases are problematic. This may be particularly pronounced in medical education settings, such as psychiatry programmes, where it is essential that learning approaches and best practices are rooted in empirical evidence (Rubin & Zorumski, 2003). To that end, Zhai recommends that teachers use ChatGPT’s responses as a starting point, adding their own knowledge and understanding to make results more inclusive and comprehensive (Zhai, 2023). Per this approach, ChatGPT does not replace the teacher, but rather compliments and supports learning objectives. This notion is exemplified in Task 2 of our results, where it was necessary to ask a very specific question with distinct parameters for the chatbot to produce practical material relevant to social psychiatry.

However, it should be noted that as chatbots advance and adapt using human feedback, their sophistication will improve, alongside the breadth of knowledge from which they draw (Perrigo, 2023). For instance, a more advanced upgrade to ChatGPT (version 4.0) has recently been released (Hern & Bhuiyan, 2023). Were we to speculate, it is not inconceivable that language models could eventually incorporate materials from PubMed and other research libraries, meaning that the shortcomings we illustrated become less pronounced or are even no longer evident. Based on this assumption, we advise educators to prepare for the application of AI in pedagogical practices and become attuned to its potential implementation and use-cases. Accordingly, we encourage other researchers in social psychiatry and beyond to practically test the applicability of chatbots and disseminate their own perspectives with the aim of developing a more rigorous debate and comprehensive expertise within this area.

We recognise that this study explores hypothetical scenarios for ChatGPT in supporting educational methods in social psychiatry and reflects our own experiences using the language model; consequently, more detailed empirical investigations are needed. Future research could involve qualitative interviews with students and teachers, eliciting first-hand insights of chatbots in this context. For this purpose, Kaur et al. (2021) provide a useful example for how such projects could be conducted. Testing the reliability of ChatGPT’s knowledge of social psychiatry would also inform its possible uses. With that in mind, longitudinal studies may be required given the evolving nature of language models. These could help chart future developments in chatbots and enable the scientific community to better understand student and instructor perceptions as the technology becomes more widely adopted.