Descriptive Analysis of Student Diagnostic and Documentation Skill Development in a low Fidelity Simulation in Psychiatric Nurse Practitioner Education

Abstract

Introduction

Nursing educators face challenges in helping registered nurses prepare to be psychiatric nurse practitioners. Nursing educators must consider the variation of students’ skill sets, experience levels, and the shortage of preceptors. Simulations can help overcome limited clinical sites and standardize skill acquisition. High-fidelity simulations can be helpful but are labor-intensive and expensive. Low-fidelity simulations are adaptable to fit the learning needs of psychiatric nurse practitioner students but little research has evaluated these simulations.

Objective

This paper evaluates a series of low-fidelity interventions used to help students learn diagnostic criteria, treatment decisions, and documentation.

Methods

A retrospective descriptive analysis was performed on the results of all students in five cohorts in a psychiatric nurse practitioner course (n=68) on 45 low-fidelity online simulations over five years during the course on advanced psychiatric nursing of adults. Diagnostic notes were graded on four criteria by a psychologist or a psychiatric nurse practitioner. A representative sample of grades on eight notes were analyzed for this manuscript. Survey responses from the students in one cohort were obtained to evaluate their confidence levels and perceptions of simulation utility and responses were analyzed using descriptive and qualitative methods.

Results

The overall mean of description, detail, completeness, and professionalism increased from 7.6 (out of 10) to 9 over the semester with statistically significant improvements in all four areas. The strongest correlation was between detail and completeness and the weakest correlation was between professionalism and description. Students reported that they found significant value in the low-fidelity simulation exercises.

Conclusion

This analysis demonstrates one of the first efforts to analyze low-fidelity simulations for psychiatric nurse practitioner students. The data indicate these simulations can help students improve their confidence and skills in several areas.

Keywords

nursing education simulation psychiatric nurse practitioner

Introduction

The evolving nature of technology continuously challenges nursing educators (Stroup, 2014). This concept is especially true in the graduate psychiatric nurse practitioner programs (Brown, 2019; Felton & Wright, 2017). Since evidence-based treatments are constantly advancing at a rapid pace due to technology, nursing educators must be well-informed of these changes along with mastering the skills required to deliver this type of knowledge to the student population. It is required that new graduate psychiatric nurse practitioners provide the best evidence-based care and complete documentation to meet government or insurance regulations in addition to providing interprofessional communication (Brown, 2019; Sharpnack & Madigan, 2012; Stroup, 2014). However, many nursing instructors are faced with a shortage of preceptors with many sites that limit active student participation including documentation (American Association of Colleges of Nursing, 2015). Despite these obstacles, new psychiatric nurse practitioners are expected to be prepared to deliver high-quality care often relatively independently (Abram & Forbes, 2019; Brown, 2019; Felton & Wright, 2017).

Consequently, it is imperative for nursing educators to acquire a paradigm shift toward student-centered learning methods, which usually involves the use of technology, versus the traditional teaching methods (i.e., lectures, writing essays, and reading assignments) (Murray, 2014; Williams et al., 2017). This shift will help foster a more seamless transition from the student role to the professional role as a psychiatric nurse practitioner (Brown, 2019; Felton & Wright, 2017). Therefore, the implementation of innovative educational strategies within the graduate psychiatric nursing curriculum is more important than ever.

Effective engagement of graduate psychiatric nurse students requires educators to create realistic learning opportunities congruent to current clinical experiences within the psychiatric profession. One potential solution is using simulation activities to enhance the knowledge and skill acquisition necessary for the psychiatric field. In addition, numerous simulations may promote student confidence. By implementing simulation activities based on timely patient-case scenarios, students can refine their psychiatric assessment and diagnostic skills in addition to improving their expertise in other psychiatric treatment modalities (i.e., psychotropic medication and therapy techniques) (Brown, 2019; Felton & Wright, 2017; Williams et al., 2017). Moreover, students can practice the various simulation activities until they feel capable of demonstrating the necessary skill or learning objective in the actual clinical setting. On-time constructive feedback along with immediate guidance in a non-threatening environment has been shown to foster student learning by building confidence with skill performance and information retention (Sharpnack & Madigan, 2012; Stroup, 2014).

Research on the evaluation of simulation activities in graduate psychiatric nurse practitioner programs is sparse. It is imperative for psychiatric educators to thoroughly evaluate the effectiveness of simulation as a feasible learning strategy for psychiatric nurse practitioner students. Recent research encourages and promotes simulations as a promising approach to enriching graduate nursing education (Schwindt & McNelis, 2015; Williams et al., 2017).

Review of Literature

Technology has become increasingly pervasive in the lives of most students. Consequently, educators may better serve them by developing more active and meaningful learning opportunities within their curriculum while incorporating technology. Simulation activities can serve as a non-threatening way to engage students and augment learning for the mastery of specific skills, such as critical thinking, clinical reasoning, and risk assessment (Brown, 2015; Garvey et al., 2021; Sharpnack & Madigan, 2012). Simulation has been identified as a teaching strategy used to replicate real-life experiences (Brown, 2015; Garvey et al., 2021). Therefore, this type of learning modality allows students to experience realistic situations without having to endure the potential consequences of irrevocable risks (Brown, 2008; Donovan & Mullen, 2019). Moreover, studies have shown a reduction in the students’ anxiety levels when simulation activities are implemented throughout the course versus being solely delivered as the final evaluation for competency skills (Brown, 2008; Hall, 2017).

At present, nurse educators have several obstacles within today's clinical learning environment. These challenges include highly restrictive healthcare liability policies, a reduction in available clinical hours for the students, and an increase in patient-acuity levels (Brown, 2008; Hall, 2017; Tosterud et al., 2013). As a result, the nursing student's scope of learning is stifled due to the minimal “hands-on” opportunities involving patient care. With regards to stigma and safety concerns, this notion is especially true in the psychiatric nursing programs. For instance, some clinical experiences are considered too unpredictable or unsafe for students, particularly when they are at the novice level (Hall, 2017; Tosterud et al., 2013). Additionally, the current shortage of clinical preceptors creates a learning barrier for students since they may have limited opportunities to take an active role with real patients (American Association of Colleges of Nursing, 2015).

The shortage of mental health care providers is a widely known problem to psychiatric educators, mainly when trying to locate qualified preceptors for their graduate students (Abram & Forbes, 2019). To circumvent this obstacle, simulations can replicate vital aspects of many clinical situations, specifically those unique experiences that students may not observe during their designated clinical rotations (Hall, 2017). In addition, simulations can provide a way for students to become more adept with various mental health issues prior to encountering them in the clinical setting (Garvey et al., 2021). Studies have shown that students who practice simulation activities achieved higher scores in self-efficacy and confidence when compared to those who did not have this type of opportunity (Sharpnack & Madigan, 2012; Stroup, 2014). However, studies have found that simulations are seldomly used in psychiatric academic programs (Brown, 2008; Garvey et al., 2021; Hammer et al., 2014). Most simulation research has examined high-fidelity simulation models within the nursing profession, primarily in undergraduate nursing programs (Lee et al., 2018).

High-fidelity simulation experiences mimic real-life scenarios and require a high level of interactivity by the student (Stroup, 2014; Yong-Shian et al., 2016). Examples of high-fidelity simulation experiences include manakins comprised of intricate technology or standardized patients who are trained actors portraying real-life patients (Doolen et al., 2014). In fact, over 50 years ago, the first type of simulation consisted of using standardized patients when educating and training health care professionals. This type of training was established primarily for medical students and residents (Brown, 2008; Hall, 2017). The use of manakins has been the most common simulation approach used within nursing programs (Brown, 2008; Hall, 2017). Given the fact that manakins lack many human-like qualities and traits, including nonverbal communication, it was clearly determined that their use in the psychiatric curriculum was inappropriate (Brown, 2008). Furthermore, these types of simulation experiences are expensive and labor-intensive, which often dissuades their use by nursing educators beyond limited summative examinations. More recent programs have started to use virtual reality to develop more immersive simulations for students (Lee et al., 2020). However, some research has found significant challenges with the implementation and refinement of such simulations (Kim et al., 2021). By contrast, studies have found that low-fidelity simulation models can improve critical thinking skills, increase clinical competencies, and support the transition of nursing theory into the clinical care setting (Sharpnack & Madigan, 2012; Stroup, 2014). These findings fostered the creation of broader and more comprehensive designs of simulation, specifically tailored for various educational programs (Brown, 2008). At present, simulation-based learning strategies are now being widely accepted by nursing educators at a rapid pace (Brown, 2015; Donovan & Mullen, 2019; Hall, 2017).

Low-fidelity simulation experiences can offer little realism and often involve static models in which the experience progresses with pre-determined prompts regardless of the student's responses. By contrast, high-fidelity simulations are often dynamic (e.g., an actor responding to a students questions) (Tosterud et al., 2013). Low-fidelity simulations focus on helping the student sharpen specific skills (Kluwer, 2018; Munshi et al., 2015; Stroup, 2014). An example of a low-fidelity simulation would include a computer program that provided a snapshot or vignette of a patient's scenario and/or health status. The students would then analyze the situation and apply their class learnings to the patient's scenario. Spontaneous critical questions would be asked throughout the learning activity based on the students’ answers. Dependent on the students’ decision-making processes and responses, the patient's health status could either change for the better or worse. Finally, the students would have to appropriately document their findings, determine an accurate diagnosis, and clearly identify the best treatment approach. Overall, the students would be able to discuss the key determinants that lead to their final decisions regarding treatment planning and referrals to other resources (Munshi et al., 2015).

When comparing various simulation methods, it was found that the majority of nursing students actually preferred the low-fidelity simulation models over the more complex ones (Hall, 2017; Tosterud et al., 2013). This concept was true regardless of the students’ education level (Hall, 2017; Kirkbakk-Fjær & Skundberg-Kletthagen, 2018; Tosterud et al., 2013). According to research, there were several reasons as to why the students preferred low-fidelity simulation models over other learning strategies. These preferences were mainly related to the ease of access of various learning activities, the option for skill repetition until mastery, timely feedback for improvement, and the allowance for critical decision-making in a safe environment (Hall, 2017; Kirkbakk-Fjær & Skundberg-Kletthagen, 2018).

Although low-fidelity simulations are readily accessible, few researchers have examined their use in graduate nursing programs (Abram & Forbes, 2019; Hammer et al., 2014). This concept is especially true regarding the use of low-fidelity simulation experiences within graduate psychiatric nurse practitioner programs. The acquisition of knowledge can be further facilitated when students engage with uncertain or unfamiliar experiences that change their preconceived notions (Brown, 2008; Hall, 2017; Marshall et al., 2018). This process can be safely accomplished through the use of low-fidelity simulations. In essence, low-fidelity simulations allow students to carefully explore their emotional and cognitive decisions associated with their future professional responsibilities (Brown, 2008; Hall, 2017; Marshall et al., 2018). As a result, reflective learning is promoted which is one of the primary goals of quality education (Brown, 2008; Hall, 2017). Successful learning outcomes can be further achieved through meticulous attention to simulation design and detail, along with providing precise learning objectives to the students (Hammer et al., 2014). In addition, even though the anecdotal feedback of students has been consistently affirmative, more research is necessary to empirically justify the use of simulation as an educational strategy (Brown, 2008; Hammer et al., 2014; Kirkbakk-Fjær & Skundberg-Kletthagen, 2018; Marshall et al., 2018). Lastly, simulation acceptability in nursing education has been identified as a significant gap in recent literature (Gause et al., 2022). The current study takes a step in filling this gap by evaluating two student surveys developed to evaluate these exercises.

Objective: The overall purpose of this study was to evaluate a series of low-fidelity simulations created to assist in the instruction of psychiatric nurse practitioner students.

Aim 1: Assess ratings of student simulation performance over the semester including improvements in description, detail, completeness, and professionalism

Hypothesis: Ratings will show trends of improving over the semester

Aim 2: Assess ratings of student confidence in critical psychiatric assessment, diagnostic, and documentation skills

Hypothesis: Students will report high levels of confidence in identified psychiatric skills

Aim 3: Assess student ratings of the utility of the simulations

Hypothesis: Students will report that the simulations helped them develop important psychiatric skills

Methods

Study Design and Participants

This study is a retrospective descriptive analysis of diagnostic notes written by students in online low-fidelity psychiatric simulations. All students in five cohorts of psychiatric nurse practitioner program (n = 68) were included in the analysis. Data were collected between January 2019 and May 2023. Class sizes commonly range from nine to 14 students. The responses and feedback from all students in the program at the University of Nebraska Medical Center College of Nursing over five years are included in the analysis. Nebraska is one of 27 states in the United States thus far which has granted nurse practitioners the authority to practice to the full scope of their preparation. In these states, nurse practitioners can work independently to evaluate, diagnose, and provide treatment. Treatment most often focuses on the prescription of medications and providing psychotherapy but may also include administering transcranial magnetic stimulation and electroconvulsive therapy. Independent practice helps reduce barriers to providing patient care but may also increase pressure on faculty to adequately prepare students.

Demographics were not collected for this study but as the analysis included data from all 68 students in the program over five cohorts, it can be argued that these results likely represent a fair approximation of students in psychiatric nurse practitioner programs at large. Inclusion criteria consisted of students enrolled in the psychiatric nurse practitioner program and no individuals were excluded.

Power Analysis

Our feasibility test is to ensure that this study can evaluate 50 participants in the study period to assess the preliminary effect size of the intervention. This sample size is based on guidelines for behavioral treatment development research. The research objective is to obtain reasonable estimates of the magnitude of the effect of the intervention for planning the needed sample size for a subsequent efficacy trial. Thus, the researchers performed a power analysis in NCSS PASS 14. Utilizing a sample size of 50, the statistical power of the analysis is determined to be 70%. This level of power enables the detection of a change in slope amounting to 0.05, assuming a two-sided statistical hypothesis. The analysis is conducted with a significance level set at 0.05 and considers additional parameters: the standard deviation of variable X is 2.5, the standard deviation of variable Y is 0.510, the standard deviation of residuals is 1, and the coefficient of determination (R²) is established at 0.04. These specifications collectively contribute to the robustness and precision of the statistical findings.

Intervention

The first author developed a series of 45 exercises that allow students to practice assessment and documentation beyond the objective structured clinical exams already conducted (see Figure 1 for study procedures flowchart). In the simulations analyzed in this project, students were presented with information about a simulated patient that did not change based on the student's responses. The primary investigator wrote basic dialogues between a patient and provider containing the review of symptoms portion of the interview. Voice-over actors made audio recordings to make the scenarios as realistic as possible. Each scenario focused on only one disorder as the exercises were used in the program's introductory diagnostic and treatment course. The audio tracks were short, generally ranging from 2 to 4 min.

Figure 1.

Study procedures flowchart revised.

The audio tracks were then placed into quizzes in Canvas®. Starting in the third week of the course, students were assigned between two and seven exercises most weeks depending on the number of major disorders covered in the weekly content. The exercises first asked the students to listen to the dialogue (see Figure 2 for flowchart of student activities in simulation). They were then asked to identify the most likely diagnosis for the patient. The next page asked students to identify the diagnostic criteria they noted during the recording. In some weeks, this was done through a checklist, and in other weeks a free-text list was elicited. However, student responses to these questions were not included in this analysis. Students were then required to write out the portion of the visit note in which they reviewed the presenting symptoms, related them to DSM criteria, and justified their diagnosis. The final portion of the exercise consisted of students identifying treatments they would consider for the patient. In some weeks, with fewer exercises, the patients called back or returned with side effects or other responses to the treatment. Sequentially, the students were then asked to identify and determine their next steps in treatment process.

Figure 2.

Flowchart of student activities in simulation revised.

The simulations were intended to encourage students to work with the diagnostic criteria using active learning methods that would facilitate information retention. The simulated patient presentations were also designed to supplement their clinical experiences in showing how symptoms might manifest in different patients, as well as examine cases of disorders they may not see in clinicals. Students also wrote clinical notes for the majority of the simulations and received detailed feedback from a clinician who knew all the details of the patient interaction (unlike typical supervision where faculty must rely largely on the student's description).

Eight of the 45 simulations were selected for analysis in this study based on three main criteria. First, one disorder was selected from each major category from the DSM covered in this class (anxiety, obsessive-compulsive, depressive, bipolar, schizophrenia spectrum, substance use, dementia, and trauma disorders). Second, in most cases, the most significant and/or common disorder was selected (e.g., major depressive disorder rather than disruptive mood dysregulation disorder). Third, the selected disorders were spread out over the course of the semester as evenly as possible to facilitate the analysis of changes over time.

Data Collection

The primary investigator downloaded the notes from the five cohorts of students from Canvas®. The first four cohorts were rated by a psychologist and the last by a psychiatric nurse practitioner. Notes were rated on a one-to-ten scale based on four categories. “Description” included a rating of how well the note detailed the ways in which the patient met each DSM criteria (e.g., “has high anxiety almost every day that peaks in the afternoon” versus “is often anxious”). “Detail” included giving as much detail as provided by the patient as possible (e.g., “having difficulty staying asleep and getting five hours of sleep on average” versus “not getting enough sleep”). “Completeness” included documenting all areas discussed in the simulation (e.g., including the discussion of all symptoms of depression discussed even when only five would be needed for the diagnosis). “Professionalism” included the organization, style, and terminology of the note (e.g., avoiding colloquial language, using the most specific professional terms, etc.).

A score of 10 on “description” would reflect a note that included a complete description of how the patient met each DSM criteria whereas a score of five would indicate they included approximately half of the description provided in the simulation, etc. A score of 10 on “detail” would indicate the student included every detail provided in the simulation whereas a score of five would reflect a note that listed approximately half of the details, etc. A score of 10 on “completeness” would indicate that the student listed all areas discussed in the simulation whereas a score of five would reflect that they listed only approximately half of the topics or criteria discussed, etc. A score of 10 on “professionalism” would indicate the note used professional style and terminology throughout whereas a score of five would indicate a note that had significant professional lapses, etc. Throughout the semester, students were given their numerical ratings and narrative feedback on how to improve.

The criteria of description, detail, completeness, and professionalism were selected based on criteria used to make sure patient encounter notes are sufficient to send for insurance reimbursement. In addition, insurance companies often examine these criteria when determining whether a higher-cost treatment is justifiable in a given situation. For example, the faculty emphasize to students that unless they document all of the patient's symptoms completely, a patient with treatment-resistant major depressive disorder may not be granted more expensive treatment (e.g., transcranial magnetic stimulation) that may hold promise for them. At the beginning of each semester, faculty and raters discussed the criteria and reviewed examples.

Surveys

The primary investigator developed two surveys that were used in this study. The first survey assessed the students’ confidence in aspects of assessing, diagnosing, and documenting patient interactions. Students were asked to rate their confidence in seven skills on a scale of zero (“Not confident at all”) to four (“Completely confident”) (see Supplemental file Educational Surveys for full details). The results of the survey can be used to assess the student's development, inform future clinical placements based on needed skill development, and provide feedback on the simulations. The first survey had a Cronbach's Alpha of 0.809.

The second survey was designed to assess the students’ perceptions of the utility of the simulation exercises. The survey had six statements (i.e., the exercises caused me to relate my patient's symptoms to the diagnostic criteria more directly) and students were asked to evaluate how much they agreed or disagreed with the six statements. These statements regarded the students’ perceptions of the simulation's utility and rated on a scale of zero (“Strongly disagree”) to four (“Strongly agree”) (see Supplemental file Educational Surveys for full details). Lastly, the survey had a free-text section where students were asked to provide any additional feedback or comments. The second survey had a Cronbach's Alpha of 0.896. Responses to the second survey were collected to assess the need for simulation revision. The surveys were developed primarily for programmatic evaluation with face validity. The surveys were reviewed and critiqued by two content experts who were not involved in this teaching or research. All 13 students in one cohort completed both surveys at the end of the semester.

Data Analysis

The researchers selected one simulation representing each major disorder group covered during the semester. These included generalized anxiety disorder, obsessive-compulsive disorder, major depressive disorder, bipolar I disorder (manic), schizophrenia, alcohol use disorder, Lewy body dementia, and post-traumatic stress disorder. This study obtained ratings of description, detail, completeness, and professionalism for 68 students on a total of eight trials. Pearson correlations were reported among four ratings of description, detail, completeness, and professionalism. Interrater reliability was assessed by having one therapist and one psychologist (both independent of the course) blindly rate 31 notes that had been graded by the course graders. The Cronbach's Alpha was 0.504. A separate linear mixed model was performed to compare the four ratings across eight trials. A random intercept model was used to account for the correlation of rating within the subject. Statistical analyses were performed using SAS 9.4 (Cary, NC). The statistical significance was claimed at 95% confidence level (p-value < .05).

Surveys were administered to only one cohort of 13 students. The standard techniques of qualitative content analysis were utilized for this study. The researchers analyzed the participants’ free-text responses from the surveys through a process of open-thematic inquiry followed by focused coding to identify pertinent themes. Subsequently, the investigators compared their results, distinguished consistencies, and agreed upon the predominant themes and patterns. In addition, repetitious comparisons were made on the relationships between the codes and the derived themes.

Results

Aim 1: Assess ratings of student simulation performance over the semester including improvements in description, detail, completeness, and professionalism

Students scored the lowest on the first exercise of the semester, with an overall mean of 7.3 (see Table 1 for complete results). The overall mean on the last exercise was 8.9. Statistically significant improvements were seen in description, detail, completeness, and professionalism. Ratings on description increased at the rate of 0.06 ± 0.02 (p = .02), partly due to the relatively high score at baseline. The other three categories had larger increases at the rate between 0.13 and 0.18 with p-value < .0001.

Table 1.

Graded Note Means by Exercise.

Position in semester	Disorder	Description	Detail	Completeness	Professionalism
1	GAD	8.5 (2.5)	7.0 (2.8)	7.1 (2.9)	6.8 (2.6)
2	OCD	8.7 (1.5)	9.2 (1.3)	9.3 (1.3)	8.4 (1.5)
3	MDD	8.7 (1.5)	8.7 (2)	9 (1.7)	8.3 (1.5)
4	Bipolar Disorder	9.1 (1)	9.2 (1.2)	9.2 (1.2)	8.6 (1.2)
5	Schizophrenia	8.6 (1.9)	9.1 (1.8)	9.3 (1.6)	8.6 (1.7)
6	Alcohol Use Disorder	9.1 (1)	9.1 (1.2)	9.2 (1.2)	8.9 (1.1)
7	Lewy Body Dementia	8.8 (1.7)	9 (1.4)	9.1 (1.5)	8.8 (1.6)
8	PTSD	9.1 (1.1)	8.9 (1.5)	8.9 (1.5)	8.7 (1.5)
	Rate of Change (β)^a	0.05 ± 0.02	0.16 ± 0.03	0.14 ± 0.03	0.19 ± 0.03
	Linear Trend p-value^a	.02	<.0001	<.0001	<.0001

Note. mean (standard deviation).

Separate general linear mixed model was performed for each outcome variables in the column. Linear regression analysis was performed with week (1–8) as the explanatory variable. Random intercept model was contracted to account for the within-subject variation over time. Regression coefficient and p-values for the linear trend were reported.

The strongest correlation was between detail and completeness (r = .93, p < .0001). Correlations between description-detail, description-completeness, detail-professionalism, and professionalism-detail were all significant and had similar values ranging from r = .68 to .72. The weakest correlation was seen between description and professionalism (r = .61, p < .0001). The overall standard deviation of the scores on the first exercise was 2.7 and this was reduced to 1.4 on the last simulation.

Aim 2: Assess ratings of student confidence in critical psychiatric assessment, diagnostic, and documentation skills

In the surveys, students reported their confidence level between “Somewhat confident” and “Fairly confident” on all skills (see Table 2 for full results). Means ranged from 2.31 to 2.85. Students were least confident in their ability to make an accurate diagnosis and most confident in their ability to document accurately.

Aim 3: Assess student ratings of the utility of the simulations

Table 2.

Survey Responses of Students Rating Their Confidence in Critical Skills.

Confidence area	Student responses
Your knowledge of the diagnostic criteria	2.62 (0.65)
Your ability to assess responses for psychiatric symptoms	2.77 (0.73)
Your ability to relate responses to psychiatric criteria	2.54 (0.66)
Your ability to make an accurate diagnosis	2.31 (0.75)
Your ability to select appropriate treatments	2.54 (0.78)
Your ability to document accurately	2.85 (0.55)
Your ability to document completely	2.77 (0.6)

Note. 0 (“Not confident at all”) to 4 (“Completely confident”).

Mean (standard deviation).

Student feedback regarding the utility of the simulations was largely positive, with the mean on most questions centering around three (“Agree”) (see Table 3 for full results). The lowest rated question was “These exercises provided me with valuable opportunities to practice charting,” and the highest rated question was “These exercises caused me to relate my patient's symptoms to diagnostic criteria more directly.”

Table 3.

Student Perceptions of Simulation Utility.

Statements	Student responses
These exercises caused me to relate my patient's symptoms to diagnostic criteria more directly	3.23 (0.44)
These exercises helped me to better understand how to relate my documentation to the patient's presentation	2.92 (0.95)
These exercises helped me better understand how to relate my documentation to the patient's diagnosis	3 (1)
These exercises provided me valuable opportunities to practice charting	2.46 (1.45)
I believe I will be better able to assess patients as a result of these exercises	3 (0.71)
I believe I will be better able to document my patients’ conditions as a result of these exercises	3.08 (0.76)

Note. 0 (“Strongly disagree”) to 4 (“Strongly agree”).

Mean (standard deviation).

The data analysis revealed three essential themes based similar statements made by the majority of students. In the first theme, most students reported that the exercises enhanced their learning process. Frequent comments were made that the exercises accelerated their overall understanding of the DSM-5 diagnostic criteria and they apply to their clinical experiences. The second theme that emerged was that the students saw the exercises as an opportunity to practice specialty skills safely. This is especially true when assessing patients’ mental health conditions and considering their unique social and environmental circumstances. This theme highlights the complexity of diverse patient care, especially within the psychiatric milieu. The students often saw the exercises as an opportunity to practice assessment and documentation in a standardized situation. The third theme that emerged was a request for the simulations to be used more frequently throughout the curriculum. The students highlighted the growth facilitated by the exercises and compared the utility of the simulations favorably to other instructional methods such as discussion boards.

Discussion

Nursing educators have many options for simulation in their programs including manikins, standardized patients, role-playing, skills stations, and computer-based simulations (Sofer, 2018; Verkuyl & Mastrilli, 2017). Simulations are common in many areas of nursing education, but have not been widely explored in the training of psychiatric nurse practitioner students.

Most research studies have primarily focused on the high-fidelity simulation models which have proven to be difficult to implement within the nursing programs and often nonrelevant to the psychiatric nursing curriculum (Sharpnack & Madigan, 2012; Stroup, 2014). Given this fact, this article is the first identified formal analysis of simulations used to help students develop certain techniques which include interview analysis, diagnostic reasoning, and clinical documentation skills. Furthermore, this article also provides an example of how simple, low-fidelity simulations can foster the development of critical skills in psychiatric nurse practitioner students. Students in psychiatric nurse practitioner programs are diverse, ranging from almost no experience to over a decade of work as a therapist or nurse practitioner in another specialty. As a result, the students’ individual needs vary significantly, and instructional methods need to ensure that all students reach a level of beginning competence from which they can build as they gain experience.

Summative assessments such as objective structured clinical examinations are critical in ensuring students are ready to start practice. However, such intensive simulations are often labor-intensive and expensive, precluding their routine use throughout a program. Low-fidelity formative assessments, such as those analyzed in this study, can help ensure students develop essential skills throughout their educational program.

In this analysis, students demonstrated consistent improvements over the semester. As may have been expected, the students who scored lower initially improved the most, resulting in more consistent performance across students at the end of the semester. This finding corresponds with past studies that demonstrate how low-fidelity models can improve students’ critical thinking skills, including sharpening specific skill sets along with strengthening their reflective learning process (Sharpnack & Madigan, 2012; Stroup, 2014). Correlations indicate that some skills (i.e., detail and completeness) are closely related and are likely learned simultaneously. Conversely, other skills (i.e., description and professionalism) are more diverse and require particular attention and feedback. The quantitative and qualitative results from the surveys showed that while most students spent many hours on the simulations, the learners regarded the exercises as a valuable opportunity for their development. Therefore, this finding is similar to past studies considering the students’ preferences for low-fidelity models over high-fidelity models, regardless of their educational or skill level (Hall, 2017; Tosterud et al., 2013). This learning modality allows students to engage in realistic situations and to safely practice navigating high-risk situations before entering the clinical setting (Brown, 2008; Donovan & Mullen, 2019).

Ethical Considerations

Consent was not obtained from students as the University of Nebraska Medical Center Institutional Review Board concluded that this study did not constitute human subjects research. In addition, the exercises were originally implemented for educational purposes with possible research implications being conceptualized only years later. As a result, most students had graduated by the time the research analysis started, at which time obtaining consent would not have been feasible. Once the team designed the study, the primary investigator (who is the coordinator of the course) informed all subsequent students that the analysis would be taking place. The individuals who graded the notes were other faculty members or outside mental health professionals. Students may have felt some pressure to give positive reviews in the surveys but these were completed after the course had concluded. In addition, it could be assumed that students would feel no more pressured to give positive responses than they would in any other anonymous course feedback.

Strengths and Limitations

This article offers the first identified analysis of low-fidelity simulations in psychiatric nurse practitioner education. It reviews the development of the intervention and grading procedures. The analysis also reviews student acceptability and confidence levels.

This analysis has several limitations. Findings would be more robust if results from a larger sample size were available for analysis. In addition, pre-and post-surveys would be helpful for analysis of changes. The raters may have expected student performance to improve over the semester but they also provided students with ongoing feedback to facilitate skill development which likely led to significant increases in student performance. Having only one rater likely limited reliability and the Cronbach's alpha was questionable. However, the researchers believe the actual graders had a higher level of consistency as their abilities were developed over many notes across the semester.

Implications for Practice

The education of psychiatric nurse practitioners presents significant challenges which often include a shortage of clinical preceptors, limited opportunities for students to practice in the primary interview and documentation role, and lack of instructor ability to directly observe student performance. Low-fidelity simulations appear to serve as an effective way to help students gain critical skills they will need as new nurse practitioners. Such efficient and standardized formative assessments may help to prepare students for more costly and labor-intensive high-fidelity simulations serving as summative assessments.

Conclusion

Low-fidelity simulations use few resources and can be used frequently throughout programs to promote active learning. Students improved their performance in all assessed areas. They also saw the simulations as a valuable part of their learning process. Simulations such as the ones analyzed in this study can supplement other forms of learning in helping students develop skills essential for beginning psychiatric nurse practitioners.

Supplemental Material

sj-xlsx-1-son-10.1177_23779608241281443 - Supplemental material for Descriptive Analysis of Student Diagnostic and Documentation Skill Development in a low Fidelity Simulation in Psychiatric Nurse Practitioner Education

Supplemental material, sj-xlsx-1-son-10.1177_23779608241281443 for Descriptive Analysis of Student Diagnostic and Documentation Skill Development in a low Fidelity Simulation in Psychiatric Nurse Practitioner Education by Nicholas Guenzel, Hongying Dai and Lyndsay Dean in SAGE Open Nursing

Footnotes

Acknowledgments

The authors would like to thank the students for their participation in efforts to improve instruction methods.

Availability of Data and Materials

A copy of the data set has been uploaded.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Nicholas Guenzel

Supplemental Material

Supplemental material for this article is available online.

References

Abram

M. D.

Forbes

M. O.

(2019). High-fidelity simulation: An application to psychopharmacological training for the psychiatric nurse practitioner student. Issues in Mental Health Nursing, 40(3), 260–267. https://doi.org/10.1080/01612840.2018.1519621

American Association of Colleges of Nursing (2015). Re-envisioning the clinical education of advanced practice registered nurses. White Paper. Http://www.Aacnnursing. Org/Portals/42/News/White-Papers/APRN-Clinical-Education

Brown

A. M.

(2015). Simulation in undergraduate mental health nursing education: A literature review. Clinical Simulation in Nursing, 11(10), 445–449. https://doi.org/10.1016/j.ecns.2015.08.003

Brown

J. E.

(2019). Graduate nurses’ perception of the effect of simulation on reducing the theory-practice gap. SAGE Open Nursing, 5, 237796081989696. https://doi.org/10.1177/2377960819896963

Brown

J. F.

(2008). Applications of simulation technology in psychiatric mental health nursing education. Journal of Psychiatric and Mental Health Nursing, 15(8), 638–644. https://doi.org/10.1111/j.1365-2850.2008.001281.x

Donovan

L. M.

Mullen

L. K.

(2019). Expanding nursing simulation programs with a standardized patient protocol on therapeutic communication. Nurse Education in Practice, 38, 126–131. https://doi.org/10.1016/j.nepr.2019.05.015

Doolen

Giddings

Johnson

de Nathan

G. G.

Badia

L. O.

(2014). An evaluation of mental health simulation with standardized patients. International Journal of Nursing Education Scholarship, 11(1), 55–62. https://doi.org/10.1515/ijnes-2013-0075

Felton

Wright

(2017). Simulation in mental health nurse education: The development, implementation and evaluation of an educational innovation. Nurse Education in Practice, 26, 46–52. https://doi.org/10.1016/j.nepr.2017.06.005

Garvey

Willetts

Sadoughi

Olasoji

(2021). Undergraduate nursing students’ experience of mental health simulation post-clinical placement: A qualitative study. International Journal of Mental Health Nursing, 30(1), 93–101. https://doi.org/10.1111/inm.12801

10.

Gause

Mokgaola

I. O.

Rakhudu

M. A.

(2022). Technology usage for teaching and learning in nursing education: An integrative review. Curationis, 45(1) 1–19, https://doi.org/10.4102/curationis.v45i1.2261

11.

Hall

(2017). Simulation-based learning in Australian undergraduate mental health nursing curricula: A literature review. Clinical Simulation in Nursing, 13(8), 380–389. https://doi.org/10.1016/j.ecns.2017.04.002

12.

Hammer

Fox

Hampton

M. D.

(2014). Use of a therapeutic communication simulation model in pre-licensure psychiatric mental health nursing: Enhancing strengths and transforming challenges. Nursing and Health, 2(1), 1–8. https://doi.org/10.13189/nh.2014.020101

13.

Kim

M. J.

Kang

H. S.

De Gagne

J. C.

(2021). Nursing students’ perceptions and experiences of using virtual simulation during the COVID-19 pandemic. Clinical Simulation in Nursing, 60, 11–17. https://doi.org/10.1016/j.ecns.2021.06.010

14.

Kirkbakk-Fjær

Skundberg-Kletthagen

(2018). Nursing students’ in mental health nursing shown by the debriefing phase in low-fidelity simulation. Issues in Mental Health Nursing, 39(9), 773–778. https://doi.org/10.1080/01612840.2018.1455774

15.

Kluwer

(2018). Increasing fidelity and realism in simulation for nursing students. https://www.wolterskluwer.com/en/expert-insights/increasing-fidelity-and-realisim-in-simulation

16.

Lee

C. A.

Pais

Kelling

Anderson

O. S.

(2018). A scoping review to understand simulation used in interprofessional education. Journal of Interprofessional Education & Practice, 13, 15–23. https://doi.org/10.1016/j.xjep.2018.08.003

17.

Lee

Kim

S. K.

Eom

M.-R.

(2020). Usability of mental illness simulation involving scenarios with patients with schizophrenia via immersive virtual reality: A mixed methods study. PloS One, 15(9), e0238437. https://doi.org/10.1371/journal.pone.0238437

18.

Marshall

Bliss

Evans

Dukhan

(2018). Fostering transformation by hearing voices: Evaluating a 6-s, low-fidelity simulation. Journal of the American Psychiatric Nurses Association, 24(5), 426–432. https://doi.org/10.1177/1078390317750749

19.

Munshi

Lababidi

Alyousef

(2015). Low-versus high-fidelity simulations in teaching and assessing clinical skills. Journal of Taibah University Medical Sciences, 10(1), 12–15. https://doi.org/10.1016/j.jtumed.2015.01.008

20.

Murray

B. A.

(2014). The use of high-fidelity simulation in psychiatric and mental health nursing clinical education. International Journal of Health Sciences Education, 2(1), 1. https://doi.org/10.59942/2325-9981.1005

21.

Schwindt

McNelis

(2015). Integrating simulation into a reflection-centered graduate psychiatric/mental health nursing curriculum. Nursing Education Perspectives, 36(5), 326–328. https://doi.org/10.5480/15-1614

22.

Sharpnack

P. A.

Madigan

E. A.

(2012). Using low-fidelity simulation with sophomore nursing students in a baccalaureate nursing program. Nursing Education Perspectives, 33(4), 264–268. https://doi.org/10.5480/1536-5026-33.4.264

23.

Sofer

(2018). The value of simulation in nursing education. The American Journal of Nursing, 118(4), 17–18. https://doi.org/10.1097/01.NAJ.0000532063.79102.19

24.

Stroup

(2014). Simulation usage in nursing fundamentals: Integrative literature review. Clinical Simulation in Nursing, 10(3), e155–e164. https://doi.org/10.1016/j.ecns.2013.10.004

25.

Tosterud

Hedelin

Hall-Lord

M. L.

(2013). Nursing students’ perceptions of high-and low-fidelity simulation used as learning methods. Nurse Education in Practice, 13(4), 262–270. https://doi.org/10.1016/j.nepr.2013.02.002

26.

Verkuyl

Mastrilli

(2017). Virtual simulations in nursing education: A scoping review. Journal of Nursing and Health Sciences, 3(2), 39–47. https://doi.org/10.1016/j.nepr.2018.05.007

27.

Williams

Reddy

Marshall

Beovich

McKarney

(2017). Simulation and mental health outcomes: A scoping review. Advances in Simulation, 2(1), 2. https://doi.org/10.1186/s41077-016-0035-9

28.

Yong-Shian

Selvarajan

Chng

M.-L.

Tan

C.-S.

Yobas

(2016). Using standardized patients in enhancing undergraduate students’ learning experience in mental health nursing. Nurse Education Today, 45, 167–172. https://doi.org/10.1016/j.nedt.2016.08.005

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.05 MB