Abstract
Nine counselling psychology students were enrolled in a 12-week pilot practicum (i.e. a work placement) for either one hour of course credit (six students) or three hours (three students). Group supervision was provided both in-person and by videoconferencing. Each trainee completed a measure evaluating their satisfaction with supervision (Supervisory Satisfaction Questionnaire, SSQ) and the supervisory relationship (Supervisory Working Alliance Inventory – Trainee Version, SWAI-T). The student's self-efficacy was also tracked during the semester (Counselling Self-Estimate Inventory, COSE). Trainees rated their satisfaction with videoconferencing similarly to the in-person format. The supervisory relationship also did not appear to be affected by the videoconferencing format. The COSE scores indicated that the students increased in counsellor self-efficacy by the end of the semester. Trainees reported that their supervisory needs were met and believed that videoconferencing was a viable format for supervision, although such a format still needed to be augmented by in-person contact. Providing better access to supervision and professional support using technology is one step towards improving health care in rural areas.
Introduction
There is a shortage of mental health professionals in rural areas. 1 One-third of America's most rural counties lack any psychologists and an even larger proportion lack any kind of specialist mental health service. 2 Due to the scarcity of psychologists in rural areas, the opportunities for collaboration, referrals and support between therapists is limited. 3 Without good supervision and a professional support system, rural mental health professionals often feel overwhelmed, 4 report lower job satisfaction, become susceptible to burnout and provide lower quality services. 5 However, travelling long distances for supervision takes time and money, and leaves less time to provide services for a population that is already dramatically underserved.
The use of telehealth to provide clinical supervision, sometimes called telesupervision, has been suggested as an efficient and affordable way to address the problems of training and professional support. Telesupervision offers the opportunity to maximize the use of direct supervision time while simultaneously ensuring proper credentialing and professional education and training in specific areas of supervision. 6
Telesupervision programmes have been developed both domestically 7–9 and internationally. 10 Miller et al. described an internship programme located in a rural setting that had difficulty finding qualified supervisors. 7 To maintain supervisory accreditation standards, a telehealth-based model of supervision was developed that used videoconferencing, email and other techniques to provide clinical supervision. The response to this programme was rated favourably by interns who used it, which is consistent with other telesupervision research. For example, those in remote areas who were supervised by videoconferencing reported high levels of satisfaction 10 and feelings of reduced isolation. 11
Telesupervision has been implemented internationally as a successful means of increasing retention rates of psychologists, training interns in rural areas such as northern Canada 12 and outlying regions of Norway, 13 as well as helping to train residents at academic institutions in Norway. 14 Despite concerns about reduced non-verbal cues, all participants in the latter study were satisfied with, and would recommend videoconferencing as an acceptable adjunct to face-to-face supervision for some supervision sessions. 14 Many of those supervised reported that paradoxically, the limited visual cues imposed by videoconferencing improved the quality of their experiences due to the increased emphasis on the verbal aspects of communication. Respondents stated that videoconferencing increased their insight, their ability to perceive cues embedded in verbalizations and their dexterity in comprehending chaotic feelings and thoughts.
Research in telesupervision has provided generally positive results. However, much of
the literature is theoretical, descriptive or lacks controlled studies that have
utilized reliable measures. The purpose of the present study was to answer the
questions: Are those supervised via videoconferencing as satisfied as those supervised
face-to-face? Does the supervisory relationship differ between supervision formats? Is counsellor self-efficacy affected by supervision format? Does the level of clinical training influence trainee perceptions of the
supervision modalities?
Methods
The study participants were nine graduate students (eight females). There were two African-American and seven White participants. The mean age of the participants was 29 years (SD = 6). The students were enrolled in a 12-week pilot practicum (i.e. a work placement) during summer 2008 for either one (n = 6) or three (n = 3) hours of course credit. Students were graduate masters (n = 2) and doctoral students (n = 7) in a counselling psychology programme at a large southeastern university. The students completed the practicum in a variety of field settings, including a university counselling centre, a summer programme for at-risk youths and a community mental health centre. The supervisor for the practicum was a Latino male and an adjunct faculty member. He was a licensed psychologist with 10 years of experience as a supervisor.
Measures
There were three main measures: Supervisory Satisfaction Questionnaire (SSQ). The SSQ is an 8-item scale,
using a four-point scale (1 = low to 4 = high) to measure satisfaction with
various components of supervision.
15
Ladany et al. reported that the SSQ had
high internal consistency (Cronbach's alpha = 0.96) and found that
satisfaction was positively related to supervisee disclosure in the
supervision process.
15
For the present study, the coefficients for the first and last
administrations were 0.97 and 0.94, respectively; Supervisory Working Alliance Inventory – Trainee Version (SWAI-T). The
SWAI-T is a 19-item measure of the supervisees' perspectives of the
supervisory relationship.
16
The measure consists of two subscales: Rapport (e.g. ‘I feel
comfortable with my supervisor’) and Client Focus (e.g. ‘My supervisor helps
me work within a specific treatment plan with my clients’). Each subscale is
rated on a 7-point Likert-type scale (1 = almost never to 7 = almost
always). Confirmatory and discriminatory evidence was also presented in the
Efstation et al. study, finding significant
correlations with the Supervisory Styles Inventory and lower correlations
with a supervisor version of the same measure.
16
The internal consistency coefficients for the Rapport and Client
Focus scales were 0.90 and 0.77, respectively. Patton and Kivlighan
17
found the scales to correlate highly, and therefore used the scales
as a composite in their study. As in their study, we combined both scales.
The coefficients for the present sample were 0.97 (first administration) and
0.99 (last administration); Counselling Self-Estimate Inventory (COSE). The COSE is a 37-item scale
using a 6-point Likert scale (1 = strongly disagree to 6 = strongly agree).
18
The COSE is designed to measure a counsellor's self-efficacy across
five counselling areas: Microskills, Process, Dealing with Difficult Client
Behaviours, Cultural Competence and Awareness of One's Values. Larson
et al. reported that the five factors
accounted for 36% of the variance and demonstrated moderate correlations
with a measure of self-concept (convergent validity) and weak correlations
with measures of personality and social desirability (discriminant validity).
18
Internal consistency estimates in the same study ranged from 0.88
(Microskills) to 0.62 (Values) and demonstrated an overall coefficient of
consistency of 0.93. Based on the recommendations of the scale's author,
only the composite score was used in the present study. The coefficient for
the total scale scores for the first and last administrations of the COSE in
the present study were 0.93 and 0.98, respectively.
Procedure
The students enrolled in the pilot practicum met over a 12-week period with the same faculty supervisor. The course rotated every three weeks between using in-person supervision and distance supervision via videoconferencing. We decided to start with the in-person format to facilitate the supervisory alliance. Trainees enrolled for one hour of course credit were required to attend one supervisory session per rotation for a total of four supervisory sessions. The students enrolled for three hours of course credit were required to attend all 12 classes.
In both the in-person and videoconference formats, class time comprised a didactic component, case presentations by the students and discussion/observation of their work with clients. Trainees videotaped their work (i.e. psychotherapy sessions) in their respective practicum sites. The recorded video was brought to class and observed over the videoconferencing network, enabling all participants to see and discuss the recorded video simultaneously. The majority of class time was focused on case presentations and discussion of the trainees' work with clients. All supervisory meetings were in a group format and lasted 2.5 hours.
Videoconferencing was conducted at a bandwidth of 384 kbit/s using the Kentucky Tele-Linking Network. The supervisor was in a distance learning technology classroom that had two 69-cm monitors side-by-side, one monitor to view students and the other for self-view of the supervisor. The students were in a distance technology classroom at a local community and technical college elsewhere in the same city as the supervisor. The distance technology room for the students had two 132-cm monitors side-by-side (one to view the supervisor and one that allowed trainees to view themselves) with push-to-talk microphones. The student room had a camera that automatically tracked the student who activated a microphone.
During the last rotation (videoconferencing format), six of the trainees (3 one-hour and 3 three-hour enrollees) agreed to participate in a supervisory session using PC-based conferencing (Skype). The session was conducted via the group chat format, where only audio was available. Trainees participated on separate computers. The measures provided did not evaluate this session, only the in-person and videoconferencing formats. However, students were asked about this experience qualitatively.
All students completed a package of measures at the end of each rotation (or the class that was attended for each rotation for the one hour enrollees). Each trainee completed a measure evaluating their satisfaction with supervision (SSQ) and the supervisory relationship (SWAI-T). The student's self-efficacy was also tracked across the semester (COSE). Data from the initial rotation (in-person format) was not included because the trainees enrolled for one hour only had to attend the initial class. Most of the first class was an introduction and overview of the course, with little actual clinical supervision being provided. Therefore, measuring satisfaction and the supervisory alliance was not appropriate. At the end of the semester both the supervisor and student completed a structured qualitative interview as used in the Gammon et al. study that evaluated supervision via videoconferencing. 14
Results
Satisfaction with supervision
Trainees rated their satisfaction with videoconferencing similarly to the in-person format. When comparing the two videoconferencing rotations to the in-person formats, the means were almost identical (see Table 1). There were no differences between the in-person format and the first distance learning rotation, t = 1.9, P = 0.10 (95% CI−1.2 to 0.1) or second distance learning rotation, t = 0.9, P = 0.40 (95% CI−0.7 to 1.6). The confidence intervals for both analyses lend support for the null hypothesis. The trainees reported a high level of satisfaction with supervision in both formats.
Mean values for the COSE, SSQ and SWAI-T by supervision format
*COSE, Counseling Self-Estimate Inventory; SSQ, Supervisory Satisfaction; SWAI-T, Supervisory Working Alliance Inventory – Trainee
Supervisory relationship
The supervisory relationship also did not appear to be affected by the videoconferencing format (see Table 1). There were no significant differences between the in-person format and either the first videoconferencing rotation, t = 0.8, P = 0.46 (95% CI−8.0 to 4.0) or the second videoconferencing rotation, t = −0.2, P = 0.86 (95% CI−9.0 to 7.7). The composite scores indicate that trainees felt that they had a good supervisory relationship with their supervisor and were consistent with, if not a little higher than other studies assessing the supervisory relationship. 19
Counsellor self-efficacy
Previous research has demonstrated that counsellor self-efficacy tends to increase with clinical supervision. 20 The same trend was observed in the present study (see Table 1) where utilizing the videoconferencing format did not seem to reduce counsellor self-efficacy. The counselling self-efficacy scores for both beginning trainees (four students with no previous practicum experience) versus more advanced trainees (five students with at least two practicums completed) indicated that both groups increased in counsellor self-efficacy by the end of the semester (see Figure 1).
Counsellor self-efficacy scores for in-person and videoconferencing formats. IP, in-person format; VC, videoconferencing format
Perceptions of supervision
Counsellor trainees have different supervisory needs depending upon their level of development. 21 The more advanced students rated their supervision experience a little higher than the beginning trainees (see Figures 2 and 3). However, there were no significant differences when comparing the beginning and advanced trainees' satisfaction, t = −0.6, P = 0.60 (95% CI−6.9 to 4.3) and the supervisory relationship, t = −0.9, P = 0.39 (95% CI−47 to 21) for the second distance learning rotation. These confidence intervals were larger due to more variability in the scores of the beginning trainee group.
Supervision satisfaction scores for in-person and videoconferencing formats. IP, in-person format; VC, videoconferencing format
Supervisor relationship scores for in-person and videoconferencing formats. IP, in-person format; VC, videoconferencing format
Qualitative data
The follow-up structured interview was consistent with the satisfaction and supervisory relationship measures and provides additional evidence that trainees found videoconferencing to be a viable alternative to in-person supervision. A summary of the themes expressed based on the content areas of the interview is presented below:
(1) Recommendation to someone who does not want to travel to receive in-person supervision. Trainees unanimously recommended that videoconferencing was a viable format for supervision and would be adequate for meeting supervisory needs. Examples of comments included: ‘I would say that videoconferencing supervision could be adequate and effective as long as the technology works correctly’ and ‘I received the same amount of support and guidance in both formats.’ The feedback from the supervisor reiterated that videoconferencing supervision was useful, providing the videoconferencing equipment was reliable.
(2) Differences between in-person and videoconferencing formats. Trainees felt that the videoconferencing format was more structured and more rigid. Several commented that there was more emphasis on staying on task and being clear verbally. However, trainees generally thought this was acceptable and a few believed that supervision was more efficient and goal-directed because of the format. This is consistent with the results of the Gammon et al. study that found some of the limitations of videoconferencing supervision to have paradoxical benefit. 14 Examples of comments included: ‘I think more time in videoconferencing went toward more intellectual and less personal content; taking care of business so to speak’ and ‘…it did not make me feel uncomfortable that we got right down to the supervision and neglected to chit-chat. I felt we actually accomplished more during the videoconferencing supervision.’ Trainees stated that the videoconferencing condition was generally less personal, but did not believe that quality or the goals of supervision were compromised. The interview with the supervisor revealed his belief that this format promoted more of a reporting type of interaction rather than discussion.
(3) Supervisory relationship. In general, trainees said that they felt comfortable with their supervisor via videoconferencing but also commented that it might be important to have an established relationship in person before engaging in a videoconferencing supervisory relationship. Most trainees, although stating that videoconferencing led to supervision that met their needs, were not comfortable with having a supervisory relationship that was strictly in the videoconferencing format. It was the supervisor's opinion that some of the emotional elements were lost in videoconferencing supervision, but he attributed this difference to technology limitations. He listed decreased self-disclosure, less emotional expression and subtleties that made role-play less effective as negatively affecting emotional content and intimacy. He added that the loss of emotion could have been a combined effect of the group supervision format and videoconferencing, both of which could make displays of vulnerability in supervision more intimidating.
(4) Videoconferencing concerns and recommendations. Most of the positive comments were tempered with concerns about the technology, saying videoconferencing was viable and useful ‘if the technology is reliable.’ Some of the students expressed mild frustration with the quality of the picture and having to repeat things due to transmission delays. Technology concerns listed by the supervisor included an echo on the audio; occasional freezing of the picture; loss of subtle nonverbal cues due to poor picture quality; and excessive delays in making the video link connection.
One suggestion was that videoconferencing should only be used for more advanced trainees, and that videoconferencing may not be appropriate as the sole form of supervision for beginning trainees. Other concerns/suggestions centred on keeping the group small and other behavioural setting matters (e.g. having a circular table to sit around), having the format structured to facilitate trainees not talking over one another, and making sure the trainees and supervisor were familiar and comfortable with the technology. The audio group chat format (i.e. Skype) was also well-received and participants believed that it was a viable alternative and that the convenience and low cost were the most attractive features. However, they felt this format would only be acceptable as an adjunct to forms of supervision that have visual images.
Discussion
Rural areas are typically underserved for mental health services. The reasons include the increased likelihood of living in poverty and the paucity of services available. This puts enormous pressure on those who do provide services in remote, rural areas. They often have little support or supervision. New professionals not only face these concerns, but also the need for supervision for licensure. Telehealth offers the possibility of addressing such matters and perhaps assisting with attracting new professionals and retaining existing mental health professionals.
Our exploratory study investigated a pilot clinical supervision course that alternated between an in-person and videoconferencing format over a 12-week period. Although the sample size was small, the trainees clearly felt that supervision via videoconferencing was a viable alternative to in-person supervision. Satisfaction with supervision and the supervisory relationship were similar in both formats. Evidence was further strengthened by qualitative data that trainees felt they were able to have their supervisory needs met in the videoconferencing format. This is consistent with other studies comparing the same formats. One difference, however, is that the present study was conducted using a group supervision format. Many students in the present study indicated that they would have liked to have experienced individual supervision using videoconferencing.
Although the feedback was favourable for videoconferencing, trainees did not feel that it should be the only format, especially for trainees with less experience. Trainees expressed having less intimacy with their supervisor, but this was not captured by a measure of the supervisory relationship nor was it clarified how less intimacy affected their supervisory experience. Less intimacy may have led to a reluctance of trainees to disclose perceived concerns or weaknesses regarding performance or to share other information that may have increased vulnerability. Openness to feedback and a willingness to discuss perceived weaknesses in supervision are typically valued trainee attributes in the supervisory process. In addition to experience, these attributes should be considered for supervision via videoconferencing. Students who commented on liking the formality of videoconferencing may have been those who were more avoidant of intimacy and disclosure. Videoconferencing was considered a viable but not equal substitute to in-person supervision. Some of the trainees indicated that they had to work harder in the videoconferencing format, and perhaps felt that meeting in-person simply required less energy to communicate.
The present study had limitations because of the small sample size and there being only one supervisor. In addition, it was not ideal that some students only attended four sessions (those enrolled for one hour of course credit). However, their data did not differ from the students who attended every session. Another limitation of the present study that may have influenced the results was the problems that occurred with the technology. The equipment used was obsolescent and if picture and sound quality had been better, trainees might have perceived even fewer differences. Larger studies need to be conducted that have more statistical power and allow for random assignment to one condition or the other. A confounding factor in the study was that trainees were able to establish a relationship in person with the supervisor and also knew that they would see him in person. Therefore, it is hard to generalize how trainees or new professionals would feel about having a relationship that was predominately or entirely conducted via videoconferencing.
In this study there were several logistical glitches that could have affected the results. Because it was not possible to use the distance education room when not using the distance education equipment, a different location had to be used for the in-person meetings. Therefore, the results may have been influenced by the suitability of each location for supervision. This shifting back and forth of rooms also led to at least one absence due to confusion about location. On two occasions, the distance education link was delayed significantly (once for over 30 minutes). This delay sometimes led to the supervisor being preoccupied during the actual supervision which may have reduced the quality of supervision. The delays may also have interfered with building the supervisory relationship. The quantitative results for both aspects in this case, however, did not appear to be affected.
On one occasion a student made a case presentation over the video link using an accompanying PowerPoint presentation. During the presentation there was a malfunction and her slides stayed frozen at the receiving end, despite no apparent problems at the transmitting end. She therefore described 10 slides to the audience without knowing that they were still viewing an earlier slide. Other problems that occurred were temporary freezing of the picture at the supervisor's end and a sound delay that contributed to people talking over each other.
In the Skype trial, the sound quality was good but the connection continually dropped out every 5–7 minutes. Although these technical problems caused little confusion, they could have had deleterious effects on the willingness of students to self-disclose or talk about more sensitive matters. For more rigorous investigation into telesupervision to take place, these types of problems need to be solved. Additional variables should also be measured. One way to look at quality of supervision would be to gather outcome data on the trainees' work with their clients. Outside raters who are blind to the experimental conditions could also be employed. Lastly, focus should be placed on measuring the quality of discussion in the supervision, examining variables such as trainee self-disclosure, risk-taking, and development of the students' ability to present meaningful case material. Such studies are difficult to conduct, given the logistics and requirements. Further research also needs to be conducted on the question of trainee experience. In the present study, there appeared to be some evidence that trainees with less experience were less satisfied with the videoconferencing format.
It is hoped that our study will stimulate interest in larger, random assignment studies to investigate the potential advantages that technology can offer to rural practitioners. Technological advances offer many exciting possibilities and have made a ‘virtual supervisory’ relationship more plausible. Providing better access to supervision and professional support using technology is one step towards improving health care in rural areas.