Implementation of a conversational,videoconferencing-based therapy group for postpartum depression and anxiety symptoms: A pragmatic evaluation

Abstract

Background

Group psychotherapy is an effective treatment for postpartum depressive and anxiety symptoms, and interpersonal connection and support through the group process can aid recovery. Little is known about the implication of the delivery of interpersonally oriented group therapy in this population through videoconferencing.

Objective

To pragmatically evaluate the implementation of a conversationally-oriented postpartum videoconferencing psychotherapy group for depression and anxiety within the clinical setting.

Methods

Over 8 weeks, five to six patients and one therapist facilitator (closed group) meet weekly for 1 hour via a secure videoconferencing platform. We evaluated group adoption metrics for all postpartum videoconferencing psychotherapy groups offered during the evaluation period (October 2021–August 2022), and offered patients the opportunity to complete baseline and post-group quality improvement surveys to evaluate outcomes including acceptability (Satisfaction with Therapist and Therapy Scale-Revised, STTS-R), group process (Group Questionnaire, GQ), and effectiveness (Edinburgh Postnatal Depression Scale, EPDS).

Results

Of 153 patients (n = 26 groups), most (72.5%) attended >70% of group sessions. Of 137 patients (n = 24 groups) who were sent surveys, n = 50 (36.5%) completed both baseline and post-group surveys. Mean (SD) ratings were high for acceptability (STTS-R-therapy: 25.0/30 (3.1); STTS-R-therapist: 27.6/30 (2.3)) and group process with GQ ratings of 81.4/91 (7.8) (positive bond), 34.1/56 (3.8) (positive working relationship) and 23.5/63 (4.4) (negative relationship). Patients with probable depression (EPDS ≥ 13) significantly decreased from n = 23 (50%) to n = 19 (41.3%, p < .001), although the absolute score difference was minimal.

Discussion

Videoconferencing-based group therapy can be implemented with a robust group process and acceptability in the postpartum period. Impact on clinical outcomes should be further investigated.

Keywords

eHealth telehealth women's health anxiety depression online group therapy postpartum

Introduction

Symptoms of postpartum depression and anxiety affect 15–20% of new mothers and birthing parents.^1,2 Effective treatments exist, yet as few as one in 10 of those in need of treatment are able to access them,³ leaving mother and child vulnerable to the effects of untreated illness. Psychotherapy, with or without medication, is a first-line treatment option for depression in the postpartum period and can be effectively delivered individually or in a group setting.^4,5 Participants in group therapy benefit from opportunities to engage in shared learning,^6–8 modelling others’ behaviours,⁷ normalizing experiences,⁸ and improving social support.⁶ In the postpartum context, an often isolated and lonely period, social and peer support from group therapy may have compounded benefits.^9–12 Group therapy also offers health system advantages, including cost effectiveness^6,7 and resource efficiency.^13,14 Feasibility of in-person group therapy is limited by low attendance,^8,15 with high attrition among new mothers attributed to barriers such as childcare,^16–18 transportation,^17,19 and scheduling constraints.^16,18

Due in part to pandemic-related restrictions on in-person services, there has been a growing use of group therapy delivered via videoconferencing. The available literature on videoconferencing therapy groups shows that they trend towards improved clinical outcomes, both in the postpartum period^20–23 and in others,^24–27 and that there are acceptable levels of satisfaction and feasibility^24,25,28,29 in varied populations. We also observe that there are knowledge gaps related to group process outcomes for therapy delivered through this modality. In the general population, the evidence on the positive and negative impacts of the videoconferencing modality on group processes is highly varied^25,26,29 and to our knowledge this has not been specifically examined in the postpartum context. There is some evidence around the value of interpersonal support from peers in a videoconferencing context.²² Given the importance of social support in the postpartum period, generating knowledge of how conversational and socially oriented group therapy programs can be delivered through videoconferencing may help us understand how they can be leveraged to address gaps in care in this population. Finally, most literature is focused on the use of videoconference-based group therapy as a scientific exercise or of its delivery as an immediate response to the pandemic; literature on the integration of such programs within a clinical care program beyond the immediate needs of the pandemic is lacking.²³ Exploring implementation outcomes related to postpartum group therapy delivered by videoconferencing can help us understand how and whether videoconferencing psychotherapy groups can be integrated into standard mental health care service delivery, and to inform future work into their optimization.

Our large, urban tertiary academic hospital has a strong vision and mandate to develop, implement and study virtual solutions to support the equitable delivery of high-quality, publicly funded healthcare services. We securely embedded Zoom^® into the hospital's electronic medical record and patient-facing portal as part of an institutional-wide initiative to offer synchronous, videoconferencing-based group health interventions (‘videoconferencing groups’) to hospital patients during the pandemic and beyond; infrastructure sustainability was a core component of these efforts.³⁰ This enables secure access for patients and providers as well as complete clinical integration for documentation, care planning, and follow-up. A specialized reproductive mental health program within the Department of Psychiatry at the hospital continues to deliver interpersonally oriented group psychotherapy for postpartum depression and anxiety through this modality as standard of care due to patient request, even as restrictions to in-person therapy have lifted. As a major focus of this group when it was offered in person was the interpersonal (i.e. social) experience, ensuring that this aspect of the group (i.e. the ability to engage in conversation and socialization within the group setting) can be implemented in the transition to a virtual platform is paramount.

The goal of the current study was to pragmatically evaluate the implementation of the postpartum videoconferencing psychotherapy group within the clinical setting.

Methods

Study design

The study design was informed by the Virtual Group Evaluation Framework³⁰ (VGEF), which spans implementation, quality and effectiveness domains; it was developed and used by our institution to guide the evaluation of videoconferencing groups. We primarily focused this evaluation on the implementation (i.e. its adoption, acceptability, appropriateness, and feasibility) of the videoconferencing group as it was most relevant to our research question and pragmatic in terms of data collection. Additionally, we looked at other outcomes related to group dynamics and development (‘group process’), given that the interaction between group members was a core element of the in-person group, as well as the group's efficiency, effectiveness, safety, and overall experience from the patient perspective. This study was part of a pragmatic, real-world quality improvement evaluation.

Setting and intervention

The psychotherapy group's content and approach loosely follow an interpersonal therapy (IPT) model, but it is not strictly an IPT group.³¹ The interpersonally oriented basis of the group is the idea that improving relationships with others can buffer the effect of major stressors on a person's psychological health.^32,33 Therapy focuses on relationship management in four areas: conflict in relationships, life changes, grief and loss, and difficulties in starting and maintaining relationships.³³ The interpersonally oriented approach is particularly appropriate for the postpartum period where people undergo significant life changes, manage new expectations of relationships, and when social isolation is a major risk factor for mental illness. During the first group session, the facilitator and patients collaboratively develop a list of specific topics of interest that are discussed over the remaining sessions. Over the course of the group, in addition to elements of traditional IPT (i.e. education and discussion on losses and gains, and the impact of transition and change), the facilitator and patients practice guided mindfulness and discuss the mutually agreed-upon topics relevant to the role transition to parenthood. Prior to this evaluation, patients often anecdotally indicated that they found the in-person group to provide validation and social connection at a time when they struggled with isolation due to their new baby and their mental health challenges. There were three primary individuals who facilitated the groups, each of whom was either a trained social worker therapist or a registered psychotherapist, and all of whom had a specialist background in perinatal mental health. One group was facilitated by a trainee under the supervision of one of the three primary facilitators.

Originally developed as an in-person psychotherapy group, it has been offered virtually since 2020 (originally due to pandemic-related restrictions) as standard of care. Our institution integrated the hospital's electronic medical record (Epic) with Zoom to facilitate ease and security of clinical care planning, documentation, and follow-up.³⁰ From the patient's perspective, it is accessed through the patient portal, which is accessible via desktop computers and laptops, and is mobile-friendly. Training materials on the use of the patient portal to access and participate in the videoconferencing group were developed by the hospital and can be accessed online. Specific changes were made to the group's structure when it was adapted from an in-person to a videoconferencing format. The duration of the group was reduced from 10 weeks to 8 weeks for practical reasons related to waitlist management during the pandemic and ensuring timely access to care. The composition of the group was reduced from 10–12 patients and two highly trained psychotherapists to 5–6 patients and one highly trained psychotherapist (either a social worker therapist or a registered psychologist) in the videoconferencing format as it was thought that a smaller group size would allow for greater social engagement among patients since this was a central aspect of the group.

Participants

Upon referral to the reproductive mental health program, patients are assessed by a perinatal psychiatrist. Together, the patient and perinatal psychiatrist collaboratively form a treatment plan that may include psychotherapy and/or medication. Group psychotherapy is the main therapeutic modality offered to patients 0–12 months postpartum with depressive or anxious symptoms. Postpartum people (anyone who delivered a baby within the past 1 year or who self-identifies as a mother of an infant <1 year of age, inclusive of cisgender and transgender individuals) with symptoms of depression and anxiety (based on the psychiatrist's assessment) are eligible to participate in the group. Patients would be excluded from the group and offered individual psychotherapy only if deemed clinically appropriate for the patient (e.g. due to diagnosis, illness severity). Referral to the postpartum videoconferencing psychotherapy group does not require a diagnosis of a major depressive or anxiety disorder, nor a specific symptom threshold. All patients of the clinical program who participated in the videoconferencing psychotherapy group as part of their clinical treatment plan were included in the quality improvement evaluation.

Data sources, collection and storage

Data on group adoption metrics (i.e. usage) was extracted from the electronic medical record for all patients registered in the postpartum videoconferencing psychotherapy group during the evaluation period (October 2021–August 2022). All patients in the postpartum videoconferencing psychotherapy groups during this period were asked to optionally complete a baseline survey before the group, and again at group completion to evaluate the self-reported outcomes of the videoconferencing group (Table 1). Sociodemographic variables were collected via self-report at baseline, and group facilitators completed a treatment log to describe the primary topics of discussion, education provided, and/or skills taught for each group session.

Table 1.

Overview of patient-reported experience measures and outcome measures.

Study variable	Measure	Purpose^a	Survey data
Study variable	Measure	Purpose^a	Baseline	Post-group
Acceptability	STTS-R, therapist subscale	I	-	X
Acceptability	STTS-R, therapy subscale	I	-	X
Appropriateness	STTS-R, outcome improvement	I	-	X
Feasibility – Technological	eHEALS	I	X	-
	Device availability, comfort with technology, training intention	I	X	-
	Device(s) used, technical issues	I	-	X
Feasibility – Practical	Confidence in participation, anticipatory concerns	I	X	-
Feasibility – Practical	Issues experienced	I	-	X
Safety	Issues with privacy, confidentiality or safety	Q	-	X
Efficiency	Time and money impact	Q	-	X
Effectiveness	Clinical outcomes: EPDS, GAD-7, SPS-10, NGSE	Q	X	X
Group process	GQ	GD&E	-	X
Experience	Net promotor score	O&I	-	X
Experience	Overall experience rating	O&I	-	X
Other	Sociodemographics	C	X	-
Other	Future preferences	C	-	X

Overview of patient-reported experience measures and outcome measures, described in terms of the study variables, the measures used for their assessment, the purpose of each measure within the Video Group Evaluation Framework (VGEF), and data collection timepoint.

^aLegend: I = implementation, Q = quality, GD&E = group dynamics and environment, O&I = outcomes and impact, C = covariate.

Surveys were digitally administered, and data were collected and managed using Research Electronic Data Capture (REDCap) hosted at our institution.^34,35 REDCap is a secure, web-based software platform designed to support data capture for research studies, providing an intuitive interface for validated data capture, audit trails for tracking data manipulation and export procedures, automated export procedures for seamless data downloads to common statistical packages, and procedures for data integration and interoperability with external sources.

Outcomes

Implementation outcomes

The primary objective was to pragmatically evaluate the implementation of the interpersonally oriented postpartum videoconferencing psychotherapy group in the clinical setting using measures of implementation outcomes^36,37 relevant to this work as per the VGEF.³⁰

Adoption (i.e. uptake of the videoconferencing group): Data on service use was extracted from hospital medical records for all groups provided during the evaluation period. We defined active participants as those who attended ≥70% of group sessions (i.e. at least 6/8 or 5/7 sessions, depending on group length), and drop-outs as not attending the last two sessions of the group series.

Acceptability and appropriateness: Acceptability (i.e. perception that the treatment and modality were satisfactory) and appropriateness (i.e. perceived fit of the videoconferencing group to address symptoms of postpartum depression and anxiety) were measured in the post-group survey by administering the Satisfaction with Therapist and Therapy Scale-Revised (STTS-R).³⁸ The STTS-R is used to evaluate satisfaction with a group therapy intervention and has been used for both in-person and online groups.³⁹ It is comprised of two, six-item subscales (satisfaction with therapy, and satisfaction with therapist), and one additional item on perceived outcome improvement (‘How much did this treatment help with the specific problem that led you to therapy?’).³⁸ Each item in the STTS-R is scored on a 5-point Likert scale, with higher scores indicating greater levels of satisfaction. Subscale items are summed to a maximum of 30 per subscale and the perceived outcome improvement question is analysed independently. A score of ≥21 has been used in the literature to indicate a reasonable level of satisfaction on each subscale.⁴⁰

Technical and practical feasibility (i.e. the extent to which the videoconferencing group can be successfully used): Technical feasibility was measured in the pre-group survey by administering the eHealth Literacy Scale (eHEALS)^41,42 to assess readiness for successful engagement with health information technology. Each of the eight items on the eHEALS is scored on a Likert scale (from 1, strongly disagree to 5, strongly agree) and summed; higher scores (to maximum of 40) indicate higher levels of literacy. In the literature, the mean eHEALS score reported in adult populations ranges from 28 to 30.⁴³ Additionally, at baseline, we asked about the availability of technical devices in the home, comfort with technology in day-to-day life on a Likert scale (from 1, excellent to 5, poor), and plans to look at training materials. In the post-group survey, we ascertained which technical devices were used to access the group, whether there were technical issues that made participation difficult, and if they needed to contact technical support at the hospital to participate. Practical feasibility was measured at baseline by asking about confidence in participation on a 5-point Likert scale and anticipatory concerns related to participation. In the post-group survey, we asked if any practical issues were experienced using a binary question and provided patients the opportunity to expand on their experience in a free-text response.

Other outcomes

Group process: The group dynamic in a videoconferencing context was measured by looking at the group process using the Group Questionnaire (GQ)⁴⁴ in the post-group survey. Modifications were made so the questionnaire reflected the group overall, not singular sessions. Each of the 30 items on the GQ is scored on a 7-point Likert scale.⁴⁵ Scores are summed for each of three subscales on relational constructs: positive bonding relationship (out of 91), positive working relationship (out of 56), and negative relationship (out of 63). Each subscale construct is related to four key elements of group functioning (cohesion, alliance, empathy, and group climate) and are assessed from the perspective of members to each other, to the group, and to the facilitator.⁴⁴ We also asked whether patients felt it was easy to interact with the facilitator and other group members online, each on a 5-point Likert scale.

Efficiency: Efficiency was measured by asking patients in the post-group survey about whether their participation saved them time and/or money in comparison to an in-person group on a 5-point Likert scale and for their free-text perspectives on group benefits and drawbacks. Patients were asked about the comfort and ease of use of the videoconferencing group platform and the video group itself, and its impact on access to care, using Likert scale-based responses.

Effectiveness (i.e. clinical outcomes): To ascertain the effectiveness of the postpartum videoconferencing psychotherapy group, patients completed the Edinburgh Postnatal Depression Scale (EPDS),⁴⁶ seven-item screening tool for Generalized Anxiety Disorder (GAD-7),⁴⁷ Social Provisions Scale, Short Version (SPS-10),⁴⁸ and New General Self-Efficacy Scale (NGSE).⁴⁹ Each of the 10 items on the EPDS is scored 0–3; sum scores ≥13 indicate probable depression. Each item on the GAD-7 is scored on a 4-point Likert scale; a sum score >10 indicates probable moderate anxiety. The SPS-10 is scored on a 4-point Likert scale; a sum score of ≥30 indicates high levels of support. Each of the eight items on the NGSE is scored on a 5-point Likert scale and an average overall score is calculated.

Safety: It was assessed by asking patients a binary question in the post-group survey about whether they experienced anything that made them worried for their privacy, confidentiality or safety during the group along with the opportunity to elaborate. They were also asked about their general comfort with participation on a Likert scale.

Experience: Overall experience was measured by calculating a Net Promotor Score (NPS), which is used to provide an overall benchmark of performance within a comprehensive feedback process.⁵⁰ To calculate the videoconferencing group's NPS, each patient rated their willingness to recommend the group on a scale of 1–10, and each rating was categorized into detractors (rating 0–6), passives (rating 7–8), and promotors (rating 9–10). The NPS is calculated by determining the difference between the percentage of promotors and the percentage of detractors, for a possible range from −100 to +100. Additionally, patients were asked to rate their experience with the technology and with the videoconferencing group overall, each on a scale of 1–10.

Future preferences: Future preferences regarding videoconferencing groups were collected in the post-group survey.

Data analysis

Descriptive analyses were conducted for each measure. Standardized scales (STTS-R, eHEALS, GQ, EPDS, GAD-7, SPS-10, NGSE, and NPS) were scored as described above. Pre- and post-group scores for the EPDS, GAD-7, SPS-10, and NGSE were compared using a paired samples t-test. For each of these four scales, each patient's score was also dichotomized based on its cut-off score, and differences between the baseline and post-group scores were compared using a Pearson chi-square test of association. Quantitative analyses were conducted using SPSS 29.0. Free-text responses and content from treatment logs were thematically coded⁵¹ by NHS in NVIVO 12.0.

Ethical considerations

This initiative was reviewed and approved under the Assessment Process for Quality Improvement Projects program (File #2021-0059-E). As the postpartum videoconferencing psychotherapy group is a standard-of-care clinical program and this was a quality improvement evaluation, patients were not required to consent to the collection of EMR data (which was anonymized) to participate in the group, nor did we withhold group participation from anyone who declined to complete the surveys, which were optional. Clinical care providers were blinded to whether patients had completed the surveys. Before completing their surveys, patients reviewed an informed consent page, which outlined that completing the survey signified their implied consent to self-reported data collection. There were no benefits to patients for completing the surveys.

Results

Data on group attendance was available for all 153 patients (26 groups) during the study period; all but two groups were 8 weeks long (Figure 1). Of 137 patients (24 groups) who were sent e-surveys, 50 patients (36.5%) completed both the pre- and post-group surveys and are included in the analysis. Surveys were not sent to patients (n = 12) in two groups due to facilitator workload capacity. Sociodemographic data for patients included in the analysis are outlined in Table 2. Facilitators completed treatment logs for 24 out of 26 groups (two groups did not have treatment logs completed due to facilitator workload capacity). Details of topics covered are in Appendix 1.

Figure 1.

Patient flow.

Table 2.

Sociodemographics.

	n (%)
Age of infant in months, median (IQR)	6 (5–8)
Gender Identity
Woman	49 (98.0)
Self-described	1 (2.0)
Sexual orientation (n = 49)
Heterosexual	39 (79.6)
Bisexual	7 (14.3)
Self-described	2 (4.1)
Prefer not to say	1 (2.0)
Marital status
Married	39 (78.0)
Co-habitating	9 (18.0)
Single	2 (4.0)
Household income (CAD)
≤$59,999	4 (8.0)
$60,000–$89,999	5 (10.0)
$90,000–$119,999	6 (12.0)
$120,000–$149,999	13 (26.0)
≥$150,000	14 (28.0)
Do not know or prefer not to say	8 (16.0)
Number of people supported by household income (n = 43), mean (SD)	3.2 (0.7)
Trouble making ends meet at the end of the month (n = 49)
Always	1 (2.0)
Sometimes	3 (6.1)
Occasionally	5 (10.2)
Never	35 (71.4)
Do not know or prefer not to say	5 (10.2)
Current employment status
Employed (full time or part time)	6 (12.0)
On leave (e.g. parental, medical, etc.)	37 (74.0)
Unemployed or other	7 (14.0)
Ethnicity
White	26 (52.0)
Asian (e.g. East Asian, South Asian, Southeast Asian)	11 (22.0)
Mixed heritage	6 (12.0)
Latin American	2 (4.0)
Middle Eastern	1 (2.0)
Black	1 (2.0)
Other (did not specify further)	1 (2.0)
Do not know or prefer to not say	2 (4.0)
Self-rated ability to speak and understand English (n = 49)
Very well	46 (93.9)
Well	3 (6.1)
Not well	-
Not at all	-
Self-rated overall physical and mental health
Excellent	-
Very good	11 (22.0)
Good	24 (48.0)
Fair	11 (22.0)
Poor	4 (8.0)

Sociodemographic data of survey respondents as reported in the pre-group survey, reported as n (%), unless otherwise specified (N = 50).

Implementation outcomes

Adoption: There were n = 8 (5.2%) registered patients who did not attend any group sessions, and over one-third (35.5%, n = 54) had perfect attendance. Patients in groups that were 8 weeks in length (n = 24 groups, n = 142 patients), attended a median of 7 (interquartile range, IQR: 6–8) group sessions. Overall, 72.5% (n = 111) were active participants and only 13.7% (n = 21) dropped-out (Table 3).

Table 3.

Video group implementation outcomes.

	n (%)
Adoption
Attendance
Did not attend any sessions	8 (5.2)
Attended all sessions	54 (35.3)
Number of sessions attended (n = 142)^a, median (IQR)	7 (6–8)
Active participants^b	111 (72.5)
Drop out^c	21 (13.7)
Acceptability and Appropriateness
Satisfaction with Therapist and Therapy Scale-Revised (STTS-R)
STTS-R Therapy Subscale^d (n = 49)	25.0 (3.1)
STTS-R Therapist Subscale^d (n = 49)	27.6 (2.3)
Perceived outcome improvement
Made things a lot better	10 (20.0)
Made things somewhat better	37 (74.0)
Made no difference	2 (4.0)
Made things somewhat worse	1 (2.0)
Made things a lot worse	-
Feasibility: technical feasibility
eHEALS score (n = 48), mean (SD)	25.8 (4.1)
Access to technological devices in the home^e
Laptop computer	44 (88.0)
Smartphone	43 (86.0)
Tablet	25 (50.0)
Personal assistant device (e.g. Google Home, Alexa, etc.)	14 (28.0)
Smartwatch	11 (22.0)
Desktop computer	11 (22.0)
Landline telephone	3 (6.0)
Technological devices used to access the video group^e
Laptop	40 (80.0)
Smartphone	14 (28.0)
Desktop computer	6 (12.0)
Tablet	6 (12.0)
Comfort with technology in day-to-day life
Excellent	29 (58.0)
Very good	13 (26.0)
Good	7 (14.0)
Fair	1 (2.0)
Poor	-
Plan on looking at training material before the first session	13 (28.0)
Experienced technical issues that made it hard to participate (n = 49)	4 (8.2)
Feasibility: practical feasibility
Pre-group confidence in the ability to participate in videoconferencing group
Very confident	31 (62.0)
Somewhat confident	14 (28.0)
Neutral	2 (4.0)
Not very confident	3 (6.0)
Not at all confident	-
Anticipatory practical concerns with participation	11 (22.0)
Experienced practical barriers to participation	8 (16.0)

Indicators of implementation, as recorded in the hospital electronic medical record (N = 153) for adoption, and as self-reported by patients in the pre- and post-group surveys (N = 50), for other domains, as n (%) unless otherwise specified.

^aBased on 24 groups that were 8 weeks in length; two groups were 7 weeks long.

^bDefined as attending at least 6 of 8, or 5 of 7 group sessions, depending on group length.

^cDefined as not attending the final two sessions of the group.

^dEach of the six items in each subscale is scored on a Likert scale from 1 to 5 and summed to a maximum score of 30.

^eSelect all that apply format, answers do not add up to 100%.

Acceptability and appropriateness: The mean scores on the STTS-R Therapy Subscale and Therapist Subscale were 25.0 (SD: 3.1) and 27.6 (SD: 2.3) out of 30, respectively (Table 3). When asked about the perceived impact of participating in the video therapy group, most patients felt it made their problem at least somewhat (n = 37, 74.0%) better.

Technical and practical feasibility: The mean (SD) eHEALS score in our sample was 25.8 (4.1). Patients had access to a range of devices that would be appropriate for accessing the video group and most accessed it using a laptop (n = 40, 80.0%) and/or smartphone (n = 14, 28.0%) (Table 3). Few patients planned to look at the video group training materials (n = 13, 28.0%). Technical issues that made it difficult to participate were uncommon and no patients contacted the hospital's technical support to participate. Before the start of the group, most felt confident in their ability to participate in a videoconferencing group (very confident: n = 31, 62.0%). Anticipatory concerns were reported by 22.0% (n = 11) and mostly related to practical issues (e.g. childcare, technology-related concerns, and general concerns about interacting with others online). Issues that were experienced were similar (e.g. Internet connection, problems logging in to portal, childcare, having a private space) but experienced by fewer in proportion (n = 8, 16.0%).

Other outcomes

Group process: Patients reported a mean positive bond score of 81.4/91 (SD: 7.8), a mean Positive Working Relationship score of 34.1/56 (SD: 3.8) and a mean negative relationship score of 23.5/63 (SD: 4.4) (Table 4). Most felt it was easy to interact online with other group members (agree: n = 20, 40.0%) and the facilitator (agree: n = 24, 48.0%).

Table 4.

Additional videoconferencing group outcome measures.

	n (%)
Group process
Group Questionnaire (GQ), mean (SD)
Positive bond (n = 47)	81.4 (7.8)
Positive work (n = 49)	34.1 (3.8)
Negative relationship (n = 49)	23.5 (4.4)
It was easy to interact with other group members online
Strongly agree	16 (32.0)
Agree	20 (40.0)
Neutral	12 (24.0)
Disagree	2 (4.0)
Strongly disagree	-
It was easy to interact with the group facilitator online
Strongly agree	18 (36.0)
Agree	24 (48.0)
Neutral	8 (16.0)
Disagree	-
Strongly disagree	-
Efficiency
Videoconferencing group saved time
Strongly agree	35 (70.0)
Agree	12 (24.0)
Undecided	-
Disagree	2 (4.0)
Strongly disagree	1 (2.0)
Videoconferencing group saved money
Strongly agree	32 (64.0)
Agree	10 (20.0)
Undecided	2 (4.0)
Disagree	4 (8.0)
Strongly disagree	2 (4.0)
Videoconferencing platform was easy to use
Strongly agree	25 (50.0)
Agree	24 (48.0)
Neutral	-
Disagree	1 (2.0)
Strongly disagree	-
I felt comfortable using the videoconferencing platform to participate in the group
Strongly agree	26 (52.0)
Agree	23 (46.0)
Neutral	1 (2.0)
Disagree	-
Strongly disagree	-
Having the videoconferencing platform improved my ability to access care
Strongly agree	24 (48.0)
Agree	21 (42.0)
Neutral	2 (4.0)
Disagree	3 (6.0)
Strongly disagree	-
Safety
Anticipatory concerns
Privacy concerns	2 (4.0)
Confidentiality concerns	2 (4.0)
Experienced issues related to privacy, confidentiality or safety	2 (4.0)
Comfort with participating in group
Very comfortable	27 (54.0)
Somewhat comfortable	23 (46.0)
Neutral	-
Not very comfortable	-
Not at all comfortable	-
Future Preferences
I would be happy to participate in a videoconferencing group again
Strongly agree	28 (56.0)
Agree	17 (34.0)
Undecided	3 (6.0)
Disagree	1 (2.0)
Strongly disagree	1 (2.0)
If the videoconferencing group was not available, I would have attended an in-person group
Strongly agree	8 (16.0)
Agree	18 (36.0)
Neutral	10 (20.0)
Disagree	12 (24.0)
Strongly disagree	2 (4.0)
If given a choice, I would choose a videoconferencing group over an in-person group
Strongly agree	18 (36.0)
Agree	9 (18.0)
Undecided	14 (28.0)
Disagree	7 (14.0)
Strongly disagree	2 (4.0)
I would like to have the option of a virtual group
Strongly agree	31 (62.0)
Agree	13 (26.0)
Undecided	4 (8.0)
Disagree	1 (2.0)
Strongly disagree	1 (2.0)

Additional outcome measures for the videoconferencing group, as self-reported by patients (N = 50), as n (%) unless otherwise specified.

Efficiency: Patients agreed that participating in the video group compared to an in-person group saved them time (strongly agree: n = 35, 70.0%) and money (strongly agree: n = 28, 56.0%) (Table 4). In free-text responses, patients reported that the benefits of the group included having the flexibility to participate from home, being able to maintain their infant's schedule, reduced exposure to communicable illness, not having to commute (e.g. costs of driving and parking, avoiding travel on public transit, avoiding weather-related concerns, etc.), ease of childcare (e.g. either self or a partner at home), and less pressure to appear ‘presentable’ to others. Drawbacks of participating in the videoconferencing group included difficulty reading and responding to body language, trouble with conversational flow, distractions at home, loss of incidental interactions with fellow group members, and the lack of excuse to leave the house. Nearly all patients reported that the videoconferencing platform was easy to use and that they felt comfortable using it to participate in the group; they agreed that having the videoconferencing platform improved their ability to access care (Table 4).

Effectiveness: Self-reported mental health symptoms improved for patients (Table 5). The mean EPDS score (n = 46) fell from 12.7 (SD: 4.1) to 11.2 (SD: 4.3) in the post-group survey (p = .004). The number of patients with probable depression (EPDS ≥13) significantly decreased from n = 23 (50%) to n = 19 (41.3%) patients (p < .001). The mean GAD-7 score (n = 49) fell from 9.0 (SD: 4.4) to 8.3 (SD: 4.6) in the post-group survey, a non-statistically significant difference (p = .223); the number of patients with probable anxiety (GAD-7 ≥ 10) significantly decreased from n = 22 (44.9%) to n = 15 (30.6%; p = 0.04). Patients (n = 48) reported high mean levels of social support on the SPS-10 in the pre-group survey (34.2, SD: 4.8) which significantly improved post-group (35.7, SD: 4.5; p = .002). The proportion of those reporting high levels of social support (SPS-10 ≥ 30) significantly increased from n = 38 (79.2%) to n = 42 (87.5%; p < .001). Similarly, self-efficacy increased from a mean NGSE score of 3.7 (SD: 0.7) to 3.9 (SD: 0.7; p < .001) and the number of patients reporting high levels of self-efficacy (NGSE ≥3.8) increased from n = 25 (50.0%) to n = 29 (58.0%; p < .001).

Table 5.

Effectiveness scales.

	Pre-group	Post-group	X ²	p
Edinburgh Postnatal Depression Scale (EPDS) (n = 46)
Raw score, mean (SD)	12.7 (4.1)	11.2 (4.3)		.004*
Probable depression (≥13), n (%)	23 (50.0)	19 (41.3)	10.9	<.001*
Seven-item Generalized Anxiety Disorder Scale (GAD-7) (n = 49)
Raw score, mean (SD)	9.0 (4.4)	8.3 (4.6)		.223
Probable anxiety (≥10), n (%)	22 (44.9)	15 (30.6)	4.1	.04*
Ten-item Social Provisions Scale (SPS-10) (n = 48)
Raw score, mean (SD)	34.2 (4.8)	35.7 (4.5)		.002*
High social support (≥30), n (%)	38 (79.2)	42 (87.5)	26.1	<.001*
New General Self-Efficacy Scale (NGSE) (n = 50)
Raw score, mean (SD)	3.7 (0.7)	3.9 (0.7)		<.001*
High self-efficacy (≥3.8), n (%)	25 (50.0)	29 (58.0)	24.6	<.001*

Self-reported outcome scales, comparing pre- and post-group scores (N = 50) using paired samples t-tests for raw scores and Pearson chi-square to compare cut-off scores. Statistical significance was defined as p < .05 and is indicated by an asterisk (*).

Safety: Few patients reported anticipatory safety concerns (n = 2, 4.0%) or actually experiencing any issues (n = 2, 4.0%) related to privacy, confidentiality, or safety during their group. All reported having felt at least somewhat comfortable participating in the videoconferencing group (Table 4).

Experience: Patients gave a mean NPS rating of 9.2 (SD: 1.2) out of 10; based on their NPS, most patients (n = 36, 72.0%) were classified as promotors. The overall NPS for the videoconferencing psychotherapy group was +70. The mean scores for the experience with the technology specifically and the videoconferencing group overall were, respectively, 8.7 (SD: 1.1) and 8.9 (SD:1.1), each out of 10.

Future preferences: Patients noted they would be happy to participate in a videoconferencing group again (agree and strongly agree: n = 45, 90.0%) yet leaned towards attending an in-person group had the virtual option not been available (agree and strongly agree: n = 26, 52.0%) (Table 4). However, if given the option of a videoconferencing or in-person group in the future, patients leaned towards choosing the videoconferencing group (agree and strongly agree: n = 27, 54.0%). Patients would like the option of participating in a videoconferencing group in the future (agree and strongly agree: n = 44, 88.0%).

Discussion

In our evaluation, we demonstrate that an interpersonally oriented therapy group for symptoms of postpartum depression and anxiety can be delivered via videoconferencing and implemented successfully, with positive and strong group processes. Overall our key findings include high levels of acceptability and appropriateness and a robust group process alongside a highly rated experience among those who participated in a psychotherapy group delivered by videoconferencing as standard of care within a clinical setting. This suggests that the interpersonal and social nature of group therapy in the postpartum period can be translated into and implemented in the virtual world.

As its primary objective, our study is unique in its examination of the implementation of an interpersonally oriented and conversationally based videoconferencing group therapy for symptoms of postpartum depression and anxiety within a clinical setting. Our drop-out rate of 13.7% was lower than that reported among videoconferencing groups elsewhere in the literature in studies on varied populations including veterans (18%²⁹), adults with anxiety disorders (15.3%⁵²), and perinatal groups (20%²⁰), although the definition of drop-out varied between studies. Studies that compared attendance to an in-person group found higher attendance in the videoconferencing format,^29,52 implying it may reduce barriers to care and support higher attendance. Many barriers to care often noted in the literature as contributing factors to treatment attrition in this population are, in part, addressed by this modality, noted by patients as benefits of the videoconferencing group. In our qualitative evaluation based on patient interviews (reported separately), patients note the tensions between finding childcare during the videoconferencing group or caregiving while participating and the added value of presenting oneself in a more authentic manner when participating online. Of critical importance, it was feasible for patients to access the group from a technical and practical standpoint and our group was highly acceptable to patients. This is particularly evident when noting that our scores on the STTS-R Therapy subscale were comparable to those found in a range of in-person studies.^53–56

There are significant challenges with the creation of strong group processes in a virtual context. The group evaluated in this study was conversational and social in nature, and our data demonstrated that a therapy group of this nature can be successfully implemented virtually. Encouragingly, our study found scores on the GQ that were comparable to those measured among in-person groups. A meta-analysis of in-person studies that used the GQ⁵⁷ determined aggregate subscale scores for positive bonding (mean: 72.91, SD: 12.37), positive working (mean: 38.11, SD: 10.52), and negative relationship (mean: 23.54, SD: 8.74), which are all comparable to our data. Based on our data, strong group processes were achieved across multiple groups with different facilitators, suggesting that the development of a strong group environment can be created through videoconferencing. Our qualitative study validates this and highlights the important role played by the facilitator in developing a sense of cohesion among group members in an online environment. The facilitator⁵⁸ and homogeneity among the group (related to either the presenting issue or identity-related factors)⁵⁹ are both associated with the cohesion of in-person groups. Inherent commonalities between group members (i.e. postpartum people with depressive and anxious symptoms) and the smaller group size (compared to the original in-person format) may have contributed to the highly rated group process and, in turn, may also explain the high levels of satisfaction observed.

Although our data shows improvement in mental health symptom scores, the small changes in absolute scores on the EPDS in our evaluation (a difference of 1.5 points from baseline to post-group) do not meet the 2-point minimum threshold for a clinically important difference.⁶⁰ This may be related to the fact that entry to the group is based on the presence of symptoms and not a symptom threshold nor formal diagnosis. Additionally, this evaluation did not focus on the impact of the group of clinical outcomes and was not powered to do so. Understanding the group's impact on clinical outcomes and well-being, particularly given that 94% of patients reported feeling that the group made their problem at least ‘somewhat better’, should be the focus of future work.

There are few studies to our knowledge that specifically examine the implementation of videoconferencing therapy groups that focus on interpersonal and social interactions within a virtual setting. Limitations of this study should be noted. There was a low response rate for completion of both surveys and we only included data from patients who completed both of the optional baseline and post-group surveys, representing approximately one-third of all of those who were registered in the group. This may have introduced bias in that feedback was not received from all of those who participated in the group and may therefore limit the generalizability of our results. With the lack of external incentives for quality improvement surveys, response and completion are a challenge; response rates to quality improvement studies (such as this one) of 50% are considered to be ‘quite high’.⁶¹ Our study achieved a response rate of only 36.5% for those who completed both the baseline and post-group surveys, although more than three-quarters of patients started at least one of the two quality improvement surveys. While we included several validated scales in our surveys, the use of questions that were not validated, and the lack of pilot testing the survey is an additional limitation. In addition, in the post-group survey we did not ask patients if they looked at the training materials, which may have better helped to understand their influence on outcomes, particularly related to the technical feasibility of the videoconferencing group. In line with the institutional approach to the development and implementation of videoconferencing groups as a treatment modality, the use of a systematic institutional-level approach to group evaluation, such as one centred on the principles of a Learning Health System,⁶² may be warranted to improve survey response rates, with the added benefit of relieving facilitator administrative burden related to sending out survey links to patients. Without a direct comparison to an in-person group, it is not possible to say with certainty how our outcomes compare to traditional in-person groups, and the homogeneity of our survey sample population may limit the generalizability of findings, particularly among marginalized groups as our sample is skewed towards a higher income group.

In conclusion, interpersonally oriented group therapy for symptoms of postpartum depression and anxiety, delivered through videoconferencing, can be implemented with robust group processes that maintain social interactivity and with general acceptability. Future work should consider an evaluation of efficacy, equity, and comparison of differing delivery modalities, to determine who might be best served from accessing care through various approaches to delivery. Based on our evidence, there is a case for maintaining its place in the ongoing delivery of standard care at our hospital. Group psychotherapy, delivered through videoconferencing can be a way through which to deliver care to postpartum people who may otherwise experience barriers to access important mental health treatment that is vital to their well-being.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241269630 - Supplemental material for Implementation of a conversational, videoconferencing-based therapy group for postpartum depression and anxiety symptoms: A pragmatic evaluation

Supplemental material, sj-docx-1-dhj-10.1177_20552076241269630 for Implementation of a conversational, videoconferencing-based therapy group for postpartum depression and anxiety symptoms: A pragmatic evaluation by Neesha Hussain-Shamsy, Lori Wasserman, Greer Slyfield Cook, Kaeli Macdonald, Keisha Greene, Lucy C. Barker, Juveria Zaheer, Geetha Mukerji, Simone N. Vigod and Emily Seto in DIGITAL HEALTH

Supplemental Material

sj-pdf-2-dhj-10.1177_20552076241269630 - Supplemental material for Implementation of a conversational, videoconferencing-based therapy group for postpartum depression and anxiety symptoms: A pragmatic evaluation

Supplemental material, sj-pdf-2-dhj-10.1177_20552076241269630 for Implementation of a conversational, videoconferencing-based therapy group for postpartum depression and anxiety symptoms: A pragmatic evaluation by Neesha Hussain-Shamsy, Lori Wasserman, Greer Slyfield Cook, Kaeli Macdonald, Keisha Greene, Lucy C. Barker, Juveria Zaheer, Geetha Mukerji, Simone N. Vigod and Emily Seto in DIGITAL HEALTH

Footnotes

Contributorship

NHS reviewed the literature, conceived of, and designed the study under the guidance and supervision of ES, SV, GM and JZ as part of her doctoral work. LW, GSC, KM and KG provided input into the design of the study and the processes for data acquisition. GSC, KM and KG facilitated data collection. NHS, ES, SV, GM and JZ interpreted the data. LB assisted with the interpretation of the data and provided critical revisions to the manuscript. All authors reviewed, edited, and approved the final version of the manuscript.

Declaration of conflicting interests

SV reports royalties from UpToDate Inc. for authorship of materials on depression and pregnancy.

Ethical approval

This initiative was formally reviewed by institutional authorities at Women's College Hospital and was approved under the Assessment Process for Quality Improvement Projects program (File #2021-0059-E).

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Women's College Hospital (grant number AFP Innovation Fund WCH-22-005).

Guarantor

NHS.

ORCID iD

Neesha Hussain-Shamsy

Supplemental material

Supplemental material for this article is available online.

References

Wang

, et al. Mapping global prevalence of depression among postpartum women. Transl Psychiatry 2021; 11: 543.

Fawcett

Fairbrother

Cox

, et al. The prevalence of anxiety disorders during pregnancy and the postpartum period: a multivariate Bayesian meta-analysis. J Clin Psychiatry 2019; 80: 18r12527.

Bowen

Butt

, et al. Patterns of depression and treatment in pregnant and postpartum women. Can J Psychiatry 2012; 57: 161–167.

Parikh

, et al. Canadian network for mood and anxiety treatments (CANMAT) 2016 clinical guidelines for the management of adults with Major depressive disorder: section 2. Psychological treatments. Can J Psychiatry 2016; 61: 524–539.

NICE. Antenatal and postnatal mental health: clinical management and service guidance , 2020.

Green

Haber

Frey

, et al. Cognitive-behavioral group treatment for perinatal anxiety: a pilot study. Arch Womens Ment Health 2015; 18: 631–638.

Oei

TPS

Dingle

. The effectiveness of group cognitive behaviour therapy for unipolar depressive disorders. J Affect Disord 2008; 107: 5–21.

Thimm

Antonsen

. Effectiveness of cognitive behavioral group therapy for depression in routine practice. BMC Psychiatry 2014; 14: 292.

Razurel

Kaiser

. The role of satisfaction with social support on the psychological health of primiparous mothers in the perinatal period. Women Heal 2015; 55: 167–186.

10.

Leahy-Warren

McCarthy

Corcoran

. First-time mothers: social support, maternal parental self-efficacy and postnatal depression. J Clin Nurs 2012; 21: 388–397.

11.

Leahy-Warren

. First-time mothers: social support and confidence in infant care. J Adv Nurs 2005; 50: 479–488.

12.

Leahy-Warren

Newham

Alderdice

. Perinatal social support: panacea or a pitfall. J Reprod Infant Psychol 2018; 36: 219–221.

13.

Biggs

, et al. Challenges in the design, planning and implementation of trials evaluating group interventions. Trials 2020; 21: 116.

14.

Joyce

Tasca

Ogrodniczuk

. Group psychotherapy in Canada. Int J Group Psychother 2015; 65: 583–593.

15.

Barkowski

Schwartze

Strauss

, et al. Efficacy of group psychotherapy for anxiety disorders: a systematic review and meta-analysis. Psychother Res 2020; 30: 965–982.

16.

Ayres

, et al. Engagement with perinatal mental health services: a cross-sectional questionnaire survey. BMC Pregnancy Childbirth 2019; 19: 170.

17.

Rai

Vigod

Hensel

. Barriers to office-based mental health care and interest in E-communication with providers: a survey study. JMIR Ment Heal 2016; 3: e35.

18.

Goodman

. Women’s attitudes, preferences, and perceived barriers to treatment for perinatal depression. Birth 2009; 36: 60–69.

19.

Kopelman

, et al. Barriers to care for antenatal depression. Psychiatr Serv 2008; 59: 429–432.

20.

Green

Inness

Furtado

, et al. Evaluation of an augmented cognitive behavioural group therapy for perinatal generalized anxiety disorder (GAD) during the COVID-19 pandemic. J Clin Med 2021; 11: 209.

21.

Paul

Dardar

River

, et al. Telehealth adaptation of perinatal mental health mother-infant group programming for the COVID-19 pandemic. Infant Ment Health J 2022; 43: 85–99.

22.

Pollack

Broudy

Coon

, et al. The holding space: An experiential–dynamic group therapy for mothers with postpartum anxiety and depression. Gr Dyn Theory Res Pract 2023. doi: https://doi.org/10.1037/gdn0000205.

23.

Wright

, et al. A virtual mother-infant postpartum psychotherapy group for mothers with a history of adverse childhood experiences: open-label feasibility study. BMC Psychiatry 2023; 23: 950.

24.

Preuhs

Klein Velderman

van Empelen

. Possibilities and challenges of delivering health-related small group interventions online: scoping review. Interact J Med Res 2023; 12: e43783.

25.

Gentry

Lapid

Clark

, et al. Evidence for telehealth group-based treatment: a systematic review. J Telemed Telecare 2019; 25: 327–342.

26.

Marton

Kanas

. Telehealth modalities for group therapy: comparisons to in-person group therapy. Int J Group Psychother 2015; 66: 145–150.

27.

Mirabito

Verhaeghen

. Remote delivery of a Koru mindfulness intervention for college students during the COVID-19 pandemic. J Am Coll Heal 1–8. doi: https://doi.org/10.1080/07448481.2022.2060708.

28.

Lopez

Rothberg

Reaser

, et al. Therapeutic groups via video teleconferencing and the impact on group cohesion. Mhealth 2020; 6: 13.

29.

Fortier

, et al. Online telehealth delivery of group mental health treatment is safe, feasible, and increases enrollment and attendance in post-9/11 U.S. veterans. Behav Ther 2022; 53: 469–480.

30.

Hussain-Shamsy

, et al. Operationalizing and evaluating synchronous virtual group health interventions: wide-scale implementation at a tertiary care academic hospital. J Med Internet Res 2022; 24: e29841.

31.

Sockol

. A systematic review and meta-analysis of interpersonal psychotherapy for perinatal women. J Affect Disord 2018; 232: 316–328.

32.

Markowitz

Weissman

. Interpersonal psychotherapy: principles and applications. World Psychiatry 2004; 3: 136–139.

33.

Group Interpersonal Therapy (IPT) for Depression. World Health Organization, 2016.

34.

Harris

, et al. Research electronic data capture (REDCap) – A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inf 2009; 42: 377–381.

35.

Harris

, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform 2019; 95: 103208.

36.

Proctor

, et al. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health 2009; 36: 24–34.

37.

Proctor

, et al. Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Heal Ment Heal Serv Res 2011. doi: https://doi.org/10.1007/s10488-010-0319-7.

38.

Oei

TPS

Green

. The satisfaction with therapy and therapist scale-revised (STTS-R) for group psychotherapy: psychometric properties and confirmatory factor analysis. Prof Psychol Res Pract 2008; 39: 435–442.

39.

Littleton

Grills

Kline

, et al. The from survivor to thriver program: RCT of an online therapist-facilitated program for rape-related PTSD. J Anxiety Disord 2016; 43: 41–51.

40.

Gould

, et al. Acceptance and commitment therapy for older people with treatment-resistant generalised anxiety disorder: the FACTOID feasibility study. Health Technol Assess 2021; 25: 1–150.

41.

Norman

Skinner

. eHEALS: the eHealth literacy scale. J Med Internet Res 2006; 8: e27.

42.

Hennemann

Beutel

Zwerenz

. Ready for eHealth? Health Professionals’ acceptance and adoption of eHealth interventions in inpatient routine care. J Health Commun 2017; 22: 274–284.

43.

Estrela

Semedo

Roque

, et al. Sociodemographic determinants of digital health literacy: a systematic review and meta-analysis. Int J Med Inform 2023; 177: 105124.

44.

Krogel

, et al. The group questionnaire: a clinical and empirically derived measure of group relationship. Psychother Res 2013; 23: 344–354.

45.

Thayer

Burlingame

. The validity of the group questionnaire: construct clarity or construct drift? Gr Dyn 2014; 18: 318–332.

46.

Cox

Holden

Sagovsky

. Detection of postnatal depression: development of the 10-item Edinburgh postnatal depression scale. Br J Psychiatry 1987; 150: 782–786.

47.

Spitzer

Kroenke

Williams

, et al. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006; 166: 1092–1097.

48.

Orpana

Lang

Yurkowski

. Validation of a brief version of the social provisions scale using Canadian national survey data. Heal Promot Chronic Dis Prev Canada 2019; 39: 323–332.

49.

Chen

Gully

Eden

. Validation of a new general self-efficacy scale. Organ Res Methods 2001; 4: 62–83.

50.

Adams

Walpola

Schembri

, et al. The ultimate question? Evaluating the use of net promoter score in healthcare: a systematic review. Heal Expect Int J Public Particip Heal Care Heal Policy 2022; 25: 2328–2339.

51.

Graneheim

Lundman

. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today 2004; 24: 105–112.

52.

Milosevic

Cameron

Milanovic

, et al. Face-to-face versus video teleconference group cognitive behavioural therapy for anxiety and related disorders: a preliminary comparison. Can J Psychiatry 2022; 67: 391–402.

53.

Chu

Paton

Bradley

, et al. Preliminary evaluation of an acceptance and commitment therapy group for transgender and gender expansive adults. Prof Psychol Res Pract 2023; 54: 378–387.

54.

Shapiro

Gros

. Acceptability of a transdiagnostic behavior therapy in veterans with affective disorders. Psychol Serv 2021; 18: 643–650.

55.

Yarns

, et al. Emotional awareness and expression therapy achieves greater pain reduction than cognitive behavioral therapy in older adults with chronic musculoskeletal pain: a preliminary randomized comparison trial. Pain Med 2020; 21: 2811–2822.

56.

Alfonsson

Olsson

Hursti

. The effects of therapist support and treatment presentation on the clinical outcomes of an internet based applied relaxation program. Internet Interv 2015; 2: 289–296.

57.

Janis

Burlingame

Olsen

. Developing a therapeutic relationship monitoring system for group treatment. Psychotherapy 2018; 55: 105–115.

58.

Borek

, et al. Identifying change processes in group-based health behaviour-change interventions: development of the mechanisms of action in group-based interventions (MAGI) framework. Health Psychol Rev 2019; 13: 227–247.

59.

Perrone

Sedlacek

. A comparison of group cohesiveness and client satisfaction in homogenous and heterogenous groups. J Spec Gr Work 2000; 25: 243–251.

60.

Mao

, et al. Sensitivity to change and minimal clinically important difference of Edinburgh postnatal depression scale. Asian J Psychiatr 2021; 66: 102873.

61.

Ammentorp

Rasmussen

Nørgaard

, et al. Electronic questionnaires for measuring parent satisfaction and as a basis for quality improvement. Int J Qual Heal Care 2007; 19: 120–124.

62.

Enticott

Johnson

Teede

. Learning health systems using data to drive healthcare improvement and impact: a systematic review. BMC Health Serv Res 2021; 21: 200.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.51 MB