Abstract
Introduction
Videoconferencing psychotherapy (VCP) delivers treatment to individuals with limited access to face-to-face mental healthcare. VCP's effectiveness has been demonstrated for various disorders and therapeutic interventions. However, there is contradictory evidence regarding the therapeutic alliance in VCP as compared to psychotherapy in person (PIP). This meta-analysis examines whether therapeutic alliance differs by psychotherapy's delivery format, namely VCP versus PIP.
Methods
We searched five databases for trials comparing the therapeutic alliance in VCP and PIP, wherein the therapeutic alliance was rated by either patients or therapists or both. Eighteen publications were included, and the difference between VCP and PIP was assessed. Furthermore, we tested possible moderators of the difference in therapeutic alliance between VCP and PIP by meta-regression, and we assessed the risk of bias of this meta-analysis.
Results
The meta-analysis revealed no statistically significant difference in the therapeutic alliance between VCP and PIP for alliance ratings by patients (SMD = −0.09; 95% CI = −0.26; 0.07) as well as by therapists (SMD = 0.04; 95% CI = −0.17; 0.25). No significant moderators were found.
Discussion
In this meta-analysis, VCP and PIP did not differ with respect to the therapeutic alliance as rated by either patients or therapists. Further research is required into mechanisms driving the therapeutic alliance in VCP and PIP.
Keywords
Introduction
Access to psychological treatment is sometimes impeded by social stigma or by physical distance.1,2 For example, the COVID-19 pandemic has hindered access to mental healthcare in 93% of all countries. 3 Videoconferencing psychotherapy (VCP) enables access to psychotherapy regardless of individuals’ geographic location, personal mobility, or reservations due to privacy or stigma. 1 VCP here refers to a format of individual psychotherapy in which patients and therapists are not in the same room and communicate via real-time video. VCP has been successful in treating various mental diseases including social anxiety disorder, 4 bulimia nervosa, 5 and posttraumatic stress disorder.6,7 Moreover, VCP has been used for a variety of psychological interventions including cognitive-behavioural therapy (CBT),8,9 schema therapy, 10 intensive short-time dynamic psychotherapy, 11 or acceptance-based therapy. 4
The therapeutic alliance has been found to be a central interpersonal factor in psychotherapy effectiveness,12–16 with stronger therapeutic alliance predicting better outcomes for patients. Synonyms of the therapeutic alliance are helping alliance, therapeutic relationship, rapport, or bond.2,13,17 Prominently, Bordin 18 coined the term working alliance that comprises three aspects: (a) mutual agreement between patient and therapist regarding the goals of the treatment; (b) a joint assignment of the tasks necessary to obtain said goals; and (c) an affective bond between patient and therapist, forming the base for the patient's trust and willingness to disclose their personal issues.
The therapeutic alliance in psychotherapy is often assessed by patients’ as well as by therapists’ ratings on scales such as the working alliance inventory (WAI) by Horvath and Greenberg, 19 which is based on Bordin's 18 conceptualization. WAI is the most widely used questionnaire to quantify therapeutic alliance.1,2,16
A number of studies have investigated the impact of the delivery format, that is, VCP or psychotherapy in person (PIP) on therapeutic alliance. The evidence is heterogeneous: Some studies found the therapeutic alliance in VCP to be inferior to that of PIP,20–22 whereas other studies found it to be equivalent1,2,10,17,23,24 or better25–27. In a non-inferiority meta-analysis, Norwood et al. 28 failed to demonstrate equivalence of therapeutic alliance in VCP as compared to PIP. However, Norwood et al. 28 included only trials applying CBT or related interventions on adult patients, resulting in only four data sets to be meta-analysed.
In the present meta-analysis, besides an update by more than four years in a time of rapid video-technological development and uptake, we extend Norwood et al. 28 in three aspects to arrive at more reliable and broader conclusions regarding the impact of psychotherapy's delivery format on the therapeutic alliance. First, the meta-analysis at hand is not restricted to a specific type of dyadic psychotherapy. However, in line with Norwood et al., 28 we excluded group therapy, since the group setting entails different interpersonal processes than a dyadic setting.12,29 Second, in addition to RCTs, we included non-randomized trials (NRTs). Third, we conducted separate analyses for patients’ and therapists’ ratings of the therapeutic alliance. Norwood et al. 28 performed an ‘amalgamation of the working alliance ratings’ by therapists and patients, despite evidence that therapists, unlike patients, perceive the therapeutic alliance in VCP to be generally weaker than in PIP.30,31 Moreover, therapists, unlike patients, rate the therapeutic alliance lower in VCP than in PIP, even if the sessions are the same. 22
In sum, the delivery format might make a difference with regard to the therapeutic alliance in psychotherapy, but a comprehensive summary of hitherto findings is missing. Thus, the first objective of this systematic review is to evaluate whether VCP and PIP differ with respect to the therapeutic alliance. Following Norwood et al., 28 we hypothesize that patients and therapists report lower levels of therapeutic alliance in VCP than in PIP. The second objective of the review at hand is to explore potential moderators of the difference in therapeutic alliance between VCP and PIP (i.e. the technological sophistication of video communication and the patients’ age).
Methods
We followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines 32 and prospectively registered this review at https://osf.io/v3dft.
Eligibility criteria
Eligible studies had to meet the following criteria: (a) Population: Participants were patients in psychotherapy with a clinical diagnosis according to ICD or DSM. (b) Intervention: Studies concerning individual psychotherapy (excluding group or self-help interventions and solely pharmacological or medical services) conducted by therapists with a Master's degree or higher in psychology or an equivalent degree were included. (c) Comparison: Studies compared a VCP condition and a PIP condition. The VCP condition consisted of psychotherapy conducted via real-time audio-visual digital communication only. Patients and psychotherapists were able to see and hear each other with the possibility to speak at all times during the intervention. (d) Outcomes: The strength of the therapeutic alliance was measured using validated instruments. (e) Study design: As we sought to provide a comprehensive overview of all studies on the topic, we decided to include NRTs in addition to randomized controlled trials (RCTs), and we assessed the impact of this decision by way of sensitivity analysis.
Information sources and search strategy
We searched the electronic databases APA PsychInfo, APA PsychoArticles, MEDLINE, PSYNDEX, and CINAHL from database inception to April 18, 2021. This search was updated on October 24, 2022. We did not apply language restrictions. The search string included combinations of three blocks of search terms: a VCP block, a PIP block, and a therapeutic relationship block (see supplemental material, Table S1). Furthermore, we carried out forward and backward citation searches using Web of Science.
Study selection
The study records were managed with the web application Rayyan. 33 Two reviewers with experience in meta-analysis (PDS and LS) independently performed title and abstract screening as well as the full-text screening of the identified studies. The agreement between the reviewers was high in the primary title and abstract screening (κ = .99) and substantial in the primary full-text screening (κ = .66). Disagreement was solved through discussion or consultation with SS and JCF.
Data collection process and data items
Two independent reviewers (PDS and LS) extracted relevant data from the selected studies using a standardized Excel sheet. The reviewers agreed on 98% of the collected primary data. In four cases, the data reported in the studies was insufficient. We contacted the authors via email, but the authors only provided the missing data in two cases. The remaining studies were excluded as data on therapeutic alliance were not available.
We collected data on (a) study: authors, year, design, country; (b) sample: size, diagnoses, age; (c) intervention: type and number of sessions, patient location in the VCP condition as well as data transmission rate; (d) alliance measurement: instrument, rater (patient/therapist), session number; and (e) outcome: means and standard deviations (SDs) for therapeutic alliance in VCP and in PIP to calculate standardized mean differences (SMDs). Furthermore, we rated the study quality and assessed the risk of bias.
In one case in which the therapeutic alliance was assessed with two instruments, we took the data from the WAI, 19 as it is the more common Instrument 1 . In cases in which SDs were not reported, we calculated them from the SEs or confidence intervals with the formulas provided in the Cochrane Handbook for Systematic Reviews of Interventions. 34 In studies that only reported the subscales of the WAI, we summarized them into an overall WAI score.
Whenever the therapeutic alliance was measured more than once in a trial, the last measurement was used, since later alliance measurements hold a more solid relation to the efficacy of psychotherapy. 13 For studies in which the number of therapy sessions differed among participants, we used the mean number of sessions, if available, or the middle of the session number range. If the therapeutic alliance was judged by both the patient and the therapist, both ratings were used for the analysis, as they derive from two separate samples of study participants. Two independent meta-analyses were conducted on the patients’ and the therapists’ ratings.
Risk of bias in individual studies
Two independent reviewers (PDS and LS) evaluated the risk of bias in the individual studies using the Effective Public Health Practice Project (EPHPP) Quality Assessment tool. 35 The EPHPP rates the methodological quality as good, fair, or poor across six domains (i.e. the risk of bias is reversely coded): (a) selection bias, (b) design, (c) confounders, (d) blinding, (e) data collection methods, and (f) withdrawals and drop-outs. Additionally, a global quality rating is derived for every study. However, for the global rating, we decided to not include blinding, as it is impossible to have patients or therapists blinded to their allocation to VCP or PIP. We chose the EPHPP because it can be applied to a variety of research designs. The reviewers agreed on 96% of the judgements. Disagreement was solved by discussion.
Effect measure and synthesis methods
Since Norwood et al. 28 did not find non-inferiority of VCP to PIP regarding therapeutic alliance, we hypothesized VCP to entail a weaker therapeutic alliance compared to PIP. We calculated Hedges’ g for the individual studies as a measure for the SMD between the therapeutic alliance in VCP versus PIP and aggregated the SMDs of the individual studies into an overall SMD. We employed a random effects model using the inverse-variance weighting method 36 and checked the aggregated SMD for significance using p-values and 95% confidence intervals. This was done for both therapists’ and patients’ ratings of the therapeutic alliance. We assessed the heterogeneity between studies using I2 and Q statistics. 34 We used the software R (Version 4.0.1, 37 3;8) with the packages metafor 39 and meta. 40
Moderators
We assessed the impact of two potential moderators for the difference in the therapeutic alliance between VCP and PIP using meta-regression analyses: (a) the publication year as a proxy for VCP's technological sophistication and spread, assuming that an increase in technological quality and/or spread might foster the therapeutic alliance in VCP; (b) the patients’ age, assuming that younger patients find it easier to adapt to the digital communication in VCP.41,42
Reporting bias and certainty assessment
To identify potential publication bias, we created funnel plots and examined them for asymmetry. To assess the overall quality of evidence, we used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. 43 GRADE takes into account: (a) risk of bias, (b) imprecision, (c) inconsistency, (d) indirectness, and (e) publication bias to rank the overall quality of evidence as ‘high’, ‘moderate’, ‘low’, or ‘very low’.
Sensitivity analyses
We conducted two sensitivity analyses: (a) excluding NRTs, as NRTs are considered epistemologically inferior to RCTs and (b) excluding studies that implemented psychotherapeutically divergent interventions (i.e. treatments in the realm of counselling or psychoeducation).
Results
The searches yielded a total of 2807 records. After removing 336 duplicates, the titles and abstracts of the remaining 2471 records were screened for eligibility. Of these, 60 were considered eligible for full-text screening (see Figure 1 for a flow chart of the search and selection process).

PRISMA flow chart for study selection.
For six studies,11,22,44–47 SS and JCF were consulted to decide whether the intervention was to be considered psychotherapy. A joint discussion based on the inclusion criteria resulted in a consensus: Two of those studies were included in the meta-analysis.11,47 One study 5 was excluded because the therapeutic alliance was reported in a separate publication, which was included in the meta-analysis instead. 48 Ultimately, 18 studies were included in the meta-analysis. For a description of included studies, see supplemental material, Table S2. For a description of studies that were excluded after the full-text screening, see supplemental material, Table S3.
All 18 studies reported patients’ ratings of the therapeutic alliance, and seven studies reported therapists’ ratings of the therapeutic alliance.6,27,47,49–52 Altogether, we retrieved 1619 therapeutic alliance ratings on 1245 patient-therapist dyads.
Characteristics of studies
Of the 18 included studies, 10 (56%) were RCTs.6,11,27,47–49,51,53–55 Of the NRTs, five (28%) were clinical controlled trials (CCT)23,52,55–57), two were cohort analytic designs,10,58 and one was an interrupted time series, 50 according to the classification provided by the EPHPP tool. 35 All articles were published between 1999 and 2022.
Characteristics of the participants
The sample sizes ranged from six to 186 (mean = 69.8, SD = 47.0). The overall ratio between patients in VCP versus PIP was 1:1.18. Participants’ mean age was 36.4 years and ranged from 20.0 (SD = 11.0) 51 to 46.4 (SD = 11.9) 6 . The studies encompassed samples with a variety of diagnoses, mostly anxiety and affective disorder,6,23,27,50,54,56,58 see supplemental material, Table S2. One study reported on a sample with more than one diagnosis. 10 Three studies did not report diagnoses but indicated that diagnoses were present.47,53,58
Characteristics of the interventions
Nine studies used CBT as a type of psychotherapeutic intervention.10,23,27,48,51,55,56,59 The interventions applied in the other studies varied considerably (see supplemental material, Table S2). Three studies did not specify the type of psychotherapeutic treatment.52,53,58 The treatments varied in length from one session47,53,58 to 20 sessions, 48 with a mean number of 8.6 sessions (SD = 6.1). Sixteen studies (89%) specified the location of the patients in the VCP sessions: In three studies the patients were at home,49,54,57 in eight studies in a research facility,10,47,48,50,51,53,56,59 in four studies in a local clinic,6,27,52,55 and in one study in a psychiatric prison. 58 Data transmission rate was reported in nine studies,11,27,47,48,51,54–56,59 and it ranged from 384 kbps55,56 to 1,544,000 kbps. 59
Instruments used
To assess the therapeutic alliance, 13 studies (72%) used the WAI23,27,48,53,55,59 or one of its short forms (WAI-S, WAI-SR.6,50,51,54,56–58 The remaining single studies used the California Psychotherapy Alliance Scale, 47 the CORE-ARM, which is an abbreviated version of the Agnew Relationship Measure, 10 the revised Helping Alliance Questionnaire (HAQ-II) 48 , or the Psychotherapist Alliance Questionnaire, which is a modified version of the Modified Helping Alliance Questionnaire (HAQ-M) 11 .
Meta-analyses and moderator analyses
Patients’ alliance ratings
The therapeutic alliance as rated by the patients did not significantly differ as a function of delivery format (n = 18; SMD = −0.09; 95% CI = −0.26; 0.07; p = .28; Figure 2). Hence, our hypothesis of the inferiority of VCP to PIP was rejected. The heterogeneity was moderate (I2 = 44%). The meta-regressions did not show a significant association of the difference in patient-rated therapeutic alliance between VCP and PIP with the publication year or age (all ps ≥ .05).

Forest plot of the difference between VCP and PIP in the therapeutic alliance, with the therapeutic alliance rated by patients in panel A and by therapists in panel B.
Therapists’ alliance ratings
The therapeutic alliance as rated by the therapists did not significantly differ as a function of delivery format (n = 7; SMD = 0.04; 95% CI = −0.17; 0.25; p = .71; Figure 2). Thus, our hypothesis of inferiority of VCP to PIP was rejected. The heterogeneity was low (I2 = 29%). The meta-regressions did not show a significant association of the difference in therapist-rated therapeutic alliance between VCP and PIP with the publication year or age (all ps ≥ .05).
Risk of bias within studies
The overall risk of bias across studies was low to moderate: The quality of evidence was rated as good in 55.56% of the studies, fair in 27.78%, and poor in 16.67%. An overview of the overall and domain-specific EPHPP ratings are in Figure 3. For the complete EPHPP ratings see supplemental material, Table S4.

Distribution of the ratings on five domains and the overall ratings of the EPHPP Quality Assessment Tool for Quantitative Studies 35 (N = 18).
Reporting bias and certainty assessment
Egger's regression test for funnel plot asymmetry showed no sign of publication bias for patients’ ratings (t = −0.34; p = .74; Figure 4). Egger's regression test was not applied to the therapists’ ratings (Figure 4), as it is not recommended if there are fewer than 10 studies. 34 The overall quality of evidence rated with the GRADE approach was low for the patient-rated therapeutic alliance and very low for the therapist-rated therapeutic alliance (see supplemental material, Table S5).

Funnel plot of the SMDs between VCP and PIP in the therapeutic alliance, and SEs, with the therapeutic alliance rated by patients in panel A and by therapists in panel B. Note. The triangle indicates the area in which 95% of studies are expected to be if no publication bias is present. SMD: standardized mean difference; VCP: videoconferencing psychotherapy; PIP: psychotherapy in person; SE: standard error.
Sensitivity analysis 1 (exclusion of NRTs)
We excluded eight studies that were not described as RCTs,10,23,50,52,55–58 leaving 10 RCTs reporting patients’ ratings of the therapeutic alliance and five RCTs reporting therapists’ ratings.
Meta-analyses and moderator analyses
Patients’ alliance ratings
With only RCTs included in the analysis, there was no significant effect of the delivery format on the therapeutic alliance as rated by patients (n = 10; SMD = −0.03; 95% CI = −0.27; 0.20; p = .78; Figure S1). The heterogeneity was substantial (I2 = 64%). The meta-regressions did not show a significant association of the difference in patient-rated therapeutic alliance between VCP and PIP with the publication year or age (all ps ≥ .05).
Therapists’ alliance ratings
With only RCTs included in the analysis, there was no significant effect of the delivery format on the therapeutic alliance as rated by therapists (n = 5; SMD = 0.00; 95% CI = −0.22; 0.22; p = .97; Figure S1). The heterogeneity was very low (I2 = 0%). The meta-regressions did not show a significant association of the difference in therapist-rated therapeutic alliance between VCP and PIP with the publication year or age (all ps ≥ .05).
Reporting bias and certainty assessment
Egger's regression test for funnel plot asymmetry showed no sign of publication bias for patients’ ratings in the RCTs (t = −0.59; p = .57; Figure S2). Egger's regression test was not applied to the therapists’ ratings (Figure S2), as it is not recommended for analyses including fewer than 10 studies. 34
Sensitivity analysis 2 (exclusion of diverging treatments)
We excluded four studies that although having met the inclusion criteria of the meta-analysis diverged from a standard psychotherapeutic set-up: Hungerbuehler et al. 54 examined psychoeducation, medication plus counselling, Morgan et al., 58 Netter et al. 57 examined only one session of psychotherapy, and Stevens et al. 47 included merely a general psychiatric assessment. The exclusion of these four studies left 14 studies reporting patients’ ratings of the therapeutic alliance, and six studies reporting therapists’ ratings.
Meta-analyses and moderator analyses
Patients’ alliance ratings
When only including studies with a standard psychotherapeutic set-up, there was no significant effect of the delivery format on the patients’ ratings of therapeutic alliance (n = 15; SMD = −0.09; 95% CI = −0.30; 0.12; p = .33; Figure S3). The heterogeneity was moderate (I2 = 56%). The meta-regressions did not show a significant association of the difference in patient-rated therapeutic alliance between VCP and PIP with the publication year or age (all ps ≥ .05).
Therapists’ alliance ratings
When only including studies with a standard psychotherapeutic set-up, there was no significant effect of the delivery format on the therapists’ ratings of therapeutic alliance (n = 6; SMD = 0.07; 95% CI = −0.15; 0.29; p = .54; Figure S3). The heterogeneity was moderate (I2 = 36%). The meta-regressions did not show a significant association of the difference in therapist-rated therapeutic alliance between VCP and PIP with the publication year or age (all ps ≥ .05).
Reporting bias and certainty assessment
Egger's regression test for funnel plot asymmetry showed no sign of publication bias for patients’ ratings (t = −0.40; p = .71; Figure S4). Egger's regression test was not applied to the therapists’ ratings, as it is not recommended for analyses including fewer than 10 studies. 34
Discussion
We conducted a meta-analysis comparing videoconferencing psychotherapy (VCP) and PIP with respect to the therapeutic alliance. Eighteen studies with 1619 therapeutic alliance ratings were summarized. The ratings for the therapeutic alliance did not differ between VCP and PIP, neither for patient-rated (SMD = −0.09) nor for therapist-rated therapeutic alliance (SMD = 0.04). Moreover, neither the publication year nor the patients’ age had a significant impact on the difference in therapeutic alliance between VCP and PIP.
The finding that the ratings of the therapeutic alliance did not differ between VCP and PIP is in line with existing reviews,1,2,17,24,60 suggesting that VCP is a promising way to provide people with psychotherapeutic treatment, especially where access to PIP is restricted. The finding that the ratings of the therapeutic alliance did not differ between VCP and PIP is at odds with Norwood et al., 28 who failed to demonstrate the equivalence of therapeutic alliance in VCP to PIP in a non-inferiority meta-analysis. However, the two meta-analyses differ in several ways, whereby the present one is more up-to-date and comprises more primary studies. That being said, however, although the main finding indicates that the therapeutic alliance is equally strong in VCP and PIP, this meta-analysis does not prove it. First, this is because absence of evidence does not equal evidence of absence, and second, the review at hand might have lacked statistical power to detect a significant difference between VCP and PIP.
The facts that the heterogeneity among summarized studies was low to moderate and that there were no indications of publication bias speak to the generalizability and stability of the main outcome. However, several facts that this review uncovered about the current state of research dampen the insights into therapeutic alliance as a function of psychotherapy format that can be gleaned up to today: A moderate number of 18 trials qualified to be included in this review, while power was lower with regard to moderator analyses. Moreover, 44% of included studies provided a moderate to low overall quality of evidence. In sum, further studies into the difference in therapeutic alliance as a function of psychotherapy format are needed.
Strengths and limitations of the study
In the way this study was conducted, it has several strengths. To the best of our knowledge, it is the first systematic review to provide separate meta-analyses for therapists’ and patients’ ratings of the therapeutic alliance. Furthermore, the review at hand extends previous research 28 by including psychotherapeutic interventions other than CBT and by including NRTs. Furthermore, we established a transparent and traceable research process by pre-registering the study. Two independent reviewers performed the screening, data extraction, and risk of bias assessment, ensuring consistency. Limitations of this review are that only one reviewer conducted the GRADE rating and that no grey literature beyond the aforementioned databases was searched.
Implications for future research
Technological factors, including the visual and acoustic quality of the therapeutic medium with regard to the temporal and spatial resolution, are deemed important for the interpersonal relationship in video communication,61–63 and further studies are required to examine their influence as well as other possible moderators such as diagnosis, comorbidity, socioeconomic status, study quality etc. Future meta-analyses on larger study bases should also assess influences involving culture, ethnicity and equity, and they may target psychotherapeutic settings with more than one patient (e.g. group formats, family therapy). Furthermore, the influence of delivery format is of special interest in psychodynamic therapy, as here the therapeutic relationship is the focus of the therapy.
This research unburied the curious finding that most VCP interventions were delivered in a research building or clinic, despite the fact that a major advantage of VCP is deliverability to the patient's home. This is probably due to methodological considerations regarding internal validity. However, this weakens the external validity of the findings, that is, the generalizability of the present results to in-home VCP. To draw conclusions about the social mechanisms present in the clinical use of VCP, it needs to be studied in a more realistic context (i.e. in patients’ private homes). In doing so, research needs to take into account additional factors: VCP in the patients’ homes is easily accessible, but with some patients in their home, there might be less privacy or even safety. PIP requests the patients to leave the house and sometimes to cover a long distance but this may have the advantage of activation, which may entail a beneficial side effect and may also influence the therapeutic relationship.
Conclusion
In summary, we did not find the therapeutic alliance to be significantly different in VCP compared to PIP. Further studies are needed to assess technological factors that benefit or hinder the therapeutic alliance in VCP. In future research on VCP, emphasis should be placed on the treatment of patients in their own home.
Supplemental Material
sj-docx-1-jtt-10.1177_1357633X231161774 - Supplemental material for Therapeutic alliance in videoconferencing psychotherapy compared to psychotherapy in person: A systematic review and meta-analysis
Supplemental material, sj-docx-1-jtt-10.1177_1357633X231161774 for Therapeutic alliance in videoconferencing psychotherapy compared to psychotherapy in person: A systematic review and meta-analysis by Patrik D Seuling, Johannes C Fendel, Lukas Spille, Anja S Göritz, and Stefan Schmidt in Journal of Telemedicine and Telecare
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
