Primary Care Physician Adherence to Telepsychiatry Recommendations: Intermediate Outcomes from a Randomized Clinical Trial

Abstract

Objective:

To compare clinical recommendations given by psychiatrists and the adherence to these recommendations by primary care physicians (PCP) following consultations conducted by asynchronous telepsychiatry (ATP) and synchronous telepsychiatry (STP).

Materials and Methods:

ATP and STP consultations were compared using intermediate data from a randomized clinical trial with adult participant enrollment between April 2014 and December 2017. In both study arms, PCPs received written recommendations from the psychiatrist after each encounter. Independent clinicians reviewed PCP documentation to measure adherence to those recommendations in the 6 months following the baseline consultation.

Results:

Medical records were reviewed for 645 psychiatrists’ consult recommendations; 344 from 61 ATP consultations and 301 from 62 STP consultations. Of those recommendations, 191 (56%) and 173 (58%) were rated fully adherent by two independent raters for ATP and STP, respectively. In a multilevel ordinal logistic regression model adjusted for recommendation type and recommended implementation timing, there was no statistically significant difference in adherence to recommendations for ATP compared with STP (adjusted odds ratio = 0.91, 95% confidence interval = 0.51–1.62). The profiles of recommendation type were comparable between ATP and STP.

Conclusions:

This is the first PCP adherence study comparing two forms of telemedicine. Although we did not find evidence of a difference between ATP and STP; this study supports the feasibility and acceptability of ATP and STP for the provision of collaborative psychiatric care. Clinical Trial Identifier NCT02084979.

Introduction

In mental health care, live two-way videoconferencing, or synchronous telepsychiatry (STP) has been shown to be clinically effective in diagnosing and treating mental health patients while increasing access to care during a time of physician shortages.^1

–7 Although uptake of STP has been increasing,^8,9 especially during the COVID-19 pandemic,^10

–14 STP had historically been hindered by logistical issues such as coordination of two physician's schedules or implementation into workflow.¹⁵ Asynchronous telepsychiatry (ATP) was developed as an easier-to-implement, data-rich alternative to STP.¹⁶ For this article we defined STP as an encounter where the primary care physician (PCP) and consulting psychiatrist conduct a live consultation while the patient is visiting their PCP.¹⁷ In contrast, in ATP—sometimes referred to as “store-and-forward” telemedicine—a video of a patient encounter is captured and sent to the consulting psychiatrist.^16,18 The consulting psychiatrist then views the video and sends a written assessment with recommendations to the PCP. Although STP has been used and evaluated for over 30 years, there is a dearth of studies evaluating ATP as a clinically effective alternative to STP.¹⁸ Most ATP literature remains in the realm of general feasibility,^19,20 cross-lingual/transcultural feasibility,^21,22 and cost-effectiveness.^23,24 To date, one study has found that use of ATP was associated with improved Patient Health Questionnaire-9 scores at 6 and 12 months compared with baseline.²⁵

Both models of care shift the responsibility of managing care from the psychiatrist to the PCP, which is part of a larger trend for increased PCP engagement in mental health care.^17,25
–27 Because of this responsibility shift, it is important to understand PCP adherence to psychiatrist consultation recommendations. Across different specialties, increased adherence to consultation recommendations is associated with improved patient outcomes.^29

–33 In a randomized study of PCP adherence to psychiatrist recommendations after inpatient discharge, an aggregated measure of adherence across five types of recommendations—medication, psychotherapeutic (including supportive PCP talks), psychosocial intervention, diagnostic action, and referral to specialist psychiatric outpatient treatment—was associated with lower depression scores 6 weeks after discharge, after adjusting for symptom severity.³³

To our knowledge, no studies have investigated PCP adherence to psychiatrist recommendations in the outpatient setting. In addition, few studies have investigated PCP adherence during consultations utilizing e-health technologies (i.e., teleconsultations),^34
–36 despite the increased usage of these technologies for formal and informal consultations.^37,38 To date, there have not been any clinical trials evaluating adherence to specialist advice during teleconsultations.^38,39 The aim of this study was to estimate PCP adherence in ATP and STP with regard to medication, diagnostic action, and psychotherapeutic recommendations in the 6 months following the baseline psychiatrist consultation. We hypothesized that ATP would result in greater PCP adherence than STP.

Materials and Methods

STUDY DESIGN AND STUDY POPULATION

This study was a secondary analysis of a parallel randomized clinical trial. The parent study compared clinical outcomes of ATP with STP.^40,41 PCPs and patients were recruited at two primary care clinic sites with telemedicine capabilities. To be eligible for the study, the adult patient (>18 years) must have had nonurgent psychiatric symptoms. After informed consent and a preliminary diagnosis, the patient was randomized to each intervention arm such that Axis 1 diagnoses were evenly balanced between the consultation intervention type. Although randomization was only assigned at the patient level, this randomization scheme ensured that by extension, a patient's PCP was randomized to the consultation intervention type in each patient–PCP dyad. Additional details on the parent study are described elsewhere^25,41 and on the registry ClinicalTrials.gov.⁴⁰ University of California Davis’ Institutional Review Board approved this study.

INTERVENTION

In the STP arm, a live videoconference was arranged with the psychiatrist while the patient was visiting the PCP. In the ATP arm, trained members of the research team conducted a standardized interview that was video recorded. Later, a psychiatrist reviewed the video, the notes provided by the interviewer, and previous electronic medical records (EMR) to write an assessment and psychiatric treatment plan that was sent to the patient's PCP. In both arms, the PCP could consult the psychiatrist by telephone or e-mail at any time during the study. Furthermore, the PCPs in this study included trained physicians and did not include mid-level providers.⁴¹

PCP ADHERENCE

Using the Consultation-Liaison framework,^33,42,43 expert physicians and telemedicine researchers created an expanded framework for classifying psychiatrist recommendations and rating PCP adherence to those recommendations. Evaluators reviewed charts from the baseline psychiatrist consultation to extract and classify recommendations into three major groups (“Medication,” “Diagnostic Action,” and “Psychotherapeutic”) and 11 total subgroups. The recommended implementation timing was then categorized as “short-term” (<2 months), “long-term” (>2 months or contingent on the results of another recommendation), or “timing not designated.”

PCP adherence was rated using the subsequent PCP encounters documented in the EMR. Because this study was in an outpatient setting, we chose endpoints longer than the previously used endpoints of discharge⁴³ and 96 h.⁴² We chose a 6-week primary endpoint because we expected this to be enough time for the patient to be evaluated by the PCP for medication changes. Six months was chosen as the second endpoint because it was the scheduled time of the second telepsychiatrist visit in the parent trial. PCP adherence was measured on an ordinal scale (“Fully,” “Partially,” or “Not”) as described in Supplementary Appendix SA1.

There were certain circumstances where a psychiatrist made a recommendation, but the recommendation was not implemented because of patient preference or symptoms. For example, a patient may not have wanted to change their current medication regime despite the psychiatrist's recommendation to increase dosage. In these cases, the PCP was rated “Fully Adherent” because the PCP must have discussed the recommendation with the patient to result in that documented action plan. A fourth option was added (“Cannot Evaluate”) for recommendations that the evaluators felt unable to judge. For example, the recommendation to increase the medication dosage could be marked “Cannot Evaluate” if that specific medication had not been initiated.

DATA COLLECTION AND ELECTRONIC MEDICAL REVIEW

Three evaluators (two psychiatry resident physicians, one fourth-year medical student) extracted recommendations and rated PCP adherence in four phases: (1) training, (2) recommendation extraction, (3) adherence rating, and (4) data reconciliation. The first phase consisted of training on 11 randomly chosen patients. The remaining 132 patients were randomly assigned 1 evaluator for the second phase and 2 evaluators for the third phase.

During the second phase—recommendation extraction—one of the three evaluators extracted recommendations from the baseline psychiatrist consultation note. In the third phase—adherence rating—the other two evaluators rated PCP adherence in relation to the extracted recommendations instead of using the baseline psychiatrist consultation note. This process blinded the evaluators in the third phase from the patients’ study arm, which was written in the baseline psychiatrist consultation note.

The fourth and final phase—data reconciliation—occurred if any given recommendation of any of the patients had two diametrically opposed ratings. Diametrically opposed ratings occurred when one rating was “Not Adherent,” and the other was “Fully adherent.” In this phase, the recommendation and rating were discussed between the first author and the two evaluators for that patient. After the discussion, the evaluators could update their ratings.

Each of the three evaluators participated in all four phases. The final analyses used arithmetic means. These data were collected in a REDCap database hosted at University of California, Davis.⁴⁴ Patient demographic data were pulled from the parent study.⁴¹

INTERRATER RELIABILITY

Interrater reliability (IRR) was determined using a two-way mixed, absolute, average measures intraclass correlation (ICC).⁴⁵ Before reconciliation, the resulting IRR was in the good range for the 6-week endpoint (IRR = 0.70) and 6-month endpoint (ICC = 0.70). After reconciliation, the ICCs were in the excellent range IRR = 0.97 and IRR = 0.98.⁴⁶ This indicates that these the raters had a high degree of agreement and suggests that there was minimal measurement error of adherence. The average adherence ratings were deemed suitable for use in hypothesis tests in this study.

COVARIATES

Covariates included patient demographics (age, sex, race/ethnicity); education and employment status; and psychiatric medication(s), psychiatric therapy, and presence of at least one comorbidity at baseline. In addition, PCP, PCP clinic, PCP specialty, psychiatrist, and number of concurrent recommendations were also considered.

POWER CALCULATIONS

Before conducting the study, power calculations were calculated using G*Power v3.1.9.7 for a two-group comparison of proportions with an α of 0.05, 77 in the ATP group and 81 in the STP group.⁴⁷ Assuming that the STP group had 50% full PCP adherence, there was a 57% power to detect an OR of 2 and 91% power to detect an OR of 3.

DATA AND STATISTICAL ANALYSES

Student t-tests and chi-square tests were used for bivariate comparisons of patient, referring physician and recommendation characteristics. Nonparametric trend tests compared adherence among the different recommendation types in unadjusted analyses. In adjusted analyses, a multilevel ordinal logistic regression compared adherence between ATP and STP. The Brant statistic was calculated to ensure that models did not violate the parallel regression assumption. Intraclass correlation was calculated using $\frac{π^{2}}{3}$ as the individual recommendation variance.⁴⁸ Covariates of interest were limited to variables that were associated with adherence, with a p-value <0.2. These covariates of interest (primary diagnosis, recommended implementation time, and recommendation type) were assessed for confounding during the modeling phase by running models with and without them and looking for a 10% or more change in the intervention mode beta coefficient. Variance inflation factors were used to confirm the absence of substantial multicollinearity (VIF >10).⁴⁹ For each covariate, exploratory models were fitted with and without an interaction term between the covariate and telemedicine mode. These models were compared using Bayesian Information Criteria. Models without interaction terms were ultimately used because all the models with interaction terms had higher Bayesian information criteria values. In addition, separate models were fit with and without a term for the number of concurrent recommendations. All tests were conducted using a two-sided significance level of alpha = 0.05. Statistical analyses were conducted using R version 3.6.1⁵⁰ and STATA version 15.⁵¹ This article was prepared with the assistance of the Consolidated Standards of Reporting Trials.⁵²

Results

PARTICIPANT CHARACTERISTICS

Of the initial 184 patients in the parent trial, there were 143 patients with accessible EMR charts. There were 20 patients who were randomized but did not have an initial baseline psychiatrist visit, yielding a final study sample of 123 (Supplementary Appendix SA2). Compared with the individuals included in this study, patients who could not be included were less likely to be on a psychiatric medication (n = 11, 55% vs. n = 102, 83%; p = 0.01) and less likely to be non-Hispanic white (n = 15, 75% vs. n = 114, 93%; p = 0.03). There were no statistical differences in the assigned study arm, primary diagnosis, or any of the other covariates.

In this study, there were 3 psychiatrists and 17 CPs (5 family physicians and 12 internal medicine physicians). However, there were similar distributions of family and internal medicine represented at the encounter level in ATP and STP. Between ATP and STP, there were no statistical differences in patient age, gender, ethnicity, and other covariates (Table 1). There were 19 patients who did not have any PCP visits pertaining to their psychiatric diagnoses: 8 (13%) in ATP and 11 (18%) in STP.

Table 1.

Baseline Patient, Physician, and Clinic Characteristics by Consult Modality

VARIABLE	ATP ( n = 61)	STP ( n = 62)	p
PATIENT CHARACTERISTICS	n (%)	n (%)	p
Patient age in years, median (IQR)	58 (43–66)	53 (42–65)	0.94
Female gender	41 (67)	40 (65)	0.90
Race/ethnicity			0.54
Hispanic	2 (3)	4 (6.5)
Non-Hispanic, Caucasian	59 (97)	55 (89)
Non-Hispanic, African American	0 (0)	2 (3.2)
Non-Hispanic, other	0 (0)	1 (1.6)
Education			0.79
High school diploma or less	7 (12)	8 (13)
Some college/2-year college	29 (48)	34 (55)
4-Year college	12 (20)	9 (15)
Graduate/professional school	13 (21)	11 (18)
Currently employed^a	25 (41)	26 (41)	0.83
Diagnosis			0.94^b
Mood disorder	42 (69)	44 (71)
Anxiety disorder	10 (16)	8 (13)
Substance abuse	1 (1.6)	1 (1.6)
Other	8 (13)	9 (14)
Comorbid medical condition	39 (64)	39 (63)	>0.99
Current mental health therapy	13 (21)	18 (29)	0.43
Current psychiatric medication(s)	50 (80)	52 (84)	0.97

PCP CLINIC CHARACTERISTICS	n (%)	n (%)
Clinic site			0.90
Site 1	44 (72)	43 (69)
Site 2	17 (28)	19 (31)

PHYSICIAN CHARACTERISTICS	n (%)	n (%)
Referring physician specialty			0.66
Family medicine	33 (54)	35 (57)
Internal medicine	28 (46)	27 (44)
Referring physician			0.66
PCP 1	14 (23)	16 (26)
PCP 2	8 (13)	10 (16)
PCP 3	5 (8)	8 (13)
Other PCP	34 (56)	28 (45)
Psychiatrist			0.09
Psychiatrist 1	18 (30)	29 (47)
Psychiatrist 2	21 (34)	20 (32)
Psychiatrist 3	22 (36)	13 (21)

ENCOUNTER CHARACTERISTICS	MEDIAN (IQR)	MEDIAN (IQR)
No. of concurrent recommendations	6 (4–7)	5 (3–6)	0.07

Frequency missing = ATP: 4 (7%), STP: 6 (10%).

Fisher's exact test instead of Pearson's chi-square test.

ATP, asynchronous telepsychiatry; IQR, interquartile range; PCP, primary care physician; STP, synchronous telepsychiatry.

As given in Table 2, at the recommendation level, there were similar distributions in recommended implementation timing and recommendation type. The most common type of recommendations was medication recommendations in both ATP and STP encounters (n = 227, 66% vs. n = 190, 63%). The most common subtypes of medication recommendations were to start medications or to adjust medication dosage/schedule.

Table 2.

Recommendation Characteristics

VARIABLE	ATP ( n = 344)	STP ( n = 301)	p
VARIABLE	n (%)	n (%)	p
Recommended implementation timing			0.54
Early	117 (34)	111 (37)
Later/contingent	90 (26)	68 (23)
Not specified	137 (40)	122 (41)
Recommendation type			0.73
Medication	227 (66)	190 (63)
Diagnostic	28 (8.2)	25 (7.7)
Psychotherapeutic	89 (26)	86 (29)
Recommendation subtype			0.68^a
Medication
Start	70 (20)	60 (20)
Adjust	92 (27)	83 (28)
Continue, no change	30 (8.7)	29 (9.6)
Discontinue	32 (9.3)	17 (5.6)
Don't give (avoid)	3 (0.9)	1 (0.3)
Diagnostic
Referral for consultation	4 (1.1)	5 (1.6)
Procedure or exam	2 (0.6)	1 (0.3)
Lab test	22 (6.4)	19 (6.3)
Psychotherapeutic
Therapist, facility	38 (11)	33 (11)
PCP talks, patient education	38 (11)	32 (11)
Support group, exercise	13 (3.8)	21 (7.0)

Fisher's exact test instead of Pearson's chi-square test.

PCP ADHERENCE

As given in Table 3, at 6 weeks, 159 (46%) recommendations in ATP and 147 (49%) recommendations in STP were rated full adherence by both raters. Adherence rates were not statistically different for all the recommendations combined or for each recommendation subtype. Similarly, at 6 months, distributions of recommendation type were comparable. The number of recommendations rated fully adherent by both raters was 56% (191) for ATP and 58% (173) for STP. In the unadjusted, ordinal logistic regression, ATP was not associated with changed adherence at 6 weeks (odds ratio [OR] = 0.95; 95% confidence interval [CI] = 0.69–1.30) or at 6 months (OR = 1.02; 95% CI = 0.74–1.41) (Table 4) compared with STP.

Table 3.

Adherence in Relation to Telepsychiatry Mode

	6 Weeks			6 Months
	ATP (n = 344)	STP (n = 301)		ATP (n = 344)	STP (n = 301)
RECOMMENDATION TYPE	n (%)	n ( %)	p^a	n (%)	n ( %)	p^a
Overall			0.75			0.90
Not adherent (0)	144 (42)	125 (42)		109 (32)	100 (33)
(0.5)	1 (0.3)	5 (1.7)		2 (0.6)	6 (2.0)
Partial adherence (1)	6 (1.7)	10 (3.3)		9 (2.6)	8 (2.7)
(1.5)	4 (1.2)	3 (1.0)		4 (1.5)	2 (0.7)
Full adherence (2)	159 (46)	147 (49)		191 (56)	173 (58)
Cannot evaluate	30 (8.7)	11 (3.7)		28 (8.1)	12 (4.0)

	n =#x2009;227	n =#x2009;190		n =#x2009;227	n =#x2009;190
Medication			0.20			0.11
Not adherent (0)	79 (35)	55 (29)		58 (26)	38 (20)
(0.5)	1 (0.4)	3 (1.6)		2 (0.9)	3 (1.6)
Partial adherence (1)	3 (1.3)	8 (4.2)		4 (1.8)	5 (2.6)
(1.5)	1 (0.4)	2 (1.1)		2 (0.9)	0 (0)
Full adherence (2)	115 (51)	112 (59)		135 (60)	133 (70)
Cannot evaluate	28 (12)	10 (5.3)		26 (12)	11 (5.8)

	n =#x2009;28	n =#x2009;25		n =#x2009;28	n =#x2009;25
Diagnostic			0.56			0.19
Not adherent (0)	12 (46)	13 (52)		7 (25)	10 (40)
(0.5)	0 (0)	0 (0)		0 (0)	0 (0)
Partial adherence (1)	1 (3.9)	1 (4.0)		1 (3.6)	1 (4.0)
(1.5)	0 (0)	0 (0)		0 (0)	1 (4.0)
Full adherence (2)	13 (46)	10 (40)		18 (64)	12 (48)
Cannot evaluate	2 (7.1)	1 (4.0)		2 (7.1)	1 (4.0)

	n =#x2009;89	n =#x2009;86		n =#x2009;89	n =#x2009;86
Psychotherapeutic			0.35			0.13
Not adherent (0)	53 (60)	57 (66)		44 (49)	52 (61)
(0.5)	0 (0)	2 (2.3)		0 (0)	3 (3.5)
Partial adherence (1)	2 (2.3)	1 (1.2)		4 (4.5)	2 (2.3)
(1.5)	3 (3.4)	1 (1.2)		3 (3.4)	1 (1.2)
Full adherence (2)	31 (35)	25 (29)		38 (43)	28 (33)
Cannot evaluate	0 (0)	0 (0)		0 (0)	0 (0)

Nonparametric trend tests, excluded the “Cannot Evaluate” category.

Table 4.

Unadjusted and Adjusted Models of Adherence

	6 WEEKS ( n = 604)		6 MONTHS ( n = 604)
	OR (95% CI)	p	OR (95% CI)	p
Unadjusted
Consultation mode
STP	Ref		Ref
ATP	0.95 (0.69–1.30)	0.75	1.02 (0.74–1.41)	0.90

Minimally adjusted^a
Consultation mode
STP	Ref		Ref
ATP	0.83 (0.49–1.39)	0.48	0.94 (0.57–1.56)	0.82

Adjusted^a
Consultation mode
STP	Ref		Ref
ATP	0.80 (0.45–1.42)	0.45	0.91 (0.51–1.62)	0.74
Recommendation type
Medication	Ref		Ref
Diagnostic	0.70 (0.33–1.52)	0.37	0.69 (0.31–1.51)	0.35
Psychotherapeutic	0.29 (0.15–0.55)	<0.001	0.20 (0.10–0.38)	<0.001
Recommended implementation timing
Early	Ref		Ref
Later/contingent	0.26 (0.15–0.46)	<0.001	0.33 (0.19–0.58)	<0.001
Not specified	0.49 (0.25–0.94)	0.032	0.54 (0.27–1.07)	0.079

Includes nested random intercepts for patient and primary care physician.

Three-level models were fitted with random intercepts for patient and PCP. At 6 weeks, the between-patient correlation was 27.9% and the between-PCP correlation was 1.3%. At 6 months, the intraclass correlation for patients and PCPs was 27.2% and 0.4%, respectively. After fitting a three-level model and adjusting for recommendation type and recommended implementation timing, ATP was still not associated with statistically significant changes in adherence compared with STP (adjusted odds ratio [aOR] = 0.80, 95% CI = 0.45–1.42) at 6 weeks or at 6 months (aOR = 0.91, 95% CI = 0.57–1.62). None of these models violated the Brant test.

Discussion

In this study of 123 patients and 645 recommendations, we were unable to conclude that PCP adherence to psychiatrists’ recommendations was substantively different among patients evaluated by ATP versus STP. The inconclusive findings remained when looking at adherence at 6 weeks and 6 months and when accounting for recommendation type, recommended implementation timing, and adjusting for clustering at the patient and PCP level. Overall, we also found that PCPs were fully adherent to the psychiatrist recommendations more than half the time, a level that is clinically acceptable. Adherence increased over time, which may be more reflective of the timing of the follow-up PCP visit.

We also found no differences in the types of psychiatrist recommendations between ATP and STP. The use of ATP did not alter or impoverish the recommendations made by the psychiatrist or the clinical management decisions made by the PCP and patient. Most notably, our data suggest that psychiatrists felt comfortable using ATP to make medication recommendations without having a real-time interaction with the patient; and likewise, PCPs felt comfortable acting on those recommendations. Other studies have shown ATP to have additional benefits that may extend beyond PCP adherence such as the ability to postprocess the captured video with translation from Spanish to English.^21,22,24 Furthermore, the parent study did not find ATP to be superior in terms of more clinically relevant outcomes such as the clinician-rated Clinical Global Impressions scale and patient-rated Patient Health Questionnaire-9.²⁵ This lack of superiority may be related to the lack of differences in the upstream PCP adherence. PCP adherence has been previously described as being associated with improved psychiatric symptoms.³³

There is limited research investigating PCP adherence to psychiatrist recommendations in coordinated care and few studies have investigated PCP adherence in relation to consultation modality using e-health technologies. In 2016, Burian et al. found an increase in PCP adherence using a “warm”/live telephone handoff in comparison with a written discharge note and with written summary of the psychiatric consultation in addition to the usual discharge note.³³ Although our study was inconclusive with respect to PCP adherence between ATP and STP modalities, the contexts between these two studies differed. In our study, the PCP was engaging in a consultation rather than a referral; the PCP was the agent in charge of managing the care. In the Burian study, the patients were being discharged from an inpatient stay in which care was being resumed by the PCP. In 2017, Lowenstein et al. described the content of the first 50 psychiatric e-consults requested by outpatient PCPs at a large tertiary care center but did not compare the adherence with a control arm.³⁶

Our study found lower proportions of full adherence for ATP (6 weeks: 46%, 6 months: 56%) and STP (6 weeks: 49%, 6 months: 58%) compared with previous studies that have found relatively higher rates of adherence to specialist recommendations. For example, Lowenstein et al.³⁶ reported 76% adherence and Burian et al. reported 79% adherence for medication and 66% adherence for psychotherapeutic recommendations.³³ Of note, our study also had stricter definitions of measuring adherence than other studies and relied on multiple raters rather than single raters. Similar to non-e-health studies in psychiatry⁴² and other specialties,^29,53,54 adherence was higher for medication recommendations than diagnostic recommendations and psychotherapeutic recommendations. This may reflect the difficulties of integrating psychotherapy such as cognitive behavioral therapy into a busy primary care practice or the lack of PCP comfort in conducting such interventions.^55
–57

LIMITATIONS

Our study assumed that the recommendations made by the psychiatrist were “correct” and needed to be discussed with the patient and implemented by the PCP. Adherence is also, by nature, a subjective measure despite our efforts to utilize previously operationalized measures. In addition, the EMR may inaccurately document the actions the PCP conducted in the visit. The way in which we split complex recommendations may have overestimated the adoption of recommendations. Furthermore, this study may have benefitted from a larger sample size to detect a smaller difference in adherence. This study may also have limited generalizability. The academic center has a long history of delivering clinical services and researching telemedicine. However, telemedicine is still gaining traction in the outpatient setting and only certain physicians and patients may be interested in this health care delivery model. Although this should not affect the internal validity of the study, this may limit the acceptance and implementation of a similar model elsewhere. Additional research is needed to evaluate the effectiveness of ATP in additional patient populations. Finally, this study could blind neither the psychiatrist, PCP, nor patient to the intervention.

Conclusions

Our findings support the feasibility and acceptability of ATP and STP for both PCPs and psychiatrists. We were unable to determine superiority for either arm. Regardless, this study supports both modes as viable technological and clinical processes for the provision of collaborative psychiatric care.

Footnotes

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ, NCATS, NIH, or University of California.

Acknowledgments

The authors indicate this work was presented as follows: (1) “The Impact of Telemedicine on Psychiatry Consultations.” Presented at the Western Medical Research Conference, January 25, 2020, Carmel, CA (Oral presentation) and (2) “Characterization of Telepsychiatry Consultation Recommendations.” Presented at the National Telehealth Research Symposium, September 12, 2019, Chicago, IL (Oral presentation).

Disclosure Statement

No competing financial interests exist.

Funding Information

This study was supported by the U.S. Department of Health and Human Services’ Agency for Healthcare Research and Quality (AHRQ), award R01HS024949-01. Dr. Lieng was supported by the University of California Dissertation Year Fellowship and by the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health (NIH), through grant number UL1 TR001860 and linked award TL1TR001861.

Supplementary Material

Supplementary Appendix SA1

Supplementary Appendix SA2

References

Hilty

, Ferrer

, Parish

, Johnston

, Callahan

, Yellowlees

. The effectiveness of telemental health: A 2013 review. Telemed J E Health, 2013; 19:444–454.

Hubley

, Lynch

, Schneck

, Thomas

, Shore

. Review of key telepsychiatry outcomes. World J Psychiatry, 2016; 6:269–282.

Naslund

, Mitchell

, Joshi

, Nagda

, Lu

. Economic evaluation and costs of telepsychiatry programmes: A systematic review. J Telemed Telecare. 2020 [Epub ahead of print]; DOI: 10.1177/1357633X20938919.

Sunjaya

, Chris

, Novianti

. Efficacy, patient-doctor relationship, costs and benefits of utilizing telepsychiatry for the management of post-traumatic stress disorder (PTSD): A systematic review. Trends Psychiatry Psychother, 2020; 42:102–110.

Guaiana

, Mastrangelo

, Hendrikx

, Barbui

. A systematic review of the use of telepsychiatry in depression. Community Ment Health J, 2021; 57:93–100.

Hassan

, Sharif

. Efficacy of telepsychiatry in refugee populations: A systematic review of the evidence. Cureus, 2019; 11:e3984.

Salmoiraghi

, Hussain

. A systematic review of the use of telepsychiatry in acute settings. J Psychiatr Pract, 2015; 21:389–393.

Spivak

, Spivak

, Cullen

, et al. Telepsychiatry use in U.S. Mental Health Facilities, 2010–2017. Psychiatr Serv, 2019; 71:121–127.

Freeman

, Boggs

, Zachrison

, et al. National study of telepsychiatry use in U.S. Emergency Departments. Psychiatr Serv, 2021; 71:540–546.

10.

Colle

, Ait Tayeb

AEK

, de Larminat

, et al. Telepsychiatry in the post-COVID-19 era: Moving backwards or forwards?. Psychother Psychosom, 2021; 90:69–70.

11.

Malla

, Joober

. COVID-19 and the future with digital mental health: Need for attention to complexities. Can J Psychiatry, 202; 66:14–16.

12.

Di Carlo

, Sociali

, Picutti

, et al. Telepsychiatry and other cutting-edge technologies in Covid-19 pandemic: Bridging the distance in mental health assistance. Int J Clin Pract, 2021; 75:e13716.

13.

Chen

, Chung

W-J

, Young

, et al. COVID-19 and telepsychiatry: Early outpatient experiences and implications for the future. Gen Hosp Psychiatry, 2020; 66:89–95.

14.

Melamed

, Hahn

, Agarwal

, Taylor

, Mulsant

, Selby

. Physical health among people with serious mental illness in the face of COVID-19: Concerns and mitigation strategies. Gen Hosp Psychiatry, 2020; 66:30–33.

15.

Lipschitz

, Hogan

, Bauer

, Mohr

. Closing the research-to-practice gap in digital psychiatry: The need to integrate implementation science. J Clin Psychiatry, 2019; 80:18com12659.

16.

Odor

, Yellowlees

, Hilty

, Parish

, Nafiz

, Iosif

A-M

. PsychVACS: A system for asynchronous telepsychiatry. Telemed J E Health, 2011; 17:299–303.

17.

Hilty

, Yellowlees

, Nesbitt

. Evolution of telepsychiatry to rural sites: Changes over time in types of referral and in primary care providers’ knowledge, skills and satisfaction. Gen Hosp Psychiatry, 2006; 28:367–373.

18.

O'Keefe

, White

, Jennings

. Asynchronous telepsychiatry: A systematic review. J Telemed Telecare, 2021; 27:137–145.

19.

Yellowlees

, Odor

, Parish

, Iosif

A-M

, Haught

, Hilty

. A feasibility study of the use of asynchronous telepsychiatry for psychiatric consultations. Psychiatr Serv Wash DC, 2010; 61:838–840.

20.

Balasinorwala

, Shah

, Chatterjee

, Kale

, Matcheswalla

. Asynchronous telepsychiatry in maharashtra, India: Study of feasibility and referral pattern. Indian J Psychol Med, 2014; 36:299–301.

21.

Yellowlees

, Odor

, Parish

. Cross-lingual asynchronous telepsychiatry: Disruptive innovation?. Psychiatr Serv, 2012; 63:945.

22.

Yellowlees

, Odor

, Iosif

A-M

, et al. Transcultural psychiatry made simple—Asynchronous telepsychiatry as an approach to providing culturally relevant care. Telemed J E Health, 2013; 19:259–264.

23.

Butler

, Yellowlees

. Cost analysis of store-and-forward telepsychiatry as a consultation model for primary care. Telemed J E Health, 2012; 18:74–77.

24.

Barrera-Valencia

, Benito-Devia

, Vélez-Álvarez

, Figueroa-Barrera

, Franco-Idárraga

. Cost-effectiveness of synchronous vs. asynchronous telepsychiatry in prison inmates with depression. Rev Colomb Psiquiatr , 2017; 46:65–73.

25.

Yellowlees

, Parish

, Gonzalez

, et al. Clinical outcomes of asynchronous versus synchronous telepsychiatry in primary care: Randomized controlled trial. J Med Internet Res, 2021; 23:e24047.

26.

Gopalan

, Shenai

, Dunn

, Bilderback

. Healthcare utilization in patients pre-and post-telepsychiatry consultation compared to in-person consultation-liaison sites. Gen Hosp Psychiatry, 2020; 67:154–155.

27.

Olfson

, Kroenke

, Wang

, Blanco

. Trends in office-based mental health care provided by psychiatrists and primary care physicians. J Clin Psychiatry, 2014; 75:247–253.

28.

Cerimele

, LePoire

, Fortney

, Hawrilenko

, Unützer

, Bauer

. Bipolar disorder and PTSD screening and telepsychiatry diagnoses in primary care. Gen Hosp Psychiatry, 2020; 65:28–32.

29.

Sellier

, Pavese

, Gennai

, Stahl

J-P

, Labarère

, François

. Factors and outcomes associated with physicians’ adherence to recommendations of infectious disease consultations for inpatients. J Antimicrob Chemother, 2010; 65:156–162.

30.

Fariñas

M-C

, Saravia

, Calvo-Montes

, et al. Adherence to recommendations by infectious disease consultants and its influence on outcomes of intravenous antibiotic-treated hospitalized patients. BMC Infect Dis, 2012; 12:292.

31.

Vehreschild

, Morgen

, Cornely

, et al. Evaluation of an infectious disease consultation programme in a German tertiary care hospital. Infection, 2013; 41:1121–1128.

32.

Marques

, Calderaro

, Yu

, et al. Impact of cardiology referral: Clinical outcomes and factors associated with physicians’ adherence to recommendations. Clin Sao Paulo Braz, 2014; 69:666–671.

33.

Burian

, Franke

, Diefenbacher

. Crossing the bridge—A prospective comparative study of the effect of communication between a hospital based consultation-liaison service and primary care on general practitioners’ concordance with consultation-liaison psychiatrists’ recommendations. J Psychosom Res, 2016; 86:53–59.

34.

Georgeton

, Aubert

, Pierrard

, Gaborieau

, Berrut

, de Decker

. General practitioners adherence to recommendations from geriatric assessments made during teleconsultations for the elderly living in nursing homes. Maturitas, 2015; 82:184–189.

35.

Wrenn

, Catschegn

, Cruz

, Gleason

, Gonzales

. Analysis of an electronic consultation program at an academic medical centre: Primary care provider questions, specialist responses, and primary care provider actions. J Telemed Telecare, 2017; 23:217–224.

36.

Lowenstein

, Bamgbose

, Gleason

, Feldman

. Psychiatric consultation at your fingertips: Descriptive analysis of electronic consultation from primary care to psychiatry. J Med Internet Res, 2017; 19:e279.

37.

Fortney

, Pyne

, Turner

, et al. Telepsychiatry integration of mental health services into rural primary care settings. Int Rev Psychiatry Abingdon Engl, 2015; 27:525–539.

38.

Flodgren

, Rachas

, Farmer

, Inzitari

, Shepperd

. Interactive telemedicine: Effects on professional practice and health care outcomes. Cochrane Effective Practice and Organisation of Care Group, ed. Cochrane Database Syst Rev , 2015; 15:CD002098–CD002583.

39.

Hilty

, Torous

, Parish

, et al. A literature review comparing clinicians’ approaches and skills to in-person, synchronous, and asynchronous care: Moving toward competencies to ensure quality care. Telemed J E-Health, 2021; 27:356–373.

40.

University of

California

, Davis. A Controlled Trial of Patient Centered Telepsychiatry

Interventions

. clinicaltrials.gov; 2019. Available at https://clinicaltrials.gov/ct2/show/NCT02084979 (last accessed September 21, 2020 ).

41.

Yellowlees

, Burke Parish

, González Á, et al. Asynchronous telepsychiatry: A component of stepped integrated care. Telemed J E Health, 2018; 24:375–378.

42.

Popkin

MK.

Consultation-Liaison Outcome Evaluation System. Arch Gen Psychiatry, 1983; 40:215.

43.

Seward

, Smith

, Stuart

. Concordance with recommendations in a consultation-liaison psychiatry service. Aust N Z J Psychiatry, 1991; 25:243–254.

44.

Harris

, Taylor

, Minor

, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform, 2019; 95:103208.

45.

Hallgren

KA.

Computing inter-rater reliability for observational data: An overview and tutorial. Tutor Quant Methods Psychol, 2012; 8:23–34.

46.

Cicchetti

DV.

Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess, 1994; 6:284–290.

47.

Faul

, Erdfelder

, Lang

A-G

, Buchner

. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods, 2007; 39:175–191.

48.

Austin

, Merlo

. Intermediate and advanced topics in multilevel logistic regression analysis. Stat Med, 2017; 36:3257–3277.

49.

O'brien

RM.

A caution regarding rules of thumb for variance inflation factors. Qual Quant, 2007; 41:673–690.

50.

R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing; 2020. Available at https://www.R-project.org/ (last accessed on July 24, 2021 ).

51.

Stata Statistical Software: Release 15. StataCorp LLC, 2017.

52.

Schulz

, Altman

, Moher

. CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol, 2010; 63:834–840.

53.

Sellier

, Labarere

, Gennai

, Bal

, Francois

, Pavese

. Compliance with recommendations and clinical outcomes for formal and informal infectious disease specialist consultations. Eur J Clin Microbiol Infect Dis, 2011; 30:887–894.

54.

Pavese

, Sellier

, Laborde

, Gennai

, Stahl

J-P

, François

. Requesting physicians’ experiences regarding infectious disease consultations. BMC Infect Dis, 2011; 11:62

55.

Zhang

, Borhneimer

, Weaver

, et al. Cognitive behavioral therapy for primary care depression and anxiety: A secondary meta-analytic review using robust variance estimation in meta-regression. J Behav Med, 2019; 42:1117–1141.

56.

Dorflinger

, Fortin

, Foran-Tuller

. Training primary care physicians in cognitive behavioral therapy: A review of the literature. Patient Educ Couns, 2016; 99:1285–1292.

57.

Cameron

, Habert

, Anand

, Furtado

. Optimizing the management of depression: Primary care experience. Psychiatry Res, 2014; 220 Suppl 1:S45–S57.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.23 MB

0.00 MB