Primary Care Practitioner Training in Child and Adolescent Psychiatry (PTCAP): A Cluster-Randomized Trial

Abstract

French

Objectives

Rural primary care practitioners (PCPs) have a pivotal role to play in frontline pediatric mental health care, given limited options for referral and consultation. Yet they report a lack of adequate training and confidence to provide this care. The aim of this study was to test the effectiveness of the Practitioner Training in Child and Adolescent Psychiatry (PTCAP) program, which was designed to enhance PCPs’ pediatric mental health care confidence. The program includes brief therapeutic skills and practice guidelines PCPs can use to address both subthreshold concerns and diagnosable conditions, themselves.

Methods

The study design was a pilot, cluster-randomized, multicenter trial. Practices were randomly assigned to intervention (n practices = 7; n PCPs = 42) or to wait-list control (n practices = 6; n PCPs = 34). The intervention involved 8 hr of training in practice guidelines and brief therapeutic skills for depression, anxiety, attention deficit hyperactivity disorder, and behavioral disorders with case discussion and video examples, while the control practiced as usual. A linear random-effects model controlling for clustering and baseline was carried out on the individual-level data to examine between-group differences in the primary (i.e., confidence) and secondary (i.e., attitude and knowledge) outcomes at 1-week follow-up.

Results

Findings were a statistically significant difference in the primary outcomes. Compared to the control group, the intervention group indicated significantly greater confidence in managing diagnosable conditions (d = 1.81) and general concerns (d = 1.73), as well as in making necessary referrals (d = 1.27) and obtaining consults (d = 0.74). While the intervention did not significantly impact secondary outcomes (attitudes and knowledge), regression analysis indicated that the intervention may have increased confidence, in part, by ameliorating the adverse impact of negative mental health care attitudes.

Conclusion

PTCAP enhances PCPs’ child/youth mental health care confidence in managing both general and diagnosable concerns. However, an 8-hr session focused on applying brief therapeutic skills was insufficient to significantly change attitudes and knowledge. Formal testing of PTCAP may be warranted, perhaps using more intensive training and including outcome assessments capable of determining whether increased PCP confidence translates to more effective management and better patient outcomes.

Keywords

primary care practitioner rural pediatric mental health confidence

Integrated care models that combine mental health services with primary care have a critical role to play in addressing the gap between the high demand for pediatric mental health support in Canada and the lack of accessible and affordable specialist services.¹ These models are especially critical in rural and remote communities where specialist mental health resources are scarce and primary care practitioners (PCPs; e.g., family physicians, pediatricians, and nurse practitioners) must, therefore, play a central role in frontline care provision.² Yet, despite this critical role, families who turn to their family physician for mental health can feel dismissed and unheard.^3,4

–8 Moreover, rural PCPs report lacking the confidence and adequate training to provide this care themselves.^9,10

There may be an association between PCPs’ mental health care confidence and the actual care they provide. For example, Miller and colleagues conducted a survey of a representative sample of family physicians in British Columbia and found that PCPs who report less comfort/skills with pediatric mental health care provision are more likely to report referring out.¹¹ However, a reliance on specialist referrals in rural primary care increases the burden on families in terms of wait times and in terms of costs associated with specialist services and travel to access care in urban centers. Critically, Miller et al. also found that practitioners with previous mental health care training and more positive attitudes toward mental health care provision are more likely to provide care themselves.

Teaching, training, and other forms of mental health knowledge translation lie along a continuum of integrated care approaches, from minimal to full integration,¹² and provide important avenues through which primary care providers and other nonspecialists come to confidently act as first responders in the provision of frontline mental health care. One example of an integrated care approach is the Chronic Care Model (CCM), which has provided the framework for most previous mental health training efforts in primary care. The model emphasizes PCPs’ adoption of evidence-based practice guidelines and intervention protocols for managing specific, diagnosable conditions.¹³ This approach is consistent with a “medical model” in which an illness is first diagnosed and then treated with an intervention targeted to the specific condition and level of severity.

Despite some success of the CCM in managing diagnosable conditions, including major depression in adults,¹³ and despite ready access to disorder-specific practice guidelines and treatment protocols online,¹⁴ PCPs (even those trained as specialist pediatricians) continue to report a lack of confidence and skills to manage pediatric mental health concerns in their practices.^9,15
–17 One reason may be that pediatric CCM models, which have relied on co-located mental health professionals,^1,18 may not be feasible in smaller rural settings with limited resources. Also, a medical approach may not translate easily to the pediatric mental health care context where the majority of presenting concerns are emergent, comorbid, and/or subthreshold, yet clinically significant, as they may well be disrupting life at home and school.¹⁹ Given these challenges, PCPs may need an approach that allows them to build confidence in managing and supporting children and families themselves when specialist resources are unavailable, a single diagnostic label does not apply, families are not receptive to a diagnosis-driven approach, or they are waiting on a diagnosis or referral.^20
–22

Common factors theory proposes that the personal and interpersonal aspects of therapy (e.g., therapeutic alliance and client engagement and expectations for positive outcomes) account for a large part of mental health treatment outcomes, distinct from the specific diagnosis and diagnosis-targeted therapy. Estimates are in the range of 30% to 70% of patient outcomes that are accounted for by these common factors across diverse diagnoses.^23
–25 In other words, these common factors themselves provide a large part of the benefits of specialist psychotherapy interventions and can be leveraged to engage families, provide hope, and promote positive mental health outcomes across a range of symptom clusters, prior to, or without, a formal diagnosis. Common factors theory proposes that PCPs can be taught to use common skills and common elements to bring about these factors and enhance outcomes.^22,25
–27

Common skills include interviewing and counseling techniques derived from solution-focused, family and motivational interviewing therapies that are broadly applicable across mental health symptoms, such as mutually deciding on an action plan to address concerns and addressing barriers to plan implementation.^22,26 Unlike common skills, which PCPs can use themselves to engage children and families in a mental health conversation, common elements are brief therapeutic techniques common to multiple, complex interventions targeting the same or related diagnoses, which PCPs can teach parents to use. Common elements can be applied broadly, not only to address a specific diagnosis, but also to address symptoms that tend to occur across diagnoses (e.g., tangible rewards or parent praise applied to both attention/hyperactivity problems and anxiety problems).²⁸

Taken together, these common factors skills and elements can be considered broad-based therapeutic techniques that can be matched to child and parent concerns, delivered prior to completion of a formal diagnostic process,²⁹ and potentially feasibly applied within a primary care medical visit.^22,26,30,31 While the efficacy of common skills training for improving pediatric mental health outcomes and processes in primary care has been demonstrated in a series of studies from the same research group,^{22,26,32
–34} further research is needed to determine whether training in common factors skills and elements, more broadly, is an effective approach to integrating pediatric mental health in primary care.

Brown and Wissow have proposed a framework for improving PCPs’ mental health care practice and confidence through training that combines common factors skills and elements with disorder-specific practice guidelines.²⁷ Our team carried out a national needs assessment that supports the use of such an approach. Of 847 rural family physicians and pediatricians surveyed, 89% indicated a lack of confidence and skills to provide mental health care for children and youth and a desire for training in both clinical skills and practice guidelines.^9,10 Based on this national needs assessment, as well as a focus group conducted with a family practice team in Southwestern Ontario, feedback from a national collaborative of mental health professionals,³⁵ the aforementioned training framework, evidence in support of common skills training for PCPs and evidence behind practice guidelines,^{22,26,32
–34} we designed a stakeholder- and evidence-informed training program for rural PCPs (i.e., Practitioner Training in Child and Adolescent Psychiatry; PTCAP).³⁵ From an integrated knowledge translation perspective,³⁶ engaging stakeholders throughout the research process in this way is likely to enhance program uptake and effectiveness. Program materials are freely available online to registered users.³⁷

Improvement in pediatric (under age 18 years) mental health care confidence was the primary endpoint of this study, which we operationalized as self-reported confidence in managing diagnosable conditions and more general concerns, as well as in making referrals and obtaining consults, when necessary. We hypothesized that as compared to PCPs randomized to the wait-list control arm of the study (treatment-as-usual), PCPs in the PTCAP intervention arm would demonstrate increased confidence in managing concerns themselves using common factors skills and elements for general concerns, and practice guidelines for diagnosable conditions. Given that positive mental health care attitudes are associated with PCPs’ mental health care confidence,¹¹ we also hypothesized that the impact of training on post-intervention confidence would depend on baseline attitudes.

Methods

Setting and Sample

The protocol for the PTCAP evaluation trial has previously been published.³⁸ Here we provide methodological details of the trial pertinent to this particular study focused on the self-report outcome measures.

We attempted to inform all rural (100,000 or fewer residents, outside of large metropolitan areas, with limited access to specialized services)³⁹ family health teams (i.e., primary health care teams) in Southwestern Ontario about the opportunity to participate in the PTCAP intervention and evaluation. Using a modified Dillman technique,⁴⁰ family health teams listed on the Ministry of Health and Long-Term Care website⁴¹ were contacted at least 3 times, first by phone and then by email. Based on an Internet search and referrals from other practices and family doctors, rural community health centers and private practices were also invited to participate (all health teams/centers/practices are referred to as “practices” from here on in). Practices were recruited from September 2014 to March 2015.

Participants

Inclusion and exclusion criteria

While PCPs (family physicians, pediatricians, residents, and nurse practitioners), social workers, and mental health leads were invited to participate in the program, only PCPs completed the evaluation.

Power

In order to detect a significant difference in our primary outcomes should one exist, given a moderate effect size of d = 0.40 (estimated based on Ref. ⁴²) and 80% power to detect an effect with a standard deviation of 0.5, we aimed for a total sample of 50 PCPs.⁴³ Adjusting for clustering using an inflation rate of approximately 15%, our target sample size was 56 PCPs, consistent with the target sample size of a common skills training trial for PCPs demonstrating significant effects.^22,26 Recruitment ceased once we reached this target.

Procedures

We conducted a stratified, cluster randomization at the practice level to avoid contamination within sites. Practices were grouped into one of three geographical locations within Southwestern Ontario, that is, (1) North, (2) South East, and (3) South West, and then randomized (1:1) within each geographic stratum, with approximately 1/3 of the clusters allocated to each stratum. Practice identity was concealed by S.G. and an online program⁴⁴ was used to generate a random list to which practices were allocated by S.E. The primary contact for each practice site was informed of their intervention status.

Interventions

PCPs in the intervention received the PTCAP program. PTCAP is the first mental health education program in Canada based on the expressed training needs and preferences of rural PCPs. Our national needs assessment of PCPs indicated a need for training in attention deficit hyperactivity disorder (ADHD), behavioral disorders, depression, and anxiety, and a preference for onsite delivery by a child and adolescent psychiatrist and rural family physician pair.⁹ The program includes 8 hr of mixed, didactic, and interactive training, including video examples of how to apply common factors skills and elements, and practice guidelines. Common factors training was designed to enhance PCPs’ confidence for addressing general mental health concerns using common skills and elements to initiate and engage families in discussing mental health concerns, to mutually formulate an action plan to address concerns and barriers to care, and to provide hope and relief of symptoms prior to a formal diagnosis. Training in practice guidelines is designed to increase PCPs’ confidence in assessing, diagnosing, and managing (e.g., providing brief psychotherapy and pharmacotherapy) diagnosable conditions in children and youth. Region-specific resources are provided as part of the guidelines to support consultation and referral, and family resources are provided to foster self-help (see Figure 1). PTCAP training sessions were scheduled according to site preference to maximize participation. Three different physician–psychiatrist pairs equally delivered the PTCAP program.

Figure 1.

Practitioner Training in Child and Adolescent Psychiatry curriculum outline and implementation schedule for 2-day delivery format.

PCPs in the control arm were encouraged to deliver care as usual, which includes assessing and managing mental health issues brought to the practitioner’s attention through psychoeducation, pharmacological intervention, and/or referral to specialists. Control practices were offered the PTCAP intervention following study completion.

Outcome Measures

Outcomes were assessed at the practitioner level. Data were collected from control sites according to the data collection schedule of the intervention site they were matched with in terms of location and number of participants. Baseline surveys were mailed to the intervention and matched control sites, which participants were asked to complete 1 week prior to receiving the intervention. Participants were provided with the post-intervention survey in person or by mail, which they were asked to complete 1 week following the intervention and to return via a pre-stamped, self-addressed envelope. Missing surveys were picked up in person or collected by email or phone.

Primary outcome measures

Two instruments were used to measure PCP confidence in providing mental health care themselves for (1) general concerns and (2) diagnosable conditions. The Provider Confidence Scale (Pr-CS)^45,46 is an 11-item scale rated from 1 (not at all comfortable) to 5 (comfortable) that measures provider comfort with managing general mental health concerns in school-aged children and adolescents, themselves, and counseling parents in relation to these concerns, consistent with the use of common factors skills and elements. The Physician Confidence Scale (PCS) is an 8-item scale that assesses provider confidence in assessing, managing, and referring specific, diagnosable conditions in children and youth on a scale of 1 (low) to 5 (high).⁴⁷ Consistent with the use of practice guidelines, items assess confidence assessing mental health and suicidal risk, providing pharmacological treatment, and directing patients at risk to emergency.

PCP confidence in obtaining consults and referrals as needed was also measured. The Ease of Consultation Scale (ECS) is an 8-item scale assessing perceived difficulty obtaining a mental health consultation (e.g., with a social worker, clinical psychologist) rated from 1 (not available), 2 (with great difficulty) to 5 (extremely easy). The Referral Comfort Scale (RCS) includes 4 items rated from 1 (not at all comfortable) to 5 (extremely comfortable) that assess PCP comfort referring children (6 to 11 years) to specialists.

Three of the aforementioned primary outcome measures (i.e., Pr-CS, ECS, and RCS) have previously been validated in a convenience sample of pediatricians and have shown good internal consistency (αs = .81 to .95).⁴⁶ The PCS also demonstrates high reliability (α = .88), and scores on the measure improve with mental health training, indicating validity.⁴⁵

Secondary outcomes

PCP attitude was measured, in part, to understand intermediate processes in a hypothesized causal pathway connecting mental health care training with confidence in providing this care. The Physician Belief Scale (PBS) is a 14-item measure of physician’s attitudes toward treating mental health problems in primary care rated from 1 (disagree) to 5 (agree), with lower scores indicating more positive attitudes (e.g., the provision of mental health care by PCPs is seen as feasible and useful).⁴⁸ The validity of the scale has been established, and lower scores are associated with physicians’ empathic and involved conversations with their patients.⁴⁹

The Practitioner Mental Health Knowledge Scale (PMHKS) is a 45-item, true (T)/false (F) measure designed specifically for this study due to a lack of relevant existing measures. The scale measures knowledge of common factors skills (e.g., “A key predictor of treatment outcomes is the therapeutic alliance with the practitioner” [T].) and practice guidelines (e.g., “When using the ‘SNAP’ scale to screen for ADHD, it is recommended to have only the parents complete this scale” [F]).

Data Analysis

The data were analyzed with SPSS Version 20.0. An intent-to-treat analysis was used such that all PCPs were included and analyzed by allocated arm, regardless of whether they received the full intervention or whether they completed all measures. Baseline characteristics were examined and compared between groups using χ² tests for categorical variables and analysis of variance for continuous variables.

To test our hypothesis that the PTCAP intervention would improve mental health care confidence using common factors skills and practice guidelines, we carried out a linear mixed models with random intercepts to accommodate for clustering at the practice level. That is, the data were analyzed at the individual level accounting for the clustering of practitioners within practices. The effects of the intervention on the primary and secondary outcomes were assessed by examining whether the main effect of intervention was statistically significant after controlling for baseline confidence. Increases reflected positive changes for all indicators, except for increases on the PBS, which reflected a negative change. One participant was missing data on all measures, and another three participants were missing data only on one measure at T1. Available data for these participants were included in the analyses. Attrition at follow-up was 20%.

To test our hypothesis that baseline attitudes would moderate the main effects of intervention on mental health care confidence, two stepwise linear regression models were carried out regressing confidence (Pr-CS and PCS) on baseline attitudes (PBS), intervention status, and on the interaction term, using forced entry.

Results

We reached 28 of 49 eligible practices identified, for which we found current contact information, with 22 (79%) practice leads expressing interest in participating. Of these 22 practices, 13 participated (11 family health teams, 1 private practice, and 1 nurse practitioner clinic) (see Figure 2 for flow of practices/participants through the trial).

Figure 2.

Consort diagram: Flow of participants through the trial.

Of the 13 practices that agreed to participate by the trial start date, 7 were randomized to the intervention arm of the study. Four received the intervention as one full-day session and three received two half-day sessions. All but one participant completed the intervention. Achieving our target sample size, a total of 76 PCPs participated in the intervention sessions including 59 family physicians/pediatricians/residents and 17 nurse practitioners. A range of 3 to 9 PCPs participated per practice ( $\bar{x}$ = 6).

Outcomes

With the exception of gender, no marked imbalances in distribution of baseline variables were found (see Table 1 for baseline characteristics). Therefore, gender was included as a covariate in the following regression models, only when including gender-improved model fit (i.e., reduced the Akaike Information Criteria and degrees of freedom.

Table 1.

Baseline Characteristics.

Variables	Interventionn (%/SD)	Controln (%/SD)	χ² (df)
Age range			1.98 (2)
20 to 39 years	18 (42.9)	11 (32.4)
40 to 59 years	18 (42.9)	20 (58.8)
60 or more years	6 (14.3)	3 (8.8)
Ethnicity			1.24 (2)
Caucasian	33 (78.6)	30 (88.2)
Aboriginal, African, Asian, West Indian, Other	9 (21.4)	4 (11.9)
Professional degree			0.57 (2)^a
Medical doctor	31 (73.9)	23 (71.9)
Nurse practitioner	10 (23.8)	8 (25.0)
Doctor of osteopathic medicine	1 (2.4)	1 (3.1)
Time in practice			1.40 (2)
<5 years	23 (54.8)	14 (41.2)
5 to 9 years	7 (16.7)	7 (20.6)
10 or more years	12 (28.6)	13 (38.2)
Previous mental health training			2.09 (2)
Yes	15 (35.7	7 (20.6)
No	27 (64.3)	27 (79.4)
% Practice, children/youth			0.13 (2)
≤30%	37 (88.1)	29 (85.3)
>30%	5 (11.9)	5 (14.7)
% Practice time, mental health care			1.25 (2)
≤30%	42 (100.0)	33 (0.0)
>30%	0 (0.0)	1 (2.9)
Gender			6.99 (2)**
Male	21 (50.0)	7 (20.6)
Female	21 (50.0)	27 (79.4)

^a Data missing from two control arm participants.

**P < 0.01.

Results of the mixed linear regressions revealed significant main effects of the PTCAP intervention on the primary outcome of interest. Consistent with our hypothesis, compared to those in the wait-list control, PCPs in the intervention arm of the study indicated enhanced confidence to address general concerns (Pr-CS: $\bar{x}$ intervention = 3.72 (0.52); $\bar{x}$ control = 2.55 (0.83), P < 0.001; Cohen’s d = 1.73) and diagnosable conditions (PCS: $\bar{x}$ intervention = 4.21 (0.47); $\bar{x}$ control = 3.16 (0.69), P < 0.001; Cohen’s d = 1.81; see Table 2). In addition, we found a significant main effect of the PTCAP intervention on PCP confidence in making referrals (RCS: $\bar{x}$ intervention = 3.97 (0.69); $\bar{x}$ control = 3.04 (0.77); Cohen’s d = 1.27) and in obtaining consults (ECS: $\bar{x}$ intervention = 3.14 (0.66); $\bar{x}$ control = 2.72 (0.48); Cohen’s d = 0.74).

Table 2.

Mean Raw Scores and Standard Deviations at Baseline and Follow-Up for Intervention and Control.

Measures	Training	Baseline Mean (SD)	Follow-Up (SD)	Estimate^a	df	SE	d
Confidence using common factors skills and elements for general concerns (Provider Confidence Scale)	Intervention	2.96 (0.64)	3.72 (0.52)	0.64***	9.95	0.12	1.73
	Control	2.62 (0.74)	2.55 (0.83)	0.64***	9.95	0.12	1.73
Confidence using practice guidelines for diagnosable conditions (Physician Confidence Scale)	Intervention	3.38 (0.64)	4.21 (0.47)	0.95***	9.23	0.14	1.81
	Control	3.10 (0.71)	3.16 (0.69)	0.95***	9.23	0.14	1.81
Referral Comfort Scale	Intervention	3.35 (0.67)	3.97 (0.69)	0.75***	57.00	0.15	1.27
Referral Comfort Scale	Control	3.09 (0.86)	3.04 (0.77)	0.75***	57.00	0.15	1.27
Ease of Consultation Scale	Intervention	2.96 (0.58)	3.14 (0.66)	0.23*	56.00	0.12	0.74
Ease of Consultation Scale	Control	2.76 (0.40)	2.72 (0.48)	0.23*	56.00	0.12	0.74
Attitudes towards mental health care (Physician Belief Scale)^b	Intervention	1.95 (0.63)	1.85 (0.40)	−0.21^†	55.00	0.13	0.28
	Control	2.04 (0.52)	2.01 (0.73)	−0.21^†	55.00	0.13	0.28
Knowledge: CFS and practice guidelines (Practitioner Mental Health Knowledge Scale)	Intervention	31.48 (3.69)	32.64 (4.11)	0.84	8.59	1.09	0.23
	Control	31.24 (5.38)	31.63 (4.63)	0.84	8.59	1.09	0.23

Note. df = degrees of freedom; SE = standard error; CI = 95% confidence intervals for main effect of intervention on follow-up scores (controlling for baseline); d = effect size.

^a Estimate represents test of between-group difference at follow-up, holding baseline constant.

^b Lower scores represent more positive attitudes.

^†P < 0.10. *P < 0.05. ***P < 0.001.

The effect of the intervention on attitudes (PBS) and knowledge (PMHKS) was not significant, although there was a trend toward improved attitudes for the intervention group (PBS: $\bar{x}$ intervention = 1.85 (0.40); $\bar{x}$ control = 2.01 (0.73); Cohen’s d = 0.28).

Supporting our hypothesis that baseline attitudes would moderate the impact of training on mental health care confidence, results of the stepwise linear regression revealed a significant interaction between baseline mental health care attitudes and intervention status on confidence to provide care for specific diagnoses (PCS), β = 0.81, P < 0.01. Somewhat unexpectedly, baseline attitudes influenced post-intervention confidence only for the control, not the intervention group (Figure 3). In contrast, confidence in addressing general mental health concerns (Pr-CS) was not affected by baseline attitudes (β = −0.27, P < ns); in this case, after controlling for baseline confidence, only the intervention effect was significant (β = 1.65, P < 0.05).

Figure 3.

Effect of intervention status on the association between baseline attitudes (higher scores = more negative attitudes) and confidence using practice guidelines at follow-up (Physician Confidence Scale).

Discussion

The goal of this study was to examine the effectiveness of the PTCAP intervention for improving rural PCPs’ mental health care confidence. Findings support the effectiveness of the intervention in terms of enhancing PCPs’ confidence in addressing general concerns and in managing diagnosable conditions. Findings also support the efficacy of the intervention for improving PCPs’ confidence in making referrals and in obtaining consultations, as necessary. Our findings add to a growing body of research that supports educational innovations aimed at fostering common factors skills for enhancing PCP mental health care confidence and practice^{22,25,26,32
–34} This study also extends past findings by providing support for educational initiatives that combine training in common factors skills with common elements and targeted practice guidelines, in line with the education framework outlined by Brown et al.²⁷

Although the main effects of the PTCAP intervention on the secondary outcomes of interest were not significant, there was a positive shift in attitudes for the intervention group with a small to medium effect (d = 0.28). Also, we found an interaction between baseline attitudes and intervention assignment such that baseline attitudes determined post-intervention confidence only for the control group. One possibility is that training undermined the association between positive baseline attitudes for the intervention group and confidence at follow-up. Assuming that the control group represents the state of the intervention group before training, it seems more likely, however, that the intervention actually “ameliorated” the previously observed adverse impact of negative baseline mental health care attitudes on confidence to provide this type of care, which remained evident for the control group. In support of the latter interpretation, PCPs in the intervention group with the most negative baseline attitudes indicated higher levels of confidence at follow-up than control PCPs with even the most positive baseline attitudes. In line with this finding, a recent trial of an adult mental health educational intervention for PCPs found a similar interaction effect between PCP mental health care stigma and intervention status on PCP mental health care confidence.⁵⁰

In contrast to confidence in addressing diagnosable conditions, which we found was dependent on attitudes prior to training, confidence in addressing general mental health concerns using common factors skills and elements was dependent on training rather than attitudes. This finding perhaps speaks to the broad applicability of these common factors skills and elements for improving practitioner confidence regardless of their perspective on the role of PCPs in mental health care provision, which is apt to vary. Also, while acquiring and readily applying common factors skills and elements may require targeted training and practice, confidently adopting practice guidelines may depend more on encouraging practitioners’ views of integrated care as useful and feasible in practice.

Although, the PTCAP intervention did not enhance pediatric mental health care knowledge, this finding was not unexpected as the PTCAP intervention was designed to target skills confidence rather than knowledge, particularly given the wide availability of practice guidelines online and PCPs’ expressed interest in skills training over a focus on knowledge of diagnostic criteria.^14,31 While this finding may reflect a mismatch between the assessment and the training, it may also indicate that brief trainings may actually be changing confidence, and attitudes to some degree, more so than knowledge, so that it may prove to be more feasible to build the latter over time, once PCPs are potentially more ready to engage.

The feasibility of implementing the PTCAP program was supported by the ease with which we garnered interest in the program. Through email and telephone advertisement alone, 22 practices (79% of practices contacted) expressed interest in participating within 5 months. In addition, after incorporating early feedback around making the program more interactive, we found that acceptability of the program was high and remained so throughout implementation. Program adaptability and acceptability is also supported by the post hoc finding that intervention delivery structure (one full day vs. two half days) was not a significant predictor of any of the outcome measures. Although future trials should consider the impact of course delivery structure a priori, it is our belief that the PTCAP intervention must be tailored to the practice workflow and needs, and so we should continue to offer at least this degree of flexibility in delivering the program to maintain participation at a high level.

Overall, the findings of this study fit with the conclusion of a recent comprehensive critical review that effective educational interventions are likely to be those that improve the use of common factors skills and therapeutic elements that can be practically and effectively applied by PCPs themselves.¹³ An emerging consensus regarding the necessary elements of effective integration programs includes evidence-based guidelines for managing common conditions that can be easily diagnosed by PCPs, as well as clinical counseling and therapeutic skills for engaging in discussions around mental health with families and for providing frontline care for mild to moderate cases.^{18,26,28,32
–34}

Strengths and Limitations

Results obtained through model fitting using fewer than 20 clusters per intervention group should be interpreted with caution.⁴⁶ In addition, the measures used to assess confidence have previously been validated with a convenience sample in only one study, and missing data may have inflated the findings. However, the inclusion of multiple measures of confidence lends converging support to the increased confidence. Given that follow-up was carried out after 1 week, we do not know whether the increased confidence was maintained beyond this time period.

These limitations notwithstanding, this study provides preliminary support for the feasibility and utility of combining teaching in common factors skills and elements, along with and practice guidelines, in mental health education interventions for first-line PCPs. An important aspect of the PTCAP program was the inclusion of stakeholders (i.e., PCPs) in the intervention design. To our knowledge, this is the first training program in Canada to engage stakeholders throughout the process of designing and evaluating a mental health care education program (for a U.S. example see Ref. ⁵¹) and the first to test a child/youth mental health educational intervention for PCPs using a randomly controlled design and accounting for the effect of clustered data. Physician practice behavior and skill was not assessed in this study, and it cannot be assumed that measures of short-term confidence shifts will translate to physician behavior change. To give more power to this current study, we are currently evaluating the longer term effect of the PTCAP intervention (1-month follow-up) on observed PCP practice and on child/youth mental health outcomes. These findings will be published separately.

Conclusion

PTCAP enhances rural PCP confidence for managing general, yet impairing mental health concerns, as well as diagnosable conditions, themselves. We recommend that future research attempts to disaggregate the effects of training in common factors skills, common elements, and practice guidelines to achieve a more nuanced understanding of best approaches to enhancing integrated care capacity.

Footnotes

Authors’ Note

Data are available upon request.

Acknowledgments

We would like to thank all of the practitioners and families who graciously provided us with their time and personal information to make this research possible. We extend special thanks to Pamela Horne, Ajit Ninan, Stephanie Rabenstein, Connie Robinson, Neal Stretch, Kellie Scott, and members of the PTCAP collaborative (Sabina Abidi, Genevieve Auclair Terrence Callanan, Tamison Doey, Wil Fleisher, Bobbi Jo Flynn, Sarah Fotti, Sarosh Khalid-Khan, Lorraine Lazier, Vitaly Liashko, Harold Lipton, Lucie Nadeau, Declan Quinn, Lindsay Hope-Ross, Helen Spenser, Neal Stretch, Richard Zayed) for assisting in the design and implementation of the training program.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this research was provided by the Children’s Health Foundation.

ORCID iD

Margaret Steele, MD, FRCPC, MEd, DFCPA, CCPE, FCAHS

References

Asarnow

Rozenman

Wiblin

Zeltzer

. Integrated medical-behavioral care compared with usual primary care for child and adolescent behavioral health: a meta-analysis. JAMA Pediatr. 2015;169(10):929–937.

Polaha

Dalton

Allen

. The prevalence of emotional and behavior problems in pediatric primary care serving rural children. J Pediatr Psychol. 2011;36(6): 652–660.

Boydell

Pong

Volpe

Tilleczek

Wilson

Lemieux

. Family perspectives on pathways to mental health care for children and youth in rural communities. J Rural Health. 2006;22(2):182–188.

Triggs

Perrin

. Listening carefully. Improving communication about behavior and development. Recognizing parental concerns. Clin Pediatr (Phila). 1989;28(4):185–192. doi: 10.1177%2F000992288902800407.

Zimmermann

Del Piccolo

Finset

. Cues and concerns by patients in medical consultations: a literature review. Psychol Bull. 2007;133(3):438–463. doi: 10.1037/0033-2909.133.3.438.

Finset

. ‘I am worried, doctor!’ Emotions in the doctor–patient relationship. Patient Educ Couns. 2012;88(3):359–363. doi: 10.1016/j.pec.2012.06.022.

Wissow

Larson

Anderson

Hadjiisky

. Pediatric residents’ responses that discourage discussion of psychosocial problems in primary care. Pediatrics. 2005;115(6):1569–1578. doi: 10.1542/peds.2004-1535.

Krippeit

Belzer

Martens-Le Bouar

Mall

Barth

Communicating psychosocial problems in German well-child visits. What facilitates, what impedes pediatric exploration? A qualitative study. Patient Educ Couns. 2014;97(2):188–194. doi: 10.1016/j.pec.2014.07.032.

Steele

Zayed

Davidson

, et al. Referral patterns and training needs in psychiatry among primary care physicians in Canadian rural/remote areas. J Can Acad Child Adolesc Psychiatry. 2012;21(2):111–123.

10.

Zayed

Davidson

Nadeau

, et al. Canadian rural/remote primary care physicians perspectives on child/adolescent mental health care service delivery. J Can Acad Child Adolesc Psychiatry. 2016;25(1):24–34.

11.

Miller

Johnston

Klassen

Fine

Papsdorf

. Family physicians’ involvement and self-reported comfort and skill in care of children with behavioral and emotional problems: a population-based survey. BMC Fam Pract. 2005;6(1):12.

12.

Collins

Hewson

Munger

Wade

. Evolving models of behavioral health integration in primary care. New York (NY): Milbank Memorial Fund; 2010.

13.

Woltmann

Grogan-Kaylor

Perron

Georges

Kilbourne

Bauer

. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. Am J Psychiatry. 2011;169(8):790–804.

14.

American Academy of Pediatrics. Primary care tools. American Academy of Pediatrics; 2018 [accessed 2018 Feb 14]. https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/Mental-Health/Pages/Primary-Care-Tools.aspx.

15.

Ferrari

Suzanne

. Youth mental health, family practice, and knowledge translation video games about psychosis: family physicians’ perspectives. J Can Acad Child Adolesc Psychiatry. 2017;26(3):184–189.

16.

Heneghan

Garner

Storfer-Isser

Kortepeter

Stein

Horwitz

. Pediatricians’ role in providing mental health care for children and adolescents: do pediatricians and child and adolescent psychiatrists agree? J Dev Behav Pediatr. 2008;29(4):262–269.

17.

Vail

Adams

Gilbert

Nettleingham

Buckingham

. Investigating mental health risk assessment in primary care and the potential role of a structured decision support tool, GRiST. Ment Health Fam Med. 2012;9(1):57–67.

18.

Kolko

Perrin

. The integration of behavioral health interventions in children’s health care: services, science, and suggestions. J Clin Child Psychol. 2014;43(2):216–228.

19.

Briggs-Gowan

Owens

Schwab-Stone

Leventhal

Leaf

Horwitz

. Persistence of psychiatric disorders in pediatric settings. J Am Acad Child Adolesc Psychiatry. 2003;42(11):1360–1369.

20.

Durey

Thompson

. Reducing the health disparities of Indigenous Australians: time to change focus. BMC Health Serv Res. 2012;12(151):1–11.

21.

Towle

Godolphin

Alexander

. Doctor-patient communication in the Aboriginal community: toward the development of educational programs. Patient Educ Couns. 2006;62(3):340–346.

22.

Wissow

Gadomski

Roter

, et al. A cluster-randomized trial of mental health communication skills for pediatric generalists. Pediatr. 2008;121(2):266–275.

23.

Imel

Wampold

. The importance of treatment and the science of common factors in psychotherapy. Handb Couns Psychol. 2008;2(4):249–266.

24.

Duncan

Miller

Wampold

Hubble

. The Heart and Soul of Change: delivering what works in therapy. 2nd ed. Washington (DC): American Psychological Association; 2010.

25.

Gadomski

Wissow

Slade

Jenkins

. Training clinicians in mental health communication skills: impact on primary care utilization. Acad Pediatr. 2010;10(5):346–352.

26.

Wissow

Gadomski

Roter

Larson

Lewis

Brown

. Aspects of mental health communication skills training that predict parent and child outcomes in pediatric primary care. Patient Educ Couns. 2011;82(2):226–232.

27.

Brown

Wissow

. Rethinking the mental health treatment skills of primary care staff: a framework for training and research. Adm Policy Ment Health. 2012;39(6):489–502.

28.

Wissow

Anthony

Brown

, et al. A common factors approach to improving the mental health capacity of pediatric primary care. Adm Policy Ment Health. 2008;35(4):305–318.

29.

Weisz

. Testing standard and modular designs for psychotherapy with youth depression, anxiety, and conduct problems: a randomized effectiveness trial. Arch Gen Psychiatr. 2012;69(3):274–282.

30.

Chorpita

Daleiden

Weisz

. Identifying and selecting the common elements of evidence-based interventions: a distillation and matching model. Ment Health Serv Res. 2005;7(1):5–20.

31.

Chorpita

Becker

Daleiden

Understanding the common elements of evidence-based practice: misconceptions and clinical examples. J Am Acad Child Adolesc Psychiatry. 2007;46(5):647–652.

32.

Baum

King

Wissow

. Outcomes of a statewide learning collaborative to implement mental health services in pediatric primary care. Psychiatr Serv. 2019;70(2):123–129. doi: 10.1176/appi.ps.201800163.

33.

Christensen

Brown

Wissow

Cook

. Spillover of ratings of patient- and family-centered care. J Ambul Care Manage. 2016;39(4):308–3015. doi: 10.1097/JAC.0000000000000133.

34.

Brown

Wissow

Cook

Longway

Caffery

Pefaure

. Mental health communications skills training for medical assistants in pediatric primary care. J Behav Health Serv Res. 2013;40(1):20–35. doi: 10.1007/s11414-012-9292-0.

35.

Gotovac

Espinet

Lingard

, et al. Child and adolescent mental health training for primary care practitioners: applying flexibility within fidelity. in progress.

36.

Henderson

Brownlie

Rosenkranz

Chaim

Beitchman

. Integrated knowledge translation and grant development: addressing the research practice gap through stakeholder-informed research. J Can Acad Child Adolesc Psychiatry. 2013;22(4):268–274.

37.

PTCAP: Practitioner Training in Child and Adolescent Psychiatry. PTCAP; c2018 [accessed 2018 Feb 14]. www.ptcap.ca.

38.

Espinet

Gotovac

Knight

Zwarenstein

Lingard

Steele

. A study protocol for the “practitioner training in child and adolescent psychiatry” cluster-randomized pilot study. J Can Acad Child Adolesc Psychiatry. 2018;27(4):236–244.

39.

Pong

Pitblado

. Geographic distribution of physicians in Canada: beyond how many and where. Ottawa (ON): Canadian Institute for Health Information; 2005.

40.

Dillman

DA.

Mail and Internet surveys: the tailored design method—2007 update with new Internet, visual, and mixed-mode guide. Hoboken (NJ): John Wiley; 2007.

41.

Ministry of Health and Long-Term Care: Family health teams. Ontario, Canada: Queen’s Printer for Ontario; c2009-2010 [accessed Feb 14 2018]. http://www.health.gov.on.ca/en/pro/programs/fht/fht_progress.aspx.

42.

MacCarthy

. Mental health practice and attitudes of family physicians can be changed! Perm J. 2013;17(3):14–17.

43.

Sample-Size Net: Sample Size Calculators. UCSF Clinical and Translational Science Institute; c2018 [accessed 2018 Feb 4]. http://www.sample-size.net/means-sample-sizeclustered/.

44.

RANDOM: Random drawings. Dublin, Ireland: Randomness and Integrity Services; c1998-2018 [accessed 2018 Feb 14]. www.random.org

45.

Brown

Riley

Wissow

. Identification of youth psychosocial problems during pediatric primary care visits. Adm Policy Ment Health. 2007;34(3):269–281.

46.

Duncan

Jones

Moon

. Context, composition and heterogeneity: using multilevel models in health research. Soc Sci Med. 1998;46(1):96–117.

47.

Garcia-Ortega

Kadlec

Kutcher

Hollander

Kallstrom

Mazowita

. Program evaluation of a child and youth mental health training program for family physicians in British Columbia. J Can Acad Child Adolesc Psychiatry. 2013;22(4):290–296.

48.

McLennan

Jansen-McWilliams

Comer

Gardner

Kelleher

. The Physician Belief Scale and psychosocial problems in children: a report from the pediatric research in office settings and the ambulatory sentinel practice network. J Dev Behav Pediatr. 1999;20(1):24–30.

49.

Levinson

Roter

. Physicians’ psychosocial beliefs correlate with their patient communication skills. Gen Intern Med. 1995;10(7): 375–379.

50.

Beaulieu

Patten

Knaak

Weinerman

Campbell

Lauria-Horner

. Impact of skill-based approaches in reducing stigma in primary care physicians: results from a double-blind, parallel-cluster, randomized controlled trial. Can J Psychiatry. 2017;62(5):327–335.

51.

King

Wissow

Baum

. The role of organizational context in the implementation of a statewide initiative to integrate mental health services into pediatric primary care. Health Care Manage Rev. 2017;43(3):206–217.