Pediatric Psychotropic Polypharmacy: An Evaluation of the Correlates and Prevalence Across Assessment Cycles in the National Health and Nutrition Examination Survey

Abstract

Background:

Pediatric psychotropic polypharmacy (PPP) is the prescription of more than one medication targeting psychiatric disorders among people younger than 18 years. Recent data suggested that PPP rates may be plateauing. Few studies have evaluated this question in large, nationally recruited samples.

Objective:

The National Health and Nutrition Examination Survey was used to examine the correlates and prevalence of PPP across assessment cycles. Independent assessments were obtained biannually between 2013 and 2018.

Methods:

Eleven thousand four hundred thirty-nine participants (4–17 years; M_age = 8.69 years; standard deviation = 5.16) were included in analyses. The Anatomic Therapeutic Chemical coding scheme was employed to classify medications, and participants were characterized as taking psychotropic medication if the medication was associated with a psychiatric diagnosis code. Participants self-reported past month medication use. Logistic regressions were used to examine correlates of pediatric psychotropic monotherapy compared with psychotropic polypharmacy.

Results:

Across assessments, 1.2% of respondents reported using two or more psychotropic medications. This estimate is lower than has been observed in specialized samples, but higher than other work using national samples. There was a small, significant difference in PPP across assessment cycles, such that rates of PPP were higher at the latter assessments. Correlates of PPP accorded with prior work, including male gender, increasing age, and markers of low socioeconomic status. The most robust predictor was having seen a mental health professional in the past year.

Conclusions:

This study documents that ∼1% of U.S. participants from a nationally recruited sample endorsed PPP. Findings are situated in the broader literature and the need for additional, prospective data to better characterize those trends in the United States and around the world.

Key Takeaway Points

It is known that many children and adolescents in the United States take more than one psychotropic medication, although few studies have examined trends in large, nationally recruited datasets.

This study adds to this literature by documenting the prevalence of pediatric psychotropic polypharmacy in a large, unselected sample (i.e., 1.2%) and shows that rates were slightly higher at subsequent assessment intervals.

Plain Language Summary

Many kids take more than one medication for psychological problems. We analyzed data from ∼11,000 children and adolescents from across the United States, evaluated between 2013 and 2018. The number of kids taking multiple medications for psychological problems was different (higher) when measured later in time. Being a boy, being older, living in poverty, and having seen a mental health professional in the past year were associated with taking multiple medications for psychological problems.

Implications for Managed Care Pharmacy

These findings suggest rates of pediatric psychotropic polypharmacy (PPP) remain high in the United States, and correlate with male gender, poverty, and having recently seen a mental health professional. Relative to White children and adolescents, Black participants were less likely and Hispanic participants more likely to endorse PPP. Policy considerations include fully educating families and practitioners about the benefits as well as potential downsides of PPP and additional intervention options for mental health problems.

Introduction

Mental, behavioral, or developmental disorders affect ∼16% of U.S. children 2–8 years of age (Bitsko et al, 2016), and 22% of adolescents 13–18 years of age meet diagnostic criteria for at least one mental disorder characterized by severe impairment and/or distress (Merikangas et al, 2010). Furthermore, rates of some disorders, such as anxiety and depression, have increased over time (Bitsko et al, 2018; Cybulski et al, 2021). These trends are alarming, given that mental health problems are associated with ongoing impairment across multiple domains (e.g., school, social, family) (Jaycox et al, 2009) and the treatment of mental disorders represents one of the most costly conditions among children and adolescents, with one recent estimate suggesting $13.9 billion spent in annual direct costs in the United States (Soni, 2015).

Psychotropic medications are commonly used to treat mental health problems in pediatric populations, and a significant minority of children and adolescents are prescribed more than one psychotropic medication (e.g., Safer and Zito, 2012). This approach is not in and of itself harmful, and may be indicated in contexts such as addressing limited treatment response, treating co-existing conditions, or offsetting adverse effects of another drug (Jureidini et al, 2013). Indeed, comorbidity and complex cases are common in psychiatry, necessitating the use of multiple medications in some cases. However, there are a number of risks associated with prescribing multiple psychotropic medications to children and adolescents, including drug-drug interactions, problems with adherence, as well as relatively few controlled studies supporting efficacy (Auger et al, 2019; Horace et al, 2020; Safer et al, 2003).

There is significant variability in the definition of polypharmacy in the published literature, depending on parameters such as the length of the assessment period, number of overlapping days the patient is exposed, and the type of polypharmacy assessed (e.g., within class, multiclass) (Zito et al, 2021). A recent scoping review concluded that polypharmacy entails the “prescription or consumption of two or more distinct medications for at least one day” (Bakaki et al, 2018, p. 16). This study adopts this definition. It is also concerned with polypharmacy among children and adolescents younger than 18 years, which we refer to as pediatric psychotropic polypharmacy (PPP).

Although the use of varying operationalizations of polypharmacy makes comparability across studies somewhat difficult, research with large U.S. samples in the 1990s suggested increasing PPP over time at pediatric visits (Bhatara et al, 2004; Comer et al, 2010). In the 2000s, findings from a State Children's Health Insurance claims database (Zito et al, 2008) as well as a series of studies using Medicaid databases (Fontanella et al, 2014a; Kreider et al, 2014; Saucedo et al, 2018; Zhang et al, 2021) similarly suggested increasing rates of PPP, a pattern mirrored in some large international samples (Lagerberg et al, 2019). There are, however, exceptions to this trend. For instance, PPP trends were stable in a large Medicaid-enrolled sample of U.S. preschoolers (Fontanella et al, 2014a) and among psychostimulant users drawn from a national cohort study in Ireland (Boland et al, 2015).

Indeed, in a recent review of nearly 300 studies of polypharmacy broadly defined (e.g., polypharmacy for physical and mental health conditions), authors observed a decrease in global polypharmacy over the past several decades, a trend they attributed to a decline in the prescription of second-generation antipsychotics following efforts to publicize potential adverse effects (Baker et al, 2019).

However, the majority of the studies included in this review utilized outpatient samples, and the population-level studies included did not examine trends in PPP over time (Kovess et al, 2015; Reed et al, 2004; Zoëga et al, 2009), or they focused on specific groups such as children and adolescents receiving Medicaid (e.g., Allaire et al, 2016; Fontanella et al, 2014a,b), enrolled in foster care (Brenner et al, 2014), diagnosed with specific conditions such as attention-deficit/hyperactivity disorder (Hauck et al, 2017), or taking medications from a specific class such as antipsychotic, stimulant, or antidepressant medications (Bergendal et al, 2015; Bhatara et al, 2004; Fegert et al, 2006). An important contribution to this literature is, therefore, to evaluate PPP compared to psychotropic monotherapy within an unselected, nationally recruited sample of U.S. children and adolescents.

Differences in the operationalization and measurement of PPP, age of participants, date of data collection, and sample type also obscure prevalence estimates. National survey estimates range from 2% to 3% among Medicaid recipients generally (dosReis et al, 2005), with much smaller rates (<1%) in unselected samples drawn from national databases in the United States and abroad (Kovess et al, 2015; Zito et al, 2008; Zoëga et al, 2009), and higher rates (9%–27%) in selected samples such as children and adolescents visiting the pediatrician where a psychotropic medication was prescribed (Comer et al, 2010), Medicaid recipients with behavioral health or psychiatric diagnoses (Lohr et al, 2018; Medhekar et al, 2019), or those in foster care (Fontanella et al, 2014b; Keast et al, 2019).

In addition to psychiatric status, findings from these studies generally converge on male gender, race (i.e., White), urban residential status, increasing age, and markers of poverty as factors that relate positively to PPP. Finally, at least one study of children and adolescents in foster care found that enrollment in psychotherapy reduced the odds of PPP (Keast et al, 2019).

The goal of this study is to uniquely extend this area of work by evaluating PPP in a recent, nationally recruited sample of children and adolescents from the United States. Data were drawn from the National Health and Nutrition Examination Survey (NHANES), collected by the U.S. National Center for Health Statistics. These data were utilized to examine (1) differences in the frequency of PPP across assessment cycles and (2) predictors of PPP versus psychotropic monotherapy within the sample as a function of a series of sociodemographic variables (sex, age, poverty ratio, race/ethnicity, having seen a mental health provider, and insurance status).

Methods

Participants and procedures

The NHANES was designed to estimate the prevalence of health, nutrition, and health risk factors using surveys with a complex, nationally representative, stratified, clustered, multiprobability sample of the U.S. population, in 2-year cycles, since 1999. This study was concerned with children and adolescents 17 years of age and younger (M_age = 8.69 years [standard deviation = 5.16], range: 0–17 years), and examined data collected between 2013 and 2018. In 2013–2014, 14,442 individuals (including adults) were selected for the NHANES from 30 different survey locations; of those, 10,715 completed the interview. In 2015–2016, 15,327 individuals (including adults) were selected for the NHANES from 30 different survey locations; of those, 9971 completed the interview.

In 2017–2018, 16,211 individuals (including adults) were selected for the NHANES from 30 different survey locations; of those, 9254 completed the interview. Given that this study was concerned with children and adolescents younger than 18 years, a total of 11,439 responses were included in this sample (weighted N = 74,418,155). The youngest child taking one psychotropic medication in the sample was 3 years old, and the youngest child taking two or more psychotropic medications was 5 years old. However, one analysis in this project (i.e., logistic regression) only includes children and adolescents between the ages of 4 and 17 years because only participants 4 years of age and older were queried about seeing a mental health provider, a correlate examined in the analysis. For participants younger than 16 years, a proxy (i.e., parent/guardian) provided participant information.

Measures

Medication

Medication lists were obtained for all NHANES participants during an in-home interview where participants who reported medications for which a prescription was needed during the past month were asked to show their medication containers for verification. Participants/proxies were also asked to provide information about the condition the medicine was used for. In this study, the World Health Organization's (2020) Anatomic Therapeutic Chemical (ATC) coding scheme was utilized to classify medications into one of the following categories: stimulants, antipsychotics, lithium, anxiolytics, hypnotics and sedatives, antidepressants, anticonvulsant mood stabilizers, drugs used in addictive disorders, and other (drugs used for an identified psychological problem where the drug was not included in the ATC codebook; i.e., Guanfacine; Metformin; Clonidine; Desmopressin; and Amantadine). In line with precedent (Lohr et al, 2018), psychotropic medication use was defined as medications from these categories associated with a psychiatric International Classification of Diseases code.

The International Classification of Diseases (ICD-10), developed by The World Health Organization, is a formal system of assigning codes to somatic and psychiatric diagnoses. The United States developed a clinical modification based on the ICD-10 called The International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM). Psychiatric diagnoses referred to in this study included any of the ICD-10-CM Mental, Behavioral, and Neurodevelopmental disorders (codes F01–F99; please see Supplementary Table S2 for additional details).

If participants reported that they were taking a psychotropic medication for a nonpsychological reason (e.g., seizures), that medication was not included in their psychotropic medication count. For example, if a participant reported taking one psychotropic medication for a psychiatric ICD coded diagnosis (e.g., depression), and one psychotropic medication for a nonpsychiatric ICD-coded reason (e.g., seizures), then the participant was coded as taking one psychotropic medication. The number of psychotropic medications was summed and participants were categorized as taking zero, one (psychotropic monotherapy), or two or more (polypharmacy) medications.

Sociodemographic characteristics

Self-reported sociodemographic characteristics were collected during an in-home interview. In these analyses, sociodemographic data included the following: gender, race/ethnicity (non-Hispanic White, non-Hispanic Black, non-Hispanic Asian, Mexican American, Other-Hispanic, and other race [including Multiracial]), ratio of income to poverty (below one indicates the family is below poverty and higher scores indicate more familial affluence), if the participant had seen a mental health provider in the past year (i.e., “During the past 12 months, … {have you/has sample person [SP]} seen or talked to a mental health professional such as a psychologist, psychiatrist, psychiatric nurse or clinical social worker about {your/his/her} health?”), if the participant was receiving Medicaid benefits, and if the participant had private insurance.

Analytic plan

Survey weights were employed in all analyses to account for the complex survey design. Frequency and percentages were calculated for all categorical variables and for the number of psychotropic medications that participants were taking. The frequency of various classes of psychotropic medications was also evaluated. A one-way analysis of variance was planned to examine the number of psychotropic medications being used as a function of survey cycle to evaluate differences over time. A Levene's test revealed significant group differences in variance with regard to the number of psychotropic medications being used in each cycle, F (2,3727385) = 40,912.27, p < 0.001. Given that the assumption of the homogeneity of variance required for conducting parametric statistics was not met, a nonparametric statistic, the Kruskal-Wallis test with survey weights, was used to examine the difference in the number of psychotropic medications being used as a function of survey cycle.

This test computes a sum of ranks for each cycle (number of meds from lowest to highest), and then looks at the average ranks for each cycle. Pairwise comparisons were used as follow-ups to see if average rank increased over the waves. Because all participants were taking at least one psychotropic medication in this analysis, significant pairwise comparisons suggest increasing PPP. Pearson's r was selected as an index of effect size with small, medium, and large effect sizes represented by r-values of 0.01, 0.30, and 0.50, respectively (Cohen, 1988).

Next, binary logistic regression analyses were used to examine how sociodemographic variables were associated with psychotropic monotherapy versus polypharmacy among children and adolescents who reported taking at least one psychotropic medication. In these analyses, participants taking one psychotropic medication were coded as “0” and those taking two or more psychotropic medications were coded as “1.” Odds ratios (ORs) were used as indices of effect size (Cohen, 1988; Field, 2013) for individual predictors, and Nagelkerke R ² (Nagelkerke, 1991) was used as the index of effect size for the entire model. For comparisons of no psychotropic versus any psychotropic medication use, please see Supplementary Table S1.

Results

Frequency of psychotropic medication use in the overall sample

Between 2013 and 2018, 3.8% of children and adolescents in the overall sample reported using one psychotropic medication, while 1.2% of participants in the overall sample reported using two or more psychotropic medications. Breaking that down further, 95.0% of participants reported taking zero psychotropic medications, 3.8% reported taking one psychotropic medication, 0.9% reported two psychotropic medications, 0.2% reported three psychotropic medications, 0.1% reported taking four psychotropic medications, and 0.001% of participants reported taking five psychotropic medications. Collectively, 24.0% of participants taking psychotropic medications took more than one. See Table 1 for information about the prevalence of psychotropic medication use by 2-year data collection cycles.

Table 1.

Frequency of Pediatric Psychotropic Polypharmacy by National Health and Nutrition Examination Survey Cycle

Number of psychotropic medications
	0, n (%)	1, n (%)	2, n (%)	3 or more, n (%)
Data release cycle
2013–2014	23,483,544 (95.6)	770,603 (3.1)	221,105 (0.9)	80,372 (0.3)
2015–2016	23,885,563 (94.3)	1,157,300 (4.6)	236,968 (0.9)	55,250 (0.2)
2017–2018	23,308,585 (95.1)	908,551 (3.7)	213,550 (0.9)	90,764 (0.4)

All estimates were weighted to be nationally representative.

Differences in PPP as a function of survey wave

Next, the average sum of ranks reflecting psychotropic medication use among participants on at least one medication was compared across waves. Analyses indicated that there was a significant effect of survey wave [H(2) = 23,739.93, p < 0.001]. Pairwise comparisons with adjusted p-values showed that there was a significant difference between the 2013–2014 cycle and the 2015–2016 cycle, as well as the 2017–2018 cycle, such that youth in the 2013–2014 cycle reported taking fewer psychotropic medications than youth in the 2015–2016 (p = 0.000, r = 0.08) as well as 2017–2018 cycle (p = 0.000, r = 0.03).

Finally, there was a significant difference between the 2015–2016 data cycle and the 2017–2018 data cycle such that youth in the 2015–2016 cycle reported taking fewer psychotropic medications than youth in the 2017–2018 cycle (p = 0.000, r = 0.05). Collectively, these findings point to small increases in PPP across assessment cycles.

Frequency of types of psychotropic medications among children and adolescents on at least one psychotropic medication

Among participants on at least one psychotropic medication, the most commonly used psychotropic medications were stimulants (77.2% of participants on at least one psychotropic medication), followed by antidepressants (21.8% on at least one psychotropic medication), “other meds” (14.2% on at least one psychotropic medication), and antipsychotics (13.2% on at least one psychotropic medication). Please see Table 2 for information about the frequency of each type of medication.

Table 2.

Frequency of Type of Psychotropic Medication Used Among Children and Adolescents on at Least One Psychotropic Medication

	Number of participants (weighted percentage of participants on at least one psychotropic medication)
Stimulants	2,881,353 (77.2%)
Antidepressants	813,219 (21.8%)
Other drugs	531,350 (14.2%)
Antipsychotics	491,133 (13.2%)
Anticonvulsant mood stabilizers	154,457 (4.1%)
Anxiolytics	86,888 (2.3%)
Lithium	5762 (0.2%)
Hypnotics and sedatives	0 (0.0%)
Drugs used in addictive disorders	0 (0.0%)

Names of drugs in “other drugs” category	Number of participants (weighted percentage of participants on at least one psychotropic medication)
Guanfacine	350,250 (9.4%)
Clonidine	161,955 (4.3%)
Desmopressin	21,779 (0.6%)
Amantadine	8019 (0.2%)
Metformin	2824 (0.1%)

All estimates were weighted to be nationally representative.

Among participants who reported taking one or more psychotropic medications, the most common medication combination was the combination of a stimulant medication and a medication falling into the “other” category (6.0% on at least one psychotropic medication), followed by the combination of a stimulant with an antipsychotic (4.8% on at least one psychotropic medication) and the combination of a stimulant with an antidepressant (3.6% on at least one psychotropic medication). Please see Table 3 for more information about the frequency of each combination of two, three, four, and five psychotropic medications among participants on at least one psychotropic medication.

Table 3.

Frequency of Medication Combinations Among Children and Adolescents on at Least One Psychotropic Medication

	Number of participants (weighted percentage of participants on at least one psychotropic medication)
Two medication combinations
Stimulants and other drugs	227,478 (6.0%)
Stimulants and antipsychotics	177,434 (4.7%)
Stimulants and antidepressants	136,329 (3.6%)
Antipsychotics and antidepressants	36,171 (0.9%)
Antipsychotics and other drugs	29,299 (0.7%)
Antidepressants and other drugs	22,434 (0.6%)
Antidepressants and anticonvulsant mood stabilizers	26,046 (0.6%)
Anxiolytics and antidepressants	19,382 (0.5%)
Antipsychotics and anxiolytics	8714 (0.2%)
Stimulants and anxiolytics	8,714 (0.2%)
Stimulants and anticonvulsant mood stabilizers	8528 (0.2%)
Three medication combinations
Antipsychotics, stimulants, and other drugs	52,642 (1.4%)
Stimulants, antidepressants, and other drugs	39,119 (1.0%)
Antipsychotics, antidepressants, and anticonvulsants	24,526 (0.6%)
Antipsychotics, stimulants, and antidepressants	10,818 (0.3%)
Stimulants, anticonvulsants, and other drugs	10,381 (0.3%)
Antipsychotics, anxiolytics, and other drugs	6576 (0.2%)
Antipsychotics, antidepressants, and other drugs	4597 (0.1%)
Stimulants, antidepressants, and anticonvulsants	3457 (0.1%)
Four medication combinations
Antidepressants, anxiolytics, stimulants, and antipsychotics	21,878 (0.6%)
Anticonvulsants, stimulants, antipsychotics, and other drugs	16,291 (0.4%)
Anticonvulsants, antipsychotics, antidepressants, and other drugs	14,838 (0.4%)
Anticonvulsants, antipsychotics, anxiolytics, and other drugs	10,106 (0.3%)
Five medication combinations
Antidepressants, lithium, antipsychotics, stimulants, and other drugs	5762 (0.1%)

All estimates were weighted to be nationally representative. Drugs that were not reported as used in combination are not included in the table. Medication Combinations are listed in descending frequency.

Sociodemographic predictors of PPP

Please see Table 4 for information about the frequency of psychotropic medication use as a function of sociodemographic factors. Table 5 shows results of the logistic regression. With respect to the prediction of PPP, sociodemographic variables accounted for 24% of the variance in one versus multiple medications [χ² (11) = 627,226.14, p < 0.001; Nagelkerke R ² = 0.242]. Specifically, PPP was linked with increasing age and male gender. With regard to race/ethnicity, Mexican American participants (OR: 1.03, 95% confidence interval [CI]: 1.02–1.04) and Other-Hispanic participants (OR: 2.09, 95% CI: 2.06–2.12) were more likely to report PPP than White participants. In contrast, non-Hispanic Black participants (OR: 0.44, 95% CI: 0.43–0.44), Asian American participants (OR: 0.57, 95% CI: 0.56–0.59), and Multiracial participants (OR: 0.85, 95% CI: 0.84–0.86) were less likely to report PPP than White participants.

Table 4.

Characteristics of Children and Adolescents by Number of Psychotropic Medications Being Used, National Health and Nutrition Examination Survey (2013–2018)

	No medication	One medication	Two or more medications
	N (weighted percentage of participants in the given category [i.e., no medication, one medication, two or more medications])
Overall	70,683,692 (95%)^a	2,836,454 (3.8%)^a	898,008 (1.2%)^a
Race/Ethnicity
Non-Hispanic White	35,457,600 (50.1%)	1,963,563 (69.2%)	621,122 (69.2%)
Mexican American	11,653,765 (16.5%)	180,336 (6.3%)	60,581 (6.7%)
Other-Hispanic	6,066,462 (8.6%)	85,058 (3.0%)	51,743 (5.8%)
Non-Hispanic Black	9,797,372 (13.9%)	279,624 (9.9%)	59,557 (6.6%)
Non-Hispanic Asian	3,531,788 (5.0%)	22,049 (0.8%)	13,367 (1.5%)
Other-Race (including multiracial)	4,176,704 (5.9%)	305,825 (10.8%)	91,638 (10.2%)
Gender
Male	35,491,554 (50.2%)	1,947,356 (68.7%)	601,808 (67.0%)
Female	35,192,139 (49.8%)	889,098 (31.3%)	296,201 (33.0%)
Age group
0–3 Years	15,644,487 (22.1%)	9750 (0.3%)	0 (0.0%)
4–7 Years	15,662,323 (22.2%)	310,743 (11.0%)	100,484 (11.2%)
8–11 Years	15,406,497 (21.8%)	1,103,185 (38.9%)	242,805 (27.0%)
12–15 Years	16,175,579 (22.9%)	850,448 (30.0%)	411,358 (45.8%)
16–17 Years	7,794,808 (11.0%)	562,328 (19.8%)	143,361 (16.0%)
Insurance type
Private insurance	35,014,262 (49.5%)	1,523,168 (53.7%)	295,813 (32.9%)
Medicaid	22,797,340 (32.3%)	1,050,631 (37.0%)	522,433 (58.2%)
Ratio of income to poverty [M (SD)]^b	2.45 (1.67)	2.55 (1.59)	2.08 (1.59)
Seen a mental health professional in past year?	3,952,576 (5.6%)	1,102,213 (38.9%)	723,750 (80.6%)

Sample weights were used to be nationally representative.

Values in parentheses indicated weighted percentage of total population, rather than weighted percentage of participants in a given category.

Range: 0.00–5.00, M = 2.45, SD = 1.63.

SD, standard deviation.

Table 5.

Weighted Logistic Regression Models of Sociodemographic Predictors of Polypharmaceutical Use Among Children and Adolescents on at Least One Psychotropic Medication

	B	Wald	Odds ratio (95% CI)	p
Race
Non-Hispanic White	—	—	1 (Reference)	—
Mexican American	0.03	21.57	1.03 (1.02–1.04)	0.000
Other-Hispanic	0.74	12,721.07	2.09 (2.06–2.12)	0.000
Non-Hispanic Black	−0.83	25,333.14	0.44 (0.43–0.44)	0.000
Non-Hispanic Asian	−0.56	1197.61	0.57 (0.56–0.50)	0.000
Other race (including multiracial)	—0.16	1241.44	0.85 (0.84–0.86)	0.000
Gender
Female	—	—	1 (Reference)	—
Male	0.29	9118.28	1.33 (1.32–1.34)	0.000
Age (years)	0.10	52,618.23	1.10 (1.10–1.10)	0.000
Medicaid
No	—	—	1 (Reference)	—
Yes	0.34	8765.09	1.41 (1.40–1.42)	0.000
Private insurance
No	—	—	1 (Reference)	—
Yes	−0.36	8200.62	0.70 (0.69–0.70)	0.000
Ratio of family income to poverty	−0.07	3785.65	0.93 (0.93–0.93)	0.000
Seen a mental health professional in past year
No	—	—	1 (Reference)	—
Yes	1.92	363,359.15	6.83 (6.79–6.87)	0.000

CI, confidence interval.

Compared to those who were not receiving Medicaid, those receiving Medicaid were more likely to endorse PPP (OR: 1.41, 95% CI: 1.40–1.42). Similarly, compared to participants with private insurance, those without private insurance were more likely to report PPP (OR: 0.70, 95% CI: 0.69–0.70). Participants who were closer to the poverty level were more likely to report PPP than those who were farther from the poverty line (OR: 0.93, 95% CI: 0.93–0.93). Finally, participants who had seen a mental health provider over the previous year were over 6 times more likely to endorse PPP than those who had not seen a mental health provider (OR: 6.83, 95% CI: 6.79–6.87).

Discussion

This study provides a recent estimate of the prevalence and correlates of PPP within a nationally recruited sample of U.S. children and adolescents 4–17 years of age. Approximately 1% of the overall sample, and 24% of participants taking one psychotropic medication, endorsed PPP. This estimate is, not surprisingly, lower than those obtained from studies of Medicaid claims (e.g., dosReis et al, 2005; Lohr et al, 2018) and specialized samples (e.g., foster care) (Keast et al, 2019), but is higher than prior work using other national samples. For example, in their sample of more than 128,000 children and adolescents who were beneficiaries of French National Health Insurance, Kovess et al (2015) observed <1% of participants (and 10% of those using one psychotropic medication) received multiclass psychotropic treatment in 2010.

Zoëga et al (2009), in a sample of Icelandic children and adolescents, and Zito et al (2008), in a U.S. sample, observed similar overall rates of PPP in their large-scale samples, with 17.5% and 19.2% evidencing concomitant psychotropic medication use, respectively. On the one hand, these data suggest the prevalence of PPP in this NHANES sample is higher than observed in similarly designed past studies. However, when comparing PPP rates across the assessment cycles in this study, the observed effect size between assessments conducted from 2013 to 2018 was very small. Notably, as the NHANES tracked cohorts rather than individual participants over time, we are not able to speak directly to prospective change in PPP over time. This remains an important goal for future work—to systematically evaluate ongoing fluctuation in PPP in national databases across the globe.

These findings accord with Baker et al's (2019) conclusion that general polypharmacy rates appear to have largely plateaued in recent years (cf., Jureidini et al, 2013). However, a number of factors temper this conclusion. For example, in their review of the literature, Zito et al (2021) concluded that children and adolescents taking three or more psychotropic medications have increased over the past two decades, highlighting the need for more fine-grained analyses of the patterns of PPP. Furthermore, comparison between these findings and past work is obscured by methodological differences across studies. For instance, database studies often index medication use in the past year (e.g., Kovess et al, 2015; Zoëga et al, 2009), whereas participants in the NHANES reported medications taken in the past month.

In addition, a strength of this study is that participants self-reported current medication use, as opposed to medications prescribed, as is common in claims data. Collectively, however, these differences in study design make it difficult to draw clear comparisons between this pattern of findings and past work.

Findings from this study were aligned with prior work, with regard to correlates of PPP (e.g., Keast et al, 2019), including results from national samples comparing office visits involving single-class versus multiclass psychotropic office visits (Comer et al, 2010), as well as a Medicaid claims study evaluating interclass polypharmacy among participants taking psychotropic medications (Lohr et al, 2018).

In this study, male gender, increasing age, recipient of Medicaid, insurance status, and poverty emerged as significant predictors of PPP (compared to monotherapy). These effect sizes also were generally small. Also, in line with previous studies, stimulants were most commonly co-prescribed with “other” medications, antipsychotics, and antidepressants. Given boys are more likely to be diagnosed with the neurodevelopmental syndromes for which stimulants are typically prescribed (e.g., attention-deficit/hyperactivity disorder), this prescribing pattern likely drives the gender differences in PPP observed here and elsewhere. Finally, the largest effect was observed for children and adolescents who reported seeing a mental health professional in the past year (i.e., mental health professional such as a psychologist, psychiatrist, psychiatric nurse, or clinical social worker).

However, 20% of respondents taking medications for psychiatric problems did not endorse seeing a mental health professional in the past year. This is troublesome, particularly given psychotherapy is recommended as first-line or multimodal treatment for pediatric mental health issues (e.g., Andrade et al, 2019; Daley et al, 2018). That is, rarely is medication indicated as a stand-alone treatment. Instead, psychotherapy is necessary to target other factors contributing to symptoms, such as coping and problem-solving, communication, and interpersonal issues.

Participants identifying as Mexican American and Other-Hispanic were more likely than White participants to report PPP. Indeed, logistic regression analyses indicated Other-Hispanic participants were more than twice as likely than White participants to endorse PPP. To the best of our knowledge, this is the first study to document this particular ethnic difference. This finding stands somewhat at odds with prior work. On the one hand, supplemental analyses from this study (Supplementary Table S1) replicated findings from a Medicaid claims study, where Hispanic children and adolescents reported less psychotropic medication in general than White children and adolescents (Martin et al, 2003). These authors also compared PPP to monotherapy and, using a stepwise approach, found that the Hispanic ethnicity was not retained in the final model due its small effect, whereas we found that Other-Hispanic participants were significantly more likely to endorse PPP when compared to monotherapy.

Qualitative (Arcia et al, 2004) and vignette-based research (Cohen et al, 2013) suggests Hispanic parents prefer not to use medication to address their children's behavioral problems; however, there is evidence that Hispanic families consult primary care physicians for mental health concerns (Bilsky et al, 2020; Cabassa et al, 2006), and thus may be more likely to receive psychopharmacological interventions for psychological problems. Possibly, once families from this ethnic background commence pharmacotherapy, they are more inclined than their White counterparts to accept PPP, although this conclusion is highly speculative. Additional research, including participatory- and qualitatively based work, is needed to replicate and contextualize this finding. In contrast, Asian American, non-Hispanic Black, and Multiracial participants were less likely than their White counterparts to report PPP (compared to monotherapy), although the magnitude of the effect was small.

We are unaware of work evaluating PPP among Asian Americans; pending replication, this is an important area for future work. The latter findings align with other work, however, including in a large sample of Kentucky children and adolescents receiving Medicaid (Lohr et al, 2018). Using the 2004–2011 Medical Expenditure Panel Survey, Lê Cook et al (2017) found that both Black (5%) and Hispanic (3.4%) children and adolescents were significantly less likely than White (8.1%) participants to fill a psychotropic prescription in the past year, but only Black participants were less likely to use psychotropic medication, despite the presence of impairment. Although this study was not focused specifically on polypharmacy, findings comport with other work indicating Black children and adolescents are less likely to receive treatment of any kind for depression, even after accounting for income and insurance status (Cummings and Druss, 2011).

Racial and ethnic differences in mental health treatment are driven by a number of dynamic and interacting factors, including system-level barriers related to segregation and physician availability (Li and Yuan, 2022; Williams and Collins, 2001) as well as quality of care, especially culturally competent care (Kohn-Wood and Hooper, 2014). Mistrust of providers may also play a role, particularly in Black communities (Brooks and Hopkins, 2017; Whaley et al, 2001), along with a preference for spiritual interventions (Ward et al, 2013), as well as stigma associated with seeking or receiving treatment for mental health problems (Clement et al, 2015).

However, in a large, vignette-based study evaluating self-reported willingness to give prescribed medication to their child if the child had behavioral problems, there were no differences between Black and White parents in terms of willingness to medicate children after controlling for important covariates (e.g., parental use of psychotropic medication, socioeconomic status; Cohen et al, 2013). Hispanic parents were significantly less likely to endorse medicating their children for behavioral problems compared to White parents, although effects were small. Notably, this study was focused on parental willingness to the use of any medication, and not polypharmacy per se.

As detailed in Supplementary Table S1, rates of use of psychotropic medications in general were lower across most ethnic and racial groups compared to White participants in this study. Additional research is needed to better understand parental willingness to endorse PPP specifically, including as a function of ethnicity and race, and whether such attitudes translate to behavior (Armitage and Conner, 2001).

Limitations

There are a number of strengths of this study, including the use of a large, nationally recruited dataset. Offsetting that strength, however, is the fact that we were constrained to the variables measured in the NHANES study, which introduces several limitations. First, although all medications reported on in this study were linked with an ICD psychiatric code, we do not have data on symptom severity, which would paint a clearer picture of children and adolescents endorsing PPP. Relatedly, we do not have data on the prescribing practitioner, including quality of care and, therefore, cannot speak about the medical necessity of PPP for participants in the current sample or whether psychotherapeutic alternatives were either available or concomitantly utilized. Future work with richer datasets, including parental attitudes about medication-based treatments for mental illness, would provide important insight into these factors and the degree to which they influence PPP and its correlates.

Third, as each NHANES cycle includes data collected in independent waves, although it is unlikely, it is unknown if participants may have participated in multiple cycles. Fourth, these data are based on self-report, which is subject to memory and affective biases; replicating this study using medical claims data would increase confidence in the observed findings (e.g., Zuvekas and Olin, 2009). Finally, we do not have data regarding whether PPP is associated with negative consequences for participants in this study. It is important to note that, although PPP should be carefully monitored (e.g., due to potential adverse effects) (Hilt et al, 2014), it is also clearly indicated in some cases, where comorbidity or other complexities require more than one psychotropic medication to facilitate mental health.

Conclusions

This study provides a recent snapshot of PPP in a nationally recruited sample of U.S. children and adolescents. Findings suggest rates of PPP are relatively stable over the past decade, although further longitudinal work is needed to confirm that observation. Finally, this study replicated past work regarding the correlates of PPP (e.g., male gender, markers of poverty, increasing age). The most robust predictor of PPP was having seen a mental health provider in the past year, although one in five children and adolescents in this sample endorsed PPP in the absence of psychosocial intervention, pointing to important gaps in treatment for children and adolescents in the United States.

Clinical Significance

From a clinical standpoint, these data suggest that many American children and adolescents endorse PPP. Although intervention for mental health problems with multiple medications is sometimes indicated, the concerns outlined in the introductory section (e.g., limited data on medication interactions) should be seriously considered when practitioners are making prescribing decisions.

Footnotes

Acknowledgments

This study was not preregistered as it is a secondary analysis of a nationally recruited dataset. The authors would like to thank Kat Vrotsos, Alexandria Baker, Caroline Lee, and Meredith Hall for their invaluable assistance in data sourcing and preparation for this project.

Disclosures

The second and fifth author received compensation for an unrelated study from Canopy Growth Corporation. This entity had no role in the project design, identification, analysis or interpretation of data, the writing of the report, or the decision to submit the article for publication. All other authors have no competing financial interests exist.

Supplementary Material

Supplementary Table S1

Supplementary Table S2

References

Allaire

, Raghavan

, Brown

. Morbid obesity and use of second generation antipsychotics among adolescents in foster care: Evidence from Medicaid. Child Youth Serv Rev, 2016; 67:27–31; doi:10.1016/j.childyouth.2016.05.019.

Andrade

, Courtney

, Duda

, et al. A systematic review and evaluation of clinical practice guidelines for children and youth with disruptive behavior: Rigor of development and recommendations for use. Clin Child Fam Psychol Rev, 2019; 22(4):527–548; doi:10.1007/s10567-019-00292-2.

Arcia

, Fernández

, Jáquez

Latina mothers' stances on stimulant medication: Complexity, conflict, and compromise. J Dev Behav Pediatr, 2004; 25(5):311–317; doi: 10.1097/00004703-200410000-00002.

Armitage

, Conner

Efficacy of the theory of planned behaviour: A meta-analytic review. Br J Soc Psychol, 2001; 40(4):471–499; doi: 10.1348/014466601164939.

Auger

, Shah

, Davis

, et al. Counting the ways to count medications: The challenges of defining pediatric polypharmacy. J Hosp Med, 2019; 14(8):506–507; doi: 10.1278/8/jhm.3213.

Bakaki

, Horace

, Dawson

, et al. Defining pediatric polypharmacy: A scoping review. PLoS One, 2018; 13(11):e0208047; doi: 10.1371/journal.pone.0208047.

Baker

, Feinstein

, Ma

, et al. Variation of the prevalence of pediatric polypharmacy: A scoping review. Pharmacoepidemiol Drug Saf, 2019; 28(3):275–287; doi: 10.1002/pds.4719.

Bergendal

, Schiöler

, Wettermark

, et al. Concomitant use of two or more antipsychotic drugs is common in Sweden. Ther Adv Psychopharmacol, 2015; 5(4):224–231; doi: 10.1177/2045125315588647.

Bhatara

, Feil

, Hoagwood

, et al. National trends in concomitant psychotropic medication with stimulants in pediatric visits: Practice versus knowledge. J Atten Disord, 2004; 7(4):217–226; doi: 10.1177/108705470400700404.

10.

Bilsky

, Dueweke

, Bridges

. The impact of anxiety on health care utilization among Hispanic and non-Hispanic youth. J Health Psychol, 2020; 25(7):986–992; doi: 10.1177/1359105317740738.

11.

Bitsko

, Holbrook

, Ghandour

, et al. Epidemiology and impact of health care provider-diagnosed anxiety and depression among US children. J Dev Behav Pediatr, 2018; 39(5):395–403; doi: 10.1097/DBP.0000000000000571.

12.

Bitsko

, Holbrook

, Robinson

, et al. Health care, family, and community factors associated with mental, behavioral, and developmental disorders in early childhood—United States, 2011–2012. MMWR Morb Mortal Wkly Rep, 2016; 65(9):221–226; doi: 10.1558/5/mmwr.mm6509a1.

13.

Boland

, Galvin

, Reulbach

, et al. Psychostimulant prescribing trends in a paediatric population in Ireland: A national cohort study. BMC Pediatr, 2015; 15(1):1–8; doi: 10.1186/s12887-015-0435-3.

14.

Brenner

, Southerland

, Burns

, et al. Use of psychotropic medications among youth in treatment foster care. J Child Fam Stud, 2014; 23(4):666–674; doi: 10.1007/s10826-013-9882-3.

15.

Brooks

, Hopkins

Cultural mistrust and health care utilization: The effects of a culturally responsive cognitive intervention. J Black Stud, 2017; 48(8):816–836; doi: 10.1177/0021934717728454.

16.

Cabassa

, Zayas

, Hansen

. Latino adults' access to mental health care: A review of epidemiological studies. Adm Policy Ment Health, 2006; 33(3):316–330; doi: 10.1007/s10488-006-0040-8.

17.

Clement

, Schauman

, Graham

, et al. What is the impact of mental health-related stigma on help-seeking? A systematic review of quantitative and qualitative studies. Psychol Med, 2015; 45(1):11–27; doi: 10.1017/S0033291714000129.

18.

Cohen

, Dillon

, Gladwin

, et al. American parents' willingness to prescribe psychoactive drugs to children: A test of cultural mediators. Soc Psychiatry Psychiatr Epidemiol, 2013; 48(12):1873–1887; doi: 10.1007/s00127-013-0710-2.

19.

Cohen

Statistical Power Analysis for the Behavioral Sciences (2nd ed). Lawrence Erlbaum Associates: Hillsdale, NJ; 1988.

20.

Comer

, Olfson

, Mojtabai

National trends in child and adolescent psychotropic polypharmacy in office-based practice, 1996–2007. J Am Acad Child Adolesc Psychiatry, 2010; 49(10):1001–1010; doi: 10.1016/j.jaac.2010.07.007.

21.

Cummings

, Druss

. Racial/ethnic differences in mental health service use among adolescents with major depression. J Am Acad Child Adolesc Psychiatry, 2011; 50(2):160–170; doi: 10.1016/j.jaac.2010.11.004.

22.

Cybulski

, Ashcroft

, Carr

, et al. Temporal trends in annual incidence rates for psychiatric disorders and self-harm among children and adolescents in the UK, 2003–2018. BMC Psychiatry, 2021; 21(1):229; doi: 10.1186/s12888-021-03235-w.

23.

Daley

, Van Der Oord

, Ferrin

, et al. Practitioner review: Current best practice in the use of parent training and other behavioural interventions in the treatment of children and adolescents with attention deficit hyperactivity disorder. J Child Psychol Psychiatry, 2018; 59(9):932–947; doi: 10.1111/jcpp.12825.

24.

dosReis

, Zito

, Safer

, et al. Multiple psychotropic medication use for youths: A two-state comparison. J Child Adolesc Psychopharmacol, 2005; 15(1):68–77; doi: 10.1089/cap.2005.15.68.

25.

Fegert

, Kölch

, Zito

, et al. Antidepressant use in children and adolescents in Germany. J Child Adolesc Psychopharmacol, 2006; 16(1–2):197–206; doi: 10.1089/cap.2006.16.197.

26.

Field

Discovering Statistics Using SPSS (4th ed). SAGE Publications Ltd.: London, UK; 2013.

27.

Fontanella

, Hiance

, Phillips

, et al. Trends in psychotropic medication use for Medicaid-enrolled preschool children. J Child Fam Stud, 2014a;23(4):617–631; doi: 10.1007/s10826-013-9761-y.

28.

Fontanella

, Warner

, Phillips

, et al. Trends in psychotropic polypharmacy among youths enrolled in Ohio Medicaid, 2002–2008. Psychiatr Serv, 2014b;65(11):1332–1340; doi: 10.1176/appi.ps.201300410.

29.

Hauck

, Lau

, Wing

LLF

, et al. ADHD treatment in primary care: Demographic factors, medication trends, and treatment predictors. Can J Psychiatry, 2017; 62(6):393–402; doi: 10.1177/0706743716689055.

30.

Hilt

, Chaudhari

, Bell

J.F

, et al. Side effects from use of one or more psychiatric medications in a population-based sample of children and adolescents. J Child Adolesc Psychopharmacol, 2014; 24(2):83–89; doi: 10.1089/cap.2013.0036.

31.

Horace

, Golchin

, Knight

EMP

, et al. A scoping review of medications studied in pediatric polypharmacy research. Paediatr Drugs, 2020; 22(1):85–94; doi: 10.1007/s40272-019-00372-4.

32.

Jaycox

, Stein

, Paddock

, et al. Impact of teen depression on academic, social, and physical functioning. Pediatrics, 2009; 124(4):e596–e605; doi: 10.1542/peds.2008-3348.

33.

Jureidini

, Tonkin

, Jureidini

Combination pharmacotherapy for psychiatric disorders in children and adolescents: Prevalence, efficacy, risks and research needs. Paediatr Drugs, 2013; 15(5):377–391; doi: 10.1007/s40272-013-0032-6.

34.

Keast

, Tidmore

, Shropshire

, et al. Characterization of chronic multiclass psychotropic polypharmacy and psychotherapy in foster care youth in a state medicaid population. J Manag Care Spec Pharm, 2019; 25(12):1340–1348; doi: 10.1855/3/jmcp.2019.25.12.1340.

35.

Kohn-Wood

, Hooper

. Cultural competency, culturally tailored care, and the primary care setting: Possible solutions to reduce racial/ethnic disparities in mental health care. J Ment Health Couns, 2014; 36(2):173–188; doi: 10.1774/4/mehc.36.2.d73h217l81tg6uv3.

36.

Kovess

, Choppin

, Gao

, et al. Psychotropic medication use in French children and adolescents. J Child Adolesc Psychopharmacol, 2015; 25(2):168–175; doi: 10.1089/cap.2014.0058.

37.

Kreider

, Matone

, Bellonci

, et al. Growth in the concurrent use of antipsychotics with other psychotropic medications in Medicaid-enrolled children. J Am Acad Child Adolesc Psychiatry, 2014; 53(9):960.e2–970.e2; doi: 10.1016/j.jaac.2014.05.010.

38.

Lagerberg

, Molero

, D'Onofrio

, et al. Antidepressant prescription patterns and CNS polypharmacy with antidepressants among children, adolescents, and young adults: A population-based study in Sweden. Eur Child Adolesc Psychiatry, 2019; 28(8):1137–1145; doi: 10.1007/s00787-018-01269-2.

39.

Lê Cook

, Carson

, Kafali

, et al. Examining psychotropic medication use among youth in the U.S. by race/ethnicity and psychological impairment. Gen Hosp Psychiatry, 2017; 45:32–39; doi: 10.1016/j.genhosppsych.2016.12.004.

40.

, Yuan

Historical redlining and resident exposure to COVID-19: A study of New York City. Race Soc Probl, 2022; 14(2):1–16; doi: 10.1007/s12552-021-09338-z.

41.

Lohr

, Creel

, Feygin

, et al. Psychotropic polypharmacy among children and youth receiving Medicaid, 2012–2015. J Manag Care Spec Pharm, 2018; 24(8):736–744; doi: 10.1855/3/jmcp.2018.24.8.736.

42.

Martin

, Van Hoof

, Stubbe

, et al. Multiple psychotropic pharmacotherapy among child and adolescent enrollees in Connecticut Medicaid managed care. Psychiatr Serv, 2003; 54(1):72–77; doi: 10.1176/appi.ps.54.1.72.

43.

Medhekar

, Aparasu

, Bhatara

, et al. Risk factors of psychotropic polypharmacy in the treatment of children and adolescents with psychiatric disorders. Res Social Adm Pharm, 2019; 15(4):395–403; doi: 10.1016/j.sapharm.2018.06.005.

44.

Merikangas

, He

, Burstein

, et al. Lifetime prevalence of mental disorders in U.S. adolescents: Results from the National Comorbidity Survey Replication—Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry, 2010; 49(10):980–989; doi: 10.1016/j.jaac.2010.05.017.

45.

Nagelkerke

NJD.

A note on a general definition of the coefficient of determination. Biometrika, 1991; 78(3):691–692; doi: 10.1093/biomet/78.3.691.

46.

Reed

, Vance

, Luk

, et al. Single and combined psychotropic medication use in a child and adolescent mental health service. Aust N Z J Psychiatry, 2004; 38(4):204–211; doi: 10.1080/j.1440-1614.2004.01341.x.

47.

Safer

, Zito

. Pharmacoepidemiology of psychotropic medications in youth. In: Pharmacotherapy for Child and Adolescent Psychiatric Disorders. Rossenberg

, Gershon

. (eds). John Wiley & Sons, Ltd.: Chinchester, UK; 2012; pp. 7–23.

48.

Safer

, Zito

, DosReis

Concomitant psychotropic medication for youths. Am J Psychiatry, 2003; 160(3):438–449; doi: 10.1176/appi.ajp.160.3.438.

49.

Saucedo

, Liu

, Hincapie-Castillo

, et al. Prevalence, time trends, and utilization patterns of psychotropic polypharmacy among pediatric Medicaid beneficiaries, 1999–2010. Psychiatr Serv, 2018; 69(8):919–926; doi: 10.1176/appi.ps.201700260.

50.

Soni

Top Five Most Costly Conditions Among Children, Ages 0–17, 2012: Estimates for the U.S. Civilian Noninstitutionalized Population. Statistical Brief #472. Rockville, MD; 2015. Available from: https://www.meps.ahrq.gov/data_files/publications/st472/stat472.shtml [Last accessed: August 1, 2022 ].

51.

Ward

, Wiltshire

, Detry

, et al. African American men and women's attitude toward mental illness, perceptions of stigma, and preferred coping behaviors. Nurs Res, 2013; 62(3):185–194; doi: 10.1097/NNR.0b013e31827bf533.

52.

Whaley

AL.

Cultural mistrust and mental health services for African Americans: A review and meta-analysis. Couns Psychol, 2001; 29(4):513–531; doi: 10.1177/0011000001294003.

53.

Williams

, Collins

Racial residential segregation: A fundamental cause of racial disparities in health. Public Health Rep, 2001; 116(5):404–416; doi: 10.1093/phr/116.5.404.

54.

World Health Organization. WHO Collaborating Centre for Drug Statistics Methodology: ATC/DDD Index. 2020. Available from: https://www.whocc.no/atc_ddd_index [Last accessed: November 1, 2020 ].

55.

Zhang

, Spence

, Reeves

, et al. Characteristics of youths treated with psychotropic polypharmacy in the united states, 1999 to 2015. JAMA Pediatr, 2021; 175(2):196–198; doi: 10.1001/jamapediatrics.2020.4678.

56.

Zito

, Safer

, de Jong-van den Berg

, et al. A three-country comparison of psychotropic medication prevalence in youth. Child Adolesc Psychiatry Ment Health, 2008; 2(1):26; doi: 10.1186/1753-2000-2-26.

57.

Zito

, Zhu

, Safer

. Psychotropic polypharmacy in the US pediatric population: A methodologic critique and commentary. Front Psychiatry, 2021; 12:644741; doi: 10.3389/fpsyt.2021.644741.

58.

Zoëga

, Baldursson

, Hrafnkelsson

, et al. Psychotropic drug use among Icelandic children: A nationwide population-based study. J Child Adolesc Psychopharmacol, 2009; 19(6):757–764; doi: 10.1089/cap.2009.0003.

59.

Zuvekas

, Olin

. Validating household reports of health care use in the medical expenditure panel survey. Health Serv Res, 2009; 44(5p1):1679–1700; doi: 10.1111/j.1475-6773.2009.00995.x.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB