Effects of Exercise on Carotid Arterial Wall Thickness in Obese Pediatric Populations: A Meta-Analysis of Randomized Controlled Trials

Abstract

Background:

In pediatric populations, the use of carotid intima-media thickness (IMT) as a marker of cardiovascular risk has increased. However, previous studies examining the effects of exercise training on arterial structure and function in obese children and adolescents have shown inconsistent findings. The primary aim of this systematic review and meta-analysis is to expand on the current body of literature by providing a quantitative estimate of the change in carotid IMT following exercise training as well as to provide an exploratory analysis of potential moderators associated with the variation in response to an exercise training intervention in overweight and obese youth.

Materials and Methods:

A computerized search was made using three databases. The analysis was restricted to studies that examined the effect of exercise interventions on carotid IMT in pediatric obesity (6–18-year-olds). Hedges' g and 95% confidence intervals (CIs) were calculated.

Results:

Six randomized controlled trials (303 youths) were included. Exercise was associated with a small-to-moderate but significant reduction in carotid IMT (g = −0.306; 95% CI −0.540 to −0.072; p = 0.011). Likewise, exercise program duration per week significantly influenced the effect of exercise on carotid IMT (β = −0.060; p = 0.015).

Conclusions:

Exercise seems to reduce carotid IMT in childhood obesity. Therefore, encouraging obese pediatric individuals to become physically active can lead to favorable changes in the arterial wall.

Introduction

Childhood overweight and obesity represent a growing health concern in modern-day society.¹ Excess adiposity and particularly abdominal obesity are linked to raised lipid metabolism and the risk of developing insulin resistance and the metabolic syndrome, constituting the basis of impaired vascular function.² Impaired vascular function can begin in early life, with impairment of endothelial function a likely precipitating event, followed by gradual remodeling of the arterial wall.^3,4 Carotid intima-media thickness (IMT) is a validated surrogate marker that has been associated with increased risk of atherosclerosis in men and women.⁵

Carotid IMT is 10%–23% mm lower among physically active individuals compared with their inactive peers.⁶ Despite consistent evidence from cross-sectional studies indicating a positive association between sedentary behavior and carotid IMT,⁷ results of intervention studies have not yielded consistent evidence that exercise training can effectively lower carotid IMT. Fewer than half of the exercise training interventions examining this relationship produced significant improvements in carotid IMT,^8–13 and the mixed results highlight uncertainty about the ability of physical exercise alone to lower carotid IMT independently.¹⁴ Regular exercise can potentially modify metabolic hormones and is considered an important treatment of endothelial function and obesity-related conditions.¹⁵ In 2016, Cayres et al.¹⁶ published a thorough narrative review of epidemiological studies attempting to examine the effect of exercise training on carotid IMT in children and adolescents and concluded that improvement in carotid IMT occurred following exercise training. However, a number of milestone publications have suggested that exercise training can lower carotid IMT only when accompanied by a decrease in body weight and BMI,¹⁷ again yielding inconsistent and inconclusive results regarding the role of exercise in reducing carotid IMT.

Despite the prevalence of obesity and the multiple position stands promoting exercise for the treatment of obesity and vascular function, a meta-analytic approach has not previously been used to examine the effects in the obesity pediatric population. However, several studies also show inconsistent results particularly with respect to children and adolescents, where data are scarce. Because of the heterogeneity between studies in terms of results, we used a meta-analytic approach to determine the effectiveness of exercise interventions on carotid IMT in overweight and obese youth. Therefore, the primary aim of this systematic review and meta-analysis was to expand on the current body of literature by providing a quantitative estimate of the change in carotid IMT following exercise training as well as to provide an exploratory analysis of potential moderators associated with variation in response to an exercise training intervention in overweight and obese youth.

Materials and Methods

The study was undertaken in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.¹⁸ The review was registered with PROSPERO (CRD42016045232) at the University of York, United Kingdom.

Literature Search

A search of the literature was performed using the electronic databases, Cochrane Central Register of Controlled Trials (CENTRAL), EMBASE, and MEDLINE (all: until August 8, 2016). The terms used were [“Obesity” and “Overweight” OR], [“Exercise” and “Training” and “physical activity” and “sport” OR], and [“Children” and “Adolescents” OR]. All medical subject heading terms were combined with intima-media thickness* as limiter. Studies reported in languages other than English were not explored.

Study Selection and Inclusion Criteria

Two authors (A.G.-H. and R.R.-V.) independently screened the titles and abstracts of potentially eligible studies identified by the search strategy. The a priori inclusion criteria for this study were as follows: (1) children and/or adolescents (6–18-year-olds) classified as overweight or obese; (2) randomized controlled trials (RCTs) in which the control group received no physical exercise or dietary restriction intervention; (3) interventions of supervised exercise (without hypocaloric diet intervention) with or without nutritional guidance; and (4) an assessment of the IMT. Authors were contacted to provide missing data.

Data Collection

Two investigators (A.G.-H. and R.R.-V.) independently abstracted all data. For each trial, data were extracted regarding the first author's last name, year of publication, characteristics of subjects, exercise programs (type, frequency, duration, and intensity), assessment of carotid IMT, sample size, and differences in the means of two time points or postintervention mean values with corresponding standard deviations (SDs) of carotid IMT and BMI.

Risk of Bias

The quality of the studies was evaluated using the Physiotherapy Evidence Database (PEDro) criteria.¹⁹ The PEDro scale is an 11-item scale designed for rating the methodological quality of RCTs (eligibility criteria [no contribution to the total score], random allocation, concealed allocation, baseline comparability, blinded subjects, blinded therapists, blinded assessors, adequate follow-up, intention-to-treat analysis, between-group comparisons, and point estimates and variability). If studies were already listed on the PEDro database (www.pedro.org.au), these scores were adopted. A PEDro score of seven or above was considered of high quality, studies with a score of five or six were considered of moderate quality, and those with a score of four or less were deemed of poor quality.²⁰

Meta-Analysis Calculation

All analyses were carried out using Comprehensive Meta-analysis Software (second version; Biostat, Englewood, NJ) to calculate the standardized mean difference and expressed as Hedges' g to correct for possible small-sample bias.²¹ The standardized mean difference of the parameters from pre- to postintervention between groups (exercise vs. control²²) in each study was calculated and pooled using the fixed-effects model and Hartung–Knapp approach. Meta-analyses were completed for continuous data by using the change in the mean and SD of outcome measures. Values for intention to treat were preferred over per protocol values when both were present.

Heterogeneity between trial results was tested with a Cochran's Q test²³ and I² statistic. I² values of <25%, 25%–50%, and >50% are considered to represent small, medium, and large amounts of inconsistency.²⁴

Each study was deleted from the model once to analyze its influence on the overall results. Egger regression tests were performed to detect small-study effects²⁵ and possible publication bias.

Finally, we explored the heterogeneity between studies using metaregression and selected variables that may influence the association between exercise and IMT: BMI at baseline and change during study (as Hedges' g), total program duration of each study (weeks), and duration of exercise per week (min/week). Overall, significance was set at 0.05%.

Results

Study Selection

The flowchart is shown in Figure 1. The literature search resulted in 295 studies. Titles and abstracts of returned articles were searched for suitability, leading to the retrieval of 17 full texts. Of those, 11 were rejected—10 for failing the study design criterion (no control group or cross-sectional study) and one because of the type of intervention criterion (interventions with diet or no programmed exercise). Exclusion criteria and the list of excluded articles are in online Supplementary Data 1 (Supplementary Data are available online at www.liebertpub.com/chi). Finally, six RCTs met the inclusion criteria and were included in the meta-analysis.^8–13

Figure 1.

Flow chart for identification of trials for inclusion in the meta-analysis.

Description of Included Studies

Characteristics of included studies are shown in Table 1. A total of 173 boys and 130 girls were included in the meta-analysis.

Table 1.

Characteristics of the Included Trials

Study	Sample	Age (years)	BMI (percentile or kg/m²)	Type of exercise	Exercise program characteristics	Reduced intima-media thickness
Farpour-Lambert et al.	44 children (28 boys and 16 girls)	6–11	≥p97	Concurrent	Total duration: 16 weeks	No
					Frequency a week: three sessions
					Duration per session: 60-minute sessions.
					Aerobic; intensity: 55%–65% VO2max;
					Resistance; intensity: NR
Horner et al.	89 adolescents (45 boys and 44 girls)	12–18	≥p95	Aerobic or Resistance	Total duration: 12 weeks	Aerobic: no
					Frequency a week: three sessions
					Duration per session: 60-minute sessions.
					Aerobic; intensity: 60%–75% VO₂peak	Resistance: no
					Resistance; intensity: NR
Kim et al.	30 adolescents (boys)	17	25	Aerobic	Total duration: 12 weeks	Yes
					Frequency a week: three sessions
					Duration per session: 60-minute sessions.
					Intensity: NR
Meyer et al.	67 children and adolescents (34 boys and 33 girls)	11–16	≥p97	Aerobic	Total duration: 24 weeks	Yes
					Frequency a week: three sessions
					Duration per session: 70-minute sessions.
					Intensity: NR
Park et al.	29 children (14 boys and 15 girls)	12–13	≥p85	Concurrent	Total duration: 12 weeks	Yes
					Frequency a week: three sessions
					Duration per session: 70-minute sessions.
					Aerobic; intensity: 60%–70% heart rate reserve
					Resistance; intensity: 60% of their one-repetition maximum resistance
Poeta et al.	44 children (22 boys and 22 girls)	8–11	≥p95	Aerobic	Total duration: 12 weeks	Yes
					Frequency a week: three sessions
					Duration per session: 60-minute sessions.
					Intensity: 65%–85% maximum heart rate

One study included only boys,¹⁰ and five included a mixed sample of boys and girls.^8,9,11–13 Three studies used aerobic exercise,^9–11 one resistance exercise,¹² and two studies used a combined aerobic plus resistance exercise protocol (concurrent exercise).^8,13 One study used nutritional guidance without dietary restriction.¹¹ Study duration ranged from 12 to 24 weeks, and training frequency ranged from three to five times weekly, with 60–70-minute session duration. Regarding dropouts, three studies reported that no participants dropped out during the duration of the intervention.^9–11,13 Farpour-Lambert et al.⁸ and Horner et al.¹² reported three and four dropouts, respectively.

Measurement of IMT

All studies measured the carotid IMT by high-resolution B mode ultrasound at the carotid artery. We took into account the average values (Table 2). IMT measurements were obtained with the participant lying in the supine position and with the neck rotated to the opposite side of examination. Subsequently, the vessel was scanned by anterolateral view, with the transducer positioned parallel to the posterior border of the sternocleidomastoid muscle. IMT was reported as mean IMT value (6/6 studies), and maximum IMT was reported in 3/3 studies.^9–11 Only two studies considered values of intersonographer reproducibility.^9,13

Table 2.

Intima-Media Thickness Values (Baseline) and Intersonographer Parameters of the Included Trials

Study	Mean IMT, mm	Maximal IMT, mm	Site, view measure	Intersonographer reproducibility
Farpour-Lambert et al.	0.48	Not reported	CCA, longitudinal, AL	Not reported
Horner et al.	0.50	Not reported	CCA, longitudinal, AL	Not reported
Kim et al.	0.51	0.66	CCA, longitudinal, AL	Not reported
				Mean difference = 0.034–0.076
Meyer et al.	0.53	0.58	CCA, longitudinal, AL	Intraobserver = 0.2%
				Interobserver = 1.2%
Park et al.	0.43	Not reported	CCA, longitudinal, AL	Test–retest coefficient of variation = 0.6%
				Intraclass correlation coefficient = 0.5%–0.8%
Poeta et al.	0.43	0.56	CCA, longitudinal, AL	Not reported

IMT, intima-media thickness; CCA, common carotid artery; AL, anterolateral.

Risk of Bias

Only one study was considered high quality,⁸ two studies were considered moderate quality,^12,13 and three studies were considered poor quality.^9–11 All included trials had random allocation between groups and provided points and estimates of variability. Two studies performed concealed allocation.^8,12 Blinding of participants and therapists was not possible because of the nature of interventions. Assessor blinding was implemented in one of the included trials.⁸ In addition, two studies had adequate follow-up^8,13 and included an intention-to-treat analysis.^8,12 Complete details are reported in electronic Supplementary Table S1.

Change in IMT

Physical exercise was associated with a significant reduction in the fixed-effects model (g = −0.306; 95% CI −0.540 to −0.072; p = 0.011) with medium heterogeneity (I² = 47.32%; Fig. 2).

Figure 2.

Absolute changes in carotid intima-media thickness in individual studies of exercise vs. control groups.

Univariate Moderator Analysis

The independent effects of potential moderating variables were examined using metaregression and are presented in Table 3. The metaregression analysis shows that there was a greater decrease in carotid IMT in studies with longer duration (Table 3). The slope regression coefficient was −0.060 [95% CI −0.136 to −0.015] and was significant (p = 0.015). These data indicate that greater decrease in carotid IMT was observed in individuals who achieved larger interventions in terms of minutes per week.

Table 3.

Summary of Univariate Moderator Analysis

Effect moderator	Unstandardized ß	95% CI	p
Age	0.082	−0.002 to 0.167	0.057
Baseline BMI	0.096	−0.035 to 0.156	0.052
Change BMI (Hedges' g)	−0.038	−0.602 to 0.526	0.894
Exercise session duration (weeks)	0.036	−0.016 to 0.089	0.177
Exercise program duration (min/week)	−0.060	−0.136 to −0.015	0.015

BMI, body–mass index; CI, confidence interval.

Publication Bias and Sensitivity Analysis

Egger's linear regression tests provided no evidence for a potential publication bias outcome (Egger regression intercept, −4.03 [p = 0.137]). In the sensitivity analysis, with each study deleted from the model once, the results remained across all deletions, except for one study¹³ (p = 0.079).

Discussion

The cumulative evidence gathered from peer-reviewed research published until 2016 indicates that a moderated and significant decrease in carotid IMT occurs following supervised exercise training. In addition, the exploratory moderator analysis shows that there was a greater decrease in carotid IMT in studies with longer duration. This analysis indicated that these individual program design features are independently associated with the change in carotid IMT after accounting for exercise program duration.

Previous studies examining the effects of exercise training on arterial structure and function in obese children and adolescents have shown inconsistent findings.^8–13,26,27 Perhaps differences in the duration of exercise intervention, magnitude of weight loss, and/or the type of exercise training may explain some of the inconsistency in results. Our meta-analysis shows that exercise was associated with a significant small-to-moderate reduction in carotid IMT. This finding is consistent with previous narrative and systematic reviews indicating that supervised exercise training results in the largest improvement in the arterial wall for a longer duration as a result of the intervention.^6,28 This finding is clinically significant. Subclinical atherosclerosis has become the most common cause of CVD among children and adolescents in Western societies, affecting approximately between 5% and 15% of them, but increases up to 55% in obese children.²⁹ To illustrate this point, an analysis of longitudinal data from the Muscatine Study also indicated that childhood BMI, measured between the ages of 8 and 18 years, is correlated with carotid IMT and coronary artery calcification at the age of 29–43 years.¹⁷ For instance, the Bogalusa Heart Study demonstrated that in youths who died at an average age of 19.6 years, there was a direct association between the degree of subclinical atherosclerosis in the coronary arteries and the levels of antemortem cardiovascular risk factors, including BMI, lipids, and blood pressure.³⁰

Likewise, our results suggest that exercise program duration per week significantly influenced the effect of exercise on carotid IMT. Overall, improvements in carotid IMT have been reported following long-term (≥24 weeks) exercise programs with (7,24) and without diet (42) in obese youth. The findings from short-term (12 weeks) exercise intervention studies generally report no significant changes in this parameter,^8,12 except for two studies that reported significant reductions in carotid IMT,^10,13 exactly those interventions with more minutes per week of exercise (240–250 min/week).

Physiologically, physical training may delay the development of the arterial wall by increasing shear stress and augmenting nitric oxide (NO) bioavailability.³¹ In this context, several biologically plausible mechanisms could explain the effects of exercise in modulating the endothelial function and arterial stiffness. It is widely known that exercise has the potential to reduce oxidative stress by increasing the efficiency of the antioxidant system, finally improving endothelial dysfunction.³² The main physiological mechanisms involved upregulating endothelial NO synthase activity in cell culture, animal, or human studies, with subsequent reduction in the expression of nicotinamide adenine dinucleotide (phosphate) (NAD(P)H) oxidase and stimulating radical scavenging systems that include copper/zinc-containing superoxide dismutase (SOD), extracellular SOD, glutathione (GSH) peroxidase, and GSH levels.³¹ However, further research is needed to confirm these mechanisms, especially in childhood obesity during and after weight-loss exercise regimens.

Strengths and Limitations

To our knowledge, this is the first meta-analysis that evaluates the changes in carotid IMT after exercise training in overweight/obese children and adolescents. Therefore, obese youths should be considered as a prime opportunity to engage young people at risk of developing (or with established) CVD in dedicated physical activity programs as young people may benefit from such interventions even more than adults. In fact, the primary focus is on adult cardiovascular health and disease prevention, but critical to achievement of this goal is maintenance of ideal cardiovascular health from birth through childhood to young adulthood and beyond.³³ However, further research is needed to confirm these mechanisms, especially in childhood obesity during and after weight-loss exercise regimens.

Nevertheless, some important limitations with regard to our study exist. First, the number of RCTs included was small, although their homogeneity was optimized by the stringency of the inclusion criteria. Second, there is inconsistency regarding the definitions of overweight and obesity. Third, moderate statistical heterogeneity levels were detected overall for the effect estimates, and thus, caution is warranted in interpretation of our findings. In addition, we included preadolescent and postadolescent children; studies have demonstrated a very different physiological response to exercise between 6- and 17-year-old youths, largely because of age-dependent metabolic and musculoskeletal differences.³⁴ Finally, we must carefully interpret the findings due to limited number of studies included in metaregression analysis (fewer than 10 studies).²¹

In conclusion, this systematic review and meta-analysis shows that physical exercise is an effective way to change carotid IMT in overweight/obese pediatric individuals (6–18-year-olds), although the magnitude of the standardized difference could be considered small to moderate. Encouraging individuals to become physically active can lead to favorable changes in the arterial wall. High-quality studies are necessary to determine an optimal exercise program and dose and to assess the clinical relevance of the results of available research.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

PROSPERO Registration: CRD42016045232 ().

References

Ogden

, Carroll

, Lawman

, et al. Trends in obesity prevalence among children and adolescents in the United States, 1988–1994 Through 2013–2014. JAMA, 2016; 315:2292–2299.

Klöting

, Blüher

. Adipocyte dysfunction, inflammation and metabolic syndrome. Rev Endocr Metab Disord, 2014; 15:277–287.

Tuzcu

, Kapadia

, Tutar

, et al. High prevalence of coronary atherosclerosis in asymptomatic teenagers and young adults evidence from intravascular ultrasound. Circulation, 2001; 103:2705–2710.

Kahlon

, Torey

, Nordstrom

, et al. The impact of coronary artery disease risk factors on intravascular ultrasound-derived morphologic indices of human coronaries. Echocardiography, 2006; 23:308–311.

de Groot

, Hovingh

, Wiegman

, et al. Measurement of arterial wall thickness as a surrogate marker for atherosclerosis. Circulation, 2004; 109:33–38.

Thijssen

, Cable

, Green

. Impact of exercise training on arterial wall thickness in humans. Clin Sci, 2012; 122:311–322.

García-Hermoso

, Martínez-Vizcaíno

, Recio-Rodríguez

, et al. Sedentary behaviour patterns and carotid intima-media thickness in Spanish healthy adult population. Atherosclerosis, 2015; 239:571–576.

Farpour-Lambert

, Aggoun

, Marchand

, et al. Physical activity reduces systemic blood pressure and improves early markers of atherosclerosis in pre-pubertal obese children. J Am Coll Cardiol, 2009; 54:2396–2406.

Meyer

, Kundt

, Lenschow

, et al. Improvement of early vascular changes and cardiovascular risk factors in obese children after a six-month exercise program. J Am Coll Cardiol, 2006; 48:1865–1870.

10.

Kim

, Kim

, Jeon

, et al. Improved insulin resistance, adiponectin and liver enzymes without change in plasma vaspin level after 12 weeks of exercise training among obese male adolescents. Korean J Obes, 2001; 20:138–146.

11.

Poeta

, Duarte Mde

, Caramelli

, et al. Effects of physical exercises and nutritional guidance on the cardiovascular risk profile of obese children. Rev Assoc Med Bras, 2013; 59:56–63.

12.

Horner

, Barinas-Mitchell

, DeGroff

, et al. Effect of aerobic versus resistance exercise on pulse wave velocity, intima media thickness and left ventricular mass in obese adolescents. Pediatr Exerc Sci, 2015; 27:494–502.

13.

Park

J-H

, Miyashita

, Kwon

Y-C

, et al. A 12-week after-school physical activity programme improves endothelial cell function in overweight and obese children: A randomised controlled study. BMC Pediatr, 2012; 12:1.

14.

Palatini

, Puato

, Rattazzi

, et al. Effect of regular physical activity on carotid intima-media thickness. Results from a 6-year prospective study in the early stage of hypertension. Blood Press, 2011; 20:37–44.

15.

Dias

, Green

, Ingul

, et al. Exercise and vascular function in child obesity: A meta-analysis. Pediatrics, 2015; 136:648–659.

16.

Cayres

, Agostinete

, Antunes

, et al. Impact of physical exercise/activity on vascular structure and inflammation in pediatric populations: A literature review. J Spec Pediatr Nurs, 2016; 3:99–108.

17.

Davis

, Dawson

, Riley

, et al. Carotid intimal-medial thickness is related to cardiovascular risk factors measured from childhood through middle age the muscatine Study. Circulation, 2001; 104:2815–2819.

18.

Liberati

, Altman

, Tetzlaff

, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Med, 2009; 6:e1000100.

19.

Maher

, Sherrington

, Herbert

, et al. Reliability of the PEDro scale for rating quality of randomized controlled trials. Phys Ther, 2003; 83:713–721.

20.

Lemes

ÍR

, Ferreira

, Linares

, et al. Resistance training reduces systolic blood pressure in metabolic syndrome: A systematic review and meta-analysis of randomised controlled trials. Br J Sports Med, 2016; 50:1438–1442.

21.

Higgins

, Green

. Cochrane Handbook for Systematic Reviews of Interventions. Wiley Online Library, New Jersey, 2008.

22.

Morris

. Estimating effect sizes from the pretest-posttest-control group designs. Organ Res Meth, 2008; 11:364–386.

23.

Higgins

, Thompson

, Deeks

, et al. Measuring inconsistency in meta-analyses. BMJ, 2003; 327:557–560.

24.

Higgins

, Thompson

. Quantifying heterogeneity in a meta-analysis. Stat Med, 2002; 21:1539–1558.

25.

Egger

, Smith

, Schneider

, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ, 1997; 315:629–634.

26.

Woo

, Chook

, Chung

, et al. Effects of diet and exercise on obesity-related vascular dysfunction in children. Circulation, 2004; 109:1981–1986.

27.

Kelishadi

, Hashemi

, Mohammadifard

, et al. Association of changes in oxidative and proinflammatory states with changes in vascular function after a lifestyle modification trial among obese children. Clin Chem, 2008; 54:147–153.

28.

Thijssen

, Carter

, Green

. Arterial structure and function in vascular ageing: Are you as old as your arteries?. J Physiol, 2016; 594:2275–2284.

29.

Canas

, Sweeten

, Balagopal

. Biomarkers for cardiovascular risk in children. Curr Opin Cardiol, 2013; 28:103–114.

30.

Berenson

, Srinivasan

, Bao

, et al. Association between multiple cardiovascular risk factors and atherosclerosis in children and young adults. N Engl J Med, 1998; 338:1650–1656.

31.

Lessiani

, Santilli

, Boccatonda

, et al. Arterial stiffness and sedentary lifestyle: Role of oxidative stress. Vascul Pharmacol, 2016; 79:1–5.

32.

Schuler

, Adams

, Goto

. Role of exercise in the prevention of cardiovascular disease: Results, mechanisms, and new perspectives. Eur Heart J, 2013; 34:1790–1799.

33.

Steinberger

, Daniels

, Hagberg

, et al. Cardiovascular health promotion in children: Challenges and opportunities for 2020 and beyond. Circulation, 2016; 134:236–255.

34.

Boisseau

, Delamarche

. Metabolic and hormonal responses to exercise in children and adolescents. Sports Med, 2000; 30:405–422.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.01 MB