Abstract
Abstract
Background:
Physicians can play an important role in preventing and treating childhood obesity. There are currently no validated measures of medical students' self-efficacy in these skills; therefore, we sought to develop a valid and reliable computerized survey to measure medical students' self-efficacy in skills needed to prevent and treat childhood obesity.
Methods:
We developed the Childhood Obesity Prevention Self-Efficacy (COP-SE) survey with input from two expert panels and cognitive interviews with medical students. We administered the 43-item COP-SE computerized survey to a nation-wide sample of medical students.
Results:
The final sample consisted of 444 medical students from 53 medical schools. Exploratory factor analysis revealed a two-factor structure with a correlation of 0.637 between factors and high reliability within factors. The correlation between the COP-SE and a measure of general self-efficacy was moderate (0.648), and reliability within factors was high (Factor 1=0.946; Factor 2=0.927).
Conclusions:
The 18-item COP-SE is a valid and reliable measure of childhood obesity prevention self-efficacy. Factor 1 assesses self-efficacy in nutrition counseling, and Factor 2 measures self-efficacy to assess readiness to change and initiate nutrition lifestyle changes. The correlation between the COP-SE and a measure of general self-efficacy indicates that the COP-SE is a distinct, valid assessment of domain-specific self-efficacy. The high reliability of items within factors indicates the items measure the same constructs. Therefore, medical schools can use this valid and reliable instrument as a formative or summative assessment of students' self-efficacy in childhood obesity prevention and treatment.
Introduction
Childhood obesity is a national concern that affects 17% of children 2–19 years of age. 1 Obesity was recently recognized as a disease by the American Medical Association, 2 which will likely encourage the medical community to have an increased focus on early prevention and treatment. Therefore, physicians need to be equipped with skills to treat childhood obesity because they see children in frequent, regular intervals. 3
The goal of medical school is to provide future physicians with the information and skills needed to prevent and treat health problems, such as childhood obesity. Ideally, this would involve nutrition education and counseling skill building; however, nutrition education in medical school is limited to an average of only 19.6 hours of instruction over 4 years. 4 Not surprisingly, medical students are dissatisfied with nutrition education,4–7 and many medical students lack confidence in their ability to provide counseling for obese children and their families. 7
This lack of self-efficacy continues into practice. 8 Even among physicians with additional training, few feel competent in treating childhood obesity or addressing weight concerns with children. 9 Moreover, few physicians believe that they can help obese patients lose weight, 8 and they rarely use nutrition behavior change counseling 10 or heed American Academy of Pediatrics' (AAP's) Expert Committee Recommendations for assessment, prevention, and treatment of childhood obesity. 11
A greater emphasis on nutrition training and behavior change is needed in medical school curricula in order to prepare medical students to prevent and treat childhood obesity. Providing this training to medical students, rather than just residents, may be beneficial in preparing physicians of all specialties with the nutrition-specific counseling skills needed to prevent and treat childhood obesity. Recognizing the importance of this training in preventing and treating childhood obesity, the Institutes of Medicine's Committee on Accelerating Progress in Obesity Prevention recently recommended that medical schools should provide students with nutrition education and training in motivational interviewing and counseling. 12 There is no standardized behavior change counseling curriculum in medical schools; however, the transtheoretical model, motivational interviewing, and the 5A's model of behavioral counseling are commonly taught.13,14 These commonly taught principles lack utility in the absence of nutrition knowledge; therefore, medical schools need to provide their students with nutrition knowledge so they can provide nutrition advice and recommendations to their patients. Employing nutrition behavior change counseling with obese children and their parents can help promote healthier eating practices, 15 and patients report higher satisfaction with care when physicians take time to offer advice. 14 Therefore, future physicians need to possess proficiency in both nutrition knowledge and counseling practices to be able to engage patients in change.
As medical schools adapt curricula to involve both nutrition education and counseling education, they will need ways to measure the effectiveness of newly adapted programs. One option is to examine medical students' self-efficacy. Self-efficacy is defined as confidence in one's skills 16 and is important because, in general, higher self-efficacy leads to persistence in problem solving and subsequent improved performance of the target skill. The addition of nutrition-specific counseling training can also increase both future physicians' knowledge and self-efficacy.17–19
Given that self-efficacy is an important marker of proficiency in a skill, a self-efficacy survey would assist medical schools in both formative and summative evaluation of students' proficiency in preventing and treating childhood obesity. Nutrition knowledge 20 and attitudinal 21 surveys have been developed for medical students and residents, 6 measuring nutrition knowledge and attitudes as a whole; however, these surveys do not assess medical students' self-efficacy in childhood obesity-specific nutrition counseling skills. Without confidence to use behavior change counseling with patients, nutrition knowledge is just that—knowledge. A survey measuring medical students' self-efficacy in skills related to preventing and treating childhood obesity could be beneficial for medical schools to determine students' confidence before and after targeted training and what skills schools need to reinforce in students through additional training. Therefore, the aim of this study was to develop a valid and reliable computerized survey to measure medical students' self-efficacy in skills needed to prevent and treat childhood obesity.
Methods
Instrument Development
We developed the Childhood Obesity Prevention Self-Efficacy (COP-SE) survey through a rigorous multiphase process to ensure both reliability and validity. 22 During phase 1, we determined the scope of the proposed survey using results from a qualitative study of third- and fourth-year medical students' needs and barriers to preventing and treating childhood obesity. 7 A review of the literature including the AAP's Expert Committee Recommendations 23 and common behavior change theories/methodologies also informed the item content. 13 Based on this information, we developed an 88-item pool of self-efficacy and belief items.
In phase 2, we sent the 88-item pool to a panel of 10 experts in nutrition, childhood obesity, pediatrics, medical education, motivational interviewing, and survey development for a check of content validity. The expert panel rated the relevancy of each item using a 5-point Likert scale and provided additional comments, where appropriate. After calculating an average relevancy score for each item and compiling comments, we reduced the pool of items to 48.
In phase 3, we conducted cognitive interviews over the phone with five medical students to determine their thought processes in responding to each of the 48 items as a check of content validity. We conducted the interviews using a standardized semi-structured interview guide. Before each interview, students completed an online version of the survey. Using a combination of the think-aloud method and the verbal probe method, 24 we asked each student about (1) the meaning of each item, (2) what actions they would take to complete the skill, (3) their confidence in the skill, (4) their belief in the importance of the skill, and (5) how they might change the item. The institutional review board at North Carolina State University (Raleigh, NC) approved the interview protocol.
After making revisions based on the cognitive interviews, in phase 4, we shared the revised 46-item survey with eight experts not in the original panel and asked them to organize the items into six proposed subscales as a check of face validity. Applying feedback from these experts, we decided to split double-barreled items asking about “obese children and family members” into two items per concept (i.e., one item about the obese child and one about the obese child's family members). We then prepared the final survey for online administration with medical students.
Instrument
The survey consisted of 43 self-efficacy items organized by a header with the phrase “I am confident I can” followed by each item containing a stem (e.g., “Describe to an obese child's family members how to choose healthy snack alternatives”). The items were ranked on a 5-point Likert scale from “strongly disagree” to “strongly agree.”
Study Participants
The survey was administered in May–August 2013 through the Qualtrics online survey system (Qualtrics, Provo, UT) to medical students in their first, second, third, or fourth year of allopathic or osteopathic medical school. We contacted medical school administrators and medical students at all allopathic and osteopathic medical schools in the United States, providing them with a link for the online survey. Before gaining access to the COP-SE survey, students completed a 10-question demographic survey for inclusion/exclusion purposes. To be eligible for this study, students had to be over the age of 18 and planning to specialize in family medicine, internal medicine-pediatrics, or pediatrics or undecided. For each completed survey, $1 was donated to the American Cancer Society as an incentive for participation. The survey administration protocol was also approved by the institutional review board at North Carolina State University.
Statistical Analyses
Validity and reliability
We used the Qualtrics software system (Qualtrics) for data management during data collection and IBM-SPSS software (Version 21; IBM Corp, Armonk, NY) for data management and statistical analyses after data collection ended.
Data quality and descriptive statistics
A Kaiser-Meyer-Olkin (KMO) statistic was used to determine sampling adequacy, and a Bartlett's test was used to test for sphericity. 25 In order to explore trends in students' self-efficacy under different demographic conditions, we calculated means and standard deviations and conducted pooled variance t-tests. Statistical significance was defined as p<0.05.
Factor analysis
We performed an exploratory factor analysis of the 43 self-efficacy items, using a principle axis factoring with an oblique rotation (Promax), allowing for correlation between items. 25 Scree plots and communalities aided in determining the factor structure, and items with correlations below 0.30 and loading on multiple factors in the pattern matrix were deleted to achieve the final factor structure. 25
Concurrent validity
Although there is currently no validated self-efficacy survey for healthcare providers, we administered the valid and reliable New Generalized Self-Efficacy (NGSE) scale 26 at the same time as the COP-SE and calculated a correlation between the two measures as a test of concurrent validity.
Reliability assessment
For item analysis, we used IBM-SPSS software (Version 21; IBM Corp) to calculate Cronbach's alpha for each factor determined by the factor analysis, using the cutoff of 0.70 or higher for internal consistency. 24 We also used the inter-item correlation matrix and new Cronbach's alpha if an item was deleted in order to exclude poor items from each factor.
Results
Sample Characteristics
Of the 168 medical schools contacted, 53 (32%) had eligible students who completed the COP-SE survey in its entirety for a total sample size of 444 participants, with roughly equal numbers of second- and third-year medical students. More first-year and fewer fourth-year students completed the survey. Most students classified themselves as undecided or planning to specialize in pediatrics. More allopathic students completed the survey than osteopathic students and more females than males. A summary of demographics can be found in Table 1.
Participant Demographics of a Nation-Wide Sample of First-, Second-, Third-, and Fourth-Year Medical Students (n=444) Taking the Childhood Obesity Prevention Self-Efficacy (COP-SE) Survey
Factor Analysis
The KMO statistic of 0.945, classified as “superb,” confirmed that the sample size was adequate for the factor analysis, and a significant Bartlett's test of sphericity confirmed that the correlation matrix was significantly different from zero. 25
The exploratory factor analysis revealed a two-factor structure with 10 items in Factor 1 and eight items in Factor 2 (Table 2). In addition to factor loadings, Table 2 lists both initial and extraction communalities for each item, indicating the effect of the oblique rotation.
Pattern Matrix for Principal Axis Factoring Factor Analysis with Promax Oblique Rotation of the Two-Factor Solution of Items in the Childhood Obesity Prevention Self-Efficacy (COP-SE) Survey (n=444)
Ranked on a 5-point Likert scale: 1=strongly disagree; 2=disagree; 3=neutral; 4=agree; 5=strongly agree.
SD, standard deviation.
The correlation between Factor 1 and Factor 2 (r=0.637) indicated that the two factors were correlated, but not too highly correlated to result in a one-factor solution. Items that loaded on Factor 1 related to self-efficacy in nutrition counseling whereas items that loaded on Factor 2 related to self-efficacy to assess readiness to change and initiate nutrition lifestyle changes.
Reliability Assessment
A Cronbach's alpha of 0.946 and 0.927 for Factor 1 and Factor 2, respectively, indicated high internal consistency within each factor (Table 3).
Descriptive Statistics for the Two Factors in the Childhood Obesity Prevention Self-Efficacy (COP-SE) Survey (n=444)
SD, standard deviation.
Concurrent Validity
The correlation between the NGSE and the COP-SE (r=0.648) was significant (p=0.01). As expected, this correlation indicates that the COP-SE is a valid assessment of self-efficacy and not a duplication of a measurement of general self-efficacy. This finding fulfills the aim of the research: to develop a domain-specific measure of self-efficacy.
Demographic Analysis
As a whole, the mean score for items in Factor 1 was higher than Factor 2 (Table 3). This same trend was observed across all demographic groups (Table 4).
Mean Scores (SD) for Medical Student Demographic Groups on Each Factor in the Childhood Obesity Prevention Self-Efficacy (COP-SE) Survey
SD, standard deviation.
No significant differences were noted between males and females on either Factor 1 or Factor 2. For Factor 1, osteopathic students tended to score higher than allopathic students; however, there was no significant difference. For Factor 2, osteopathic students scored significantly higher than allopathic students (t=−2.31; df=442; p=0.02; r=0.11).
A trend of increasing scores was observed across the four medical school years, with first years scoring the lowest and fourth years scoring the highest. For Factor 1, significant differences were noted between students in years 1 and 3 (t=−3.07; df=274; p<0.001; r=0.18), years 1 and 4 (t=−3.32; df=207; p<0.001; r=0.22), years 2 and 3 (t=−2.59; df=233; p=0.01; r=0.17), and years 2 and 4 (t=−3.11; df=166; p<0.001; r=0.23). There were no significant differences between years 1 and 2 or years 3 and 4. Similar results were observed for Factor 2, with significant differences being noted between students in years 1 and 3 (t=−2.92; df=274; p<0.001; r=0.17), years 1 and 4 (t=−3.14; df=207; p<0.001; r=0.22), years 2 and 3 (t=−2.15; df=233; p=0.03; r=0.14), and years 2 and 4 (t=−2.51; df=166; p=0.01; r=0.19). There were also no significant differences between years 1 and 2 or years 3 and 4 for Factor 2.
For Factor 1, those intending to specialize in family medicine scored significantly higher than undecided students (t=2.64; df=278; p=0.01; r=0.16) and those intending to specialize in pediatrics scored significantly higher than undecided students (t=3.06; df=305; p<0.001; r=0.17). Similar results were observed for Factor 2 with those intending to specialize in family medicine scoring significantly higher than undecided students (t=3.12; df=278; p<0.001; r=0.18) and those intending to specialize in pediatrics scoring significantly higher than undecided students (t=2.87; df=305; p<0.001; r=0.16). No other significant differences were noted with regard to specialty.
Discussion
In this study, we created a valid and reliable survey to assess medical students' self-efficacy for childhood obesity-specific prevention and treatment counseling skills, resulting in the 18-item COP-SE survey. The final factor structure of the COP-SE survey included two dimensions: (1) self-efficacy in providing nutrition counseling and (2) self-efficacy in assessing readiness and initiating nutrition lifestyle change. Though the two subsets of skills are correlated, they are distinct. Factor 1 requires medical students to have specific nutrition knowledge, whereas Factor 2 relates to the students' comfort with general tenets of behavior change counseling theories related to nutrition. Reliability analysis confirmed high rates of internal validity for both factors. Validity checks were used for content and face validity in the development of the survey, and a test of concurrent validity with the NGSE confirmed that the COP-SE is a valid assessment of domain-specific self-efficacy. A moderate correlation between the two factors indicates that the COP-SE measures a construct similar to the NGSE. Because the NGSE is a general self-efficacy survey and the COP-SE is domain specific, we expected that the two would not be highly correlated. The purpose of developing the COP-SE survey was to create a domain-specific measure of self-efficacy because it could be argued that high general self-efficacy might not translate to high domain-specific self-efficacy in the absence of adequate training and experience.
Items loading on Factor 1 related to describing diet-related topics, including how to choose healthy foods and snacks, eat more fruits and vegetables, and what a healthy diet should include as well as being able to discuss health impacts of obesity and benefits of making lifestyle changes. Factor 2, however, requires a more advanced skill related to nutrition behavior change counseling. Items loading on this factor related to using motivational interviewing, determining when a patient recognizes that a change needs to occur, determining readiness to change, and helping patients overcome barriers to change.
It is logical that students scored higher on Factor 1 than Factor 2 because more-advanced counseling techniques require more practice and skill. Our previous qualitative exploration of medical students' needs to prevent and treat childhood obesity indicated that students feel inadequate in their nutrition knowledge and counseling skills and would benefit from additional training. 7 This inadequacy is also confirmed by moderate scores on Factor 1.
There were no significant differences in gender. Previous studies that have found that males report higher social sciences academic self-efficacy, of which behavior change counseling is a subset. 27 Female physicians tend to score higher on counseling self-efficacy,10,28 which may be because female physicians tend to spend longer with patients than male physicians. 28 Additionally, osteopathic students scored significantly higher than allopathic students on Factor 2 only, which could be explained by the emphasis of holistic healthcare by osteopathic medical schools 29 and potentially more practice with behavior change counseling as a result.
We observed statistically significant differences between years 1 and 3, years 2 and 3, years 1 and 4, and years 2 and 4 on both Factor 1 and Factor 2. These differences in self-efficacy follow a natural progression of acquisition of knowledge. It can be assumed that first-year medical students feel less confident in providing nutritional advice and counseling because they have not yet learned what advice to give or best practices in counseling. Medical schools typically provide students with the majority of nutrition instruction during their first 2 years, 4 but students may not feel efficacious until they have a chance to master skills outside of the classroom. 16 Further, only roughly one quarter of students had completed their pediatric clerkship where they would typically be able to practice these techniques. 4 It is important to note that we did not see significant differences between years 1 and 2 and years 3 and 4, indicating that self-efficacy changes can likely be attributed to the progression from didactic to clinical training.
Although self-efficacy did not differ across most specialties, we did observe a statistically significant difference between students intending to specialize in family medicine and those who were undecided, with those planning to specialize in family medicine rating their self-efficacy as higher than the undecided students. The same trend was noted between students intending to specialize in pediatrics and those who were undecided. This could have been because students do not tend to choose a specialty until their third year of medical school or because many medical students change their intended specialty over the course of medical school. 30 It is also possible that undecided students may not actually pursue careers where they will work with obese children, perhaps leading them to focus less on learning nutrition behavior change counseling techniques for pediatrics.
Although the COP-SE is a valid and reliable survey, the factor structure needs to be confirmed by a large nation-wide sample of medical students of all intended specialties, which would confirm the structure in the medical school population as a whole, potentially leading to widespread use of the survey in medical education. Because the survey is self-report, it does not measure skills of students, only their perceived self-efficacy. Therefore, a performance assessment would be useful to compare self-efficacy to performance. Additionally, there could have been a social desirability response in which students believed they should be confident in these skills and therefore rated themselves higher than their actual self-efficacy or skill level. This is a limitation common in self-report measures; however, our rigorous methodology attempted to limit this effect.
Conclusions
The COP-SE is a valid and reliable survey that could be used to assess self-efficacy in childhood obesity-specific counseling practices, but future research should involve the confirmation of the factor structure. It is our hope that medical schools will start to use this valid and reliable instrument to assess their students' self-efficacy in counseling skills to inform future curricular changes or as a pre/post measure of self-efficacy to evaluate the effectiveness of their training programs.
Footnotes
Acknowledgment
The procedures followed were in accord with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008.
Author Disclosure Statement
No competing financial interests exist.
