Using Audience Response Systems to Enhance Chronic,Noncancer Pain Knowledge Acquisition Among Veterans

Abstract

Background: Educating patients on factual information related to chronic pain and self-management improves knowledge, pain outcomes, and compliance. The current study assesses changes in pain knowledge among veterans who attended a 12-week “Pain Education School” and examines whether or not the addition of an active learning component (i.e., use of an audience response system [ARS]) leads to greater increases in pain knowledge. Materials and Methods: A sample of 102 veterans 24–84 years of age who elected to participate in the 12-week pain education program was evaluated. Only a subgroup of veterans (n=69) had access to the ARS. Results: A 2×2 repeated-measures multivariate analyses of variance was conducted to examine changes in pain knowledge, pain beliefs, stage of readiness to adopt a self-management approach, and depressive symptoms over time and to explore the impact of ARS use on pain knowledge acquisition. Participants reported a statistically significant difference between their pre- and posttest measures of pain beliefs (p=0.04), stage of readiness to adopt a self-management approach (p=0.00), and depressive symptoms (p=0.00). Veterans who used the ARS demonstrated significantly greater increases in pain knowledge acquisition after completing the pain education program than those who did not (p=0.03). Conclusions: Findings provide support for the incorporation of ARS in pain education programming to facilitate active learning and to increase knowledge acquisition. Findings also provide evidence for the effectiveness of pain education programs in improving veterans' attitudes about their pain, increasing readiness to engage in self-management, and decreasing depressive symptoms.

Introduction

Research has shown that patients who receive clear information regarding their pain and treatment are more likely to be satisfied with their care and symptom improvement, more likely to adhere to treatment recommendations, and more likely to use self-management strategies.¹ Educating patients on factual information related to chronic pain and self-management improves knowledge, pain outcomes, and compliance.^2

–5 Without proper pain education, patients may be misinformed or maintain unrealistic expectations regarding their pain prognosis and treatment.^6
–8 Patients who maintain impractical beliefs and attitudes about their pain are less likely to engage in self-management behaviors and more likely to seek more potent medications and risky procedures for pain relief.^9,10 Therefore, understanding patients' pain knowledge and determining ways to enhance accurate knowledge acquisition have become focuses of chronic pain research.

Many health education programs deliver information via traditional lecture-based classes—a process characterized as passive learning.¹¹ The Cone of Learning Model was adapted from Edgar Dale in 1969 and demonstrates how people learn and retain knowledge (Fig. 1).¹² The model shows that after 2 weeks, humans remember only 10% of what they read but can remember 90% of what they are actively engaged in, or do. According to this model, active learning encompasses a variety of education strategies that facilitate pupil involvement in the learning process, whereas passive learning consists of the traditional lecture-based approach to teaching.^13

–16 Active learning strategies have been associated with increased pupil engagement, improved performance on assessments of course material, and increased motivation.¹⁵

Fig. 1.

The Cone of Learning Model adapted from Edgar Dale.¹²

Past research has used different types of technology-assisted education systems in order to promote a more active learning environment and simultaneously obtain additional measurements of change in knowledge.^17
–19 These audience response systems (ARSs) have been shown to improve participant engagement and attention,²⁰ increase attendance,²¹ reinforce key information,²² become a conduit for communication between pupil and instructor to improve instruction,²³ and improve learning performance and retention of presented material.^24,25 ARSs consist of visual presentation software, remote hand-held input devices, and a wireless receiver that connects to a computer containing the presentation software.²⁶ Although more traditional active learning approaches (e.g., class discussion) may be dominated by participants who are more vocal or less fearful of public humiliation, the anonymity of ARSs is appealing and promotes equal participation among all attendees.^27,28

The Department of Veterans Affairs (VA) has proven to be an ideal setting to develop and implement formal, self-management education programs.²⁹ Past research has found that veterans who elected to complete a pain education program reported a statistically significant decrease in pain intensity, increase in stage of readiness to adopt a self-management approach, decrease in negative experiences of pain, and decrease in depressive symptoms.³⁰ Research investigating the use of ARSs for education programs catered to veterans is lacking. In concurrence with the VA's mission to improve performance through the use of innovative technologies and evidence-based medical practices, the primary outcome of the current research study was to examine whether the use of ARSs promoted increased comprehension of material and motivation among veterans who participated in a pain education program. Secondary outcomes included whether veterans who participated in a pain education program would acquire more factual information about intractable pain, demonstrate positive changes in their attitudes or beliefs about their own pain problem, increase their readiness to adopt a self-management approach, and demonstrate a decrease in depressive symptoms.

Materials and Methods

Participants and Procedures

Five hundred twenty-four veterans were originally referred to the 12-week “Pain Education School” program at a Midwestern VA Medical Center between November 4, 2011 and October 26, 2012. The current study had no exclusion criteria. The referring provider submitted a consult in the computerized patient record system. Potential candidates were then added to the waiting list for the next available monthly mandatory introduction class. From this target population, a sample of 243 veterans enrolled in the program and attended the introduction class. The majority of veterans (68%; n=165) attended the program at the main VA hospital, whereas 32% (n=78) participated at two affiliated community-based outpatient clinics (CBOCs). Each veteran in this sample was subsequently scheduled for 11 weeks of 1-h classes. The current study protocol was reviewed and approved by the affiliated university's Institutional Review Board and the VA's Research & Development Office. A waiver of informed consent was submitted because the current investigation is an archival study.

Intervention

The “Pain Education School” was developed in November 2009 at a Midwestern VA Medical Center using the Agency for Healthcare Research and Quality's principles and the VA's National Center for Health Promotion and Disease Prevention's manual.²⁹ A goals-based evaluation of the development and implementation of the program is available in the literature.³¹ The program consisted of an introduction class followed by 12 weekly 1-h classes that were led by invited presenters from 23 different disciplines within the facility that provided a menu of treatment modalities—the presenters rotated on a schedule, not the veterans. Presenters from each discipline shared information about chronic or persistent, noncancer pain from their perspective, available treatments within their service, and how to access their respective clinics.

ARSs

An ARS was used in the current study to facilitate more active learning among participants at the main VA hospital (not at the two CBOCs). The enabling technology used was TurningPoint^® from Turning Technologies, LLC (Youngstown, OH). The ARS software is an interactive application that offers a simple interface for polling in Microsoft^® (Redmond, WA) PowerPoint^®. Prior to the start of each class, wireless keypads that linked to the ARS software were distributed to each veteran in attendance at the main VA hospital only. These keypads permitted class attendees to answer four multiple-choice questions displayed on a screen before the speaker began his or her presentation. Veterans responded to the same four questions before and after each presentation. The presenter revealed the correct answers after the responses were retabulated at the end of their lesson. The presenter also displayed a graph generated by the ARS software that depicted the change in percentage of correct responses from pre- to postassessment. This information offered feedback to both the participants and the presenter regarding areas requiring further clarification.¹⁸

Outcome Measures

As part of the introduction and conclusion of the program, all veterans completed a pre- and posteducation assessment. The assessments included multiple measures, including the Readiness Questionnaire (M. Jensen, personal communication, August 23, 2004), the Pain Information and Beliefs Questionnaire (PIBQ),⁸ and the Patient Health Questionnaire-2 (PHQ-2).³² The Readiness Questionnaire and the PHQ-2 were chosen based on their brevity and ease of administration as well as their reliability and validity based on prior research.³⁰

Readiness Questionnaire

Veterans were asked to select one statement out of five items that best described their stage of readiness to adopt a self-management approach to their pain. Scores ranged from 1 to 5, with larger numbers indicating more readiness to adopt a self-management approach. The temporal stability of this measure (r=0.627) has been found to be reliable.³⁰ The measure was sampled from the stages of change from the Transtheoretical Model, the intended domain of content that provides content validity.

PIBQ

The PIBQ assesses two constructs: factual knowledge (19 true-or-false questions) and attitudes or beliefs (nine 6-point Likert scale questions) independently. The estimated internal consistency reliability of the PIBQ measure was reported as α=0.84.⁸

PHQ-2

The PHQ-2 inquires about the frequency of depressive symptoms over the past 2 weeks, scoring each as 0=not at all to 3=nearly every day. Scores ranged from 0 to 6. A PHQ-2 score ≥3 has a sensitivity of 83% and a specificity of 92% for major depression.³²

Data Analyses

A one-way analysis-of-variance and independent-samples t tests identified differences on demographic and outcome variables at baseline. Chi-squared tests were used to compare demographic variables of participants at the main VA hospital versus the CBOCs. The primary intervention outcome analysis was a 2×2 repeated-measures multivariate analyses of variance, with “Intervention Condition” as the between-subjects factor and “Time” as the within-subjects factor. The dependent variables were pain knowledge, pain beliefs, stage of readiness to adopt a self-management approach, and evidence of depressive symptomology. Outcome analyses used an efficacy subset analysis strategy, which selects the subset of the patients who received the intended programming and who did not drop out for any reason. A last-observation-carried-backward approach³³ was used for missing data on the pre-education assessment, and a last-observation-carried-forward approach³⁴ was used for missing data on the posteducation assessment. The Power and Sample Size Program³⁵ was used to calculate sample size using an anticipated effect size (Cohen's d) of 0.5, a desired statistical power level of ≥0.80, and a probability level of p≤0.05. The minimum total sample size (pairs of subject scores) was n=33. SPSS version 20 software (SPSS, Inc., Chicago, IL) was used for all outcome analyses.

Results

Sample Characteristics

Of the 228 veterans who completed the pre-education measure, nearly half (45%; n=102) also elected to complete the posteducation assessment, and their responses were included in the current study (Fig. 2). This rate substantiates data found in a recent VA Evidence-based Synthesis Program report (VA intranet communication, December 2012), which indicated that group visits focusing on education for the management of chronic conditions in veterans tend to suffer from high levels of attrition. Most veterans were African American (58%) and male (82%). Most of the veterans were 55–64 years old (41%), and the youngest returning veterans (17–24 years old) accounted for less than 1% of the sample. Thirteen percent (n=31) of veterans only attended the introductory class.

Fig. 2.

Flowchart of study participants. ARS, audience response system; CBOC, community-based outpatient clinic.

Differences at Baseline

Consistent with past research findings, women veterans reported higher levels of pain knowledge than male veterans at baseline (p=0.00),³⁶ significant differences in readiness to adopt a self-management approach (p=0.04) and pain knowledge (p=0.00) were found for race,^37,38 significant differences in pain knowledge were found among age groups (p=0.05),³⁹ and the youngest veterans (17–34 years old) were less likely to complete the program (p=0.01).^40,41 No differences were found between veterans who elected to complete the pre- and posteducation measures and those who only completed the pre-education measures on readiness to adopt a self-management approach, symptoms of depression, pain knowledge, or pain beliefs (Table 1).

Table 1.

Baseline Outcome Measure Scores by Patient Characteristics (n=228)

	READINESS TO CHANGE			KNOWLEDGE			BELIEFS			DEPRESSION			NUMBER OF SESSIONS
	M (SD)	F	P	M (SD)	F	P	M (SD)	F	P	M (SD)	F	P	M (SD)	F	P
Race		2.92	0.04^a		6.92	0.00^b		1.82	0.15		0.60	0.61		2.42	0.07
White	2.93 (1.31)			14.26 (1.82)			27.65 (5.31)			3.23 (1.99)			6.42 (3.95)
African American	2.71 (1.32)			13.02 (2.02)			26.52 (5.60)			3.41 (1.89)			7.49 (3.49)
Other	3.50 (0.58)			15.67 (0.58)			25.75 (2.63)			2.25 (2.63)			8.75 (5.19)
Unknown	1.57 (0.79)			12.67 (2.07)			31.40 (4.56)			3.57 (1.90)			5.14 (3.76)
Age (years)		0.94	0.47		2.28	0.05^a		1.95	0.08		1.20	0.31		2.92	0.01^b
17–24	4.00 (0.00)			0.00 (0.00)			34.00 (0.00)			0.00 (0.00)			1.00 (0.00)
25–34	2.81 (1.38)			14.67 (1.40)			26.75 (3.73)			3.88 (2.09)			5.29 (4.19)
35–44	2.91 (1.27)			13.94 (1.78)			25.29 (5.51)			3.36 (1.99)			6.88 (3.44)
45–54	2.89 (1.36)			13.88 (2.15)			27.73 (5.10)			3.43 (1.82)			6.80 (3.85)
55–64	2.56 (1.27)			13.21 (2.08)			27.38 (5.35)			3.44 (1.96)			6.92 (3.61)
65–74	2.83 (1.36)			12.93 (2.00)			25.18 (6.71)			2.71 (1.85)			8.59 (3.43)
75+	3.43 (1.27)			13.00 (1.58)			31.40 (4.98)			2.50 (2.26)			10.00 (1.83)

	M (SD)	T	P	M (SD)	T	P	M (SD)	T	P	M (SD)	T	P	M (SD)	T	P
Ethnicity		−1.54	−0.13		−0.79	0.43		−1.56	0.12		0.10	0.92		0.68	0.50
Hispanic	3.33 (1.44)			14.00 (1.95)			29.42 (4.27)			3.27 (1.95)			6.33 (3.94)
Non-Hispanic	2.74 (1.30)			13.50 (2.04)			26.89 (5.52)			3.33 (1.94)			7.08 (3.72)
Gender		−1.24	0.22		−3.15	0.00^b		−0.21	0.83		0.54	0.59		−0.42	0.67
Male	2.72 (1.31)			13.35 (2.09)			27.01 (5.75)			3.36 (1.91)			7.01 (3.72)
Female	3.00 (1.30)			14.38 (1.47)			27.18 (3.87)			3.18 (2.06)			7.27 (3.78)
ARSs		0.35	0.73		−2.40	0.02^a		−0.88	0.38		−1.54	0.13		0.84	0.40
Hospital	2.80 (1.32)			13.28 (2.10)			26.83 (5.51)			3.18 (1.99)			7.15 (3.71)
CBOCs	2.74 (1.31)			14.08 (1.77)			27.55 (5.41)			3.62 (1.80)			6.71 (3.85)
Assessments		0.96	0.34		−0.64	0.52		−0.96	0.34		1.11	.27		13.15	0.00^b
Completers	2.87 (1.30)			13.43 (2.12)			26.68 (5.27)			3.48 (1.94)			9.71 (1.96)
Noncompleters	2.70 (1.32)			13.64 (1.93)			27.42 (5.66)			3.18 (1.94)			5.03 (3.51)

Significant at the p<0.05 level.

Significant at the p<0.01 level.

ARS, audience response system; CBOC, community-based outpatient clinic; M (SD), mean (standard deviation).

Further analyses of baseline data indicated that veterans who attended “Pain Education School” at the CBOCs only evidenced significantly greater pain knowledge on pre-education measures (p=0.02) when compared with veterans at the main VA hospital. Significant differences were also found for race between these two groups (χ² _{3, 241}=20.96, p=0.00). These findings suggest that the majority racial groups at these locations (white at the CBOCs and African American at the main hospital) differ in knowledge. This pattern of racial distribution across the locations is consistent with demographic data from the U.S. Census.^42,43

Program Outcomes

A 2×2 repeated-measures multivariate analysis of variance found a significant multivariate “Condition×Time” interaction (Wilks' λ=0.89, F _{4, 90}=2.81, p=0.03). Post hoc univariate analysis of variance revealed that veterans who used the ARS at the main hospital yielded significantly greater increases in knowledge scores than veterans at the CBOCs (F _{1, 93}=5.29, p=0.02) (Table 2). In fact, pain knowledge decreased among those who did not have access to the ARS at the CBOCs, but this difference was not statistically significant (t ₂₈=0.69, p=0.50). Post hoc univariate analysis of variance also revealed that veterans at the CBOCs evidenced significantly greater increases in readiness to adopt a self-management approach than veterans at the main hospital (F _{1, 93}=4.90, p=0.03). No significant differences were found for “Intervention×Time” on pain beliefs (F _{1, 93}=0.07, p=0.79) or depression (F _{1, 93}=0.67, p=0.42).

Table 2.

Repeated-Measures Multivariate Analysis of Variance Results

	MAIN HOSPITAL (ARS)		CBOCS (NO ARS)		P VALUE FOR
VARIABLE	PRETEST	POSTTEST	PRETEST	POSTTEST	INTERACTION	TIME
Pain knowledge	13.20 (2.29)	13.98 (2.03)	14.10 (2.03)	13.83 (1.97)	0.024^a	0.271
Pain beliefs	26.52 (5.74)	28.24 (5.55)	27.10 (4.44)	28.45 (3.49)	0.792	0.037^a
Readiness to adopt self-management	3.03 (1.31)	3.64 (1.05)	2.59 (1.18)	3.86 (0.88)	0.029^a	0.000^b
Depression	3.32 (1.97)	2.74 (1.82)	3.76 (1.88)	2.86 (1.53)	0.416	0.000^b

Data are mean (standard deviation) values.

Significant at the p<0.05 level.

Significant at the p<0.01 level.

ARS, audience response system; CBOC, community-based outpatient clinic.

The results of the repeated-measures multivariate analysis of variance also indicated main effects for “Time” (Wilks' λ=0.63, F _{4, 90}=13.03, p=0.00). Post hoc univariate tests revealed significant main effects for “Time” on pain beliefs (F _{1, 93}=4.50, p=0.04), stage of readiness to adopt a self-management approach (F _{1, 93}=38.68, p=0.00), and depressive symptoms (F _{1, 93}=14.06, p=0.00). No significant main effect was found for “Time” on pain knowledge (F _{1, 93}=1.23, p=0.27).

Discussion

The Institute of Medicine has placed an emphasis on the need for pain education programs to fill the gaps in patient pain knowledge and to promote self-management of chronic pain.⁴⁴ Previous research has demonstrated that pain education positively impacts patient knowledge of pain, compliance with treatment recommendations, satisfaction with care, and pain outcomes.^1

–5 Military veterans comprise a population of individuals for whom improvements in effective pain management has become both a necessity and a priority. Previous studies have shown that veterans who endorse experiencing pain report lower self-rated health, increased affective distress, greater health risk behaviors, and higher utilization of outpatient services.⁴⁵ The VA's national pain management strategy has encouraged the creation and implementation of pain education programs. Pain education programs often take the form of traditional lecture-based classes or alternatively take a “passive learning” approach.¹¹ Research has suggested that “active learning” methods lead to greater participant engagement, increased interest, improved information retention, and increased motivation.¹⁵ Therefore, the current study used ARSs as a means to facilitate active learning among veterans enrolled in a 12-week “Pain Education School” at a Midwestern VA Medical Center.

The primary outcome finding of the current study was consistent with previous research suggesting the use of ARSs promotes active learning among participants.^15,24 The current results indicated that veterans at the main VA hospital who used the ARS demonstrated significantly greater increases in pain knowledge than those without the ARS at the CBOCs (accounts for 5% of variance). In contrast, veterans at the CBOCs who did not use the ARS evidenced a significantly greater increase in readiness to adopt a self-management approach than veterans who used the ARS at the main hospital (accounts for 5% of variance). These findings are inconsistent with previous research that found that active learning approaches increase motivation among pupils.¹⁵ At baseline, veterans at the CBOCs were in contemplation. During this stage, people become aware of the potential benefits of making a change, but the costs tend to outweigh them. This conflict creates a strong sense of ambivalence about changing. In comparison, the veterans at the main VA hospital were already preparing to adopt a self-management approach. Following the intervention, the CBOC veterans' readiness to adopt a self-management approach increased to an active level similar to that of veterans at the main VA hospital.

The secondary findings from the current study suggest that veterans who participated in the “Pain Education School” program demonstrated positive changes in their attitudes and beliefs about pain (accounts for 5% of variance), increases in their readiness to adopt a self-management approach (accounts for 29% of variance), and reductions in depressive symptoms (accounts for 13% of variance). Previous research has found that individuals who receive pain education report more accurate and realistic attitudes and beliefs about their pain problems.⁸ The current finding also provides support for the notion that pain knowledge and pain beliefs are separate constructs that should be assessed independently in order to accurately reflect patients' perspectives on their pain and pain management.⁸ Prior research has also shown that educating patients on pain management positively impacts patients' willingness to participate in their care and to engage in self-management strategies.^1,30 Furthermore, previous research has found positive changes on measures of depression after completing pain education programming.^30,46 This is a particularly encouraging finding, given that veterans who endorse experiencing chronic pain also report high rates of affective distress. The best weapon against chronic pain and depression is education. Patient education about these conditions can protect patients from unnecessary emotional and physical pain by helping them realize that help is available.

The current study found that pain education programming (without enabling technology) does not increase retention of factual pain knowledge. The current finding is inconsistent with past research looking at pain education of patients suffering from cancer pain.^47,48 Conversely, the current findings mirrored those of previous noncancer pain research³⁰ in which veterans participating in a pain education program did not evidence increased pain knowledge when measured by a different instrument. Perhaps this may be due to the gains made by the veterans using the ARS at the main VA hospital statistically canceling out the losses in knowledge witnessed with veterans at the CBOCs (without the ARS).

The current study has some limitations. This study used a quasi-experimental, pre-/posttest design rather than a randomized control trial. Quasi-experimental designs are viable alternatives when innovative interventions are developed and preliminary testing is appropriate prior to further financial investment in a more definitive trial and system-wide dissemination. As the World Health Organization notes, “the use of randomized control trials to evaluate health promotion initiatives is, in most cases, inappropriate, misleading, and unnecessarily expensive.”⁴⁹ Veterans in the two groups self-selected to attend the education program at the location nearest to their homes. As a result, the number of participants and participant demographics differed between locations; however, the differences in sample size and in demographics are reflective of the U.S. Census data for the two areas.^42,43 Additionally, the self-selection process also aligns with the spirit of autonomy expected from self-management. The current study did not account for a provider effect, given that all veterans were referred by their VA providers. The study also did not control for whether veterans engaged in prior or ongoing pain management modalities, which may account for some of the changes in outcome measures.

Using an ARS also has some inherent disadvantages. The study period included 52 weeks of data collection. The “Pain Education School” was canceled on 4 of these weeks because of national holidays. Of the remaining 48 sessions, ARS data were available for 30 sessions (63%). The missing data from 18 sessions were due to facilitator absence (50%), technological glitches (22%), and unsaved data (28%). These issues mirror those disadvantages commonly described in the literature, such as the need for trained personnel, the potential for technological malfunctions, and the logistics of saving and transferring the data from the ARS programs.¹⁷

The current study yielded additional data supporting the effectiveness of VA pain education programming in increasing readiness to adopt a self-management approach, improving pain beliefs, and decreasing depression scores among veterans. Furthermore, the study highlighted the benefits of incorporating enabling technology to improve outcomes among veterans. The use of the ARS allowed for a collaborative, active learning approach to pain education. Additionally, this technology enabled the presenters to become aware of and subsequently clarify any confusion or misinformation regarding pain and approaches to pain self-management at the completion of each class. Future studies should replicate the current findings using demographically matched samples, satisfaction measures, pain intensity scales, and quality of life assessments.

Footnotes

Acknowledgments

The authors thank all the veterans and providers who made this research possible. The authors would also like to thank the Anesthesiology/Pain Clinic Department, Jesse Brown VA Medical Center, for their vision and ongoing support of the “Pain Education School” program. E.B.S. is currently completing her postdoctoral work in psychology at Loyola University Medical Center.

Disclosure Statement

No competing financial interests exist.

References

Dorflinger

, Kerns

, Auerbach

. Providers' roles in enhancing patients' adherence to pain self management. Transl Behav Med, 2013; 3:39–46.

Ferrell

, Rhiner

, Ferrell

. Development and implementation of a pain education program. Cancer, 1993; 72:3426–3432.

Johnson

. The effects of a patient education course on persons with a chronic illness. Cancer Nurs, 1982; 5:117–123.

Riddell

, Fitch

. Patients' knowledge of and attitudes toward the management of cancer pain. Oncol Nurs Form, 1997; 24:1775–1784.

Rimer

, Levy

, Keintz

, et al. Enhancing cancer pain control regimens through patient education. Patient Educ Counsel, 1987; 10:267–277.

Chen

, Yeh

, Yang

. Testing the impact of a multimedia video CD of patient-controlled analgesia on pain knowledge and pain relief in patients receiving surgery. Int J Med Inform, 2005; 74:437–445.

Ferrell

, Borneman

, Juarez

. Integration of pain education in home care. J Palliat Care, 1998; 14:62–68.

Shutty

, DeGood

. Patient knowledge and beliefs about pain and its treatment. Rehabil Psychol, 1990; 35:43–54.

DeGood

. Reducing medical patients' reluctance to participate in psychological therapies: The initial session. Prof Psychol Res Pract, 1983; 14:570–579.

10.

Gottlieb

. Predicting outcome in pain programs: A matter of cognition. Paper presented at the American Psychological Association Annual Convention, Washington, DC, August 1986.

11.

Stewart-Wingfield

, Black

. Active versus passive course designs: The impact on student outcomes. J Educ Bus, 2005; 81:119–125.

12.

Dale

. Audiovisual methods in teaching. New York: Dryden Press, 1969.

13.

Bonwell

, Eison

. Active learning: Creating excitement in the classroom. 1991 ASHE-ERIC Higher Education Reports. Washington, DC: EPIC Clearinghouse on Higher Education, 1991.

14.

Ebert-May

, Brewer

, Allred

. Innovation in large lectures: Teaching for active learning. Bioscience, 1997; 47:601–607.

15.

Michel

, Cater

III , Varela

. Active versus passive teaching styles: An empirical study of student outcomes. Hum Resour Dev Q, 2009; 20:397–418.

16.

Sarason

, Banbury

. Active learning facilitated by using a game-show format or who doesn't want to be a millionaire?. J Manag Educ, 2004; 28:509–519.

17.

Kay

, LeSage

. Examining the benefits and challenges of using audience response systems: A review of the literature. Comput Educ, 2009; 53:819–827.

18.

Latessa

, Mouw

. Use of an audience response system to augment interactive learning. Fam Med, 2005; 37:12–14.

19.

Vana

, Silva

, Muzyka

, Hirani

. Effectiveness of an audience response system in teaching pharmacology to baccalaureate nursing students. Comput Inform Nurs, 2011; 29:326–334.

20.

Hinde

, Hunt

. Using the personal response system to enhance student learning: Some evidence from teaching economics. In: Banks

, ed. Audience response systems in higher education: Applications and cases. Hershey, PA: Information Science Publishing, 2006:140–154.

21.

Bullock

, LaBella

, Clinghan

, et al. Enhancing the student–instructor interaction frequency. Physics Teacher, 2002; 40:30–36.

22.

Pelton

, Pelton

. Selected and constructed response systems in mathematics. In: Banks

, ed. Audience response systems in higher education: Applications and cases. Hershey, PA: Information Science Publishing, 2006:175–186.

23.

Caldwell

. Clickers in the large classroom: Current research and best-practice tips. Life Sci Educ, 2007; 6:9–20.

24.

El-Rady

. To click or not to click: That's the question. Innov J Online Educ, 2006; 2:1–5.

25.

Slain

, Abate

, Hodges

, et al. An interactive response system to promote active learning in Doctor of Pharmacy curriculum. Am J Pharm Educ, 2004; 68:117.

26.

Cain

, Black

, Rohr

. An audience response system strategy to improve student motivation, attention, and feedback. Am J Pharm Educ, 2009; 73:21.

27.

Graham

, Tripp

, Seawright

, et al. Empowering or compelling reluctant participators using audience response systems. Active Learn High Educ, 2007; 8:233–258.

28.

Martyn

. Clickers in the classroom: An active learning approach. Educause Q, 2007;(2):71–74.

29.

Kinsinger

, Lewis

, Strickland

. Put prevention into VA practice: A step-by-step guide to successful program implementation. 2004. Available at www.prevention.va.gov/prevention_leader_resources/stepbystepmanual.pdf (last accessed January 23, 2014 ).

30.

Cosio

, Lin

. Effects of a pain education program for veterans with chronic, non-cancer pain: A pilot study. J Pain Palliat Care Pharmacother, 2013; 27:340–349.

31.

Cosio

, Hugo

, Roberts

, et al. A pain education school for veterans with chronic non-cancer pain: Putting prevention into VA practice. Fed Pract, 2012; 29:23–29.

32.

Kroenke

, Spitzer

, Williams

. The Veteran Health Questionnaire-2: Validity of a two-item depression screener. Med Care, 2003; 41:1284–1292.

33.

Blankers

, Koeter

, Schippers

. Missing data approaches in ehealth research: Simulation study and a tutorial for non-mathematically inclined researchers. J Med Internet Res, 2010; 12:e54.

34.

Streiner

, Geddes

. Intention to treat analysis in clinical trials when there are missing data. Evid Based Ment Health, 2001; 4:70–71.

35.

Dupont

, Plummer

. PS: Power and Sample Size Calculation version 3.0. 2009. Available at http://biostat.mc.vanderbilt.edu/wiki/Main/PowerSampleSize (last accessed February 5, 2014 ).

36.

Beier

, Ackerman

. Determinants of health knowledge: An investigation of age, gender, abilities, personality, and interests. J Pers Soc Psychol, 2003; 84:439–448.

37.

Grzywacz

, Lang

, Suerken

, et al. Age, race, and ethnicity in the use of complementary and alternative medicine for health self-management: Evidence from the 2002 National Health Interview Survey. J Aging Health, 2005; 17:547–572.

38.

Williams

, Jackson

. Social sources of racial disparities in health. Health Aff (Millwood), 2005; 24:325–334.

39.

Klein

, German

, McPhee

, et al. Aging and its relationship to health knowledge and medication compliance. Gerontology, 1982; 22:384–387.

40.

Garcia

, Kelley

, Rentz

, et al. Pretreatment predictors of dropout from cognitive behavioral therapy for PTSD in Iraq and Afghanistan war veterans. Psychol Serv, 2011; 8:1–11.

41.

Erbes

, Curry

, Leskela

. Treatment presentation and adherence of Iraq/Afghanistan era veterans in outpatient care for posttraumatic stress disorder. Psychol Serv, 2009; 6:175–183.

42.

U.S. Census Bureau. State & county quickfacts. 2012. Available at http://quickfacts.census.gov/qfd/states/17000.html (last accessed May 20, 2014 ).

43.

U.S. Census Bureau. State & county quickfacts. 2012. Available at http://quickfacts.census.gov/qfd/states/18000.html (last accessed May 20, 2014 ).

44.

Institute of Medicine. Relieving pain in America: A blueprint for transforming prevention, care, education, and research. Washington, DC: The National Academies Press, 2011.

45.

Kerns

, Otis

, Rosenberg

, et al. Veterans' reports of pain and associations with ratings of health, health-risk behaviors, affective distress, and use of the healthcare system. J Rehabil Res Dev, 2003:40:371–379.

46.

Spinhoven

, Corry

, Linssen

. Education and self-hypnosis in the management of low back pain: A component analysis. Br J Clin Psychol, 2011; 28:145–153.

47.

George

, Teyhen

, Wu

, et al. Psychosocial education improves low back pain beliefs: Results from a cluster randomized trial in a primary prevention setting. Eur Spine J, 2009; 18:1050–1058.

48.

Meng

, Seekatz

, Roband

, et al. Intermediate and long-term effects of a standardized back school for inpatient orthopedic rehabilitation on illness knowledge and self-management behaviors: A randomized controlled trial. Clin J Pain, 2011; 27:248–257.

49.

WHO European Working Group on Health Promotion Evaluation. Health promotion evaluation: Recommendations to policy-makers: Report of the WHO European Working Group on Health Promotion Evaluation. 1998. Available at www.who.dk/document/e60706.pdf (last accessed January 21, 2014 ).