A Prospective Patient-Centered Data Collection Program at an Acupuncture and Oriental Medicine Teaching Clinic

Abstract

Objective:

Large-scale patient-reported outcomes research investigating the role of acupuncture and Oriental medicine (AOM) in general practice is limited, despite the growing use of AOM in the United States. This article describes the development and refinement of a prospective, patient-centered outcomes data collection program at an Oriental medicine college and presents demographic and clinical data.

Setting/location:

Individualized acupuncture treatment at the Oregon College of Oriental Medicine teaching clinic in Portland, Oregon.

Methods:

A prospective patient-centered data collection program was implemented in 2007 using the Measure Your Medical Outcomes Profile (MYMOP) questionnaire and college-developed demographic and conditions forms. The forms were completed by patients on the first and fifth clinic visit. The program was revised after two years to streamline the data entry process and to include three Patient Reported Outcome Measurement Information System (PROMIS) questionnaires measuring pain, general health, and physical functioning.

Outcome measures:

Outcome measures were patient demographics, chief complaints, MYMOP, and PROMIS data collected at first visit.

Results:

Demographics were similar to those reported in other AOM settings. The majority of patients were Caucasian females and expressed confidence in acupuncture treatment. The most common chief complaint was joint and muscle pain. Additionally, we found that mean scores at baseline for global physical and mental health and physical functioning were all lower than U.S. averages. In contrast to some studies, we found that the majority of patients had previous experience with acupuncture.

Conclusions:

An ongoing, prospective data collection program can be successfully developed and implemented at an AOM college. The program will ultimately provide large-scale, patient-reported outcomes on patients seeking AOM treatment at the student clinic.

Introduction

A cupuncture and Oriental medicine (AOM) are a growing part of the U.S. healthcare system. Between 1997 and 2007, the number of annual acupuncture visits nearly doubled in the United States.^1,2 The 2007 National Health Interview Survey, which included a comprehensive survey of complementary and alternative medicine (CAM) use by all Americans, estimated that 3.1 million adults and 150,000 children had used acupuncture in the previous year, logging more then 17.5 million visits and spending nearly 830 million dollars out of pocket.³

The body of AOM research has grown rapidly in the past 15 years as well. A recent search on PubMed identified more than 1,800 English-language randomized controlled trials investigating acupuncture as a treatment for a wide array of health concerns. Despite this relatively large number of studies, there is a lively debate within the CAM community regarding the appropriateness of randomized clinical trials (RCTs) to study complex, whole-systems interventions such as acupuncture.^4

–7 It is argued that while the RCT is an appropriate design in which to ask a narrowly focused question regarding the efficacy of an intervention, other trial designs, such as large-scale outcomes studies, better reflect the conditions of clinical practice^8,9 and contribute to a broader-based evidence profile.¹⁰

Outcomes research is well-suited for assessing treatment effects in real-world practice settings.^10,11 It can summarize the experiences of a large number of patients treated by many practitioners over prolonged periods of time, all of which are not possible with a typical RCT.⁸ This breadth and depth of data are especially useful when answering questions regarding the effectiveness of care for patients with chronic disease and when determining which treatment(s), under what circumstances, work best and for whom. Given these issues, it is argued that large-scale outcomes research at this time is particularly valuable for the further development of the CAM evidence base.^4,8,10,11

There are relatively few published studies evaluating acupuncture from this perspective, though several descriptive studies have characterized the patients treated in CAM clinics. Bullock et al.¹² reported demographic and descriptive data on 760 patients who sought CAM treatment at the Center for Alternative Medicine. Data were prospectively collected over 18 months with instruments developed by the authors. Major findings were the predominance of female patients (69.5%), and musculoskeletal pain as the most common primary complaint (65.3%). Similar observations have been reported at other CAM clinics¹³ and at AOM clinics.^14

–17

Teaching clinics at AOM educational institutions, with their high number of patient visits, are uniquely positioned to systematically gather patient demographic and clinical data. However, only two AOM schools have reported the demographic and clinical outcomes of patients treated at their respective teaching clinics.^18,19 One retrospective survey reported on the experiences of patients at the University of Salford Acupuncture Clinic. Of the respondents, the primary complaint was pain (63%), followed by depression and/or insomnia (11%). Seventy-four (88%) reported significant improvement in symptoms and 8 respondents (9%) were not satisfied with their interaction.¹⁸

In 2005, a full-scale patient outcomes assessment program was initiated at the Minnesota College of Acupuncture and Oriental Medicine.²⁰ Analysis of data collected on 485 new patients identified pain as the most common presenting condition for 267 (55.1%) respondents, followed closely by wellness care for 215 (44.3%).¹⁹ The majority of patients (89%) indicated they were satisfied with the care they had received.

The Oregon College of Oriental Medicine (OCOM) initiated a prospective patient-centered data collection program in its intern clinics in 2007. Our program was designed to answer a number of clinically relevant questions, including the following: 1) Who seeks treatment at a high-volume AOM teaching clinic? 2) For what self-described conditions do people seek treatment? 3) What specific AOM treatments and treatment regimens are provided for what conditions? 4) Which treatments and regimens work best for which patients? In addition, the ongoing program enables clinic administrators to monitor patient census and serves as an opportunity for students to learn research skills that may be applied after graduation. This article describes the development and implementation of OCOM's data collection program and summarizes patient demographics and clinical characteristics.

Materials and Methods

To facilitate the development and implementation of the data collection program, OCOM formed the College Research Committee (CRC) in 2007. The committee was composed of six faculty members with expertise in research, administration, and clinical practice. All patients in the program were informed on their first visit that their data would be used for research purposes and, if they agreed, provided their informed consent. All new patients to the OCOM teaching clinics over the age of 18 were eligible for inclusion. Exclusion criteria included inadequate comprehension of informed consent or self-report questionnaires, or refusal to participate. All aspects of care, including patient interview and assessment, TCM diagnosis, and treatment, were delivered by interns with in-room guidance by licensed supervisors with at least five years of clinical experience. Approval for the project was obtained from the OCOM Institutional Review Board in 2007, and a revised protocol was approved in 2009. The project was funded internally.

Questionnaires

The Measure Yourself Medical Outcome Profile (MYMOP)²¹ was initially selected by the CRC, and three additional questionnaires developed by the NIH Patient Reported Outcomes Measurement Information System (PROMIS) were added in 2009.^22,23 The MYMOP questionnaire was selected because it allows patients to indicate their primary reason for seeking treatment, resulting in greater patient-centered outcomes data.^21,24,25 The PROMIS questionnaires were selected because they are relatively brief forms yielding summary scores that are standardized to the general U.S. population.^23,26,27

The MYMOP is designed to measure the aspects of the illness that the patient decides are most important²¹ and is considered a sensitive tool to assess change over time.^{15,21,25,28,29} Patients identify up to two symptoms for which they are seeking help, write these symptoms in their own words in spaces provided, and score the severity on a seven-point scale. Other MYMOP questions ask the patient to: 1) rate their well-being; 2) indicate how long they've had the primary symptom; and 3) indicate if they are taking medications for their condition.

NIH PROMIS questionnaires added in 2009 were selected to assess patient-reported pain, physical functioning, and general health.^27,30
–32 These questionnaires were: 1) the “PROMIS Global Items Form”; 2) the “PROMIS Pain Impact-Short Form”; and 3) the “PROMIS Physical Functioning-Short Form.” Patients respond to questions by marking a 5-point scale. Pain severity is rated on a 0 to 10 scale.

In addition to the MYMOP and PROMIS questionnaires, the CRC included a 57-item checklist of biomedical conditions. Lastly, a general intake form was included to collect data on demographics, prior acupuncture care, and expectations of treatment. The total time to complete all forms on visit one was no more than 15 minutes.

Follow-up

The fifth return visit was selected as the most appropriate reassessment point. The experience of clinicians on the CRC suggested that response after visit four is generally a good indication of eventual outcome of treatment. Outcomes assessment was limited to intern-delivered acupuncture care. Thus, only patients who returned for five consecutive acupuncture treatments with student interns were tracked, while patients who visited either of the bodywork clinics (Shiatsu and Tuina massage) or the herbal, doctoral, and faculty practice clinics were excluded.

At their fifth acupuncture visit, patients completed a follow-up MYMOP form, and starting in 2009, follow-up PROMIS questionnaires for pain, physical functioning, and general health. Total time to complete follow-up forms was no more then 10 minutes.

Implementation

In fall of 2007, the data collection program was initiated with the MYMOP, general intake form, and 57-item biomedical conditions checklist. On first visit, the forms were completed by all new patients as part of their initial paperwork packet. Interns confirmed that all forms were completed in full and provided consultation if patients had questions. A part-time clinic data coordinator oversaw college work-study students who entered the information from the completed forms by means of a simple, computerized data entry template associated with a relational database.

During the initial two years of the project, several problems with the collection and entry of data became apparent. First, with multiple work-study students entering the free-text data from the MYMOP forms, consistency in data entry was not being achieved. Second, approximately ¼ of the MYMOP forms were missing data and/or were not correctly filled out by patients, and there was no quality control process to correct these errors prior to data entry. Third, the 57-item biomedical conditions checklist proved to be overly detailed and confusing for patients. Fourth, interns were not carefully reviewing their patients' paperwork to assure quality and completion. Fifth, the online data entry program proved slow and difficult to use. Finally, the CRC decided that additional information about patient's health status needed to be collected.

Project developments

Based on lessons learned in the initial two years, in 2009 the data entry process was amended to improve accuracy and streamlined to reduce burden on clinical operations. The project was also expanded to include three PROMIS questionnaires. These questionnaires were added to better assess changes in pain, general health, and physical functioning. In addition, the 57-item biomedical conditions checklist was condensed into a single page, 20-item checklist that was completed by the intern.

Next, a half-time clinic data entry position was created, with responsibilities that included a review phase to assure quality and completion of data recorded by patients, and entry of all data from the 1st and 5th visit forms into the relational database. When necessary, any missing responses were referred back to the intern, who would obtain the missing information from the patient. By limiting review and data entry to a single, specially trained individual, consistency and accuracy in data entry were improved.

Lastly, open-source software, written in PhP/MySQL,^33,34 was specifically designed by the OCOM IT Department for the entry and storage of all data. This newly designed software can generate downloadable Excel spreadsheets for analysis. Overall, these changes resulted in an enhanced clinic data collection program that is more accurate, streamlined, and sustainable over the long term.

Results

Patient demographics and clinical characteristics

As of September 2011, 2485 new patients consented to the use of their information for data collection. One patient declined. Table 1 summarizes patient demographics. The mean age of patients was 48.1±16.8 (SD) and nearly 70% were female. The majority were Caucasian (81.4%), followed by Hispanic (5.3%), Asian (4.8%), and Black or African-American (2.4%). Approximately 35% of patients had no previous acupuncture treatments. More than 70% had confidence that acupuncture would help their condition.

Table 1.

Patient Demographics (N=2485)

Sex	n	%
Male	767	30.9
Female	1,718	69.1
Age	48.1 (mean)	16.8 (SD)
Race/ethnicity^a	n	%
American Indian or Alaska Native	14	1.1
Asian	62	4.8
Black or African American	31	2.3
Hispanic	69	5.3
White or Caucasian	1,062	81.4
More than 1	42	3.2
Other	24	1.8
Chose not to answer	1,181	n/a
Previous acupuncture treatment	n	%
Yes, many times	613	24.7
Yes, a few times	712	28.7
No, never	873	35.1
Confidence in acupuncture treatment	n	%
Very confident	953	38.4
A bit confident	800	32.2
Neutral	572	23
A bit doubtful	59	2.4
Very doubtful	12	0.5

Some subtotals do not add up to 2485 due to missing data.

Percentages calculated for 1304 respondents.

Table 2 presents the 20 most common biomedical complaints seen. Complaints are generated by patient self-report and categorized on a 20-item checklist. Muscle/joint pain (57.6%) represented the majority of visits. Other less common complaints were anxiety (4.3%), fatigue (3.9%), headaches (3.7%), digestive issues (3.5%), depression (2.5%), women's health issues (2.5%), sleep problems (2.3%), skin problems (2.3%), and frequent cough/colds (1.8%).

Table 2.

Chief Complaints (N=2485)

MYMOP chief complaint	n	%
Muscle/joint pain	1,409	57.6
Anxiety or excess worry	105	4.3
Fatigue or low energy	95	3.9
Headache or migraine	91	3.7
Digestive or stomach problems	85	3.5
Depression or sadness	62	2.5
Women's health problems	62	2.5
Insomnia or sleep problems	55	2.3
Skin problems	55	2.3
Frequent colds, cough, sore throat	43	1.8
Allergies	32	1.3
Asthma or breathing problems	29	1.2
Vision or hearing problems	23	0.9
Obesity or weight gain	20	0.8
Addiction/substance abuse	19	0.8
Cancer (treatment or preventive care)	10	0.4
Hypertension	10	0.4
Diabetes	8	0.3
Heart disease or heart problems	6	0.3
Men's health problems	4	0.2
Other problems	223	9.1

Totals do not add up to 2,485 due to missing data.

MYMOP, Measure Your Medical Outcomes Profile questionnaire.

Baseline scores for global physical and mental health, physical functioning, and pain interference (Table 3) were assessed using PROMIS questionnaires. Raw scores on global health and physical functioning scales were converted to a standardized T-score, with 50 representing the mean of the U.S. population, and an SD of 10. Global physical health T-score was 44.9, lower than the U.S. average. Global mental health T-score was 45.8, also lower than the U.S. average. Physical functioning T-score was 43.5, substantially lower than the U.S. average. Pain interference T-score was 59.1, nearly one standard deviation worse than the U.S. average.

Table 3.

Baseline Physical and Mental Health, Physical Functioning, and Pain Interference

PROMIS domain	n	Mean (SD)	T-score (SE)^a
Global physical health^b	2402	13.7 (2.9)	44.9 (4.3)
Global mental health^b	2431	13.4 (3.3)	45.8 (3.6)
Physical functioning^c	2340	41.3 (8.2)	43.5 (1.9)
Pain interference^d	2481	15.2 (7.3)	59.1 (1.6)

T-scores are standardized to U.S. population, range 0 to 100. U.S. average=50. SD=10.

PROMIS Global Short Form v.1.0, 10 item.

PROMIS Adult Physical Function v.1.0 Short Form, 10 item.

PROMIS Adult Pain Interference v.1.0 Short Form, 6 item.

PROMIS, Patient Reported Outcome Measurement Information System questionnaire.

Analysis of the first four years of data collected revealed a high number of patients did not return for five consecutive acupuncture treatments. While baseline data were consistently collected on new patients at first visit, 60% of new patients in our database did not have a fifth visit follow-up. Implications and countermeasures are discussed below.

Discussion

This article describes an ongoing program to systematically collect patient-centered clinical data in a high-volume AOM intern clinic, and additionally presents demographic and clinical data obtained at visit one. Patient demographics and conditions seen in our AOM clinic were similar to those reported by others.^{12

–17,19} The majority of patients were white females between the ages of 35 and 60 seeking treatment for musculoskeletal pain. We found that a minority (35.13%) of new patients reported no previous treatment with acupuncture, whereas both Maiers et al.¹⁹ and Bovey et al.¹⁶ observed at their respective clinics that a majority of new patients had no previous treatment. This discrepancy may reflect geographic differences in popularity and/or acceptance of CAM therapies.¹ Baseline scores in global physical and mental health, physical functioning, and pain interference were all worse than U.S. averages, suggesting that patients visiting our AOM clinic do so with diminished overall health and not purely for preventative care.

The observation that 60% of new patients do not return for five consecutive acupuncture treatments is consistent with data from the 2007 National Health Interview Survey (NHIS)^35,36 in which 25% of respondents had only one acupuncture visit in the previous 12 months, and approximately 45% had between two and five visits. In addition, data from the National Center for Health Statistics Annual Ambulatory Medical Care Surveys demonstrate that the average number of visits to outpatient clinics per patient per year is fewer than four for patients less than 65 years old.³⁷ Despite the fact that our patient return rates are similar to the national average for outpatient visits, the possibility that many of our patients do not receive adequate treatment must be addressed.^38
–40

We believe that several factors may be relevant. Early treatment success or failure and financial considerations may influence return rates.³⁵ Discussions with clinic personnel revealed two further reasons for our observed return rate. First, because of decisions made early in the project, our database contains information on return visits of patients who received only individualized acupuncture during their first five consecutive visits. It currently does not contain fifth visit data from another 24% of patients who returned for one or more of the other forms of treatment: group acupuncture (3%), herbal consult (4%), treatment by faculty or doctoral students (11%), or bodywork clinic (6%). Additionally, 27% who did not return for five consecutive acupuncture treatments were initially seen for an acute condition that often requires fewer then five visits to resolve, though it is not possible to know for certain if this was their primary reason for discontinuing treatment. To address these issues and enable us to evaluate all treatments provided in our clinic, the data collection protocol has recently been modified to include all visit types.

Second, the failure of patients to return for five consecutive acupuncture treatments may be due in part to inadequate intern clinical education with regard to treatment planning. When interns do not establish a treatment plan with patients, it is more likely that patients who require a minimum of five treatments may leave care prior to reaching visit five.^41
–43 To address this, clinic supervisors are now training interns to implement treatment plans for all patients under their care. Preliminary findings from a recently completed operational review of treatment planning demonstrated a 59% increase in patient retention at fifth visit. With corrections to both the data collection protocol and intern treatment planning, we expect to see improvement in clinic retention of patients in 2012.

Attrition is an important issue to consider in longitudinal studies and will vary widely depending on the type of research in question.^44,45 In observational research conducted in clinical settings, in contrast to experimental research where patients are followed for a specified period of time based on a specific protocol, attaining very low attrition is less critical.⁴⁴ This is in part because one primary goal of outpatient care is to increase health to the point where treatment is discontinued. In order for us to evaluate response to acupuncture, which often requires at least five treatments, we expect to increase our patient return rates through treatment planning. As our data collection program proceeds, analyses will include evaluation of, and adherence to, the treatment plan as a covariate.

Another limitation is the use of the fifth visit as the single follow-up point. Clearly, some conditions may take more visits before improvement is detected. In the future, we may ask returning patients to complete forms at visit 10 and 20 as a means of capturing longer-term outcomes, or at intervals consistent with treatment planning.

The duration of time within which a patient returns for the fifth visit is another important factor that may influence treatment outcomes. Because the frequency of treatments will vary among patients based on individual treatment plans, our analyses will also include the duration from first to fifth visit as a covariate.

With a precise means of data collection now firmly established in the day-to-day procedures of the clinic, data can be extracted efficiently for future articles regarding patient reported outcomes. With specific information such as ICD-9 codes and TCM diagnoses obtained through retrospective chart review, outcomes of particular treatments and conditions can be evaluated, as well as the relationships between TCM patterns and outcomes. In the future, condition-specific forms may be introduced, such as the Hospital Anxiety and Depression (HADS) Questionnaire, to collect more targeted outcomes data for particular disorders.

We hope that our findings, both here and in the future, will help to further the development of the AOM evidence base, and that by sharing the ongoing process of our program, our experiences may be informative to other clinics interested in implementing similar patient-centered data collection programs.

Conclusion

Since its introduction in 2007, an ongoing, prospective data collection program was successfully developed and implemented in an AOM college. Revisions were made to enhance data entry accuracy, minimize loss to follow-up, and ensure project sustainability. This program demonstrates the feasibility of prospective data collection in a manner that does not impede patient flow through a high-volume clinic. The database will provide searchable data to support research regarding AOM healthcare outcomes, enhance clinical curriculum, and inform clinical practice. The program will ultimately generate large-scale, patient-reported outcomes on patients seeking AOM treatment at the OCOM intern clinic.

Footnotes

Acknowledgments

The authors would like to thank the members of the OCOM CRC for their vision in establishing the outcomes data collection program: Tim Chapman, PhD; Robert Kaneko, DAOM, LAc; Zhaoxue Lu, PhD, LAc; Rosa Schnyer, DAOM, LAc; and Liz Collins, BS.

Disclosure Statement

No competing financial interests exist.

References

Barnes

, Bloom

, Nahin

. Complementary and alternative medicine use among adults and children: United States, 2007. Natl Health Stat Report, 2008; 12:1–23.

Tindle

, Davis

, Phillips

, Eisenberg

. Trends in use of complementary and alternative medicine by US adults: 1997–2002. Altern Ther Health Med, 2005; 11:42–49.

Nahin

, Barnes

, Stussman

, Bloom

. Costs of complementary and alternative medicine (CAM) and frequency of visits to CAM practitioners: United States, 2007. Natl Health Stat Report, 2009; 18:1–14.

Walach

, Falkenberg

, Fonnebo

, Lewith

, Jonas

. Circular instead of hierarchical: methodological principles for the evaluation of complex interventions. BMC Med Res Methodol, 2006; 6:29.

Verhoef

, Vanderheyden

, Dryden

, Mallory

, Ware

. Evaluating complementary and alternative medicine interventions: in search of appropriate patient-centered outcome measures. BMC Complement Altern Med, 2006; 6:6–38.

Shea

. Applying evidence-based medicine to traditional chinese medicine: debate and strategy. J Altern Complement Med., 2006; 12:255–263.

Jonas

. Building an evidence house: challenges and solutions to research in complementary and alternative medicine. Forsch Komplementarmed Klass Naturheilkd, 2005; 12:159–167.

Walach

. The role of outcomes research in evaluating complementary and alternative medicine. Altern Ther, 2002; 8:88–95.

Lao

, Berman

, Ezzo

, Hammerschlag

. Considerations for the future design of clinical trials in acupuncture. Stux

, Hammerschlag

. Clinical Acupuncture: Scientific Basis. Berlin: Springer-Verlag, 2001.

10.

Riley

, Berman

. Complementary and alternative medicine in outcomes research. Altern Ther Health Med, 2002; 8:36–37.

11.

Black

. Why we need observational studies to evaluate the effectiveness of health care. BMJ, 1996; 312:1215–1218.

12.

Bullock

, Pheley

, Kiresuk

, Lenz

, Culliton

. Characteristics and complaints of patients seeking therapy at a hospital-based alternative medicine clinic. J Altern Complement Med, 1997; 3:31–37.

13.

Secor

, Markow

, Mackenzie

, Thrall

. Implementation of outcome measures in a complementary and alternative medicine clinic: evidence of decreased pain and improved quality of life. J Altern Complement Med, 2004; 10:506–513.

14.

Cassidy

. Chinese medicine users in the United States. Part I: Utilization, satisfaction, medical plurality. J Altern Complement Med, 1998; 4:17–27.

15.

Hull

, Page

, Skinner

, Linville

, Coeytaux

. Exploring outcomes associated with acupuncture. J Altern Complement Med, 2006; 12:247–254.

16.

Bovey

, Horner

, Shaw

et al. Engaging in the audit of acupuncture practice. J Altern Complement Med, 2005; 11:293–298.

17.

Day

, Kingsbury-Smith

. An audit of acupuncture in general practice. Acupunct Med, 2004; 22:87–92.

18.

Xing

, Long

. A retrospective survey of patients at the University of Salford Acupuncture Clinic. Complement Ther Clin Pract, 2006; 12:64–71.

19.

Maiers

, McKenzie

, Evans

, McKenzie

. Patient outcomes at a traditional Chinese medicine teaching clinic: a prospective data collection project. J Altern Complement Med, 2008; 14:1083–1088.

20.

Maiers

, McKenzie

, Evans

, McKenzie

. The development of a prospective data collection process in a traditional Chinese medicine teaching clinic. J Altern Complement Med, 2009; 15:305–320.

21.

Paterson

. Measuring outcomes in primary care: a patient generated measure, MYMOP, compared with the SF-36 health survey. BMJ, 1996; 312:1016–1020.

22.

Cella

, Yount

, Rothrock

et al. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care, 2007; 45,5 Suppl 1:S3–S11.

23.

NIH. PROMIS: Dynamic Tools to Measure Health Outcomes from the Patient Perspective. 2012. www.nihpromis.org. 2012 March 15.

24.

Paterson

, Britten

. pursuit of patient-centred outcomes: a qualitative evaluation of the 'Measure Yourself Medical Outcome Profile.' J Health Serv Res Policy, 2000; 5:27–36.

25.

Jolles

, Buchbinder

, Beaton

. A study compared nine patient-specific indices for musculoskeletal disorders. J Clin Epidemiol, 2005; 58:791–801.

26.

Gershon

, Rothrock

, Hanrahan

, Bass

, Cella

. The use of PROMIS and assessment center to deliver patient-reported outcome measures in clinical research. J Appl Meas, 2010; 11:304–314.

27.

Cella

, Riley

, Stone

et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. J Clin Epidemiol, 2010; 63:1179–1194.

28.

Polus

, Kimpton

, Walsh

. Use of the measure your medical outcome profile (MYMOP2) and W-BQ12 (Well-Being) outcomes measures to evaluate chiropractic treatment: an observational study. Chiropr Man Therap, 2011; 19:7.

29.

Paterson

, Langan

, McKaig

et al.

Assessing patient outcomes in acute exacerbations of chronic bronchitis: the measure your medical outcome profile (MYMOP), medical outcomes study 6-item general health survey (MOS-6A) and EuroQol (EQ-5D)

Qual Life Res, 2000; 9:521–527.

30.

Reeve

, Hays

, Bjorner

et al.

Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS)

Med Care, 2007; 45,5 Suppl 1:S22–31.

31.

Amtmann

, Cook

, Jensen

et al. Development of a PROMIS item bank to measure pain interference. Pain, 2010; 150:173–182.

32.

Hung

, Clegg

, Greene

, Saltzman

. Evaluation of the PROMIS physical function item bank in orthopaedic patients. J Orthop Res, 2011; 29:947–953.

33.

MySQL. 2011. www.mysql.com/. 2011 November 10.

34.

Achour

. PHP Manual. 2011. www.php.net/manual/en/index.php. 2011 November 20.

35.

Zhang

, Lao

, Chen

, Ceballos

. Acupuncture use among American adults: What acupuncture practitioners can learn from national health interview survey 2007? Evid Based Complement Alternat Med, 2012; 2012:710750.

36.

Hawk

, Ndetan

, Evans

Jr . Potential role of complementary and alternative health care providers in chronic disease prevention and health promotion: an analysis of national health interview survey data. Prev Med, 2012; 54:18–22.

37.

Cherry

, Lucas

, Decker

. Population aging and the use of office-based physician services. NCHS Data Brief, 2010; 41:1–8.

38.

Demark-Wahnefried

, Bowen

, Jabson

, Paskett

. Scientific bias arising from sampling, selective recruitment, and attrition: the case for improved reporting. Cancer Epidemiol Biomarkers Prev, 2011; 20:415–418.

39.

Touloumi

, Babiker

, Pocock

, Darbyshire

. Impact of missing data due to drop-outs on estimators for rates of change in longitudinal studies: a simulation study. Stat Med, 2001; 20:3715–3728.

40.

Krishnan

, Murtagh

, Bruce

, Cline

, Singh

, Fries

. Attrition bias in rheumatoid arthritis databanks: a case study of 6346 patients in 11 databanks and 65,649 administrations of the Health Assessment Questionnaire. J Rheumatol, 2004; 31:1320–1326.

41.

Bosch-Capblanch

, Abba

, Prictor

, Garner

. Contracts between patients and healthcare practitioners for improving patients' adherence to treatment, prevention and health promotion activities. Cochrane Database Syst Rev, 2007; 2:CD004808.

42.

Suarez-Almazor

. Patient-physician communication. Curr Opin Rheumatol, 2004; 16:91–95.

43.

Stewart

. Effective physician-patient communication and health outcomes: a review. CMAJ, 1995; 152:1423–1433.

44.

Fewtrell

, Kennedy

, Singhal

et al.

How much loss to follow-up is acceptable in long-term randomised trials and prospective studies?

Arch Dis Child, 2008; 93:458–461.

45.

Kristman

, Manno

, Cote

. Loss to follow-up in cohort studies: how much is too much? Eur J Epidemiol, 2004; 19:751–760.