How Do Perceptions of Asthma Control and Severity Relate to Indicators of Asthma Status and Treatment Recommendations by Pediatricians?

Abstract

Background:

According to National Institutes of Health (NIH) guidelines, asthma control and severity are unique constructs. Little is known about how asthma control and severity are distinguished by pediatricians and if they influence treatment recommendations.

Methods:

We conducted a random-sample survey of 500 pediatricians using patient vignettes with different asthma status indicators (recent hospitalization, parental report of bother from asthma, frequent symptoms, parental report of worsening asthma, and wheeze during physical exam) and a visual analog scale (VAS) to rate control and severity. Regression models assessed the independent effects of these indicators on asthma control and severity ratings, and the effects of these ratings on treatment recommendations.

Results:

A total of 270 respondents provided usable data. Compared to patients with well-controlled asthma: (1) medication intensity influenced only severity ratings; (2) frequent symptoms and recent hospitalization influenced control and severity ratings; (3) wheeze and bother influenced control ratings only (p<0.001 for all comparisons); (4) a report of worse asthma did not significantly affect any ratings (p>0.2). Poorer VAS control ratings were associated with recommendations to step-up treatment (odds ratio [OR] 2.61, 95% confidence interval [CI], 2.2–3.1, p<0.001), but more severe VAS ratings were not (OR 1.02, 95% CI, 0.9–1.2, p=0.8). Recommendations to step-down treatment were associated with poorer VAS control ratings (OR 0.70, 95% CI, 0.6–0.8, p<0.001) and more severe VAS ratings (OR 0.82, 95% CI, 0.7–0.9, p<0.001).

Conclusions:

Pediatricians who step-up asthma treatment base their assessments on asthma control, while assessments of both control and severity factor into their decision to step-down asthma therapy.

Introduction

The asthma guidelines of the third expert panel report (EPR-3) of the National Asthma Education and Prevention Program published by the National Institutes of Health (NIH) recommend that clinicians use the patient's level of asthma control and/or severity to make decisions to step-up or step-down asthma controller medications.¹ Asthma control is defined in the guidelines as the degree to which manifestations of asthma are minimized, while asthma severity is the intrinsic intensity of the disease process.^1–3 Although these are unique constructs, guidelines suggest that clinicians use the same indicators of asthma status, including symptoms, short-acting beta₂ agonist use, functional limitations and exacerbations requiring oral corticosteroids,^1,3 as a basis for the treatment of asthma.

We have previously shown that primary care pediatricians utilize a combination of guideline-based and non-guideline-based clinical criteria (see Fig. 1) to treat asthma.⁴ However, we did not examine how these clinical criteria influenced assessments of asthma control and severity, or how asthma control and severity influenced treatment decisions by these pediatricians. There is little published evidence about what clinical criteria pediatricians do use to determine asthma control or severity, or how they incorporate those determinations into treatment decisions. The published studies highlighting pediatrician “non-adherence” to asthma guidelines have not explored the evaluation and decision-making processes of these clinicians. It would be valuable to know if there is a disconnect between asthma guidelines and pediatricians in the notions of control and severity, and how those are to be used to treat patients with asthma.

FIG. 1.

Classification of asthma status indicators according to 2007 NAEPP EPR-3 asthma guidelines. NAEPP EPR-3, National Asthma Education and Prevention Program third expert panel report.

Additionally, a number of studies have explored the concepts of asthma control or severity, but have not examined if or how pediatricians distinguish asthma control from severity.^5–10 Furthermore, it is unknown if asthma control assessments have a greater impact on clinician treatment decision making than asthma severity assessments. The purpose of this study was to evaluate how indicators of asthma status influence physician ratings of asthma control and asthma severity, if these indicators of asthma status are used differently to distinguish asthma control from asthma severity, and how perceptions of asthma control and severity relate to physician recommendations to change asthma therapy.

Methods and Materials

Study population and recruitment procedures

This study was a cross-sectional survey using clinical vignettes to evaluate the importance of various indicators of asthma health on pediatricians' asthma management practices.^11,12 A total of 500 pediatricians, who were fellows of the American Academy of Pediatrics (AAP) and living in the United States, were randomly selected to receive the survey. For this analysis, we excluded pediatricians who reported they do not provide asthma care to children and those who did not report seeing at least one asthma patient per week in their practice. A complete description of the study design has been reported previously.⁴

Questionnaire

Standardized clinical vignettes of patients between 5 and 10 years of age returning for a 3-month follow-up clinical visit were used as a means of evaluating pediatrician assessments of asthma control and asthma severity, as well as treatment practices (to step-up or step-down asthma treatment). The vignettes were created to examine the influence of a variety of indicators of asthma status (Table 1).^4,13,14 The indicators were selected through the qualitative analysis of focus groups of: (1) adult and teenage African American asthma patients recruited through local community centers in Baltimore, Maryland, and from a patient asthma education program at Howard University, Washington, DC; (2) primary care providers (internists, family practitioners, pediatricians); and (3) specialist physicians (pulmonologists, allergists, pediatricians, and geriatricians) from the Johns Hopkins Community Physicians, Howard University and Charter Health Plan (HMO; Washington, DC) who treat asthma patients.^4,13,15 Based on feedback from these focus groups, four clinical indicators important for the assessment of asthma disease activity were established (see descriptions below). A fifth clinical indicator was developed based on subsequent semi-structured interviews with pediatric pulmonologists from the Johns Hopkins Children's Center about the content of the vignettes. The following five indicators were included in the vignettes:

Table 1.

Concept Map of Vignettes

Vignette name	Treatment intensity	Wheeze	Bother	Direction	Acute care	Frequency of symptoms
Low-intensity Reference¹	Low	No	No	Unchanged	No	Symptoms once per 2 weeks
Wheeze	Low	Yes	No	Unchanged	No	Symptoms once per 2 weeks
Bother	Low	No	Yes	Unchanged	No	Symptoms once per 2 weeks
Worse	Low	No	No	Worse	No	Symptoms once per 2 weeks
Hospitalized	Low	No	No	Unchanged	Yes	Symptoms once per 2 weeks
Frequent symptoms	Low	No	No	Unchanged	No	Symptoms 4–5 times per week
High-intensity	High	No	No	Unchanged	No	Symptoms once per 2 weeks

Bold type indicates factor of interest for each vignette.

• Frequent symptoms: wheeze and albuterol use 4 to 5 days per week;

• Wheeze: the presence of faint wheeze on physical examination;

• Hospitalization: asthma hospitalization 6 months ago;

• Bother: parental report of being bothered by the child's asthma;

• Direction: parental report of the child doing worse since the last clinic visit 3 months earlier.

A vignette without any of the above indicators (the “reference” vignette) was also created for comparison of assessments (e.g., asthma control) and treatment recommendations (e.g., to step-up treatment).

For each vignette, there were two possible medication regimens reported: (1) a low-intensity regimen of fluticasone 44 mcg (2 puffs twice daily), or (2) a high-intensity regimen of fluticasone 220 mcg (2 puffs twice daily), a long-acting inhaled β-agonist and a leukotriene modifier.

Various combinations of the indicators of asthma status and medication regimens were used to construct the vignettes (see Table 1). Each pediatrician received four vignettes, as the burden of a longer survey was expected to reduce the response rate.^16,17 Each physician received two standard vignettes without any indicators of abnormal asthma status (one each of the low-intensity medication regimen and the high-intensity medication regimen) and two randomly selected vignettes. The two standard vignettes were originally constructed to represent baseline comparisons for all other vignettes in terms of recommendations to step-up (well-controlled symptoms, low-intensity therapy, asthma unchanged from last visit, no hospitalizations in the past 6 months, and parent not bothered by asthma) and step-down treatment (similar patient on high-intensity therapy). In the current analysis, we have used the low-intensity scenario as the reference for all comparisons. The complete vignettes are provided in the online supplement.

For each vignette, pediatricians were asked to estimate the patient's level of asthma control and asthma severity on a Visual Analog Scale (VAS). Each VAS was a 10 cm line anchored by description of control (“excellent” and “poor”) and severity (“mild” and “severe”; Fig. 2). By placing a mark on the VAS, pediatricians indicated the level of asthma control and the level of asthma severity. Lower VAS scores indicated better asthma control or less asthma severity, respectively. Pediatricians were also asked to choose a treatment recommendation for each vignette: step-up treatment, step-down treatment, or to leave the current medications unchanged.

FIG. 2.

Sample Visual Analog Scale (VAS). Each VAS was a 10 cm line anchored by description of control (“excellent” and “poor”) and severity (“mild” and “severe”; Fig. 1). Respondent ratings of asthma control and severity were determined by measuring from the left anchor (“excellent” or “mild”) to the location of the “x” placed by the respondent on the VAS scale. Lower VAS scores indicated better asthma control or less asthma severity respectively.

Analysis

Means and proportions were used to describe the participant characteristics. Linear regression was used to examine influence of each asthma status indicator on clinician ratings of asthma control and severity in comparison to the reference vignette. Multivariable linear regression models were also constructed to test the independent effect of each asthma status indicator on control and severity ratings, while controlling for the other asthma indicators. Multivariable logistic regression models were constructed to assess the independent effects of mean ratings of control and severity on treatment recommendations (step-up or step-down therapy) after adjusting for asthma status indicators. Statistical computations were performed using Stata 10 statistical software (StataCorp, College Station, TX).

Results

Respondent characteristics

A total of 335 pediatricians completed the survey, giving a 69% response rate. After excluding 65 pediatricians who reported not seeing at least one asthma patient a week, there were 270 pediatricians included for this analysis. There were no significant demographic differences between excluded pediatricians and the final sample. A majority of the pediatricians were female (57.1%), were employed in a private practice (69.9%), and rated their asthma treatment experience as moderate (64.9%; Table 2). Slightly less than half of the respondents (45%) reported treating more than 10 asthma patients per week.

Table 2.

Physician Sociodemographics and Practice Characteristics

Physician characteristic (n=270)	Pediatricians %
Female	57.1
Primary employer
Private practice	69.9
University medical school	4.5
Community hospital	12.3
Government	2.6
Managed care	7.8
Other	3.0
Experience treating asthma
Extensive	33.7
Moderate	64.9
Limited	1.5
Weekly volume of asthma patients
1–5 per week	21.2
6–10 per week	33.7
11–20 per week	25.0
>20 per week	20.4

Asthma control ratings

The presence of frequent symptoms was associated with the worst asthma control ratings (Table 3). The mean VAS score for asthma control was also significantly lower in the vignette with patient report of bother (3.07; 95% CI, 2.85–3.58) compared to the low intensity reference vignette (2.25; 95% CI, 2.02–2.49, p<0.001). This is similar in magnitude to the effect of report of wheeze 4–5 days per week (3.21; 95% CI, 2.85–3.58, p<0.001). Asthma control was also rated significantly more poorly than in the reference vignette for a report hospitalization 6 months earlier (p<0.001), but not if a parent reported that the child's asthma was worse than at the prior visit (p=0.8).

Table 3.

Mean VAS Ratings ^a of Asthma Control and Asthma Severity For Each Asthma Status Indicator

Asthma status indicators	Mean VAS asthma control rating (95%CI)	p value^b	Mean VAS asthma severity rating (95%CI)	p value^b
Low intensity medication regimen	2.25	—	2.88	—
(reference vignette) (n=186)	(2.02, 2.49)		(2.60, 3.15)
Wheeze	3.21	<0.001	3.12	0.348
(n=79)	(2.85, 3.58)		(2.69, 3.54)
Bother	3.07	<0.001	3.20	0.209
(n=77)	(2.70, 3.43)		(2.77, 3.63)
Worse	2.19	0.798	2.55	0.241
(n=61)	(1.78, 2.61)		(2.07, 3.03)
Acute care	3.42	<0.001	4.81	<0.001
(n=61)	(3.01, 3.83)		(4.33, 5.30)
Frequent symptoms	8.30	<0.001	6.47	<0.001
(n=61)	(7.89, 8.71)		(5.99, 6.96)
High intensity medication regimen	2.13	0.406	6.02	<0.001
(n=269)	(1.93, 2.32)		(5.79, 6.25)

Ratings were obtained from multivariable linear regression analyses that included each asthma status indicator in the model.

For comparison of each indicator of asthma status to the reference vignette.

VAS, visual analog scale.

Asthma severity ratings

As with control ratings, frequent symptoms were associated with the most severe asthma ratings (p<0.001) compared to the reference vignette. High medication intensity and prior hospitalization were also associated with significantly more severe asthma ratings (p<0.001), while parental reports of worse asthma, being bothered about the child's asthma, and wheeze on exam did not significantly influence asthma severity ratings.

Influence of asthma control and severity ratings on treatment recommendations

Stepping-up and stepping-down treatment were both significantly associated with control ratings, while severity ratings were associated only with recommendations to step-down treatment. Specifically, a 1 cm increase in the VAS ratings toward poorer control was independently associated with physician recommendations to step-up treatment (odds ratio [OR] 2.61, 95% CI, 2.2–3.1, p<0.001), while a 1 cm increase in the VAS ratings toward more severe asthma was not significantly associated with recommendations to step-up treatment (OR 1.02, 95% CI, 0.9–1.2, p=0.8). In terms of recommendations to step-down treatment, a 1 cm increase in VAS ratings toward poorer control (OR 0.70, 95% CI, 0.6–0.8, p<0.001) and toward more severe asthma (OR 0.82, 95% CI, 0.7–0.9, p<0.001) was independently associated with recommendations to step-down therapy.

Discussion

This study provides evidence that pediatricians incorporate multiple dimensions of asthma health into their assessments of asthma control and severity. Some of these dimensions of asthma health are reflected in current asthma guideline assessment algorithms (acute care, frequent symptoms, wheeze on exam, and medication intensity), while “bother” is not. Previous findings indicate that “bother” is influential in treatment decisions by pediatricians,⁴ while our current findings indicate that “bother” is influential in pediatrician perceptions of disease severity. The EPR-3 asthma guidelines recommend monitoring patient quality of life, and state that “… perceptions and experiences of patients must be assessed directly and not imputed from measures of clinical status.”¹⁸ Before creating the vignettes, we conducted patient focus groups from whom the suggestion of assessing disease burden or “bother” is important; it is one means of learning the patient's personal experience with asthma that is meaningful to them. The focus group participants indicated that “bother” is important for doctors to know about when treating them. We realize that further work is needed to better understand and define the concept of “bother,” but the fact that it does influence asthma assessments and treatment decisions by pediatricians suggests that it is worth considering.

Of note, there was one clinical indicator—direction (a parental report that the child's asthma had worsened)—that did not significantly influence control or severity ratings. However, in a previous analysis of this data, a parental report of “worse” asthma was associated with reluctance to step-down treatment.⁴ This suggests that the treatment decisions made by this group of pediatricians are not mediated solely by perceptions of asthma control or severity. “Direction” of asthma status is not an asthma-guideline-based criterion for determining control or severity, nor is it a quantifiable measure of disease status, suggesting that subjective patient evaluations can also be meaningful determinants of treatment decisions by pediatricians. Subjective evaluations of patients are likely made by clinicians,^19–23 but their influence on treatment decisions in asthma remains to be further explained.

We also observed that pediatricians do distinguish asthma control from asthma severity. Given that the terms have been used interchangeably in the past, it is valuable to know that there is agreement between practicing pediatricians and the EPR-3 asthma guidelines that asthma control and severity are distinct constructs. When all else is equal, these pediatricians used treatment intensity to assess a person's level of severity but not control. Although this approach is recommended in current asthma guidelines, our findings provide evidence to support this practice among primary care pediatricians, who care for the majority of children with asthma in the United States. Pediatricians use a combination of guideline- and non-guideline-based criteria (see Fig. 1) when assessing asthma control, while they exclusively use guideline-based criteria to determine asthma severity. Specifically, a combination of symptom frequency, history of hospitalizations for asthma, wheezing on exam, and a parental report of disease burden (“bother”) are incorporated into assessments of control, while excluding “bother” and wheeze to determine severity. This approach would be consistent with distinguishing a labile construct (control) from a fixed construct (severity). So although there seems to be evidence for agreement on the concepts of control and severity between the pediatricians in this study and the EPR-3 guidelines, how criteria are utilized to make a determination differs slightly, with the use of “bother” as an additional dimension of control not incorporated into current guideline assessment algorithms.

Symptom frequency had the greatest change in both control and severity ratings, suggesting that these pediatricians rely heavily on symptom reporting when determining a patient's control and severity and to make treatment decisions. Accurate assessments of symptom frequency, therefore, may be crucial to treatment decision making by clinicians. A measure that includes valid assessments of symptom frequency and other asthma indicators used by clinicians may hold clinical value as a strategy for increasing adherence to guideline recommended therapy and reducing asthma morbidity.

This study does suggest that asthma control and severity are mediating factors in the treatment decisions by pediatricians. In a previous analysis of this data, we explored how the clinical indicators of asthma status are directly linked to treatment decisions.⁴ In this analysis, we have shown how these clinical indicators relate to assessments of control and severity, and how control and severity assessments determine treatment decisions. These results are consistent with and provide empirical support for the guideline-based approach of treatment based on evaluations of asthma control and severity. However, our findings also suggest that asthma control and severity are used differently by clinicians in titrating asthma medications. As suggested by NIH guidelines, we found that ratings of poorer asthma control resulted in significantly higher odds of stepping-up treatment, as well as lower odds of stepping-down. However, current asthma guidelines do not offer direction on how to incorporate severity assessments into decisions to titrate treatment for those already on controller medications (e.g., how much to step-down or how quickly to step-down for patients with varying degrees of disease severity in the face of well-controlled asthma). The pediatricians in this study indicated that severity may be a permanent consideration throughout the duration of a patient–physician relationship and that more severe disease warrants a more cautious approach to stepping-down treatment—as evidenced by a generalized reluctance to step-down when a patient is perceived as having more severe disease. Current EPR-3 national asthma guidelines¹ do not incorporate disease severity into medication titration decisions, so we cannot state whether the decisions made by these respondents are correct or incorrect.

In this study, we chose to have participants evaluate asthma control or severity using a VAS in order not to force people to respond to arbitrary labels (e.g., intermittent, moderate persistent, etc.) or groupings that may not reflect the respondent's thinking. To capture fully the range of responses, we analyzed the VAS scores as continuous variables. Some studies have also used this analytic approach,^24–26 while others have created “categories” of ranges VAS scores.²⁷ We reevaluated VAS scores in this study by creating categories (0–2.5;>2.5–5.0;>5.0–7.5;>7.5–10 cm) and observed the same effect of clinical factors on pediatrician ratings of control and severity (data not shown). Others have observed no difference findings when comparing a continuous and categorical approach to analyzing VAS scores.²⁸ We have opted for the continuous approach because we do not know if equal divisions of the VAS scale actually represent what the physician or NIH asthma guidelines intended.

There are potential limitations to this study. First, a vignette-based approach may not reflect all of the realities of a clinical encounter (e.g., poor patient–physician communication). However, vignettes are useful for isolating factors of interest and understanding how physicians incorporate such information into decision making.^13,29–34 Second, our findings may not be generalizable to non-AAP pediatricians and to non-pediatricians, but AAP pediatricians represent a significant portion of clinicians providing care to children in the United States.³⁵

In conclusion, our study suggests that the criteria used to classify asthma control could be expanded to include subjective patient valuations (i.e., bother), that a standardized approach to incorporating severity determinations into titration of controller medications would be consistent with current pediatrician practices and that when provided with sufficient patient information, pediatricians do make clinical decisions consistent with asthma guidelines. Therefore, a clinical decision-support tool to facilitate the accurate and complete collection of patient data during routine clinical encounters could be beneficial in assuring more accurate assessments of control and severity, making it easier for clinicians to provide appropriate treatment of patients with uncontrolled asthma, and reconciling differences between current and recommended asthma care.

Footnotes

Acknowledgments

This project was supported in part by funding from the National Institutes of Health and the following grants: P01 R-826724 from the Environmental Protection Agency; P01 ES09606 from the National Institute of Environmental Health Sciences; HL04266, 5U01 HL072455, HL67850, NIH T32 HL072748 from the National Heart Lung and Blood Institute.

Author Disclosure Statement

No competing financial interests exist.

References

Expert Panel Report 3 (EPR-3):. Guidelines for the diagnosis and management of asthma—summary report 2007. J Allergy Clin Immunol, 2007; 120:S94–138.

Guidelines for the diagnosis and management of asthma. National Heart, Lung, and Blood Institute. National Asthma Education Program. Expert Panel Report. J Allergy Clin Immunol, 1991; 88:425–534.

Urbano

. Review of the NAEPP 2007 Expert Panel Report (EPR-3) on asthma diagnosis and treatment guidelines. J Manag Care Pharm, 2008; 14:41–49.

Okelo

, Patino

, Riekert

et al. Patient factors used by pediatricians to assign asthma treatment. Pediatrics, 2008; 122:e195–e201.

Bacharier

, Strunk

, Mauger

, White

, Lemanske

Jr , Sorkness

. Classifying asthma severity in children: mismatch between symptoms, medication use, and lung function. Am J Respir Crit Care Med, 2004; 170:426–432.

Boulet

, Boulet

, Milot

. How should we quantify asthma control? A proposal. Chest, 2002; 122:2217–2223.

Boulet

, Phillips

, O'Byrne

, Becker

. Evaluation of asthma control by physicians and patients: comparison with current guidelines. Can Respir J, 2002; 9:417–423.

Bush

, De Benedictis

, Hedlin

, Paton

, Wennergren

, Wilson

. Re: A new perspective on concepts of asthma severity and control. Eur Respir J, 2009; 33:705–706.

Madsen

. A new perspective on concepts of asthma severity, control and optimum treatment. Eur Respir J, 2009; 33:704–705.

10.

Schatz

, Zeiger

, Yang

et al. Relationship of asthma control to asthma exacerbations using surrogate markers within a managed care database. Am J Manag Care, 2010; 16:327–333.

11.

Clark

, Dodge

, Shah

, Thomas

, Andridge

, Awad

. A current picture of asthma diagnosis, severity, and control in a low-income minority preteen population. J Asthma, 2010; 47:150–155.

12.

Shekelle

, Kravitz

, Beart

, Marger

, Wang

, Lee

. Are nonspecific practice guidelines potentially harmful? A randomized comparison of the effect of nonspecific versus specific guidelines on physician decision making. Health Serv Res, 2000; 34:1429–1448.

13.

Patino

, Okelo

, Rand

et al. The Asthma Control and Communication Instrument: a clinical tool developed for ethnically diverse populations. J Allergy Clin Immunol, 2008; 122:936–943.

14.

Diette

, Patino

, Merriman

et al. Patient factors that physicians use to assign asthma treatment. Ann Intern Med, 2007; 167:1360–1366.

15.

Patino

, Riekert

, Quartey

. Howard-Hopkins Center for Reducing Asthma Disparities. Development of the Asthma Control and Communication Instrument (ACCI) Am J Respir Crit Care Med, 2005; A254.

16.

Cummings

, Savitz

, Konrad

. Reported response rates to mailed physician questionnaires. Health Serv Res, 2001; 35:1347–1355.

17.

Jepson

, Asch

, Hershey

, Ubel

. In a mailed physician survey, questionnaire length had a threshold effect on response rate. J Clin Epidemiol, 2005; 58:103–105.

18.

Expert Panel Report 3 (EPR-3):. Guidelines for the diagnosis and management of asthma—summary report 2007. J Allergy Clin Immunol, 2007; 120:S94–S138.

19.

van Ryn

, Burke

. The effect of patient race and socio-economic status on physicians' perceptions of patients. Soc Sci Med, 2000; 50:813–828.

20.

Macrae

, Bodenhausen

. Social cognition: categorical person perception. Br J Psychol, 2001; 92:239–255.

21.

Roter

. The enduring and evolving nature of the patient–physician relationship. Patient Educ Couns, 2000; 39:5–15.

22.

Hall

, Horgan

, Stein

, Roter

. Liking in the physician–patient relationship. Patient Educ Couns, 2002; 48:69–77.

23.

Cooper

, Roter

, Johnson

, Ford

, Steinwachs

, Powe

. Patient-centered communication, ratings of care, and concordance of patient and physician race. Ann Intern Med, 2003; 139:907–915.

24.

Withers

, Sklar

, Crandall

. Impairment and severity: how ED physicians decide to override an impaired patient's refusal. Am J Emerg Med, 2008; 26:803–807.

25.

Zeller

, Taegtmeyer

, Martina

, Battegay

, Tschudi

. Physicians' ability to predict patients' adherence to antihypertensive medication in primary care. Hypertens Res, 2008; 31:1765–1771.

26.

Barton

, Imboden

, Graf

, Glidden

, Yelin

, Schillinger

. Patient–physician discordance in assessments of global disease severity in rheumatoid arthritis. Arthritis Care Res (Hoboken), 2010; 62:857–864.

27.

Fazekas

, Baumhackl

, Berger

et al.

Decision-making for and impact of early immunomodulatory treatment: the Austrian Clinically Isolated Syndrome Study (ACISS)

Eur J Neurol, 2010; 17:852–860.

28.

Guyatt

, Townsend

, Berman

, Keller

. A comparison of Likert and visual analogue scales for measuring change in function. J Chronic Dis, 1987; 40:1129–1133.

29.

Cummings

, Savitz

, Konrad

. Reported response rates to mailed physician questionnaires. Health Serv Res, 2001; 35:1347–1355.

30.

Dresselhaus

, Peabody

, Luck

, Bertenthal

. An evaluation of vignettes for predicting variation in the quality of preventive care. J Gen Intern Med, 2004; 19:1013–1018.

31.

Jepson

, Asch

, Hershey

, Ubel

. In a mailed physician survey, questionnaire length had a threshold effect on response rate. J Clin Epidemiol, 2005; 58:103–105.

32.

Peabody

, Luck

, Glassman

et al. Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Ann Intern Med, 2004; 141:771–780.

33.

van

, Rickelt

, Griez

. Validation of the electronic Visual Analogue Scale of Anxiety. Prog Neuropsychopharmacol Biol Psychiatry, 2008; 32:1045–1047.

34.

Zorc

, Pawlowski

, Allen

et al. Development and validation of an instrument to measure asthma symptom control in children. J Asthma, 2006; 43:753–758.

35.

Freed

, Dunham

, Gebremariam

, Wheeler

. Which pediatricians are providing care to America's children? An update on the trends and changes during the past 26 years. J Pediatr, 2010; 157:148–152.