Psychometric Properties of the Mini Pediatric Asthma Quality of Life Questionnaire in a US Sample

Abstract

The 13-item Mini Pediatric Asthma Quality of Life Questionnaire (MiniPAQLQ) was developed to measure asthma-specific quality of life in children. However, no validation studies have been conducted in the United States. This study aimed at determining the psychometric properties of the MiniPAQLQ in a US sample. Children aged 7–17 years and with an asthma diagnosis (n = 193) were identified from primary care clinics within an integrated healthcare system in the Southeastern United States. Participants completed surveys consisting of the MiniPAQLQ and the control module of the Asthma Therapy Assessment Questionnaire (ATAQ). Convergent validity was determined based on association between the MiniPAQLQ and ATAQ scores. Internal consistency reliability was determined from Cronbach's alpha coefficients for the MiniPAQLQ subscales (symptoms, emotions, and activities). Item-convergent validity was examined based on corrected item-total correlations. Item-discriminant validity was determined by comparing corrected item-total correlations and item-to-other-scale correlations. Floor and ceiling effects were examined based on the percentage of respondents having the lowest and highest scores on the MiniPAQLQ. A negative association was observed between quality of life scores from the MiniPAQLQ and asthma control as determined from the ATAQ, providing evidence of convergent validity. Internal consistency reliability was good with Cronbach's alpha values of above 0.8 for the MiniPAQLQ subscales. Item-convergent validity was confirmed, whereas item-discriminant validity was not confirmed. Floor effects were absent, whereas ceiling effects were present. The MiniPAQLQ possesses moderately good psychometric properties among children and adolescents in the United States and could be a useful tool for asthma management in clinical practice.

Introduction

Measurement of health-related quality of life in patients with asthma complements the traditional clinical measures including forced expiratory volume and peak expiratory flow in routine assessment and has been recommended in the recent asthma management guidelines.¹ Established instruments for measuring health-related quality of life in pediatric asthma patients include the Pediatric Asthma Quality of Life Questionnaire (PAQLQ),² the Child Health Survey for Asthma,³ the Pediatric Quality of Life Inventory 3.0 Asthma Module,⁴ and the Childhood Asthma Questionnaire.⁵ The PAQLQ is the most frequently used instrument to measure asthma-specific quality of life in children.⁶ The instrument has been found to have good psychometric properties, with evidence of cross-sectional and longitudinal construct validity, responsiveness, and reliability.⁷ Although the PAQLQ is an effective instrument that is used to measure quality of life in children with asthma, it has certain limitations that restrict its usefulness in routine clinical practice. First, the PAQLQ is time-consuming to complete due to the involvement of some patient-specific questions. Second, the PAQLQ is interviewer administered and requires trained personnel for administration.⁷ The newly developed MiniPAQLQ is a shortened version of the 23-item PAQLQ. This 13-item scale is self-administered and was developed as an instrument with reduced respondent burden for ease of use in long-term clinical trials and routine clinical practice.⁷

Scant literature currently exists regarding psychometric properties of the MiniPAQLQ. A thorough review of the literature yielded only one study in this regard. Wing et al.⁷ determined the psychometric properties of the MiniPAQLQ using a sample of children receiving care at healthcare practices in Birmingham and Warwickshire in the United Kingdom. The authors found the MiniPAQLQ to have adequate criterion and construct validity, test–retest reliability, and responsiveness.⁷ Although this study provides useful information, it has certain limitations, including a small sample size (42) and a UK setting that restrict its generalizability. To support routine usage of the MiniPAQLQ in future epidemiological studies and clinical practice, it is necessary to evaluate its psychometric properties in diverse patient samples.

The objective of the current study was to examine the psychometric properties of the MiniPAQLQ in a sample of pediatric patients with asthma in the southeastern United States. In particular, the convergent validity, internal consistency reliability, item-convergent and item-discriminant validity, and floor and ceiling effects of the MiniPAQLQ were assessed.

Materials and Methods

Patients and procedure

This study was conducted at Carolinas HealthCare System, a non-profit healthcare system located in the southeastern United States. The study sample consisted of children and adolescents aged 7–17 years who received asthma care at one of the clinics within the system and had participated in a previous asthma comparative effectiveness study conducted within the system.⁸ Patients were included if they had a documented asthma diagnosis (International Classification of Diseases, Ninth Revision, Clinical Modification code of 493) in the Carolinas HealthCare System's electronic medical records. A sample of 497 eligible children and adolescents were contacted between January and March 2015 and were invited to complete a survey consisting of the MiniPAQLQ, the Asthma Therapy Assessment Questionnaire (ATAQ) control module, and a few demographic questions (race, ethnicity, and school affiliation). Participants had the option of completing paper-based or online survey (via SurveyMonkey^®). A $10 merchandise card was offered to those who completed the survey as compensation for their time. A total of 216 children responded to the survey (response rate of 43.5%). The study received approval from the Institutional Review Board of Carolinas HealthCare System.

Study measures

Mini Pediatric Asthma Quality of Life Questionnaire

The MiniPAQLQ is an asthma-specific, health-related quality of life instrument that is designed for children aged 7–17 years. It consists of 13 items, which are divided into 3 subscales: activity limitations (3 items); emotional function (4 items); and symptoms (6 items). Each item is scored on a scale of 1–7, where 1 reflects maximum impairment and 7 reflects no impairment. All items are equally weighted, and the scores on all the items associated with the subscales are averaged to determine the score for each subscale. The total quality of life score is calculated by averaging the scores on the 13 items. The subscale scores and the total score range from 1 to 7.⁷

Asthma Therapy Assessment Questionnaire for children and adolescents

The control module of the ATAQ was used for measuring asthma control. The ATAQ control module consists of 7 items related to recent (in the past 4 weeks) symptoms and consequences of asthma: wheezing during exercise; frequency of wheezing when not exercising; nocturnal awakenings; school time loss; daily activity loss; parent's assessment of child's asthma control; and frequency of quick reliever use. Responses to each of these items are dichotomized and summed to determine the total score. The total score ranges from 0 to 7, with a higher score indicating a greater number of control problems and poorer asthma control.⁹ The ATAQ control module has been found to have good internal consistency reliability and predictive and construct validity among children aged 5–17 years.^9,10

Statistical analysis

The survey data were examined for the presence of missing values. Surveys with missing data for the MiniPAQLQ or the ATAQ items were excluded from the analysis (N = 23). Convergent validity is defined as the degree to which the results of a measure are similar to a measure that it should be related to.¹¹ In the current study, convergent validity of the MiniPAQLQ was determined based on the correlation between health-related quality of life score obtained on the MiniPAQLQ and asthma control score obtained on the ATAQ control module. Asthma control has been reported to be negatively associated with health-related quality of life in asthma patients in several studies.^7,12–16 Accordingly, negative correlations between the MiniPAQLQ and the ATAQ control module scores were considered evidence for convergent validity.^12,17 Spearman's correlation was used as the statistical technique. The correlations between the total MiniPAQLQ score as well as the scores on the MiniPAQLQ subscales and the ATAQ control module score were reported. Correlations >0.5 were considered evidence for convergent validity.¹⁸ Internal consistency reliability is defined as the degree of consistency among different survey items intended to measure the same construct.¹⁹ Internal consistency reliability was determined using Cronbach's alpha for the MiniPAQLQ subscales. As per the rule of thumb, a Cronbach's alpha value between 0.7 and 0.8 was considered acceptable, that between 0.8 and 0.9 was considered good, and that above 0.9 was considered excellent.²⁰

Item-convergent validity is defined as the degree of convergence between an item and its own scale.²¹ In the current study, item-convergent validity was determined based on the corrected item-total correlations (correlation between an item and the total score on the subscale excluding that item) for each of the MiniPAQLQ items.²¹ Corrected item-total correlations of 0.4 or above provided evidence of item-convergent validity.²² Item-discriminant validity is the degree of divergence between an individual item and other scales, excluding its own scale.²¹ In this study, the item-discriminant validity of the MiniPAQLQ was determined by comparing the item-to-other-scale correlations (correlation between the score on an item in a subscale and the total score on the other subscales) and the corrected item-total correlations.^21,23 Item-discriminant validity was present when all the item-to-other-scale correlations were significantly lower compared with the corrected item-total correlations. Statistical significance for these comparisons was determined based on the 95% confidence intervals for the correlation coefficients. Non-overlapping confidence intervals indicated statistical significance.²³ Floor and ceiling effects were, respectively, considered present if the percentage of respondents who received the lowest (1) and the highest possible score (7) on the MiniPAQLQ was greater than or equal to 15%.²⁴ All analyses were conducted using SAS v9.4 (SAS Institute, Inc., Cary, NC).

Results

Sample characteristics

Table 1 lists the characteristics of the study sample. The mean age of the study respondents was 11.97 (±2.80) years. Approximately 61% of the respondents were male. The sample was ethnically diverse with ∼51% whites and ∼27.5% blacks; whereas the remaining 21.5% consisted of Asians, Hispanics, American Indians, Alaskan Natives, individuals with multiple races, and those with an unknown race. Regarding insurance, ∼63.2% of the respondents had commercial insurance; ∼27.5% had Medicaid; and ∼9.3% had charity, self-pay, or other governmental/public insurance. Average scores on the MiniPAQLQ favored better quality of life, with an overall mean score of 5.86 (±1.29) and subscale means of 5.76 (±1.45) (activities), 6.11 (±1.24) (emotions), and 5.74 (±1.37) (symptoms). The mean number of control problems obtained from the ATAQ was 1.73 (±2.03).

Table 1.

Study Sample Characteristics

Characteristic	Descriptive statistics
Age, mean (SD)	11.97 (2.80)
Male, N (%)	118 (61.14)
Race, N (%)
White	98 (50.78)
Black	53 (27.46)
Hispanic	2 (1.04)
Asian	4 (2.07)
Other^a	36 (18.65)
Insurance, N (%)
Commercial	122 (63.21)
Medicaid	53 (27.46)
Other^b	18 (9.33)
MiniPAQLQ score, mean (SD)
Total score	5.86 (1.29)
Activities	5.76 (1.45)
Emotions	6.11 (1.24)
Symptoms	5.74 (1.37)
ATAQ control module score	1.73 (2.03)

N = 193.

Consisted of American Indians, Alaska Natives, Native Hawaiians and other Pacific Islanders, individuals with multiple races, and those with unknown race.

Consisted of charity, self-pay, and other public/governmental insurance.

SD, standard deviation; MiniPAQLQ, Mini Pediatric Asthma Quality of Life Questionnaire; ATAQ, Asthma Therapy Assessment Questionnaire.

Convergent validity

Table 2 presents results of the correlation analyses between the MiniPAQLQ scores and the ATAQ control module score. All the correlations observed between the total MiniPAQLQ score [Spearman's correlation coefficient (ρ) = −0.741, P < 0.001] as well as the scores on the subscales (activities: ρ = −0.696, P < 0.001; emotions: ρ = −0.663, P < 0001; and symptoms: ρ = −0.736, P < 0.001) and the ATAQ control module score were above the acceptable value of 0.5¹⁸ and statistically significant, indicating the presence of convergent validity.

Table 2.

Correlation Between the Mini Pediatric Asthma Quality of Life Questionnaire Scores and the Asthma Therapy Assessment Questionnaire Control Module Score

MiniPAQLQ scores	ATAQ control module score
Total score	−0.741^*
Activities	−0.696^*
Emotions	−0.663^*
Symptoms	−0.736^*

The values mentioned are Spearman's correlation coefficients.

P < 0.001.

Internal consistency reliability

The Cronbach's alpha coefficients for the symptoms, emotions, and activities subscales of the MiniPAQLQ were found to be 0.94, 0.92, and 0.88 respectively. The overall Cronbach's alpha coefficient for the MiniPAQLQ was found to be 0.97 (results not shown in the tables).

Item-convergent and item-discriminant validity

The corrected item-total correlations ranged from 0.694 to 0.886 (Table 3). In terms of analyses for assessing item-discriminant validity, the item-to-other-scale correlations were significantly lower than the corrected item-total correlations for only 2 items: “bothered by wheezing” (item-to-other-scale correlation of 0.667 with the emotions subscale compared with the corrected item-total correlation of 0.822 with the symptoms subscale) and “felt frustrated because of asthma” (item-to-other-scale correlation of 0.797 with the activities subscale compared with the corrected item-total correlation of 0.886 with the emotions subscale) (Table 3).

Table 3.

Item-Total Correlations for the Mini Pediatric Asthma Quality of Life Questionnaire

Subscale	Items	Symptoms	Emotions	Activities
Symptoms	Bothered by coughing	0.764	0.682	0.682
	Bothered by wheezing	0.822	0.667^*	0.767
	Bothered by tightness in chest	0.830	0.722	0.764
	Felt out of breath	0.870	0.830	0.801
	Felt tired	0.850	0.836	0.774
	Had trouble sleeping at night	0.794	0.855	0.737
Emotions	Felt frustrated because of asthma	0.852	0.886	0.797^*
	Felt frightened or worried because of asthma	0.781	0.804	0.753
	Felt irritable because of asthma	0.782	0.844	0.758
	Felt different or left out because of asthma	0.716	0.746	0.692
Activities	Bothered doing physical activities	0.809	0.778	0.782
	Bothered being with animals	0.666	0.678	0.694
	Bothered doing activities with friends and family	0.848	0.803	0.854

Gray cells represent corrected item-total correlations (correlation between an item and the total score on the subscale excluding that item).

Significant difference between corrected item-total correlations and item-to-other-scale correlations.

Floor and ceiling effects

Approximately 20% of the respondents (39/193) had a score of 7 on the MiniPAQLQ, indicating the presence of moderate ceiling effects. Floor effects were absent, since none of the respondents had a score of 1 on the MiniPAQLQ.

Discussion

Patient-reported outcomes (PROs) such as quality of life are being increasingly used in routine asthma management.^25–27 We examined the psychometric properties of the MiniPAQLQ, a recently developed PRO instrument that measures asthma-specific quality of life in children and adolescents. Our primary finding was that there was a strong negative association between quality of life as determined from the MiniPAQLQ and asthma control as determined from the control module of the ATAQ, which confirmed the convergent validity of the MiniPAQLQ. We also observed ceiling effects, with approximately a fifth of participants obtaining the highest score on the MiniPAQLQ. Our findings add to the literature by evaluating the psychometric properties of this instrument in a US sample. This is also the first study to determine the item-convergent and item-discriminant validity and floor and ceiling effects associated with the MiniPAQLQ.

The negative association observed between asthma-specific health-related quality of life and the number of asthma control problems obtained from the ATAQ [Spearman's correlation coefficients of −0.741, −0.696, −0.663, and −0.736 (P < 0.001) for the association between the total MiniPAQLQ score and scores for the activities, emotions, and symptoms subscales, respectively, and the ATAQ control module score] in our study is consistent with the results of the study by Wing et al.⁷ That study found a significant negative association between the MiniPAQLQ scores and the asthma control score on the Asthma Control Questionnaire in their study set in the United Kingdom [Correlation coefficients of −0.81, −0.76, −0.77, and −0.73 (P < 0.001) for the association between the total MiniPAQLQ score and scores for the activities, emotions, and symptoms subscales, respectively, and the Asthma Control Questionnaire score].⁷ Prior research by Okelo et al. also found a similar association between the emotional domain of the longer PAQLQ and asthma control scores on the ATAQ in a sample of adolescents aged 11–17.²⁸

With Cronbach's alpha coefficients of 0.94, 0.92, and 0.88 for the symptoms, emotions, and activities subscales, respectively, the MiniPAQLQ was found to have good internal consistency reliability. In their study of children and adolescents residing in Birmingham and Warwickshire, United Kingdom, Wing et al.⁷ found a Cronbach's alpha coefficient of 0.79 for the symptoms and activities subscales of the MiniPAQLQ. However, the Cronbach's alpha value for the emotions subscale (0.63) was less than the acceptable limit of 0.70 in that study.⁷ A limitation of the study by Wing et al.⁷ was its low sample size (42). Javali et al. recommend a sample size of at least 50 to obtain a good estimate of internal consistency reliability.²⁹ With a sample of close to 200 patients, our study provides a better estimate of the internal consistency of the MiniPAQLQ and confirms the ability of the items within each subscale to provide coherent scores.

All the corrected-item total correlations for all the MiniPAQLQ items were above 0.4, indicating the presence of item-convergent validity. However, with non-significant differences between item-to-other-scale correlations and corrected item-total correlations, the study results did not provide evidence of item-discriminant validity for the MiniPAQLQ. A plausible explanation for this finding could be the presence of a general positive or negative feeling among the respondents about their health, which could have led them to respond similarly to all the MiniPAQLQ items. Although prior information about the item-discriminant validity of the MiniPAQLQ, the parent questionnaire PAQLQ, or the adult versions AQLQ and MiniAQLQ is not available for comparison, some studies evaluating the item-discriminant validity of the comparable St. George Respiratory Questionnaire have found similar results.^30,31 For example, in a study examining the psychometric properties of the Indonesian version of the St. George Respiratory Questionnaire, Adnan et al. found that several items in the activities subscale correlated better with the impact subscale (comparable to the emotions subscale in the MiniPAQLQ), indicating a lack of item-discriminant validity.³¹ The lack of item-discriminant validity and the excellent overall internal consistency reliability (0.97) for the MiniPAQLQ observed in this study suggest that it could be better interpreted as a measure of quality of life if used as a total scale score rather than considering the subscales. Floor effects were absent for the MiniPAQLQ. However, ceiling effects were present, indicating that there may be some challenges in using the MiniPAQLQ for measuring improvement in quality of life among those with good health.

This study had a few limitations. Considering the high mean quality of life scores observed in this study, it is possible that children responding to the survey had low asthma severity. Also, this study involved patients from the southeastern United States, primarily North Carolina. Together, these factors may limit the generalizability of our study results. Although the surveys were addressed to children, it was not possible to ensure whether parents were involved in the completion of the surveys or not. Finally, due to the cross-sectional nature of the study, we could not assess longitudinal measures such as test–retest reliability and predictive validity.

In summary, the MiniPAQLQ was found to have moderately good psychometric properties. Convergent validity was observed with the finding of negative association between asthma-specific quality of life and the number of asthma control problems on the ATAQ control module. Internal consistency reliability was good. Strong corrected item-total correlations were observed, providing evidence for item-convergent validity. Item-discriminant validity was not confirmed, as non-significant differences were observed between item-to-other-scale correlations and corrected item-total correlations. Floor effects were absent, whereas ceiling effects were present. To the best of our knowledge, this is the first study assessing the psychometric properties of the MiniPAQLQ in a US sample. Considering its short administration time, the MiniPAQLQ could be a useful tool for asthma management in clinical practice. However, the ceiling effects observed in this study suggest that clinicians should consider complementing the MiniPAQLQ scores with asking specific questions about changes in quality of life over time while making treatment decisions among patients with good asthma control.

Footnotes

Acknowledgment

This study was funded by the Carolinas HealthCare Foundation.

Author Disclosure Statement

No competing financial interest exist.

References

National Asthma Education and Prevention Program. Expert Panel Report 3 (EPR-3): Guidelines for the diagnosis and management of asthma-summary report 2007. J Allergy Clin Immunol, 2007; 120:S94–S138.

Juniper

, Guyatt

, Feeny

, Ferrie

, Griffith

, Townsend

. Measuring quality of life in children with asthma. Qual Life Res, 1996; 5:35–46.

Asmussen

, Olson

, Grant

, Fagan

, Weiss

. Reliability and validity of the children's health survey for asthma. Pediatrics, 1999; 104:e71.

Chan

, Mangione-Smith

, Burwinkle

, Rosen

, Varni

. The PedsQL™: reliability and validity of the short-form generic core scales and asthma module. Med Care, 2005; 43:256–265.

Christie

, French

, Sowden

, West

. Development of child-centered disease-specific questionnaires for living with asthma. Psychosom Med, 1993; 55:541–548.

Hafkamp-de Groen

, Raat

. Asthma and health related quality of life in childhood and adolescence. In: Pereira

, ed. Allergic diseases—highlights in the clinic, mechanisms and treatment. Rijeka, Croatia: InTech Open Access Publisher, 2012, p. 366.

Wing

, Upton

, Svensson

, Weller

, Fletcher

, Walker

. The standardized and mini versions of the PAQLQ are valid, reliable, and responsive measurement tools. J Clin Epidemiol, 2012; 65:643–650.

Tapp

, Hebert

, Dulin

. Comparative effectiveness of asthma interventions within a practice based research network. BMC Health Serv Res, 2011; 11:188.

Skinner

, Diette

, Algatt-Bergstrom

, Nguyen

, Clark

, Markson

, Wu

. The Asthma Therapy Assessment Questionnaire (ATAQ) for children and adolescents. Dis Manag, 2004; 7:305–313.

10.

Diette

, Sajjan

, Skinner

, Weiss

, Wu

, Markson

. Using the pediatric asthma therapy assessment questionnaire to measure asthma control and healthcare utilization in children. Patient, 2009; 2:233–241.

11.

Parker

. Principles of measurement, research, and practice. In: Darcy

, ed. The chiropractic theories: a textbook of scientific research. Baltimore, MD: Lippincott Williams & Wilkins, 2004, p 66.

12.

Doz

, Chouaid

, Com-Ruelle

, Calvo

, Brosa

, Robert

, Decuypère

, Pribil

, Huerta

, Detournay

. The association between asthma control, health care costs, and quality of life in France and Spain. BMC Pulm Med, 2013; 13:15.

13.

Williams

, Wagner

, Kannan

, Bolge

. The association between asthma control and health care utilization, work productivity loss and health-related quality of life. J Occup Environ Med, 2009; 51:780–785.

14.

Vollmer

, Markson

, O'Connor

, Sanocki

, Fitterman

, Berger

, Buist

. Association of asthma control with health care utilization and quality of life. Am J Respir Crit Care Med, 1999; 160:1647–1652.

15.

Pereira

, Cavalcante

, Pereira

, Lucas

, Holanda

. Asthma control and quality of life in patients with moderate or severe asthma. J Bras Pneumol, 2011; 37:705–711.

16.

Correia de Sousa

, Pina

, Cruz

, Quelhas

, Almada-Lobo

, Cabrita

, Oliveira

, Yaphe

. Asthma control, quality of life, and the role of patient enablement: a cross-sectional observational study. Prim Care Respir J, 2013; 22:181–187.

17.

Voll-Aanerud

, Eagan

, Plana

, Omenaas

, Bakke

, Svanes

, Siroux

, Pin

, Antó

, Leynaert

. Respiratory symptoms in adults are related to impaired quality of life, regardless of asthma and COPD: results from the European community respiratory health survey. Health Qual Life Outcomes, 2010; 8:107.

18.

Carlson

, Herdman

. Understanding the impact of convergent validity on research results. Organ Res Meth, 2012; 15:17–32.

19.

Tavakol

, Dennick

. Making sense of Cronbach's alpha. Int J Med Educ, 2011; 2:53.

20.

Cohen

, Setodji

, Evenson

, Ward

, Lapham

, Hillier

, McKenzie

. How much observation is enough? Refining the administration of SOPARC. J Phys Act Health, 2011; 8:1117.

21.

Yang

J-J

, Yao

. Item discriminant validity. In: Michalos

, ed. Encyclopedia of quality of life and well-being research. Berlin, Germany: Springer Reference, 2014, pp. 3414–3415.

22.

Fayers

, Machin

. Multi-item scales. Quality of life: the assessment, analysis and interpretation of patient-reported outcomes. Chichester, United Kingdom: John Wiley & Sons, 2007, pp. 109–130.

23.

Jakobsson

, Westergren

, Lindskov

, Hagell

. Construct validity of the SF‐12 in three different samples. J Eval Clin Pract, 2012; 18:560–566.

24.

Terwee

, Bot

, de Boer

, van der Windt

, Knol

, Dekker

, Bouter

, de Vet

. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol, 2007; 60:34–42.

25.

Baiardini

, Braido

, Brandi

, Canonica

. Allergic diseases and their impact on quality of life. Ann Allergy Asthma Immunol, 2006; 97:419–428.

26.

U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research, U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research, U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Life Outcomes, 2006; 4:79.

27.

van der Molen

, Foster

, Caeser

, Muller

, Postma

. Difference between patient-reported side effects of ciclesonide versus fluticasone propionate. Respir Med, 2010; 104:1825–1833.

28.

Okelo

, Wu

, Krishnan

, Rand

, Skinner

, Diette

. Emotional quality-of-life and outcomes in adolescents with asthma. J Pediatr, 2004; 145:523–529.

29.

Javali

, Gudaganavar

, Raj

. Effect of varying sample size in estimation of coefficients of internal consistency. WebmedCentral Biostatistics, 2011; 15:WMC001572.

30.

Engstrom

, Persson

L-O

, Larsson

, Sullivan

. Reliability and validity of a Swedish version of the St George's Respiratory Questionnaire. Eur Respir J, 1998; 11:61–66.

31.

Adnan

, Perwitasari

, Mulyani

. Validation of St. George respiratory questionnaire (SGRQ) into Indonesian version for tuberculosis patients in Indonesia. IJPHS, 2014; 3:179–184.