Abstract
Inhalation therapy has become the primary treatment mode for most pediatric pulmonary diseases. Errors in inhalation technique can decrease intrapulmonary drug deposition, which may lead to a decrease in therapeutic response. This study examined the ability of caregivers of young children with asthma, after being provided careful and standardized instruction, to correctly demonstrate how to use 1 of 2 different inhaler delivery systems. Caregivers of children aged 1–6 years, who were being prescribed an inhaler device for the first time, were trained by a staff asthma educator using a standardized educational checklist. One week later, the caregiver was required to demonstrate the correct preparation, use, and maintenance of their prescribed device to a different staff asthma educator, who rated their performance using the same standardized educational checklist. Errors (total and major) in carrying out the individual steps for each delivery device were recorded, and a Performance Mastery Score (measure of ability to do device steps correctly) was calculated. The total and major error rates were 24.8% and 15.6%, respectively, for the metered-dose inhalers (MDIs) with spacer, and 15.9% and 8.5%, respectively, for the nebulizer (P < 0.001). The mean Performance Mastery Score was 84.5% for the MDI with spacer and 91.9% for the nebulizer (P < 0.001). Caregivers of young children with asthma demonstrate a number of errors in device use, including major ones that can potentially result in poor lung delivery, even 1 week after standardized training by an asthma educator. Mastery of the MDI with spacer delivery system appears somewhat harder than that seen with the nebulizer. These findings suggest the need for repetition of sessions reinforcing inhalation instructions at follow-up in addition to initial education of proper device use, no matter what type of delivery system is selected.
Introduction
There is limited information available on the outcome of instructional practices by practitioners to patients with asthma regarding proper technique in administering and maintaining the inhaler device and delivery system prescribed. The few studies that have been done have not been conducted in those younger than 6 years. They have also lacked some aspects of scientific rigor, such as independent evaluators and inter-rater reliability.2–4 The present study was undertaken to understand more about these actual teaching habits and their ability to result in mastery of the inhaler device selected by the physician for the younger patient with asthma.
This study examined the ability of caregivers of young children (aged 1–6 years) with asthma to correctly demonstrate how to use 1 of 2 different inhaler delivery systems (nebulizer with mask/mouthpiece, or an MDI/spacer with mask/mouthpiece) after being provided careful and standardized instruction on that 1 prescribed delivery system. The study took place in the “real world” setting of the clinic where the young child was first prescribed their inhaler device. The specific objective was to determine overall mastery of the child's prescribed device by the caregiver, by measuring the total rate of caregiver error associated with use of their device. We hypothesized that regardless of standardized caregiver education, (1) error in recalling the proper use of inhaler delivery systems will occur and (2) certain devices are more difficult than others to train caregivers to use, resulting in a greater rate of error associated with their utilization.
Patients and Methods
Study design
The design was a 2-visit data collection event, targeting caregivers of young children who were being prescribed 1 of the 2 preidentified inhaler devices for the first time. The data collection took place at an already occurring clinic visit where the patient was receiving normal care. Ten outpatient allergy and asthma treatment centers were utilized from different geographical regions of the United States.
Eligibility for enrollment required the following criteria: (1) The child was between age 1 and 6 years, (2) the inhaler device prescription was one of the allowed devices, and (3) the inhaler device was new to the patient (ie, had not been previously used by the child). Institute Review Board approval was obtained from the Essex Institutional Review Board, Inc. (Lebanon, NJ).
Inhaler devices used
The treatment device was either (1) a pressurized MDI with spacer (with mouthpiece or mask) or (2) a nebulizer (with mouthpiece or mask). The inhaled medication could be either a quick-reliever medication, a controller medication, or both. To standardize the experience of the study population and keep sufficient homogeneity in the subgroups, the allowed type of nebulizer and spacer device were limited to certain brands (see Table 1).
Visit 1
Visit 1 included the following:
Obtaining informed consent. Providing caregiver with a self-administered questionnaire to obtain demographics and prior personal experience with asthma and inhaler devices. Providing step-by-step patient inhaler device education as a part of their normal care by clinic education staff. The staff used a study-specific standardized educational checklist (see Fig. 1) appropriate to the child's device to standardize their demonstration. The caregiver was not provided with any additional take-home materials other than the instructions that normally come with the inhaler devices.

Sample section of standardized educational checklist. MDI, metered-dose inhaler.
Visit 2
Visit 2 occurred 1 week (±1 day) after visit 1. At this second visit, the caregiver was seen by a different nurse or educator than the one who provided their training the previous week to ensure an independent assessment. During visit 2,
The caregivers demonstrated what they recalled about how to prepare, use, and maintain/clean the device that was prescribed for their child. A nurse/educator observed and rated this demonstration on the standardized educational checklist that was used during their instruction at visit 1.
Standardized teaching
Only medically trained clinic staff (eg, licensed vocational nurse [LVN], registered nurse [RN], nurse practitioner [NP]) pretrained in the specific educational content, patient education methods, and study procedures were allowed to do the visit 1 caregiver teaching. The standardized educational checklist was designed for teaching correct inhaler device technique to the caregiver. The checklist for each device was developed from review of product information, the medical literature, and national asthma guidelines. The steps for proper technique were divided into 3 stages: preparation (setup), application (use or administration), and maintenance (cleaning) of the device (see Tables 2 and 3 for steps of each stage).
Major error if step omitted or incorrectly performed.
MDI, metered-dose inhaler.
Major error if step omitted or incorrectly performed.
Inhaler device technique rating
A key part of the evaluation at visit 2 was direct observation of the caregiver's demonstration of the correct preparation, use, and maintenance of their prescribed device. As with the trainers, raters were pretrained in the specific educational content of proper MDI inhaler with spacer and nebulizer technique given at visit 1 to the caregivers. The standardized educational checklist from visit 1 was used for this rating process (see Fig. 1). Each specific step that was expected as a part of the caregiver's return demonstration was marked on the checklist as successfully demonstrated or incorrect. Only steps applicable to the device prescribed for the patient (ie, a spacer with mask instead of a spacer without mask) were demonstrated and evaluated.
Error rating
Before the scoring of the study data, an expert panel (2 clinicians specialized in pediatric asthma, 2 clinic nurses/educators) collaboratively designated steps on the checklist for each device as either major or minor. The definitions of a major versus a minor error were as follows:
A major error—an error made in any step that would lead to a significant decrease in the lung delivery of the inhaled medication. • A minor error—any other error made that was not determined to be major.
Tables 2 and 3 identify the steps, which, if not correctly performed, would result in either major or minor errors.
Data analysis
Inter-rater agreement
To assess the reliability and precision of the rating process, we assessed the percentage of inter-rater agreement on the designation of error scores. Four of 10 study sites were asked to have an additional person present to be able to simultaneously rate the caregiver return demonstration at visit 2. These 2 raters evaluated each step that had been taught during the caregiver education and how well it was performed during the return demonstration. The percent agreement between the errors reported in the paired ratings for each step was assessed with Kappa statistics. 5 Inter-rater agreement results were used to establish the extent of consensus on the occurrence of errors observed by the study personnel at each site.
Device error rate
The major, minor, and total error rates for each device were calculated by dividing the number of errors (major, minor, or total) by the total possible errors (major, minor, or total) for that device. The error rate also is specifically reported for preparation, application, and maintenance steps. Differences in rates between devices were tested using Pearson chi-square tests.
Individual Performance Mastery Score
A Performance Mastery Score was calculated for each individual. It was defined as the proportion of major steps correctly performed out of all major steps possible for the given inhaler device.
The total number of major steps is dependent on the type of device prescribed (see Table 4). Performance Mastery Scores can be compared across the 2 devices, because the number of major steps correctly performed is reported as a proportion of total major steps for that device. The Performance Mastery Scores were reported for each device as the percent of caregivers with scores in each quartile and as group means. The differences in mean scores by device were tested using a t-test for samples with unequal variance.
Additional analysis
Additional exploratory analyses using parametric and nonparametric statistics were performed to identify factors related to success in using the inhaler device on the return visit. These factors included caregiver demographic characteristics, including previous caregiver experience with asthma. Analyses were conducted using SPSS. 6
Results
Population characteristics
A total of 287 caregivers were approached to be part of the study. Of these caregivers, 39 declined participation and 78 were screened for participation but were found to be ineligible.
A total of 170 caregivers completed visit 1 and 158 of these completed visit 2, making an overall completion rate of 92.9%. Data from 4 caregivers were incomplete and, therefore, omitted, resulting in 154 caregivers with sufficient data for analysis.
The demographic information for the participants by treatment group is presented in Table 5. Many more children overall were prescribed the MDI with spacer than the nebulizer (110 versus 44). A greater number of older children received treatment with an MDI device (mean age, 4.1 years), whereas a greater number of younger children received a nebulizer (mean age, 2.9 years). The average age of caregivers was ∼33 years for each group; all caregivers were a parent of the child. Most caregivers were women and married, with an average level of education for each group of caregivers being ∼14 years. In general, other than the age of the child, the 2 groups of caregivers had similar demographics.
SD, standard deviation.
Inter-rater agreement
The 4 sites selected to participate in the inter-rater process provided paired ratings of each step for 45 patients (nebulizer n = 8, MDI n = 37). The level of agreement among raters was high: The Kappa statistic for both the nebulizer group and the MDI group was 0.64 (P < 0.001).
Error rates in device use
The error rates observed for each device are shown in Table 6. The total error rate for the MDI with spacer (24.8%) was significantly higher than that seen with the nebulizer (15.9%) (P < 0.001). The major error rate for the MDI with spacer (15.6%) was also significantly higher than that of the nebulizer (8.5%) (P < 0.001).
Statistical significance (Chi-square, P < 0.001) for all differences in major, minor, and total error rates between nebulizers and MDI with spacer.
The major error rate for each device by the stage of use (preparation, application, and maintenance) is shown in Table 7. In preparation of the device, the nebulizer had a lower rate of errors than the MDI with spacer (2.3% versus 11.4%; P < 0.001). In application of the device, the nebulizer had a similar rate of errors as the MDI with spacer (22.2% versus 20.6%; P < 0.743). In maintenance of the device, the nebulizer had a lower but not statistically significant difference in rate of errors compared with the MDI with spacer (11.4% versus 23.6%; P < 0.09).
Statistical significance (Chi-square, P < 0.001) for differences in major error rates.
Performance Mastery Scores of study sample
The mean [standard deviation (SD)] Performance Mastery Score for caregivers using the nebulizer was greater than that of caregivers using the MDI with spacer [91.9 (SD 11.3) versus 84.5 (SD 15.5); P < 0.001]. The percentages of caregivers within each level of Performance Mastery Score are shown in Fig. 2. As seen, the nebulizer group had more caregivers with higher scores than the MDI/spacer group.

Performance Mastery Scores by percentage of patients demonstrating use of nebulizer and MDI with spacer.
Factors related to device success
There were no significant correlations between the Performance Mastery Score and any of the measured demographic characteristics, including previous caregiver experience with asthma.
Discussion
The present study assessed the ability of trained caregivers to correctly administer inhaled therapy to children younger than 7 years of age with asthma to expand the existing body of literature that focuses on errors made using MDIs, DPIs, and spacers in adult patients with asthma.3,4,7,8 It mainly focused on the caregiver of the child, in particular, their ability to retain and recall instructions around the proper administration of the inhaled medication and use of the prescribed delivery device.
A major finding of the study was that despite optimal training, caregivers of young children with asthma aged 6 years or younger still demonstrate a number of errors in device use, even 1 week after training [15.9% (nebulizer) to 24.8% (MDI with spacer)]. Errors that were considered “major” (ie, more likely to interfere with lung deposition) were less common (8.5%, nebulizer; 12.6%, MDI with spacer). This error rate was demonstrated despite standardized training conducted by skilled asthma educators, in clinics having expertise in providing care for children with asthma.
Little information is available in the literature about how well parents/caregivers are able to utilize proper technique for delivery systems with very young children, the age group studied here. One study evaluated the parents of children younger than 5 years attending a pulmonology clinic for the very first time and reported that 42% of them made at least 1 error in administration of an inhaled treatment using a spacer; however, no device training and follow-up testing of proper technique took place, as was done in the present study. 9 There are no published reports of the ability of children or their parent or caregiver to follow through with the proper steps for nebulization therapy. This study is the first to undertake this type of observation. The findings presented here suggest the need for not just initial education of proper device use but also re-education, with repetition of sessions reinforcing inhalation instructions at follow-up visits.
This study provided an opportunity to compare the ability of caregivers to correctly use a nebulizer versus an MDI with spacer for delivery of their child's asthma medication after proper instructions. The number of caregivers in the MDI with the spacer group was considerably larger than that of the nebulizer group, mostly due to prescribing practices. Nevertheless, a comparison could be made. The average Performance Mastery Score was significantly higher for the nebulizer group (91.9%) than the MDI with the spacer group (84.5%), suggesting it was somewhat easier to educate caregivers on proper use of the nebulizer. The total error rate also was significantly higher for the MDI with spacer than the nebulizer, suggesting the same conclusion.
Device use in this study was divided into 3 stages: preparation (setup), application (use or administration), and maintenance (cleaning). Critical errors that can occur in any of these 3 stages may significantly impact lung deposition of medication. When the types of error were examined for both devices, more errors (total and major) were made for each device in the application stage than in the other 2. In the application stage, the nebulizer had a slightly higher (but statistically nonsignificant) rate of major errors versus the MDI with spacer; whereas the nebulizer had a lower rate of major errors versus the MDI with spacer in the preparation and maintenance stages. When all major errors from all stages were combined, a significantly lower percentage of errors was made with the nebulizer. These results indicate that education of the proper inhaler device use is needed for all stages but should particularly focus on the specific details of the application of medication.
Comparison and preference studies looking at the ability of adults and older children to master different inhaled delivery systems for asthma (eg, MDI versus DPI) have been reported.4,8,10 However, none have been conducted in young children or have used a nebulizer as a comparison, as done in this study. Knowledge in this area is important, as there is a strong recent trend toward use of the MDI with spacer with or without mask for treating pre-elementary school-aged children (and even infants and toddlers) with asthma.11–15 In the United States, nebulization has historically been the more common way to deliver inhaled asthma medications to this age group. Regarding prescribing 1 delivery system over the other, the present study results imply that (1) the MDI with spacer delivery system may have a higher error rate in use than the nebulizer and (2) whatever delivery system is selected for the young child, errors in use of the device are likely, despite careful up-front education and personal demonstration of the device to the caregiver.
It should be noted the present study has several limitations. First, it examined error rates of 2 different delivery systems but not the impact of this error rate on actual asthma control or efficacy of therapy prescribed. That is, the clinical impact of these errors, or any differences in error rate between devices, was not assessed. Additionally, the nebulizer group was smaller than had been planned; however, the differences between groups were still found to be statistically significant. Further, there was an unavoidable difference in ages between the groups (the nebulizer group was younger than the MDI group); and it could be that the better proficiency in the nebulizer group was due to the parents of infants being more concerned than the parents of older children, so greater attention was paid to the initial training. Also, it should be noted that only 1 interactive session with the caregiver at the first visit comprised the extent of instruction in this study. The level of device mastery could have possibly been improved if the initial education had been supplemented with additional educational material (written handout or video) for the caregiver to take home to study. Others have shown that video instructions for inhaler technique may work as well, or even better than individual one-on-one training. 16 A further limitation is that an assessment of caregiver technique occurred only once, at visit 2. A longer study, with ratings performed at multiple times, to see how technique recall was maintained, would provide additional insight. Finally, the definition of a major error in device use, versus a minor one, was determined by expert consensus—it was not generated from study of clinical impact on medication efficacy.
In conclusion, errors in use of delivery devices for treating asthma in young children were demonstrated by caregivers despite standardized initial education on proper use of their child's prescribed device. Some of these errors were of the types that could likely result in poor lung delivery, thereby giving less-than-optimal clinical results. Mastery of the MDI with spacer delivery system appeared to be harder than that seen with the nebulizer. Further studies are needed to see whether these differences in error rate and device mastery lead to any differences in clinical outcomes.
Footnotes
Acknowledgments
The study was funded by a grant from AstraZeneca Pharmaceuticals LP, 1800 Concord Pike, Wilmington, DE 19803.
Author Disclosure Statement
Dr. Welch serves on the AstraZeneca speakers bureau and his corporation has received research grants/support for clinical trials.
