Abstract
Introduction:
Asthma is a common condition affecting many children in child-care centers. The National Asthma Education and Prevention Program offers recommendations about creating an asthma-friendly child-care setting. However, no studies have investigated the extent to which child-care centers adhere to these recommendations. This study describes the development of a novel instrument to determine the ability of child-care centers to meet national recommendations for asthma.
Methods:
The Preparing for Asthma in Child Care (PACC) Instrument was developed using information from existing recommendations and standards, the peer-reviewed literature, site visits, and expert interviews. The survey questions were pilot-tested at 36 child-care centers throughout San Francisco.
Results:
The instrument is composed of 43 items across seven domains: smoking exposure, presence of a medical consultant and policies, management of ventilation and triggers, access to medication, presence of asthma action plans, staff training, and encouragement of physical activity.
Discussion:
The PACC Instrument is an evidence-based and comprehensive tool designed to identify areas to target to improve asthma care for children in child-care centers.
Introduction
N
Recognizing that the quality of asthma care provided to such children at child-care centers monitored by the San Francisco Department of Public Health (SFDPH) had never been fully evaluated, the SFDPH initiated a partnership with the University of California, San Francisco (UCSF), with the goal of developing an instrument for use by public health workers to quantify and compare child-care center asthma preparedness. This paper describes the development of a comprehensive and evidence-based instrument to measure the preparedness of child-care centers to prevent and manage asthma exacerbations (Figure 1). The Preparing for Asthma in Child Care (PACC) Instrument operationalizes the concept of asthma preparedness across seven domains identified as important by the National Asthma Education and Prevention Program (NAEPP). 13 These domains are smoking exposure, presence of an asthma consultant and policies, management of ventilation and triggers, access to medications, presence of asthma action plans, staff training, and encouragement of physical activity.

A schematic depicting development of the Preparing for Asthma in Child Care (PACC) Instrument.
Methods
Phase I: formative research and activities
All actions described took place between April 2013 and May 2014. The study was approved by the UCSF Committee for Human Research and an SFDPH staff member. First, existing materials related to asthma preparedness in child-care centers were identified. To this end, a search of the peer-reviewed literature and publicly available resources online was conducted. Several checklists exist. However, none was found that was both comprehensive and based on published recommendations from the NAEPP.14,15 The NAEPP recommendations, while holistic and originating from national experts, provide general suggestions rather than specific items that operationalize the concept of preparedness. 13
National and local standards were reviewed. In California, providers are required to attend a 15 h health and safety course upon hiring and a first aid and CPR training course every other year. 16 These curricula typically include a short section about inhaled medications. Additional asthma training is optional. The widely used American Academy of Pediatrics' Guidelines for Out-of-Home Child Care 17 do not include policies relating specifically to asthma (Table 1). In the Early Childhood Environment Rating Scale—Revised 18 (ECERS-R) and the Infant Toddler Rating Scale-Revised (ITERS-R), 19 certain recommendations conflict with asthma recommendations. For example, the ECERS-R awards points to child-care centers for providing aquariums and stuffed animals, which could serve as allergen reservoirs. Finally, when this project was initiated, the SFDPH lacked policies specific to asthma in child-care settings.
Asthma policies at other child-care organizations were reviewed, including the relevant sections of the Head Start Performance Standards. 20 To inform development of the instrument, a health manager serving all Head Start centers in San Francisco was interviewed, and five visits were conducted to child-care centers in San Francisco and a neighboring county. Key lessons that emerged were the lack of asthma-related policies and requirements for child-care and the importance of developing a comprehensive instrument. Because child-care providers are occupied with direct child-care activities during the workday, limiting the demands placed on the providers was a primary concern. For these reasons, the NAEPP guidelines 13 were selected as the framework, and an investigator-implemented instrument (i.e., one conducted by an outside observer) was selected as the format.
Phase 2: development of the instrument
The PACC Instrument focuses on the seven domains specified by the NAEPP recommendations (Table 2). 13 The instrument incorporates data from semi-structured interviews with the child-care manager, environmental assessments, and file and medication reviews. Cutoffs for adherence within each of the seven domains were set at either 100% of 66%, depending on the number of items included within the domain. For domains in which fewer than two items made up the score, 100% was used as the cutoff. For domains in which more than two items made up the score, 66% was used as the cutoff so as not to give too much weight to any individual factor (Table 3).
There is no safe level of tobacco smoke exposure for children. 30 The monthly cutoff was selected to distinguish between centers where smoke exposure as a rare versus regular occurrence.
Investigator observation of mold was not included in the final score because mold was not observed during visits.
To be adherent, a manager must report having at least two out of four of the NAEPP-recommended asthma policies in writing at the center.
Use of a spacer is recommended for all children, and facemasks are recommended for children younger than 4 years of age. 3
I, item assessed through in-person interview with childcare center manager; A, item assessed through environmental assessment; FMR, item assessed through file and medication review.
Phase 3: review and revision
Meetings with key informants and community stakeholders
The San Francisco Asthma Task Force, a local organization that has been investigating and influencing asthma policies since 2001, provided additional comments. A group of children's health and environmental health professionals, including pediatricians, nurses, health workers, environmental specialists, and an epidemiologist, provided regular feedback that shaped the instrument's overall direction and approach. Two child-care center managers provided feedback on the content and wording of the surveys.
Data collection
The assessment was conducted at child-care centers or in classrooms where children were primarily older than 2 years of age. Child-care centers or classrooms serving exclusively infants were excluded. Each site visit included three components: a semi-structured interview, an environmental assessment, and a file and medication review. For the interview, the manager (e.g., the person present at the center who is primarily responsible for the daily decisions at the center) was invited to participate. The child-care manager was interviewed rather than a staff member responsible for direct child care because the majority of the interview questions related to center-level policies or practices with which the manager would be most familiar. The majority (75%) of managers interviewed reported direct involvement in child care for some portion of the workday. A minority (25%) reported performing exclusively administrative duties. All interviews were conducted in English. The environmental assessment was conducted by an investigator in one classroom and other areas where children spent significant time, such as play areas. If there was more than one eligible classroom, then the manager was asked to select a classroom for the assessment. The majority of centers had only one eligible classroom. The file and medication review was completed by a trained SFDPH nurse, who reviewed all files at the center to identify children with asthma based on physician report. For each child with asthma, the nurse looked for an up-to-date asthma action plan, asthma medications, a spacer and mask or nebulizer, paperwork allowing child-care providers to administered inhaled medications, 16 and a medication administration log. 17 Site-level data were used for analysis.
Results
Preliminary pilot testing and revision
The observation and interview items were first piloted at five child-care centers not served by the SFDPH, including three Head Start sites serving a low-income population similar to the target population. Minor changes to wording and formatting were made based on these results. Experts in survey development and community-based participatory research reviewed the instrument.
Secondary pilot testing and revision
Forty centers were eligible for the second round of pilot testing. A full assessment was completed at 36 centers. Initially, 119 data points were collected. Items with the best face validity and variability between centers were used to create the final 43-item instrument through consensus with the research team. On average, completion of the interview and environmental observation required 26 min and 30 min, respectively. Completion of the file and medication review required <1 h per center. Full results were presented elsewhere. 21
Discussion
The PACC Instrument is a comprehensive and evidence-based tool to address asthma preparedness at child-care centers based on national asthma recommendations specific to child care. The instrument was developed for use by a health worker or nurse and identifies specific domains in need of improvement. Although based on pilot data from a small sample of child-care centers, this tool is an improvement over the most commonly cited asthma preparedness checklists, which offer a list of allergen-related items with some general preparedness items 14 or focus solely on allergens. 15 Moreover, because child-care providers and community partners provided input throughout the development process, the instrument is relevant for individuals who will benefit from its use.
While specific to child care, the NAEPP's own recommendations 13 for asthma in child-care settings offer general advice rather than specific items for quantifying and comparing performance. For example, the statement that the center should be “free of tobacco smoke at all times” could be interpreted differently by different users. Does this statement include secondhand smoke? If the manager has ever smelled smoke at the center during his or her career, would the center be considered nonadherent? Such questions make interpreting the NAEPP recommendations difficult. The PACC Instrument minimizes these potential ambiguities.
This instrument has several limitations. First, in condensing the instrument from 119 to 43 items, formal analyses were not performed but rather expert consensus was relied on to select items with the greatest face validity and variability between centers. Additionally, formal reliability or validity testing was not conducted. A sample size of 36 is small. However, it was selected for feasibility and to meet the needs of the SFDPH. In terms of feasibility, not all child-care centers have access to public health nurses who can review files and medication. However, it was felt that with support from a public health department or other health worker, this instrument could be implemented in a practical fashion.
Future work will focus on pilot testing the instrument in its condensed form and conducting formal reliability and validity testing with a larger sample size. Posting the instrument on the SFDPH Web site and circulating it to other public health departments and child-care organizations could increase its uptake. Additionally, converting the instrument to a self-assessment tool for use by child-care providers should be explored. An instructional video may be necessary if this approach is taken. Other successful interventions designed for child care 22 offer comprehensive web-based training to support providers with implementation.
Asthma is a significant problem for young children and their caretakers. With more children spending increasing time in child-care settings, a method to assess a center's ability to care for children with asthma is crucial for public health practitioners, child-care administrators, and parents. To the authors' knowledge, there have been only a small number of rigorous interventions targeting child-care providers and the child-care environment for asthma preparedness and management.23–25 The PACC Instrument could help facilitate additional research on this important topic.
Footnotes
Acknowledgments
The authors would like to acknowledge the San Francisco Department of Public Health Child Care Health Program nurses and health workers—Peter Vaernet, Jaleila King, Carol Thrailkill, Ivy Steverson, Yan Oi Wong, Tito Arana, and Lisa Tao-Lew—for making this project possible, and the providers at the San Francisco General Hospital Pediatric Asthma Clinic for support and feedback. This project was supported by the Clinical and Translational Research Fellowship Program (CTRFP), a program of UCSF's Clinical and Translational Science Institute (CTSI) that is sponsored in part by the National Center for Advancing Translational Sciences, National Institutes of Health, through UCSF-CTSI Grant Number TL1 TR000144 and the Doris Duke Charitable Foundation (DDCF). This publication was made possible by the National Institute on Minority Health and Health Disparities, National Institutes of Health, through Grant Number R25MD006832. An abstract of the results of this project was presented at the American Academy of Pediatrics National Conference and Exhibition in San Diego on October 11, 2014. The contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH, UCSF, or the DDCF.
Author Disclosure Statement
No competing financial interests exist.
