Clinimetric analysis of outcome measures for airway clearance in people with cystic fibrosis: a systematic review

Abstract

Background:

Airway clearance techniques (ACTs) are integral to cystic fibrosis (CF) management. However, there is no consensus as to which outcome measures (OMs) are best for assessing ACT efficacy.

Objectives:

To summarise OMs that have been assessed for their clinimetric properties (including validity, feasibility, reliability, and reproducibility) within the context of ACT research in CF.

Design and Methods:

A systematic review was conducted according to Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA) standards. Any parallel or cross-over randomised controlled trial (RCT) investigating outcome measures for ACT in the CF population were eligible for inclusion. The search was performed in five medical databases, clinicaltrials.gov, and abstracts from international CF conferences. The authors planned to independently assess study quality and risk of bias using the COnsensus-based Standards for the selection of health status Measurement InstrumeNts (COSMIN) risk of bias checklist with external validity assessment based upon study details (participants and study intervention). Two review authors (GS and MJ) independently screened search results against inclusion criteria, and further data extraction were planned but not required.

Results:

No completed RCTs from the 187 studies identified met inclusion criteria for the primary or post hoc secondary objective. Two ongoing trials were identified.

Discussion and conclusion:

This empty systematic review highlights that high-quality RCTs are urgently needed to investigate and validate the clinimetric properties of OMs used to assess ACT efficacy. With the changing demographics of CF combined with the introduction of cystic fibrosis transmembrane conductance regulator (CFTR) modulator therapies, an accurate assessment of the current benefit of ACT or the effect of ACT withdrawal is a high priority for clinical practice and future research; OMs which have been validated for this purpose are essential.

Registration:

This systematic review was registered on the PROSPERO database (CRD42020206033).

Keywords

airway clearance clinimetrics cystic fibrosis endpoints outcome measures physiotherapy

Introduction

Rationale

Cystic fibrosis (CF) is a genetic disorder of ion transport across cell membranes affecting over 10,650 people in the United Kingdom.¹ The CF genetic defect causes organ damage leading to significant morbidity and premature mortality with the most common cause of death being respiratory complications.^1,2 Within the CF lung, airways are dehydrated and viscous secretions which are difficult to clear create airway obstruction, mucus plugging and an ideal environment for infections.² Removal of these secretions using a combination of airway clearance techniques (ACTs) or chest physiotherapy and mucoactive agents is key to preventing recurrent infections and airway inflammation, which can cause lung damage leading to respiratory failure and death.²

There are several ACT techniques (also known as chest physiotherapy) in use, ranging from simple breathing techniques such as the Active Cycle of Breathing Techniques (ACBT) or Autogenic Drainage (AD), hand-held devices such as the Acapella™ or Aerobika™ to high-tech machines such as the high-frequency chest wall oscillator (HFCWO), which have been designed to aid clearance. Currently, no one technique is advocated over another for people with CF³ as data are lacking to guide this; one research study over the course of a year comparing five ACT modalities [ACBT, AD, positive expiratory pressure (PEP) and two oscillatory PEP devices] demonstrated that no one technique was superior to others.⁴

Historically, the spirometric measure, forced expiratory volume in 1 s (FEV₁) has been utilised most commonly by the CF community as a clinical and research outcome.⁵ CF ACT-specific research has followed this practice, using FEV₁ alongside expectorated sputum wet weight as outcome measures (OMs). A recent systematic review and meta-analysis of OMs used to assess the immediate effects of ACT for adults with CF identified eleven different outcomes from studies completed between 1979 to August 2021, with the most frequently used being sputum weight in 72% of studies, and spirometry, especially FEV₁ in 60% of studies.⁶ Similar to these results, a recent systematic review on outcomes and endpoints used in CF pulmonary exacerbation studies also reported a wide range of OMs used,⁷ with FEV₁ being the most common.

Despite the frequent use of FEV₁ and sputum weight as OMs for ACT research, some possible flaws in their use in this context have been reported.^3,4,8,9 These include FEV₁ being effort-dependent,¹⁰ and possibly not sensitive enough to pick up changes in subjects with well-preserved lung function;^5,9 and sputum weight being subject to the will/ability of the person to expectorate and not swallow secretions.⁸ The utility of sputum weight as an OM may also be affected by the introduction of the newly developed cystic fibrosis transmembrane conductance regulator (CFTR) modulator medications particularly the compound, elexacaftor/tezacaftor/ivacaftor (Kaftrio™/Trikafta™) which is potentially suitable for 80–90% of the CF population.¹¹ These medications target the CF defect at a protein level, correcting abnormal ion transport across cell membranes,¹² thus preventing airway dehydration within the lungs and limiting the amount of sticky mucus produced, potentially making people with CF less productive of sputum than they were before taking CFTR modulators.

Due to the potential flaws in these traditional OMs, and a paucity of data specifically evaluating these outcomes for ACT assessment, debate exists within the CF community as to what is the best OM to use for ACT research.^8,9 In fact, several Cochrane reviews regard this as an unmet need, suggesting that further work is necessary to identify the most appropriate OMs for the assessment of ACT efficacy.^8,13 There has been recent growing interest in OMs for both observational and interventional studies in CF including the lung clearance index (LCI) derived from a multiple breath washout (MBW) test and hyperpolarised gases in magnetic resonance imaging (MRI). However, whether these are appropriate to assess the effect of ACTs have not been fully confirmed.

When considering which OM to use for the assessment of ACTs, it is first key to consider the construct that is airway clearance. The COnsensus-based Standards for the selection of health status Measurement InstrumeNts (COSMIN) group advocate that for an effective outcome measurement, it is essential that the construct being measured is clearly defined.¹⁴ Based upon measurement theory,^14,15 airway clearance could be considered as a complex construct combining many different items which influence it (formative model) and which it influences (reflective model), for example, sputum production, breathlessness, and ventilation inhomogeneity (Figure 1).

Figure 1.

Overview of the various items within the construct of airway clearance.

The circle represents the construct and the rectangles represent the items. Arrows running to items represent reflective models, while formative models are represented by arrows running away from items. For example, airway clearance aims to enhance alveolar recruitment and reduce ventilation inhomogeneity, but ventilation inhomogeneity itself can influence the effectiveness of airway clearance. Figure idea adapted from De Vet et al.¹⁴

To recommend an OM for use, it should be clinimetrically assessed, which involves investigating whether the OM has been validated for use with a specific construct, for example, to assess airway clearance effect in people with CF.¹⁴ This entails investigating different measurement properties for the OM. The (COSMIN) study used a Delphi approach to reach a consensus on taxonomy, definitions, and terminology of measurement properties,¹⁶ thus offering a standardised definition for use. In broad terms, the main measurement properties can be classified as validity, reliability (sometimes called reproducibility), and responsiveness.^14,16

The OM has to be reliable, changing only due to real changes and free from measurement error.^14,16 It needs to be a valid measure for the construct, essentially it has to be measuring what it says it does, and it needs to be responsive and able to detect change over time in that construct.^14,16 Furthermore, the results of the OM need to be interpretable, so that assessors can understand what they mean to the individual.¹⁶ A known minimal important difference (MID) (i.e. how much change is required to create a meaningful change for an individual) or agreed minimal clinically important difference (MCID) determined by expert clinicians can be useful in assessing interpretability.¹⁴

In 2021, a systematic review into the measurement properties of OMs used in general cystic fibrosis studies highlighted a large variety of OMs utilised, with 118 studies investigating 74 tests included. Of particular interest to this review, airway clearance research was not specifically investigated, which leaves the question as to whether any OMs have been clinimetrically assessed for the construct of airway clearance.

With this systematic review, we sought to follow on from the work of Chapman et al.,⁶ who’s 2021 systematic review identified which OMs are used to assess the immediate effects of ACTs in adults with CF. We aimed to investigate if OMs used to assess efficacy of ACTs for the whole CF population (paediatric and adult), had been specifically evaluated for their clinimetric properties within that context. Such a review is warranted to enable clinicians and researchers to assess the appropriateness of measures they may select to analyse the effects of ACT for people with CF and to identify areas requiring further work.

Objectives of the review

The primary objective of the review was to investigate the clinimetric properties of the OMs of FEV₁ and sputum weight, in order to answer the research question ‘how reliably does a change in FEV₁ or sputum weight indicate effective airway clearance in people with CF?’.

A secondary objective was added post hoc when no eligible studies were identified to answer our primary objective, as suggested in the Cochrane guidance on reporting empty reviews.¹⁷ This secondary objective was to broaden our literature search to identify any OMs which had been assessed for their clinimetric properties in the context of airway clearance assessment.

Methods

The protocol for this review was developed following the guidance set out in the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 checklist,¹⁸ the COSMIN guideline for systematic reviews of patient-reported outcome measures (PROMs),¹⁹ and advice on assessing the quality of studies assessing measurement instruments.²⁰ The 10-step process for completing a systematic review described by COSMIN¹⁹ (Figure 2) was adapted to be followed for measurement instruments as recommended by Mokkink et al.²⁰ This systematic review was registered on the international prospective register of systematic reviews (PROSPERO) database (ID CRD42020206033).

Figure 2.

COSMIN 10 step procedure for systematic reviews of PROMs¹⁹ (open-access permission from authors).

Eligibility criteria

Criteria for considering studies for inclusion in this systematic review for the primary and secondary objectives followed the PICO approach (population, intervention, comparison, outcome) and were agreed by the authors (Table 1).

Table 1.

PICO research strategy for systematic review (population, intervention, comparison, outcome) and key search terms for the primary and secondary objective.

PICO	Description	Key Search Terms
Population	Any person with CF who has been diagnosed by sweat testing, genotyping or both. Participants will be included irrespective of gender, age, or the presence of co-morbidities. People post-lung transplant were excluded. The time period covered in the search means that some patients included were receiving CFTR modulator medications.	Cystic Fibrosis, CF, mucoviscidosis
Intervention	Any parallel or cross-over randomised controlled trial (RCT) investigating outcome measures of FEV₁ (spirometric tests) and sputum weight for airway clearance. Abstracts, case series and case studies were considered. All ACTs were eligible and included breathing exercises; hand-held devices such as the Acapella™, PEP, or Flutter™; HFCWO; postural drainage; and percussion and noninvasive ventilation or exercise as ACT. Trials specifically testing pharmacotherapies were excluded.	Physical Therapy Modalities, airway clearance technique, chest physiotherapy, respiratory physiotherapy, physical therapy, respiratory therapy, positive expiratory pressure, high frequency chest wall oscillation, positive pressure oscillation, vest. Flutter, positive pressure expiration, Aerobika, lung flute, Acapella, cornet, percussion, postural drainage, breathing exercises, active cycle of breathing techniques, autogenic drainage, exercise, non-invasive ventilation.
Comparison	Trials were considered for inclusion if the author(s) compared any outcome measures with another outcome measure for measuring the effect of an ACT.	Reproducibility of Results, validation study, Psychometrics, clinimetrics, comparative study, outcome assessment, randomised controlled trials, outcome measures, health outcome assessment, reproducibility, reliability, validity, sensitivity, specificity, test-retest, observer variation, inter-rater, generalisation, minimal clinical important difference.
Outcomes	Primary objective – spirometric tests (e.g. FEV₁) or forced vital capacity (FVC) and sputum measures (e.g. sputum wet weight expectorated, dry weight, viscoelasticity) when used to evaluate the effect of ACT.	Specific outcomes were not searched for as we aimed to include any OMs used in any trials investigating clinimetrics within the context of airway clearance in CF
	Secondary objective – Other outcome measures such as exercise capacity tests, oxygen saturations, imaging, and patient-reported outcome were included.

ACT, airway clearance techniques; CF, cystic fibrosis; CFTR, cystic fibrosis transmembrane conductance regulator; FEV₁, forced expiratory volume in 1 s; FVC, forced vital capacity; HFCWO, high-frequency chest wall oscillation; PEP, positive expiratory pressure; PICO, population, intervention, comparison, outcome; RCT, randomised controlled trial.

Literature search strategy

The review was conducted on the reporting of trials investigating the clinimetric properties of OMs used to assess the efficacy of airway clearance in people with CF. Our search strategy consisted of search terms [medical subject headings (MeSH) and text words] for each of the key PICO elements of the review: (1) the CF population; (2) the construct of airway clearance or chest physiotherapy; (3) all OMs used; and (4) measurement properties (Table 1). This search strategy was adapted from a comprehensive search filter developed for PubMed by the Patient Reported Outcomes Measurement Group, University of Oxford, which was recommended in the COSMIN methodology for systematic reviews of Patient-Reported Outcome Measures (PROMs) user manual²¹ and published on the COSMIN website.²² The medical librarian (RP) based at the Royal Brompton Hospital campus library of Imperial College assisted development of our search strategy. Table 1 includes details of the key terms included in the search strategy; the full search strategy is available in the supplementary material.

In October 2020, searches were completed of the Cochrane Central Register of Controlled Trials (CENTRAL), PEDro (Physiotherapy Evidence Database), PubMed, MEDLINE, and Science Direct databases without restrictions on year, publication status, or language. Unpublished work was identified by searching Pediatric Pulmonology and the Journal of Cystic Fibrosis for the abstracts of two major CF conferences, the European Cystic Fibrosis Conference, and the North American Cystic Fibrosis Conference. Separate searches of clinicaltrials.gov, International Standard Randomised Controlled Trial Number (ISRCTN) registry, and the World Health Organisation’s International Clinical Trials Registration Platform (WHO ICTRP) databases were performed.

Literature search results were uploaded to the Covidence^© software programme to allow collaboration among authors for screening abstracts, full-text review, and risk of bias assessment.

Study selection

Search results were independently screened against the inclusion criteria based upon their title and abstract by two review authors (GS and MJ). Screening of the full text of studies that were not excluded based upon title and abstract was completed by the same authors (GS & MJ). A third author (NJS) was available to resolve any disagreement between the authors on study inclusion. Contacting authors to clarify trial details was attempted twice for two separate studies. The selection process and reasons for trial exclusion for the primary review objective are illustrated in the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) flow diagram in Figure 3.

Figure 3.

PRISMA flow diagram of the selection process for the primary review objective.

Screening for the secondary objective was completed following the same procedure and by the same authors. The selection process and reasons for trial exclusion for the secondary review objective are illustrated in the PRISMA flow diagram in Figure 4.

Figure 4.

PRISMA flow diagram of the selection process for the secondary review objective.

Data collection process

Two review authors (GS and MJ) planned to assess the methodological quality of the selected studies and independently extract data using a standardised data collection form.²³ A third author (NJS) was available to resolve any disagreements between authors.

The data extracted was planned to include number of participants; participant characteristics; trial design (randomisation, allocations, and concealment); details of the ACT intervention (type, duration, supervision, frequency, adherence); details of OMs used; publication status; and financial support details. The authors planned to use the review software to manage and analyse this data (Covidence^©). Results were planned to be grouped based upon type of OM used. Subgroups of adult versus paediatric patients and stable versus exacerbating patients were planned.

Risk of bias in individual studies

The authors planned to independently assess study quality and risk of bias using the COSMIN risk of bias checklist^20,24 including the following domains: cross-cultural validity, reliability, measurement error, criterion validity, construct validity, and responsiveness. Studies were to be rated as very good, adequate, doubtful, or inadequate for each of the domains. Each study’s external validity would have been assessed based upon the details of study participants (e.g. mean age) and the study intervention.

Confidence in cumulative evidence

The authors planned to assess the quality of evidence for each OM using the GRADE approach considering the studies over five domains (risk of bias, directness, consistency, precision, and publication bias) to give the evidence a label of high, moderate, low or very low–quality depending upon confidence in the estimate of effect.

Differences between protocol and review

Once it was apparent that no eligible studies existed for inclusion in this systematic review, the authors followed the guidance from the Cochrane Effective Practice and Organisation of Care (EPOC) resources for review authors in the reporting of empty reviews¹⁷ and created a secondary objective to identify any OMs that had been clinimetrically evaluated for airway clearance assessment. The authors planned to use this information to suggest future work to evaluate outcome measures where insufficient evidence may have been identified.

Results

Primary objective results

The literature search identified a total of 187 studies or trials for screening, 186 were excluded after title and abstract screening was completed and one was subsequently excluded after full text review as it was a conference abstract reporting preliminary results²⁵ linked to one of the ongoing trials²⁶ (Figure 2). This resulted in there being no studies for further data extraction and analysis in this systematic review.

Of note, two ongoing studies were identified,^27,28 but publication of results of these trials is still pending. One of these trials is investigating the use of MRI scanning compared with spirometry for ACT assessment,²⁸ while the other compares the clinimetric properties of FEV₁, sputum weight, impulse oscillometry, lung clearance index derived from the multiple breath washout test and electrical impedance tomography when used for ACT assessment.²⁷ We excluded a conference abstract of preliminary results from this group²⁵ after full text review as it reported upon feasibility and reproducibility of OMS (LCI, IOS, EIT, FEV₁) across two different visits, and did not report any pre- versus post-ACT data.

Secondary objective results

The 187 studies that were identified from the original literature search were then screened against the secondary objective. 183 studies were excluded as being irrelevant to the secondary objective, four were assessed by full text review and were subsequently excluded (Figure 3). Reasons for exclusion were being a pilot study not an randomised controlled trial (RCT)²⁹ (one study); having the wrong study design as it investigated the effect of ACT upon LCI rather than if LCI could measure ACT effect³⁰ (one study); and being a conference abstract lacking detail³¹ (one study) or reporting preliminary results only²⁵ (one study). Of note, the same two ongoing studies were identified as per the primary objective,^27,28 and one of these was linked to the conference abstract preliminary results report²⁵ which were excluded for the same reason as discussed above for the primary objective.

Two studies investigating LCI in the context of ACT were excluded after full text review,^31,32 the study design and reporting of these studies was such that they were investigating the impact of ACT on LCI measurements, rather than validating LCI for use as an outcome for ACT assessment. These studies reported conflicting results, with one study of 29 participants (mean age = 21.8, range 7.3–43.7years) showing that ACT may increase or decrease LCI due to ACT changing ventilation inhomogeneity,³² while the other study of 20 participants (age range = 6–26years, no mean given) showed no effect on LCI.³¹ These studies indicate that the relationship between LCI and ACT warrants further investigation, and in the context of this review, requires specific study designs looking at the use of LCI to measure ACT effect.

The final excluded study was a pilot investigation into the use of hyperpolarised 3-Helium MRI scanning to assess the effects of ACT,²⁹ it was excluded due to a high risk of bias on randomisation as it could not be characterised as an RCT. The authors reported 3-Helium MRI scans as reproducible and highly sensitive to the effects of ACT but acknowledged their conclusions were limited as the study was underpowered due to small sample size.

Strengths and limitations of this review

To the best of our knowledge, this is the first systematic review investigating clinimetric validation studies of OMs specifically for AC effect in the CF population. It aims to give insight into issues with OMs for airway clearance raised in several Cochrane Systematic Reviews^8,13 and complements a recent systematic review reporting what OMs are used for AC⁶ by reporting on specific clinimetric assessment of OMs for use in the context of airway clearance. Our work while focusing solely on the construct of airway clearance, also complements publications by McLeod et al.^7,33 the first which reviewed OMs used in CF pulmonary exacerbations⁷ followed by a second paper describing the clinimetrics of OMs used in the wider CF context.³³

This review involved a thorough search of several medical databases and clinical trials registries, based upon a comprehensive search strategy recommended by the specialist COSMIN group²² and followed a clearly defined protocol which was published on the PROSPERO database (ID CRD42020206033).

‘Airway clearance technique’ is a broad term which encompasses many different techniques, from breathing exercises to mechanical adjuncts to exercise used as airway clearance. One limitation identified while undertaking this review was that while the literature search strategy (appendix 1) tried to include all terms for ACT, it is possible some may have been missed. A further limitation is that our search strategy focused on studies investigating the clinimetrics of OMs, and while a broad search strategy was created for this purpose following published guidance^21,22 (Appendix 1), there is a possibility that work looking at aspects of clinimetrics, but not specifically stating this within the title, abstract, or keywords could have been missed. Furthermore, this review was purposely limited to evaluation of OMs for use with AC in CF, and so studies validating OMs for use in other aspects of CF which may be relevant to ACT, or in other respiratory diseases which may share similar characteristics to CF were not included.

Discussion

The aim of this review was to systematically assess the clinimetric properties of OMs when used to measure the effect of ACTs following the guidance described by the COSMIN group.^22,34 No RCTs met the inclusion criteria for our primary or secondary objectives. This outcome was considered a possibility from the outset, as historical practice has likely been the adoption of OMs used in other aspects of CF care. Furthermore, several Cochrane reviews into ACTs have highlighted the need for more robust OMs in this area.^8,35

The COSMIN group advocates that for effective outcome measurement, it is essential that the construct being measured is clearly defined.¹⁴ Airway clearance could be considered as a complex construct combining many different items which influence it and which it influences, such as sputum production, breathlessness, and ventilation inhomogeneity. In considering this, it could be hypothesised that assessing ACT using a single measure – which may only reflect one item of the airway clearance construct – could oversimplify conclusions as to ACT effect. Ideally, an OM should be able to assess the construct of ACT as a whole or a toolbox of measures which look at various aspects of ACT would be required.

From our literature search and Chapman’s systematic review,⁶ we note that many different OMs are being used in CF ACT research in addition to the traditional measures of FEV₁ and sputum weight. These include LCI derived from MBW tests, electrical impedance tomography, and a variety of radiological techniques including 3-helium MRI. We acknowledge that within our excluded studies, there was a pilot study investigating hyperpolarized 3-Helium MRI,²⁹ and two studies investigating the effect of ACT upon LCI, suggesting preliminary work has been carried out in this area. Although there were no accepted studies evaluating any measures, these OMs may have been assessed for their clinimetric properties in regard to other aspects of CF, although identifying those validation studies was outside the scope of this review.

This review identified two ongoing studies,^27,28 which suggests that the CF research community is trying to address the lack of robust ACT OMs. The results of these trials, which are assessing the clinimetric properties of some of the emerging OMs against historical OMs (spirometry, LCI, plus two forms of MRI²⁸ and LCI, impulse oscillometry and electrical impedance compared with spirometry and sputum weight)²⁷ are urgently needed. Randomised controlled trials such as these are essential so that clinicians and researchers can identify which OMs are appropriate to use for airway clearance assessment.

The demographics of the CF population is changing; there are now more adults than children with CF as people with CF live longer.¹ The development of CFTR modulator medications is likely to dramatically change the outlook for people with CF for the future.³⁶ By correcting abnormal ion transportation across cell membranes,¹² they preventing airway dehydration within the lungs and limits the thick sticky mucus which traditionally has been a breeding ground for infection and a major cause of CF lung damage, leading to respiratory failure. It is thought that while early introduction of CFTR modulators will limit the development of CF lung disease, it will not be possible to correct established lung damage,³⁶ and sputum will still be produced. While post-CFTR modulator sputum appears to be more fluid and easier to clear, ACTs to promote clearance and prevent lung infections may well remain part of daily management for many people with CF for the foreseeable future, and thus, OMs to assess efficacy will still be required.

This positive advancement in CF management does provide a welcome problem when considering OMs for airway clearance assessment. OMs that have been used traditionally may not be relevant in this new post-CTFR modulator population, for example, sputum measures will not be helpful in people with minimal secretions, and spirometry may not be sensitive enough to pick up changes in a population with ‘normal’ values.⁵ A recent review into monitoring early stage lung disease in CF highlighted that the lack of measurable defects in spirometric values does not ‘indicate the absence of inflammation, infection and remodelling’.³⁶ OMs used to assess people with CF, and other respiratory diseases, whether postexacerbation, for monitoring or for the effects of ACT, need to be sensitive enough to pick up these early lung changes.³⁷

Furthermore, and essential for all CF healthcare professionals to consider, as the CF population continues to change, the recommendation of ‘daily ACT for all’ may change and the ability to thoroughly assess when ACT is required, for example, in times of exacerbation or declining respiratory status and equally, when it could be withdrawn, will be essential. LCI may be one measure which could be used for these assessments, having been shown to be sensitive enough to detect small changes in CF lungs.³⁶ However, although LCI has good clinimetric evidence as a long-term outcome for use in the CF population,^36,38 validation is still needed for the use of LCI to measure short-term treatment effects,³⁸ especially in relation to ACTs.^30–32 The use of LCI as an outcome may also be dependent upon disease stage, with emerging evidence that it may be impractical for those with severe lung disease due to prolonged test duration.³⁹ People with advanced disease may also exhibit paradoxical LCI results due to changes in occluded lung units causing differences in the amount of communicating lung contributing to the MBW signal.⁴⁰

Similar to Chapman et al.,⁶ a recent systematic review by Mcleod et al.⁷ investigating outcomes and endpoints used in CF pulmonary exacerbation studies reported a wide range of outcomes used,⁷ with FEV₁ the most common. The authors of this review suggested that choice of OM may have historically been influenced by cost, available expertise, and equipment.⁷ This may also be the case for previous ACT trials, with spirometry and sputum weight being cheap and relatively easy with access to spirometry equipment fairly universal in CF care.³ McLeod et al.⁷ highlighted a need for a core outcome set for use in research into pulmonary exacerbations as described by the Core Outcome Measures in Effectiveness Trials (COMET) initiative⁴¹ and emphasised that these endpoints should fulfil the desired characteristics of being both clinimetrically validated and clinically meaningful to people with CF,⁷ something which ACT research also requires.

Conclusion

Empty systematic reviews, that is, reviews which find no eligible studies for inclusion to address an objective, are sometimes considered to offer no additional information to clinicians.⁴² However, we argue that they serve to highlight the gaps in evidence for a particular area of interest;⁴³ we believe this is the case here. The assessment of the effect of ACTs is essential to enable effective management regimens for people with CF. The nature of ACTs, which can have multiple and differing effects on the individual, make them unique in measurement requirements, and using OMs which have been assessed for other aspects of CF or other respiratory diseases may not be appropriate. OMs should be clinimetrically assessed specifically in the context of ACTs and be clinically meaningful to people with CF. Identifying robust OMs which have had their clinimetric properties assessed for measuring ACT effect is essential. This systematic review highlights that there is a need for targeted RCTs that provide reliable, conclusive results to determine the most appropriate OMs to use for the assessment of ACT effect, both in the clinical and research settings.

Supplemental Material

sj-docx-1-tar-10.1177_17534666221122572 – Supplemental material for Clinimetric analysis of outcome measures for airway clearance in people with cystic fibrosis: a systematic review

Supplemental material, sj-docx-1-tar-10.1177_17534666221122572 for Clinimetric analysis of outcome measures for airway clearance in people with cystic fibrosis: a systematic review by Gemma E. Stanford, Mandy Jones, Susan C. Charman, Diana Bilton, Omar S. Usmani, Jane C. Davies and Nicholas J. Simmonds in Therapeutic Advances in Respiratory Disease

Footnotes

Acknowledgements

RP, Medical Librarian at the Royal Brompton Hospital for his help with the literature search strategy. The National Institute for Health Research (NIHR) for funding GS’s fellowship which included this work.

Declarations

ORCID iD

Gemma E. Stanford

Supplemental material

Supplemental material for this article is available online.

References

CF Trust. UK Cystic fibrosis registry annual data report 2020, https://www.cysticfibrosis.org.uk/the-work-we-do/uk-cf-registry/reporting-and-resources

Bowen

S-J

Hull

The basic science of cystic fibrosis. Paediatr Child Health 2015; 25: 159–164.

Association of Chartered Physiotherapists in Cystic Fibrosis. Standards of care and good clinical practice for the physiotherapy management of cystic fibrosis. London: Cystic Fibrosis Trust, 2020.

Pryor

Tannenbaum

Scott

, et al. Beyond postural drainage and percussion: airway clearance in people with cystic fibrosis. J Cyst Fibros 2010; 9: 187–192.

Stanojevic

Ratjen

Physiologic endpoints for clinical studies for cystic fibrosis. J Cyst Fibros 2016; 15: 416–423.

Chapman

Watson

Hatton

, et al. Methods used to evaluate the immediate effects of airway clearance techniques in adults with cystic fibrosis: a systematic review and meta-analysis. J Clin Med 2021; 10: 5280.

McLeod

Wood

Schultz

, et al. Outcomes and endpoints reported in studies of pulmonary exacerbations in people with cystic fibrosis: a systematic review. J Cyst Fibros 2020; 19: 858–867.

Bradley

Moran

Elborn

JS.

Evidence for physical therapies (airway clearance and physical training) in cystic fibrosis: an overview of five Cochrane systematic reviews. Respir Med 2006; 100: 191–201.

Main

Airway clearance research in CF: the ‘perfect storm’ of strong preference and effortful participation in long-term, non-blinded studies. Thorax 2013; 68: 701–702.

10.

Graham

Steenbruggen

Miller

, et al. Standardization of spirometry 2019 update. an official American Thoracic Society and European Respiratory Society technical statement. Am J Respir Crit Care Med 2019; 200: e70–e88.

11.

Mahase

Cystic fibrosis: triple therapy shows promising results. BMJ 2019; 367: l6347.

12.

Lopes-Pacheco

CFTR Modulators: the changing face of cystic fibrosis in the era of precision medicine. Front Pharmacol 2020; 10: 1662.

13.

Wilson

Morrison

Robinson

KA.

Airway clearance techniques for cystic fibrosis: an overview of Cochrane systematic reviews. Cochrane Database Syst Rev 2019; 1: CD011231.

14.

De Vet

Terwee

Mokkink

, et al. Measurement in medicine. New York: Cambridge University Press, 2011.

15.

Fayers

Hand

DJ.

Factor analysis, causal indicators and quality of life. Qual Life Res 1997; 6: 139–150.

16.

Mokkink

Terwee

Patrick

, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 2010; 63: 737–745.

17.

Cochrane Effective Practice Organisation of Care (EPOC). Reporting empty reviews and results from excluded studies, 2017, epoc.cochrane.org/resources/epoc-resources-review-authors

18.

Shamseer

Moher

Clarke

, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015; 349: g7647.

19.

Prinsen

CAC

Mokkink

Bouter

, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res 2018; 27: 1147–1157.

20.

Mokkink

Boers

van der Vleuten

CPM

, et al. COSMIN risk of bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study. BMC Med Res Methodol 2020; 20: 293.

21.

Mokkink

Prinsen

Patrick

, et al. COSMIN methodology for systematic reviews of Patient-Reported Outcome Measures (PROMs) user manual, 2018, https://cosmin.nl/wp-content/uploads/COSMIN-syst-review-for-PROMs-manual_version-1_feb-2018.pdf

22.

COSMIN. COSMIN (Consensus-based Standards for the selection of health Measurement instruments) 2020, cosmin.nl

23.

Higgins

JPTTJ

Chandler

Cumpston

, et al. Cochrane handbook for systematic reviews of interventions version 6.0. Cochrane, www.training.cochrane.org/handbook (Accessed July 2019).

24.

Mokkink

de Vet

HCW

Prinsen

CAC

, et al. COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res 2018; 27:1171–1179.

25.

Stanford

Cathcart

Beverley

, et al. P243 outcome measures for airway clearance in adults with cystic fibrosis (CF): a randomised controlled crossover trial. Thorax 2019; 74(Suppl. 2): A222.

26.

Stanford

Parrott

Bilton

, et al. Randomised cross-over trial evaluating the short-term effects of non-invasive ventilation as an adjunct to airway clearance techniques in adults with cystic fibrosis. BMJ Open Respir Res 2019; 6: e000399.

27.

Stanford

Davies

Usmani

, et al. Investigating outcome measures for assessing airway clearance techniques in adults with cystic fibrosis: protocol of a single-centre randomised controlled crossover trial. BMJ Open Respir Res 2020; 7: e000694.

28.

NCT. 03593434 – Airway Clearance Therapy on Hyperpolarized 129Xenon and MRI, https://clinicaltrials.gov/ct2/show/NCT03593434?term=airway+clearance&cond=cystic+fibrosis&draw=2&rank=20 (2020, accessed 16 November 2020).

29.

Woodhouse

Wild

van Beek

, et al. Assessment of hyperpolarized 3He lung MRI for regional evaluation of interventional therapy: a pilot study in pediatric cystic fibrosis. J Magn Reson Imaging 2009; 30: 981–988.

30.

Fuchs

Toussaint

Edlhaimb

, et al. Short-term effect of physiotherapy on variability of the lung clearance index in children with cystic fibrosis. Pediatr Pulmonol 2010; 45: 301–306.

31.

Kouroukli

Hatziagorou

Parisi

, et al. P121 short-term effect of physiotherapy on lung clearance index among patients with cystic fibrosis. J Cystic Fibrosis 2018; 17: S93.

32.

Pfleger

Steinbacher

Schwantzer

, et al. Short-term effects of physiotherapy on ventilation inhomogeneity in cystic fibrosis patients with a wide range of lung disease severity. J Cyst Fibros 2015; 14: 627–631.

33.

McLeod

Wood

Tong

, et al. The measurement properties of tests and tools used in cystic fibrosis studies: a systematic review. Eur Respir Rev 2021; 30: 200354.

34.

COSMIN. Guideline for Systematic Reviews of Outcome Measurement Instruments, https://www.cosmin.nl/tools/guideline-conducting-systematic-review-outcome-measures/2018

35.

Wilson

Morrison

Robinson

KA.

Airway clearance techniques for cystic fibrosis: an overview of Cochrane systematic reviews. Cochrane Database Syst Rev 2019; 1: CD011231.

36.

Nissenbaum

Davies

Horsley

, et al. Monitoring early stage lung disease in cystic fibrosis. Curr Opin Pulm Med 2020; 26: 671–678.

37.

Usmani

OS.

Calling time on spirometry: unlocking the silent zone in acute rejection after lung transplantation. Am J Respir Crit Care Med 2020; 201: 1468–1470.

38.

Kent

Reix

Innes

, et al. Lung clearance index: evidence for use in clinical trials in cystic fibrosis. J Cyst Fibros 2014; 13: 123–138.

39.

Short

Saunders

Davies

JC.

Horses for courses: learning from functional tests of pulmonary health. Pediatr Pulmonol 2020; 55: 1855–1858.

40.

Horsley

Lung clearance index in the assessment of airways disease. Respir Med 2009; 103: 793–799.

41.

Prinsen

Vohra

Rose

, et al. Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a ‘core outcome set’. Trials 2014; 15: 247.

42.

Yaffe

Montgomery

Hopewell

, et al. Empty reviews: a description and consideration of Cochrane systematic reviews with no included studies. PLoS One 2012; 7: e36626.

43.

Lang

Edwards

Fleiszer

Empty systematic reviews: hidden perils and lessons learned. J Clin Epidemiol 2007; 60: 595–597.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB