The Sinonasal Outcome Test–22 or European Position Paper: Which Is More Indicative of Imaging Results?

Abstract

Objective

The 22-item Sinonasal Outcome Test (SNOT-22) is a trusted measure of symptom severity in chronic rhinosinusitis. The European Position Paper on Rhinosinusitis (EPOS) provides widely accepted diagnostic criteria, which include sinonasal symptoms, their duration, and imaging results. Our objective was to compare these approaches to assessing symptoms to determine if either was more indicative of radiologic findings, to support decisions in telehealth.

Study Design

Observational outcomes study.

Setting

Tertiary care center.

Methods

In total, 162 consecutive patients provided a structured sinonasal history, completed the SNOT-22, and underwent sinus computed tomography (CT) within 1 month. SNOT-22 scores, EPOS-defined symptom sets, and Lund-Mackay results were assessed. To facilitate direct comparisons, we performed stepwise evaluations of sinonasal symptoms alone and combined with duration. The discriminatory capacity for imaging results was determined through areas under the receiver operating characteristic curves (ROC-AUC) for dichotomous outcomes and ordinal regression for multilevel outcomes.

Results

In ROC-AUC analyses, SNOT-22 and EPOS-defined symptoms had similar discriminatory capacity for Lund-Mackay scores, regardless of duration. Within ordinal regression analyses, SNOT-22 nasal scores were significantly associated with Lund-Mackay scores, while EPOS-defined nasal symptoms were not statistically significantly related.

Conclusions

SNOT-22 nasal scores and EPOS-defined nasal symptoms may have similar associations with imaging results when assessed via ROC-AUC, while SNOT-22 may have more association within ordinal data. Understanding the implications of discrete patterns of symptoms may confer benefit, particularly when in-person and fiberoptic exams are limited.

Keywords

rhinosinusitis patient-reported outcome measure position statement computed tomography diagnostic assessment predictive value telemedicine COVID-19

Chronic rhinosinusitis is a highly prevalent medical condition, affecting 11.6% of American adults.¹ This condition underlies more than 4.1 million visits to physicians¹ and an estimated $60 billion in health care expenditures each year.² Visits for affected patients each begin with a staple of the clinical encounter: an inquiry into individuals’ symptoms and relevant history. Prior studies of patient-reported symptoms and diagnostic impressions related to chronic rhinosinusitis have had mixed results, with some data suggesting useful specificity and sensitivity, while others suggest poor discrimination across reports.^3-10

Chronic rhinosinusitis symptoms have been conceptualized in multiple ways in clinical practice, whether through proposed diagnostic criteria or research tools. For example, the 22-item Sinonasal Outcome Test (SNOT-22) is a validated measure of patient symptom severity increasingly used in routine practice. While originally developed for research, this instrument has demonstrated properties that provide information for clinical decision making and monitoring treatment response.^11-17 Nasal-specific items within SNOT-22 comprise their own domain, based on mathematical analyses and understanding of typical patient presentations.^7,18 At the same time, the European Position Paper on Rhinosinusitis and Nasal Polyps (EPOS) has provided widely used diagnostic criteria for chronic rhinosinusitis, which incorporates a clearly defined means of cataloguing specific aspects of sinonasal symptomatology.^19,20

Particularly in the current clime of telehealth and cost considerations, understanding how specific patterns and details of symptom assessments can predict radiological imaging findings provides important information to support clinical decision making. Previous studies of the relationship between SNOT-22 overall score and imaging scores have documented no relationship,^8,21-26 although nasal domain scores may have correlation with computed tomography (CT) findings.^7-10 Data assessing EPOS-defined symptoms related to imaging findings have been more limited.^27-29 To our knowledge, the predictive ability of EPOS-defined nasal categorization has not been previously studied in comparison to SNOT-22 assessments. Given the current practice environment, it becomes of interest to assess how we can optimally assess patients’ symptom descriptions to best discern those who would have objective imaging findings. Our objective was therefore to determine whether SNOT-22 nasal domain scores or EPOS-defined symptom sets were more indicative of imaging results. The null hypothesis was that SNOT-22 nasal scores and EPOS-defined symptoms would have no difference in their associations with Lund-Mackay CT scores.

Materials and Methods

Study Design and Participants

Approval for this study was obtained from the Partners Healthcare Institutional Review Board before data collection began. Consecutive patients age 18 years or older who presented with a chief complaint of rhinosinusitis or nasal drainage to a participating hospital ambulatory otolaryngology clinic were eligible. Patients were included if they prospectively provided a structured sinonasal history, completed the entire SNOT-22 questionnaire, and underwent sinus CT imaging within 1 month. The structured history included an assessment of symptom duration. Exclusion criteria were the presence of nonnasal chief complaints or lack of either SNOT-22 completion or related imaging within the specified timeframe. Since studies of discriminatory ability ideally include patients with a range of symptom and disease states, symptom thresholds and endoscopy were not a requirement for entry.

Chronic Rhinosinusitis Symptom Assessments

SNOT-22

SNOT-22 is a 22-question validated, disease-specific instrument, which quantifies patient-reported symptoms and functional impact of chronic rhinosinusitis. Assessing both sinonasal and related health,³⁰ it has gained widespread use because of its clinical applicability, test-retest reliability, internal consistency, validity, and responsiveness to change.^30-32 Within the instrument, each symptom is scored from 0 (no problem) to 5 (as bad as it can be). The total overall possible score ranges from 0 (best) to 110 (worst). SNOT-22 focuses on the preceding 4 weeks and contains specific domains. Domain scores are calculated by averaging the scores of questions corresponding to that domain.¹⁸ A 4-domain system based on mathematical factor analysis and clinical intuition has been described in prior literature.⁷ Here, the nasal symptom domain includes nasal blockage/congestion, altered sense of smell/taste, rhinorrhea, postnasal discharge, thick nasal discharge, facial pain/pressure, nose blowing, sneezing, and cough. Although other domain systems exist,^18,33 this system was selected because it aligns with traditional clinical diagnostic concepts and has undergone psychometric assessment.⁷

EPOS

The EPOS provides vetted guidance for the management of rhinosinusitis and helpfully includes a definition of chronic rhinosinusitis.¹⁹ Within this definition, diagnostic criteria require 2 or more sinonasal symptoms, one of which must be either nasal blockage/obstruction/congestion or nasal discharge; there should also be concurrent facial pain/pressure and/or reduction or loss of smell. In addition, EPOS diagnostic criteria require a symptom duration of at least 12 weeks and evidence of demonstrable disease on nasal endoscopy or CT imaging.

Differences between SNOT-22 nasal domain and EPOS-defined symptoms

While there are similarities in these 2 approaches, EPOS differs from SNOT-22 in multiple ways. More specifically, EPOS focuses on the need for symptom pairs clustered in the pattern described above to be supportive of the diagnosis. EPOS does not include assessments of nose blowing, sneezing, or cough, which are included in the SNOT-22 nasal domain used. Moreover, EPOS focuses on a duration of 12 weeks while SNOT-22 has traditionally queried the patient experience during the prior 4 weeks.

Radiographic Analysis

CT results were scored using the Lund-Mackay scale due to its widespread, accepted use in chronic rhinosinusitis.³⁴ This scale is often favored clinically due to its high intra- and interobserver agreement and responsiveness.^35,36 In this scale, each paired paranasal sinus and osteomeatal complex is scored from 0 to 2. A score of 0 indicates no abnormality, 1 indicates partial opacification, and 2 indicates complete opacification. An overall score ranging from 0 (no abnormality) to 24 (all areas completely opacified) is then calculated.

Data Analysis

The primary outcome focused on whether either the SNOT-22 nasal domain or EPOS nasal symptom set was more indicative of radiologic findings of chronic rhinosinusitis. This relationship was assessed via receiver operating characteristic area under the curve (ROC-AUC) analyses for dichotomous outcomes and ordinal regression for multilevel, ordered outcomes. Since ROC-AUC analyses are predicated on normal/abnormal outcome data, we used a Lund-Mackay threshold of either ≥5 or ≥1 to encompass any possible chronic rhinosinusitis finding on CT. A threshold of Lund-Mackay ≥5 was used as some authors have suggested using this cutoff to exclude incidental CT findings nonsuggestive of chronic rhinosinusitis and select for patients who have true chronic rhinosinusitis.³⁷ While the rationale here is clear, this approach may also classify as “normal” some true findings of chronic rhinosinusitis that are concentrated or severe in only 1 or a few sinuses; for example, complete opacification of only the bilateral maxillary sinuses would result in a Lund-Mackay score of 4 and be considered “normal” using such a threshold but could be quite clinically significant. In a related but separate analysis, we also evaluated the Lund-Mackay score as an ordinal outcome variable, assessing for a potential relationship between SNOT-22 nasal domain or EPOS symptom results through ordinal regression, which supports analysis of multilevel results.

To facilitate the most direct comparisons possible, we performed stepwise evaluations of sinonasal symptoms alone, then symptoms combined with their duration. More specifically, ROC-AUCs were used to determine the discriminatory ability of SNOT-22 nasal domain and EPOS-defined symptom criteria for Lund-Mackay results, with and without the factor of symptom duration of at least 3 months. ROC-AUC values <0.5 suggested no discriminatory capability, 0.5 to <0.6 low capability, 0.6 to <0.7 mild capability, 0.7 to <0.9 moderate capability, and 0.9 to 1.0 high capability.³⁸ Comparisons of the ROC-AUC among the differing approaches to cataloguing sinonasal symptoms with and without duration were assessed to test the null hypothesis that there was no statistically significant difference between these different approaches.³⁹ A threshold value of 2 for the SNOT-22 nasal domain score was used in these analyses as it represents mild or worse symptoms and provides a straightforward threshold for clinicians who seek to apply these data in real time.^7,14,40-42 Comparisons between ROC-AUC values were assessed using established nonparametric methods to test the null hypothesis that there was no statistically significant difference between groups.³⁹ The minimum sample size (n = 86) to support a study with 80% power to determine a 0.15 difference between ROC-AUC results within subjects was determined prior to proceeding with the analysis. Ordinal regression was employed according to previously established techniques⁴³; within these analyses, the Lund-Mackay score was the ordinal outcome variable, while the SNOT-22 nasal domain and EPOS-defined symptom sets were predictor variables. As multiple comparisons were performed across all of these assessments, we also applied established methods to adjust P value significance,⁴⁴ which resulted in a threshold of P = .011 to achieve statistical significance.

Results

Baseline Characteristics

A total of 162 consecutive patients from participating clinics met criteria for inclusion in the study. The mean age was 46.5 years (range, 18-80), and 68.5% were female ( Table 1 ). The median overall SNOT-22 score was 37 (range, 4-102), and the median Lund-Mackay score was 4 (range, 0-17). Within the study cohort, 57.2% applied nasal saline spray and 56.0% used nasal saline rinses either with or without added medications (eg, nasal budesonide). Intranasal steroids were used in 67.2% of patients, nasal antihistamines in 13.2%, and nasal decongestants in 34.0%. In addition, usage of oral steroids, oral antihistamines, oral decongestants, and oral antibiotics was 35.5%, 64.1%, 50.9%, and 67.6%, respectively.

Table 1.

Patient Characteristics (n = 162).

Demographics	Value
Age, mean (range), y	46.5 (18-80)
Sex, % female (95% CI)	68.5 (60.9-75.3)
SNOT-22 score, median (range)
Overall score	37 (4-102)
Domain scores
Nasal	2 (0-5)
Ear	1 (0-5)
Sleep	2 (0-5)
Psychological	1 (0-4)
EPOS-defined symptoms, % positive (95% CI)	88.9 (83.0-92.9)
Lund-Mackay overall scores, median (range)	4 (0-17)

Abbreviations: EPOS, European Position Paper on Rhinosinusitis and Nasal Polyps; SNOT-22, 22-item Sinonasal Outcome Test.

Symptoms Alone

ROC-AUC analyses were used to assess the discriminatory capability of SNOT-22 and EPOS for Lund-Mackay scores, before the stepwise addition of symptom duration. When sinonasal symptoms alone were considered, relative to Lund-Mackay scores ≥5, the SNOT-22 nasal domain and EPOS nasal symptoms alone showed low discriminatory capability (SNOT-22 ROC-AUC, 0.59; EPOS ROC-AUC, 0.53; P = .193; Table 2 ). There was no significant difference between the 2 approaches when assessing sinonasal symptoms alone ( Table 2 ). The SNOT-22 nasal domain and EPOS symptom set both also had low discriminatory capability for Lund-Mackay scores ≥1 (SNOT-22 ROC-AUC, 0.59; EPOS ROC-AUC, 0.58; P = .886; Table 2 ).

Table 2.

Receiver Operator Characteristic Areas Under the Curve for SNOT-22 Nasal Domain Scores and EPOS-Defined Nasal Symptoms.

Characteristic	ROC-AUC (95% CI)
	SNOT-22 nasal domain scores	EPOS-defined nasal symptoms	P value
Lund-Mackay scores ≥5
Nasal symptoms	0.59 (0.50-0.67)	0.53 (0.49-0.58)	.193
Nasal symptoms and reported duration of symptoms ≥3 months	0.61 (0.54-0.67)	0.53 (0.49-0.58)	.024
Lund-Mackay scores ≥1
Nasal symptoms	0.59 (0.42-0.76)	0.58 (0.46-0.69)	.886
Nasal symptoms and reported duration of symptoms ≥3 months	0.61 (0.48-0.74)	0.58 (0.46-0.69)	.590

Abbreviations: EPOS, European Position Paper on Rhinosinusitis and Nasal Polyps; ROC-AUC, receiver operating characteristic area under the curve; SNOT-22, 22-item Sinonasal Outcome Test.

Within the univariate ordinal regression models, worse SNOT-22 nasal domain scores were associated with a significantly increased odds of a worse Lund-Mackay score (3.57; 95% CI, 1.89-6.74; P < .001; Table 3 ). Meeting the EPOS nasal symptom diagnostic criteria was not statistically significantly associated with CT results (odds ratio, 3.10; 95% CI, 1.27-7.59; P = .013; Table 3 ) when appropriately accounting for multiple comparisons. When both approaches to assessing symptoms were assessed simultaneously in the same regression analyses, the odds of an abnormal Lund-Mackay score were 3.11 (95% CI, 1.59-6.09; P = .001) times higher with worse SNOT-22 nasal scores. When EPOS nasal symptom criteria were met, the odds ratio was 1.87 (95% CI, 0.70-5.02; P = .213; Table 3 ).

Table 3.

Ordinal Regression Models Assessing for Association Between SNOT-22 Nasal Domain Scores or EPOS-Defined Nasal Symptoms and Lund-Mackay Imaging Scores.

	Odds ratio (95% CI)	P value
Symptoms alone
Univariate
SNOT-22 nasal score	3.57 (1.89-6.74)	<.001
EPOS nasal symptoms	3.10 (1.27-7.59)	.013
Multivariate
SNOT-22 nasal score	3.11 (1.59-6.09)	.001
EPOS nasal symptoms	1.87 (0.70-5.02)	.213
Symptoms and duration ≥3 months
Univariate
SNOT-22 Nasal Score	3.14 (1.58-6.23)	.001
EPOS Nasal Symptoms	3.95 (1.30-11.98)	.015
Multivariate
SNOT-22 Nasal Score	2.82 (1.40-5.65)	.004
EPOS Nasal Symptoms	3.08 (0.97-9.76)	.056

Abbreviations: EPOS, European Position Paper on Rhinosinusitis and Nasal Polyps; SNOT-22, 22-item Sinonasal Outcome Test.

Symptoms With a Reported Duration of at Least 3 Months

When a symptom duration of at least 3 months was also stipulated and Lund-Mackay scores ≥5 were required, the SNOT-22 nasal domain had seemingly better discriminatory capability than EPOS-defined nasal symptoms (SNOT-22 ROC-AUC, 0.61; EPOS ROC-AUC, 0.53). However, accounting for multiple comparisons, the related P value was nonsignificant, so there was no demonstrated difference among the 2 approaches (P = .0238). When a duration of at least 3 months of symptoms and a Lund-Mackay threshold of ≥1 was used (SNOT-22 ROC-AUC, 0.61; EPOS ROC-AUC, 0.58; P = .590; Table 2 ), overall, there was no difference in discriminatory capability between the SNOT-22 nasal domain and EPOS nasal symptoms.

When univariate ordinal regression was performed, the odds of an abnormal Lund-Mackay score were 3.14 (95% CI, 1.58-6.23) times higher with worse SNOT-22 nasal domain scores. These findings were statistically significant for the SNOT-22 nasal domain (P = .001) and not significant for the EPOS nasal symptom diagnostic criteria when accounting for multiple comparisons (odds ratio, 3.95; 95% CI, 1.30-11.98; P = .015). When assessed simultaneously through an ordinal regression analysis, worse SNOT-22 nasal scores were associated with a 2.82 (95% CI, 1.40-5.65) times higher odds of a worsened Lund-Mackay score (P = .004), while meeting the EPOS nasal symptom diagnostic criteria led to an odds ratio of 3.08 (95% CI, 0.970-9.76) for an abnormal Lund-Mackay score (P = .056); this latter finding was not statistically significant.

SNOT-22 Overall Scores

The SNOT-22 overall score had low discriminatory capability of SNOT-22 for Lund-Mackay scores ≥5 (ROC-AUC, 0.53; 95% CI, 0.44-0.62) and Lund-Mackay scores ≥1 (ROC-AUC, 0.52; 95% CI, 0.34-0.71). In ordinal regression analyses, the SNOT-22 overall score did not significantly increase the odds of an abnormal Lund-Mackay score (odds ratio, 1.01; 95% CI, 0.99-1.03; P = .241).

Discussion

These data suggested that the SNOT-22 nasal domain and EPOS-defined nasal symptoms had similar discriminatory capability for Lund-Mackay imaging scores, regardless of whether a 3-month duration of symptoms was also stipulated. Ordinal regression analyses, however, demonstrated that worse SNOT-22 nasal domain scores were associated with a significantly higher odds of abnormal Lund-Mackay scores. Satisfying the EPOS-defined nasal symptom diagnostic criteria also seemed to increase the odds of abnormal Lund-Mackay scores, although this was narrowly not statistically significant when accounting for multiple comparisons.

Prior studies of the discriminatory ability of patient reports have had mixed results. Some studies have demonstrated poor association with chronic rhinosinusitis, as related symptoms may be nonspecific.^3-6 For example, authors have suggested that facial pain, dental pain, ear pain, and headache could be removed from diagnostic criteria without significant changes in diagnostic sensitivity.⁵ In contrast, other authors have concluded that questionnaire results can have functional sensitivity and specificity.⁴⁵ Prior studies have also demonstrated that even within a single validated instrument, results may be mixed, with SNOT-22 nasal domain scores performing better than overall scores,^{7-10,21-26,42} also suggesting that specific approaches to symptoms and symptom clusters can alter discriminatory capacity. With regard to EPOS, 1 prior study investigated magnetic resonance imaging (MRI) and seasonality, while another assessed those who had imaging for nonrhinological reasons.^27,28 We build upon this literature by conducting a direct comparison of the SNOT-22 and EPOS symptom criteria, focusing on CT scans in a broader-based population, and discretely incorporating the variable of symptom duration, which was not the focus of prior studies.

Amid the current coronavirus disease 2019 (COVID-19) practice environment,^46-53 otolaryngologists have faced the challenge of making diagnoses without the typical reliance on concurrent fiberoptic exam. During telehealth encounters, we often need to proceed with medical decisions based on described symptoms and histories. With this in mind, we have experienced a renewed interest in the ability of isolated subjective symptom reports to predict objective findings such as imaging results. These symptoms are also often importantly used to determine whether a patient should be seen in person for further evaluation (eg, imaging) and/or surgical consideration, as well as to what degree and timing such follow-up should occur.¹³ It has thus become incumbent upon us to discern which symptom components and combinations are most useful when diagnosing conditions that would have readily relied on potentially aerosol-generating procedures such as nasal endoscopy for assessment in the past.

When taking a history, we have traditionally asked patients and families to report symptoms in a gestalt or dichotomous way (eg, How have your sinuses been? or Do you have facial pain?). This approach is often sufficient to determine whether patients meet the symptom component of diagnostic criteria such as those used in EPOS, which defines key symptoms according to their presence or absence, further qualified by a specified duration.^19,20 Over time, incorporation of validated instruments such as SNOT-22 into daily practice has become more commonplace, such that patients may routinely receive more in-depth assessments of severity across a broader set of symptoms.^54,55 These data suggest that SNOT-22 nasal domain scores and EPOS-defined symptoms have similar discriminatory capacity for Lund-Mackay results when assessed via ROC-AUC, while SNOT-22 nasal domain scores have a significant association with Lund-Mackay scores when assessed via ordinal regression models. These data suggest that SNOT-22 nasal domain scores could provide added diagnostic value. These results are particularly relevant in the current practice setting in which we first need to assess based on symptoms alone, as physical exam and imaging more frequently cannot be simultaneously obtained relative to concern for risks and/or patient deferral of in-person visits to health care facilities amid the current pandemic.^51-53

Our study also assessed symptoms alone separately from symptoms with a specified duration of at least 3 months. The relationship between subjective symptoms and objective imaging findings was similar in both cases. In other cases, however, the serial addition of diagnostic characteristics may change the ultimate odds of disease.⁵⁶ We assessed symptoms and duration in a stepwise fashion for this reason, as well as because the SNOT-22 instrument classically focuses on the preceding 4 weeks, rather than 3 months. Here, we found that the separately reported duration of sinonasal problems did not increase the successive likelihood of SNOT-22 nasal domain scores or EPOS-defined symptoms in predicting an abnormal Lund-Mackay imaging result within the current study’s context. Overall, the present data demonstrated that symptoms alone could not strongly predict CT findings of chronic rhinosinusitis, suggesting utility in corroboration of findings.

This study has limitations. These include taking place at a single academic institution and using just 2 different measures of patients’ symptoms, SNOT-22 and EPOS. Moreover, while our study has implications for telehealth delivery during the COVID-19 pandemic and beyond, it does not directly study the use of SNOT-22 and EPOS during telehealth visits. This may be a benefit as our “gold standard” is the diagnostic capability and timing we had prior to pandemic-related limitations. In addition, Lund-Mackay thresholds of import are not definitively established,^57,58 and so we evaluated 2 proposed inflection points,²⁸ which have limitations. As we continue to collect data, future studies could be conducted to assess the actual predictive value of SNOT-22 collected during a telehealth visit for CT scans in comparison to previous traditional visits. However, such a study could be limited by any extended duration between SNOT-22 completion and subsequent CT amid the pandemic, which could ultimately measure different actual severities of rhinosinusitis, which have changed over time.

Conclusions

SNOT-22 and EPOS-defined nasal symptom assessments have similar discriminatory capability for Lund-Mackay scores when evaluated via ROC-AUC, regardless of symptom duration. Ordinal regression, however, demonstrated that the SNOT-22 nasal domain has a significant association with Lund-Mackay scores, more so than the EPOS-defined symptom set. As both SNOT-22 and EPOS symptom sets alone have limitations in discriminatory ability, further inquiries into optimizing patient-reported symptom content and patterns would provide additional value.

Footnotes

Acknowledgements

We thank Bruce Zhou for statistical conversations and Samuel Wong and Thomas Lin for additional support during the project.

Author Contributions

Allen S. Zhou, contributed to data acquisition and presentation, analysis and interpretation, all manuscript drafts, approval of the final version, and agrees to be accountable for the work; Anthony A. Prince, contributed to data acquisition, draft content, revision of manuscript content, approved the final version, and agrees to be accountable for the work; Alice Z. Maxfield, contributed to data acquisition, draft content, revision of manuscript content, approved the final version, and agrees to be accountable for the work; C. Eduardo Corrales, contributed to data acquisition, draft content, revision of manuscript content, approved the final version, and agrees to be accountable for the work; Jennifer J. Shin, contributed to the study conception and design, data acquisition, analysis and interpretation, all manuscript drafts, approval of the final version, and agrees to be accountable for the work.

Disclosures

Competing interests: Jennifer J. Shin receives textbook royalties from Springer and Plural Publishing. She is the deputy editor of Otolaryngology–Head and Neck Surgery and a recipient of the American Academy of the Otolaryngology–Head and Neck Surgery Foundation Maureen Hannley grant, the Brigham Care Redesign Program and DOS Outstanding Citizenship Awards, and the Schlager Family Innovations Fund Award.

Sponsorships: None.

Funding source: None.

References

Centers for Disease Control and Prevention. Chronic sinusitis. Accessed May 11, 2020. https://www.cdc.gov/nchs/fastats/sinuses.htm

Caulley

Thavorn

Rudmik

Cameron

Kilty

SJ.

Direct costs of adult chronic rhinosinusitis by using 4 methods of estimation: results of the US Medical Expenditure Panel Survey. J Allergy Clin Immunol. 2015;136(6):1517-1522.

Bhattacharyya

Clinical and symptom criteria for the accurate diagnosis of chronic rhinosinusitis. Laryngoscope. 2006;116(7, pt 2)(suppl 110):1-22.

Hsueh

Conley

Kim

, et al. Identifying clinical symptoms for improving the symptomatic diagnosis of chronic rhinosinusitis. Int Forum Allergy Rhinol. 2013;3(4):307-314.

Hirsch

Reiter

DiNardo

Wan

Schuman

TA.

Elimination of pain improves specificity of clinical diagnostic criteria for adult chronic rhinosinusitis. Laryngoscope. 2017;127(5):1011-1016.

Bhattacharyya

Lee

LN.

Evaluating the diagnosis of chronic rhinosinusitis based on clinical guidelines and endoscopy. Otolaryngol Head Neck Surg. 2010;143(1):147-151.

Dejaco

Riedl

Huber

, et al. The SNOT-22 factorial structure in European patients with chronic rhinosinusitis: new clinical insights. Eur Arch Otorhinolaryngol. 2019;276(5):1355-1365.

Bhattacharyya

Piccirillo

Wippold

FJ.

Relationship between patient-based descriptions of sinusitis and paranasal sinus computed tomographic findings. Arch Otolaryngol Head Neck Surg. 1997;123(11):1189-1192.

Sedaghat

Gray

Caradonna

DS.

Clustering of chronic rhinosinusitis symptomatology reveals novel associations with objective clinical and demographic characteristics. Am J Rhinol Allergy. 2015;29(2):100-105.

10.

Brooks

Trope

Blasetti

, et al. Preoperative Lund-Mackay computed tomography score is associated with preoperative symptom severity and predicts quality-of-life outcome trajectories after sinus surgery. Int Forum Allergy Rhinol. 2018;8(6):668-675.

11.

Tait

Kallogjeri

Suko

Kukuljan

Schneider

Piccirillo

JF.

Effect of budesonide added to large-volume, low-pressure saline sinus irrigation for chronic rhinosinusitis: a randomized clinical trial. JAMA Otolaryngol. Head Neck Surg. 2018;144(7):605-612.

12.

Amali

Saedi

Rahavi-Ezabadi

Ghazavi

Hassanpoor

Long-term postoperative azithromycin in patients with chronic rhinosinusitis: a randomized clinical trial. Am J Rhinol Allergy. 2015;29(6):421-424.

13.

Rudmik

Soler

Mace

DeConde

Schlosser

Smith

TL.

Using preoperative SNOT-22 score to inform patient decision for endoscopic sinus surgery. Laryngoscope. 2015;125(7):1517-1522.

14.

DeConde

Mace

Bodner

, et al. SNOT-22 quality of life domains differentially predict treatment modality selection in chronic rhinosinusitis. Int Forum Allergy Rhinol. 2014;4(12):972-979.

15.

Soler

Rudmik

Hwang

Mace

Schlosser

Smith

TL.

Patient-centered decision making in the treatment of chronic rhinosinusitis. Laryngoscope. 2013;123(10):2341-2346.

16.

Hopkins

Rudmik

Lund

VJ.

The predictive value of the preoperative Sinonasal Outcome Test-22 score in patients undergoing endoscopic sinus surgery for chronic rhinosinusitis. Laryngoscope. 2015;125(8):1779-1784.

17.

Caulley

Lasso

Rudmik

Kilty

SJ.

Pre-treatment SNOT-22 score predicts response to Endoscopic Polypectomy in Clinic (EPIC) our experience in 30 adults. Clin Otolaryngol. 2017;42(3):732-734.

18.

DeConde

Bodner

Mace

Smith

TL.

Response shift in quality of life after endoscopic sinus surgery for chronic rhinosinusitis. JAMA Otolaryngol Head Neck Surg. 2014;140(8):712-719.

19.

Fokkens

Lund

Mullol

, et al. EPOS 2012: European position paper on rhinosinusitis and nasal polyps 2012: a summary for otorhinolaryngologists. Rhinology. 2012;50(1):1-12.

20.

Fokkens

Lund

Hopkins

, et al. European position paper on rhinosinusitis and nasal polyps 2020. Rhinology. 2020;58(suppl 29):1-464.

21.

Bradley

Kountakis

SE.

Correlation between computed tomography scores and symptomatic improvement after endoscopic sinus surgery. Laryngoscope. 2005;115(3):466-469.

22.

Wabnitz

DAM

Nair

Wormald

. Correlation between preoperative symptom scores, quality-of-life questionnaires, and staging with computed tomography in patients with chronic rhinosinusitis. Am J Rhinol. 2005;19(1):91-96.

23.

Ryan

Ramachandra

Hwang

PH.

Correlations between symptoms, nasal endoscopy, and in-office computed tomography in post-surgical chronic rhinosinusitis patients. Laryngoscope. 2011;121(3):674-678.

24.

Garneau

Ramirez

Armato

, et al. Computer-assisted staging of chronic rhinosinusitis correlates with symptoms. Int Forum Allergy Rhinol. 2015;5(7):637-642.

25.

Lim

Ramirez

Garneau

, et al. Three-dimensional image analysis for staging chronic rhinosinusitis. Int Forum Allergy Rhinol. 2017;7(11):1052-1057.

26.

Zheng

Zhao

, et al. Correlation between computed tomography staging and quality of life instruments in patients with chronic rhinosinusitis. Am J Rhinol Allergy. 2010;24(1):e41-e45.

27.

Dietz

Loos

Lourijsen

Wildeman

MAM

, et al. Prevalence of chronic rhinosinusitis in the general population based on sinus radiology and symptomatology. J Allergy Clin Immunol. 2019;143(3):1207-1214.

28.

del Rio

Trost

Tartaglia

O’Leary

Michael

Seasonality and incidental sinus abnormality reporting on MRI in an Australian climate. Rhinology. 2012;50(3):319-324.

29.

Zhao

Tan

, et al. Severity of rhinosinusitis: comparison between visual analog scale given by patients and otorhinolaryngologists [published online May 13, 2020]. Am J Rhinol Allergy.

30.

Morley

Sharp

HR.

A review of sinonasal outcome scoring systems—which is best?

Clin Otolaryngol. 2006;31(2):103-109.

31.

Hopkins

Gillett

Slack

Lund

Browne

JP.

Psychometric validity of the 22-item Sinonasal Outcome Test. Clin Otolaryngol. 2009;34(5):447-454.

32.

Quintanilla-Dieck

Litvack

Mace

Smith

TL.

Comparison of disease-specific quality-of-life instruments in the assessment of chronic rhinosinusitis. Int Forum Allergy Rhinol. 2012;2(6):437-443.

33.

Feng

Wesely

Hoehle

, et al. A validated model for the 22-item Sino-Nasal Outcome Test subdomain structure in chronic rhinosinusitis. Int Forum Allergy Rhinol. 2017;7(12):1140-1148.

34.

Lund

Mackay

IS.

Staging in rhinosinusitus. Rhinology. 1993;31(4):183-184.

35.

Oluwole

Russell

Tan

Gardiner

White

A comparison of computerized tomographic staging systems in chronic sinusitis. Clin Otolaryngol Allied Sci. 1996;21(1):91-95.

36.

Metson

Gliklich

Stankiewicz

, et al. Comparison of sinus computed tomography staging systems. Otolaryngol Head Neck Surg. 1997;117(4):372-379.

37.

Ashraf

Bhattacharyya

Determination of the “incidental” Lund score for the staging of chronic rhinosinusitis. Otolaryngol Head Neck Surg. 2001;125(5):483-486.

38.

Akobeng

AK.

Understanding diagnostic tests 3: receiver operating characteristic curves. Acta Paediatr. 2007;96(5):644-647.

39.

DeLong

Clarke-Pearson

DL.

Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837-845.

40.

Toma

Hopkins

Stratification of SNOT-22 scores into mild, moderate or severe and relationship with other subjective instruments. Rhinology. 2016;54(2):129-133.

41.

Chowdhury

Mace

Bodner

, et al. Investigating the minimal clinically important difference for SNOT-22 symptom domains in surgically managed chronic rhinosinusitis. Int Forum Allergy Rhinol. 2017;7(12):1149-1155.

42.

Zhou

Prince

Maxfield

Shin

JJ.

Psychological status as an effect modifier of the association between sinonasal instrument and imaging results [published online May 26, 2020]. Otolaryngol Head Neck Surg.

43.

McKelvey

Zavoina

A statistical model for the analysis of ordinal level dependent variables. J Math Sociol. 1975;4(1):103-120.

44.

Benjamini

Hochberg

Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B. 1995;57(1):289-300.

45.

Workman

Parasher

Blasetti

Palmer

Adappa

Glicksman

JT.

Accuracy of self-reported diagnosis of chronic rhinosinusitis. Otolaryngol Head Neck Surg. 2019;160(3):556-558.

46.

Zhao

Viana

Wang

Wei

H-Q

Yan

A-H

Capasso

Otolaryngology during COVID-19: preventive care and precautionary measures. Am J Otolaryngol. 2020;41(4):102508.

47.

Ralli

Greco

de Vincentiis

The effects of the COVID-19/SARS-CoV-2 pandemic outbreak on otolaryngology activity in Italy [published online April 29, 2020]. Ear Nose Throat J.

48.

Civantos

Leibowitz

Arnold

, et al. Ethical surgical triage of patients with head and neck cancer during the COVID-19 pandemic. Head Neck. 2020;42(7):1423-1447.

49.

Brody

Albergotti

Shimunov

Nicolli

Harris

Bur

AM.

Changes in head and neck oncologic practice during the COVID-19 pandemic. Head Neck. 2020;42(7):1448-1453.

50.

Tay

Lim

Loh

KS.

Sustaining otolaryngology services for the long haul during the COVID-19 pandemic: experience from a tertiary health system. Otolaryngol Head Neck Surg. 2020;163(1):47-50.

51.

Cheng

Liu

, et al. Otolaryngology providers must be alert for patients with mild and asymptomatic COVID-19. Otolaryngol Head Neck Surg. 2020;162(6):809-810.

52.

Workman

Welling

Carter

, et al. Endonasal instrumentation and aerosolization risk in the era of COVID-19: simulation, literature review, and proposed mitigation strategies. Int Forum Allergy Rhinol. 2020;10(7):798-805.

53.

Patel

Fernandez-Miranda

Hwang

, et al. Letter: precautions for endoscopic transnasal skull base surgery during the COVID-19 pandemic. Neurosurgery. 2020;87(1):E66-E67.

54.

Shin

Carroll

Prince

Landman

AB.

The utility and feasibility of extending beyond traditional patient descriptions in daily practice. Laryngoscope. 2020;130(suppl 3):S1-S13.

55.

Shin

JJ.

An electronic interface to routinize outcomes assessment and streamline clinic workflow. Laryngoscope. 2017;127(5):1058-1060.

56.

Shin

Stinnett

Randolph

GW.

Evidence-based medicine in otolaryngology part 4: everyday probabilities—nonbinary diagnostic tests. Otolaryngol Head Neck Surg. 2013;149(2):179-186.

57.

Bhattacharyya

Fried

MP.

The accuracy of computed tomography in the diagnosis of chronic rhinosinusitis. Laryngoscope. 2003;113(1):125-129.

58.

Hopkins

Browne

Slack

Lund

Brown

The Lund-Mackay staging system for chronic rhinosinusitis: how is it used and what does it predict?

Otolaryngol Head Neck Surg. 2007;137(4):555-561.