Abstract
Objectives:
Quality of life (QoL) questionnaires are widely used in clinical interviews to assess the impact of medical interventions or measure the outcomes of healthcare services. The main aim of such questionnaires is the subjective assessment of health status and its impact on QoL. This study aimed to develop an efficient, short sinonasal disease assessment instrument, the sinonasal outcomes test-12 (SNOT-12), and to compare it with the preexisting SNOT-22.
Methods:
This was a two-phase cross-sectional study. The study was performed between June 2019 and February 2020 using the electronic files of the ORL department outpatient clinics at King Fahd University Hospital, affiliated with Imam Abdulrahman Bin Faisal University. The study was performed in 2 phases: an item reduction phase, which resulted in an initial SNOT-12 scale, and a validation phase, using a comparative analysis of the initial SNOT-12 and the SNOT-22.
Results:
The developed short-form SNOT-12 maintained the 4 latent factors extracted in EFA (nasal, Sleep/extra nasal, psychological, ear/facial). It strongly correlated with SNOT-22 (r = 0.973). It had good construct reliability (0.705-0.901) and validity and a higher discrimination power than the SNOT-22.
Conclusions:
The SNOT-12 is a short, valid, and reliable instrument that may prove useful for the initial screening and monitoring of patients with chronic rhinosinusitis.
Keywords
Introduction
Chronic rhinosinusitis (CRS) is an inflammation of the nose and the paranasal sinuses. It is a common clinical condition that significantly impacts the quality of life (QoL) and individual morbidity. CRS cases represent 10.8% of all outpatient clinic visits. Therefore, patient-reported outcome measures (PROMs) are widely used in clinical interviews to assess patients’ perspectives regarding their systemic health status and its impact on QoL. 1
Morley and Sharp 2 compared 15 sinonasal disease-specific questionnaires; they considered the sinonasal outcomes test-22 (SNOT-22) the most adequate self-assessment tool for CRS with or without nasal polyps. The SNOT-22 comprises 22 items reflecting the health burden of CRS symptoms and health-related QoL measures centered on 5 distinct symptom domains: rhinological, extra-nasal, ear, facial, sleep dysfunction, and psychological disease. 3 The SNOT-22 has been translated into different languages and applied to various adult patient groups. Alanazy et al. adopted and validated a literary Arabic version of the SNOT-22. It exhibited excellent internal consistency with excellent test to re-test reliability, similar to the English version. 4
For optimum quality of care for patients with CRS, it is essential to improve PROMs through validation, making them more efficient and convenient for both patients and physicians. The SNOT-22 questionnaire has the advantage of combining items specific to sinonasal disease with general health items, which may be assessed alone or together. However, it is considered lengthy, and it includes redundant items. 5 This imposes a significant respondent burden, making it time-consuming for patients to complete and clinicians to score. Also, they do not specifically relate to the typical CRS symptomatology seen by otolaryngologists. 6
In this study, we developed and validated a short version of the preexisting SNOT-22 questionnaire by removing redundant questions and those with the least contribution to the scale validity and reliability items without reducing the overall information provided. Also, for the newly developed short version, all questions associated with higher content validity were included (ie, the instrument’s capacity to properly measure all important aspects associated with the disease at hand for patients with self-perceived symptoms of CRS to enable tracing improvements in the targeted symptoms and, hence, optimizing patient-centered care and facilitating treatment outcomes. 5
Methods
A license agreement for the use of the questionnaires was obtained from Washington University. Written consents were obtained from the participants and from parents of participants younger than 18 years.
The study was conducted between June 2019 and February 2020 using the electronic files of the ORL department outpatient clinics at King Fahd University Hospital, affiliated with IAU. In the clinics, all patients were routinely asked to complete the validated Arabic-translated SNOT-22 7 before their initial evaluation. Their demographic data, personal habits, and medical histories were also obtained. This study was conducted in 2 phases.
Phase 1: Item Reduction
The original SNOT-22 is a self-administered questionnaire designed to investigate rhinosinusitis. “All rights reserved. Copyright 2006. Washington University in St. Louis, Missouri.” It uses a six-point Likert-type response format to obtain data regarding a list of symptoms and social/emotional consequences of rhinosinusitis. Scoring ranges from 0 (No Problem) to 5 (Problem as bad as it can be); the total score ranges from 0 to 110, with higher scores indicating worse conditions.
In Phase 1, item analysis was performed on the Arabic-translated SNOT-22, 4 filled out by 75 patients with rhinosinusitis and 150 controls, that is, patients visiting the clinic for other ENT reasons. All were evaluated using history and examination, including nasal endoscopy. Student’s t-test was performed to obtain items with statistically significant different scores between the case and the control groups. Moreover, 4 rhinologists prioritized the 10 most “clinically relevant” SNOT-22 items during their initial assessment for rhinosinusitis. They had a clinical consensus conference, where options with less than 2 votes were eliminated. This resulted in 10 SNOT items considered by the group to be the most clinically robust based on their clinical experience in evaluating CRS (Table 3).
Exploratory factor analysis (EFA) was performed. The 22 questions of the SNOT-22 were factor analyzed using principal components analysis with promax rotation. This yielded 4 latent factors; we kept the item loadings at more than 0.3 7 (facial pain/pressure loading was 0.298, but it was kept because it is a cardinal item of SNOT-22). Confirmatory factor analysis (CFA) was computed using AMOS 22 to test the measurement models. The model was run many times on a trial to obtain a shorter version of SNOT-22 while keeping the 4 latent constructs obtained in the EFA.
Reasons to consider the deletion of an item were: a factor loading below 0.5 or a standardized residual covariance above 2. When high modification indices (MI) showed redundancy between similar indicators within each construct, the indicator showing smaller factor loadings was considered for deletion.8 -11 Based on the clinical experts’ consensus on the 10 most important items and the patients’ choices of the most irritating SNOT-22 symptoms, we re-ran and re-assessed the model until a 4-factor model with 12 items was obtained (SNOT-12). This SNOT-12 gave the best model fit and good validity and reliability. The model fit measurements were used to assess the model’s overall goodness-of-fit with the following acceptance levels: CMIN/DF <3, GFI, AGFI, CFI, NFI, RFI, IFI, TLI >0.9, PNFI >0.5, and SRMR, RSMEA <0.08.8 -11
Construct reliability was assessed using Cronbach’s alpha and composite reliability and established for constructs of values above .7. 12 The convergent validity scale of items was estimated using average variance extracted (AVE), with a threshold value above 0.5.12,13 Discriminant validity was assessed using the Fornell and Larcker criterion: discriminant validity was established when the square root of AVE for a construct was greater than its correlation with the other constructs. We also used the heterotrait monotrait (HTMT) ratio, where discriminant validity was established for ratios less than the required limit of 0.85. 13
Phase 2: Comparative Analysis of the Initial SNOT-12 and SNOT-22
We validated the SNOT-12 through a register-based cross-sectional survey of 613 patients who visited the ORL department. After completing the SNOT-22 questionnaire, the patients underwent a comprehensive evaluation by an ENT specialist to confirm the coded diagnosis based on EPOS 2020 defining criteria for CRS (symptoms, compatible endoscopic findings and/or computed tomography abnormalities when imaging was available). 14
We excluded the following patients from the study: those younger than 10 years, those with a systemic granulomatous disease, recurrent acute rhinosinusitis, ciliary dyskinesia, cystic fibrosis, or malignant tumors, and/or those who underwent surgery for CRS within 6 months, and patients with a psychiatric disorder.
Statistical Analyses
The SNOT-12 score was calculated by adding the scores of the 12 selected items of the initial SNOT-12 from Phase 1, and the SNOT-22 was calculated by adding the scores of the 22 items in the SNOT-22. Statistical analyses were performed using IBM SPSS Statistics for Windows, Version 26, with the significance level set at 0.05 (2-tailed).
The content validity of SNOT-12 was evaluated by correlation with SNOT-22. Construct validity was evaluated using EFA, and the reliability of the SNOT-12 was evaluated using Cronbach’s alpha. Moreover, areas under the receiver operating characteristic (ROC) curve were compared for the SNOT-12 and SNOT-22, stratified by a SNOT-22 mild/moderate/severe (MMS) classification15 -17: mild, SNOT-22 score of 8 to 20; moderate, >20 to 50; severe, >50; normal, <8. The overall model quality after ROC was indicated by the correct prediction rate for positive responses. A good model has a value >0.5; <0.5 indicates that the model is no better than random prediction. 18
Results
Phase 1
There were 75 CRS cases and 150 comparable controls (age: 10-69 years, median: 33 years). Also, 53% were males, 14.7% were smokers, and 1.8% consumed alcohol. Other medical conditions were asthma (13.8%; more in the CRS group), DM (8.4%), hypertension (8%), laryngopharyngeal reflux (8.4%), nasal trauma (21.3%; more in the control group), and other allergies (54.2%) (Table 1). The most irritating symptoms are displayed in Table 2.
Basic Characteristics of the Patients by Grouping According to Chronic Rhinosinusitis.
For the same row, a and b are significantly different at P < .05 in the 2-sided z-test of proportion adjusted for all pairwise comparisons using the Bonferroni correction.
Distribution of Patients According to the Most Irritating SNOT-22 Symptom Grouped by Chronic Rhinosinusitis Status.
For the same row, a and b are significantly different at P < .05 in the 2-sided z-test of proportion adjusted for all pairwise comparisons using the Bonferroni correction.
For EFA, the SNOT-22 questions were factor analyzed. The Kaiser–Meyer–Olkin measure was 0.915, indicating an adequate sample, and Bartlett’s test of sphericity was significant (χ2 = 2867.657, P < .0001). We used both the scree plot and eigenvalues >1 to determine the underlying components. The analysis yielded 4 latent factors explaining 66.14% of the variance: nasal, sleep issues, psychological (emotional + functional), and ear/facial. Table 3 shows the factors, the percentage of the variance explained by each factor after rotation, and the item loadings to each latent factor.
Exploratory Factor Analysis, Clinical Consensus and Confirmatory Factor Analysis Items, Loading, Reliability, and Convergent Validity Latent Factors.
% of variance explained after rotation. AVE, average variance extracted; CA, Cronbach’s alpha; CR, composite reliability.
CFA was computed using AMOS 22 to test the measurement models. We re-ran and re-assessed the model until a 4-factor model with 12 items was obtained (SNOT-12). The model yielded good fit: CMIN/DF = 2.281, GFI = 0.970, AGFI = 0.952, CFI = 0.979, NFI = 0.963, RFI = 0.949, IFI = 0.979, TLI = 0.971, PNFI = 0.7, and SRMR = 0.035, RMSEA = 0.047. The model and factor loadings are shown in Table 3 and Figure 1.
Confirmatory factor analysis model for SNOT-12, with standardized estimates.
Construct reliability: Cronbach’s alpha for each construct exceeded the required limit of .7. Composite reliabilities ranged from 0.705 to 0.901, above the 0.7 benchmark. Hence, construct reliability was established for each construct (Table 3).
Convergent validity: The AVE values were above the threshold value of 0.5, except for the “ear/facial” factor, where AVE = 0.44. However, since the construct reliability was over the required value and AVE is above 0.4, we could conclude that the “ear/facial” construct was valid. Therefore, the scales used have the required convergent validity (Table 3).
Discriminant validity: The square root of the AVE for each construct is greater than its correlation with the other constructs. Moreover, all HTMT ratios were less than the required limit of 0.85; hence, discriminant validity was established (Table 4).
Discriminant Validity using Fornell and Larcker Criterion and Heterotrait Monotrait (HTMT) Ratio.
The 12 symptoms (initial SNOT-12) resulting were: runny nose, need to blow nose, nasal blockage, thick nasal discharge, decreased sense of smell or taste, waking up at night, lack of a good night’s sleep, sadness, reduced productivity, dizziness, ear fullness, and facial pain/pressure.
Phase 2
This phase comprised 613 patients visiting ENT clinics (age: 10-70 years, median: 33 years). Patients with CRS formed 17.6% of the sample, while the rest (82.4%) complained of 1 of the following: vocal cord problems and hoarseness of voice, ear infection and problems concerning hearing, tinnitus, and imbalance, seasonal allergies, headache, CSF rhinorrhea, nasal trauma and epistaxis, septal deviation, reflux, insomnia, apnea attacks and SOB, cough, excessive tears, facial pain, sore throat, and URTI.
Both groups were comparable regarding age, sex, smoking status, alcohol consumption, and medical history. The CRS group had higher percentages of allergies and asthma (Table 1). Their most irritating symptoms, according to the SNOT-22, are presented in Table 2.
The SNOT-12 scores ranged from 0 to 60, with a median of 19, and the SNOT-22 scores ranged from 0 to 108, with a median of 32. Both were higher in the CRS group (SNOT-12 = 25 and SNOT-22 = 42) than the non-CRS group (SNOT-12 = 17 and SNOT-22 = 26).
Validation of SNOT-12
SNOT-12 and SNOT-22 had a strong significant direct correlation (r = 0.973, P = .0001). SNOT-12 had good reliability using the internal consistency (Cronbach’s alpha (CA) = .895), compared to CA = .943 for the SNOT-22 (not shown in the tables). Based on the SNOT-22 MMS classification, 8.19% of the sample was normal, 13.1% was mild, 37.9% was moderate, and 30% was severe.
The AUC for SNOT-12 and SNOT-22 were 0.625 and 0.593, respectively. Furthermore, SNOT-12 had a higher overall model quality. Stratified by a SNOT-22 MMS classification, SNOT 12 discrimination for CRS cases was comparable to the SNOT-22 in normal levels but higher in the other severity levels. SNOT-12 also gave higher accuracy for mild CRS followed by normal, moderate, and severe levels (Figure 2).
ROC curves for SNOT-22 and SNOT-12 as discrimination for chronic rhino sinusitis by mild/moderate/severe (MMS) classification.
Discussion and Conclusions
Health status, as described by the World Health Organization, is a multidimensional concept encompassing physical, mental, and social states of well-being. 17 Improvement in any of these states due to management is considered an achievable outcome. Thus, disease-specific, health-related QoL measuring tools have been developed, such as the SNOT-22, which is an indicative measure of a patient’s perspective regarding the disease. It includes different elements, such as physical limitations and functional and emotional impacts due to CSR. This instrument primarily evaluates the effectiveness and efficiency of treatment. 19
As per clinical practices and research purposes, many measures of patient-based rhinosinusitis outcomes have recently been developed.20,21 Juniper and Guyatt 22 developed a self-regulated rhinosinusitis QoL questionnaire to assess the following components: slumber, problems of practicality, nasal relations, symptoms of non-hay fever, issues of the eye, emotions, and functional activities. Moreover, this instrument requires only 5 to 10 min to determine the effects and can be used repeatedly for longitudinal assessments. In addition, the Chronic Sinusitis Survey 23 is a duration-based 6-item sinusitis-specific monitor that gives results in 2 sections: medication-based and symptom-based. Lund et al. 24 developed a symptom score instrument involving 5 symptoms: pressure or pain in the face, headaches, nasal congestion, discharge, or blocking, and an olfactory disturbance analog scale. This is considered a brief, reliable, and sensitive tool; however, the patient has to determine his priorities by ranking the 3 most disturbing symptoms. The patient lists nasal- and sinus-related symptoms and specifies their functioning limitations to generate a rhinosinusitis disability index. 25 Conversely, the SNOT-22 is a high-quality questionnaire, but it is considered lengthy and can be replaced by a shortened version, as we developed in our study. SNOT-12 has been proven valid, reliable, and sensitive for many nose and sinus diseases.
SNOT-12 is a concise tool for the initial and follow-up assessments of all patients with rhinosinusitis disease. SNOT-12 was shown in this study to be more accurate, with a higher discrimination power, than SNOT-22. Nevertheless, SNOT-12 symptom items were classified in a prior manner through factor analysis of the original SNOT-22, which describes 4 latent factors of patient-perceived CRS. Also, SNOT-12 has a strong, direct, significant correlation with SNOT-22. Previous studies by Orlandi and Terrell and Bhattacharyya et al. have addressed the CRS diagnostic criteria and the American Rhinosinusitis Task Force. They found that nasal congestion and obstruction are the most frequently reported symptom (81%-95%), followed by facial pressure and fullness (70%-85%), discolored nasal leakage (51%-83%), and hyposmia (61%-69%).26,27 Nevertheless, SNOT-22 and SNOT-12 focus on and quantify a patient’s “overall” state and the reliability of the patient’s perspective to measure the trait of interest instead of considering the performance of each item of rhino-sinuses disease. SNOT-22 focuses on broadband and general aspects of CRS, while SNOT-12 is more accurate with higher discrimination power and increased strength of direct significant correlation.
This review of the SNOT-22 and SNOT-12 scores of many patients represents an exceptional comprehensive description of the mean scores and ranges across wide variations of CRS. The clinical consensus conference and the item analysis initially selected 10 SNOT items. However, we proceeded with a 12-item questionnaire based on clinical and statistical analyses that patients can fill up in a shorter time. This saves almost half of the time (48%).
The diagnostic ability of SNOT-12 to differentiate various disease states was mainly evaluated using ROC curves. Moreover, the study revealed a highly significant correlation between SNOT-12 and SNOT-22 (P = .0001). The measure of accuracy provided by the AUC, by displaying the capacity of this diagnostic test to determine differences between the presence and absence of any disease and its sensitivity and specificity values corresponded. 27 The AUC for the total SNOT-12 score was 0.625, while it was 0.593 for the SNOT-22. This indicates that SNOT-12 can differentiate CRS cases with high accuracy. When testing the correlations with the severity factors for the different CRS symptoms, the SNOT-12 had a direct, significant correlation with a higher strength than the SNOT-22.
This work had several limitations, including the difficulty of answering the SNOT-12 questionnaire in children younger than 10 years. The control group was recruited from the ENT clinics. These patients potentially had several medical problems with overlapping symptoms with CRS, including allergy, headache, imbalance, reflux, sleep problems, and facial pain. Thus, this was not a true “normal” cohort. However, they reflected the actual population in the ENT outpatient clinics who will utilize the SNOT-12 as a screening and evaluating tool for CRS.
This study developed and validated a shortened version of the SNOT-22, a disease-specific CRS questionnaire. It also integrated the most important complaints of this group of patients. Questions related to the frequency of reported symptoms and other clinically relevant questions were added to the patient population. Quantifiable outcomes could be provided by successful validation to strengthen the clinician’s evaluation of CRS symptoms concurrently with an assessment of QoL improvement. Moreover, more studies are needed to evaluate CRS management outcomes and follow-up scores to trace the disease progression over time, before and after treatment.
In conclusion, the SNOT-12 is a short, valid, and reliable instrument that may be useful for evaluating initial screenings and monitoring patients with rhinosinusitis.
Footnotes
Acknowledgements
Thanks to our colleagues at the Otolaryngology, Head, and Neck Surgery Department and Epidemiology Division of the Family Medicine Department at King Fahad University Hospital for their constructive contributions. We are grateful to our patients for their understanding and agreement to participate in the study.
List of Abbreviations
CMIN/DF (minimum discrepancy per degree of freedom), GFI (goodness of fit index), AGFI (adjusted goodness of fit index), CFI (comparative fit index), NFI (normed fit index), RFI (relative fit index), IFI (incremental fit index), TLI (Tucker Lewis index), PNFI (parsimony-adjusted measures index), SRMR (standardized root mean square residual), RSMEA (root mean square error of approximation), AUC (area under the curve), ROC (receiver operating characteristic curve), EFA (exploratory factor analysis)
Author Contributions
Salma Saud Al Sharhan: designed the study and wrote and revised the manuscript. Maha Ismail AlSomali: participated in data collection and writing and reviewing the manuscript. Mohammad Hussain Al-Bar and Abdulmalik Saad Alsaied: reviewed the literature and participated in writing the manuscript. Moataza Abdelwahab: participated in the study design, compiled, and analyzed the data, and participated in writing the manuscript. All authors read and approved the final manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethical Approval
Ethical approval was obtained from the institutional review board (IRB) of Imam Abdulrahamn bin FaisaL University, and informed consent was obtained from all patients prior to their enrollment (IRB-2020-01-047). This study was approved by the institutional review board of Imam Abdulrahman Bin Faisal University (IAU) (IRB No. 2019-01-362).
Availability of Data and Material
The data supporting this study’s findings are available from the corresponding author upon reasonable request.