Abstract
The Kessler Psychological Distress Scale (K-10) is widely utilized to assess non-specific psychological distress. However, when applied to COVID-19 pandemic experiences, its psychometric properties have not been examined. This study investigated the factor structure, reliability, and latent profile of K-10 using data from 3032 Filipino teachers who reflected on their pandemic-related adversity prior to completing K-10. Confirmatory factor analyses were conducted to evaluate five extant models of the K-10’s structure, supporting the two-factor model (i.e., anxiety and depression) as best fitting the data. Both factors showed excellent internal reliability, and criterion-related validity was supported by correlations with anxiety and depression measures. Latent profile analysis revealed a four-class solution, identifying four distinct levels of distress severity. Overall, the contextualized K-10 demonstrated sound psychometric properties, which corroborates and extends its reliability and validity as a measure of pandemic-related distress. Further, the results offer insights into the dimensionality and distress profiles of the K-10 among teachers. Broadly, the findings highlight the importance of valid instruments in assessing contextualized psychological distress.
Introduction
The rapid spread of the 2019 novel coronavirus has significantly impacted the Philippines, one of the hardest-hit countries, with approximately 3,920,693 confirmed cases and 62,549 deaths as of 2022 (World Health Organization, 2022). The shift to online learning during the pandemic and its accompanying challenges have exacerbated the distress experienced by teachers, who were already dealing with a myriad of stressors even before the pandemic (Jones & Kessler, 2020).
Teaching has often been classified as one of the most stressful professions (Johnson et al., 2005; MacIntyre et al., 2020). Numerous studies have identified teachers’ sources of stress that included heavy workloads, time constraints, limited autonomy, excessive administrative obligations, role conflict/ambiguity, emotional labor, dread over losing control of the class, fear of evaluation, and low professional self-esteem, among others (Boyle et al., 1995; Kokkinos, 2007; MacIntyre et al., 2020; Mercer & Gregersen, 2020). Thus, when COVID-19 interrupted school communities globally, teachers gained a lengthier list of new stressors to deal with (Jones & Kessler, 2020; Mendoza et al., 2023). These myriads of stressors included the fear of possible infection by COVID-19, social isolation, economic and financial loss, lack of security of tenure, insufficient preparation for online or remote teaching, inadequate knowledge and skills in digital technology instruction, and concerns about students’ access to online technology (Kim & Asbury, 2020; MacIntyre et al., 2020; Schleicher, 2020).
An increasing number of studies have shown that the accumulated stressors during the pandemic were likely to profoundly impact teachers’ mental health and well-being (Allen et al., 2020; Alves et al., 2020; Mendoza & Dizon, 2023; Rabacal et al., 2020). As greater levels of disturbance and disruption were intensified during the pandemic (Rabacal et al., 2020), the severity of psychological distress became apparent for teachers (Al-Hanawi et al., 2020; Burke et al., 2020; Esterwood & Saeed, 2020). Thus, the need to create adaptive and relevant interventions to lower teachers’ psychological distress is imperative. However, effective treatment requires contextualized, valid, and reliable assessment. Unfortunately, there were concerns about the assessment of psychological distress, particularly on the issue of context. The most widely utilized measure of distress, the Kessler Psychological Distress Scale (K-10), was used to evaluate psychological distress in a variety of settings and populations (Easton et al., 2017; Pereira et al., 2019; Slade et al., 2011), but not in the context of the COVID-19 pandemic. While K-10 has excellent reliability as well as factorial and construct validity (Easton et al., 2017; Li & Kou, 2018), to the author’s knowledge, none so far had examined its psychometric properties contextualized in an Asian sample who have experienced a variety of pandemic-related distress. Moreover, despite the overwhelming reports of psychological distress caused by the pandemic (e.g., Al-Hanawi et al., 2020; Burke et al., 2020; Droit-Volet et al., 2020; Mendoza et al., 2022; Van der Heide et al., 2020), few have explored K-10’s symptom structure, and so far, none had specifically studied this among teachers in an Asian setting.
Examining Symptom Structure Models of Psychological Distress
Item Mapping of K-10 Models.
Note. PD, psychological distress; Dep, depression; Anx, anxiety; Model 1, general factor model; Model 2, 2-factor model based on Sunderland et al. (2012); Model 3, 2-factor model based on Lace et al. (2019) & Pereira et al. (2019); Model 4, 2-factor model based on Bu et al. (2017); Model 5, 2-factor with correlated items 2 and 3, items 5 and 6, and items 7 and 9.
Model 2, a two-factor model, is based on Sunderland et al.’s (2012) proposed model, where the six items (i.e., tired out for no good reason, nervous, so nervous that nothing could calm you down, restless or fidgety, so restless that you could not sit still, and feel that everything was an effort) loaded on the anxiety factor, while four items (i.e., hopeless, depressed, so sad that nothing could cheer you up, and feel worthless) loaded on the depression factor. While anxiety and depression have been traditionally regarded as separate entities, comorbidity between anxiety and depression is a prevalent condition (Sigdel et al., 2020). Numerous studies have claimed that symptoms of anxiety and depression are indicators of psychological distress (Kranzler et al., 2016; Lace et al., 2019; Levin et al., 2014; Norton & Paulus, 2017). This model has been shown to be better than other two-factor models in the literature (Brooks et al., 2006; Sunderland et al., 2012).
Model 3 was based on the exploratory factor analysis (EFA) model of O’Connor et al. (2012). Unlike model 2, model 3 assigned the two items “tired out for no good reason” and “feel that everything was an effort” on the depression factor along with the previously identified depression symptoms in model 2 (i.e., hopeless, depressed, so sad that nothing could cheer you up, and feel worthless). The assignment of these two items from the anxiety factor to the depression factor contends that “fatigue” and “lack of energy” are both characteristics of depression rather than features of tension (Lace et al., 2019). Model 3 has been validated in a sample of the Brazilian population (Peixoto et al., 2021).
Bu et al. (2017) proposed model 4, a two-factor model, which shifted one item “tired out for no reason” from model 3’s depression factor to the anxiety factor. The anxiety factor now contains five items: tired out for no reason, nervous, so nervous that nothing could calm you down, restless or fidgety, and so restless you could not sit still. Brooks et al. (2006) and Sunderland et al. (2012) contended that the symptom “tired out for no reason” is a manifestation of anxiety rather than an indicator of depression. This model was shown to be valid in a sample of Chinese parents of children with cancer (Bu et al., 2017).
Model 5 is a two-factor model based on Sampasa-Kanyinga et al.’s (2018) study. In this model, the “lack of energy” symptom, which falls under the depression symptom group, has been transferred to the anxiety symptom group, along with symptoms such as “fatigue,” “nervous,” “so nervous that nothing could calm you down,” “restless or fidgety,” and “so restless you could not sit still.” In contrast to models 3 and 4, this model affirmed that the “lack of energy” is not a derivation of low mood but a repercussion of anxiety issues (Brooks et al., 2006; Sunderland et al., 2012). Model 5 is similar to model 2 in terms of item assignment to the two factors, except that the residuals of some items in model 5 were correlated (i.e., “nervous” and “so nervous that nothing could calm you down”; “restless or fidgety” and “so restless you could not sit still”; “depressed” and “so sad that nothing could cheer you up”).
The Current Study
Despite the extant literature supporting the validity of various K-10 models as a measure of psychological distress (Easton et al., 2017; Li & Kou, 2018; Pereira et al., 2019), none so far has investigated its psychometric properties in a Filipino sample, particularly in light of the COVID-19 pandemic. This research gap is noteworthy, given the widespread reports of psychological distress caused by the pandemic (Al-Hanawi et al., 2020; Burke et al., 2020; Droit-Volet et al., 2020; Van der Heide et al., 2020).
Thus, the current study intended to address this gap by (1) conducting confirmatory factor analyses (CFA) on the five competing K-10 models to identify which model best represents the sample, with an emphasis on assessing distress related to COVID-19 experiences; (2) establishing criterion-related validity by relating psychological distress, including distress specifically related to COVID-19, to generalized anxiety and major depression; and (3) performing latent profile analysis (LPA) to explore the number of latent classes among the teacher participants within the backdrop of the COVID-19 pandemic.
The study also examined the LPA of the teachers’ responses to K-10 to provide further evidence for construct-related validity. LPA is an approach that identifies latent groups accounting for the distribution of individuals across item indicators (Ferguson et al., 2020) or item symptoms in this study. LPA findings provide data on each individual’s probability of belonging to a group, demonstrating the model’s validity in classifying individuals (Cheng et al., 2022). The LPA results offer construct-related evidence by revealing how well K-10’s theoretical model classifies individuals into latent groups.
Prior studies have identified classes based on psychological distress severity. For instance, Fernández et al. (2022) found a 3-class solution among volunteers: low (17.9%), mild (40.90%), and moderate/severe (41.06%). Browning et al. (2021) identified three classes in students: low (14%), moderate (40%), and high (45%) distress. Interestingly, Gewirtz-Meydan and Lassri (2021) showed a 5-class solution driven by COVID-19 concerns. Given the variation across studies, this study aimed to investigate K-10’s latent classification among Filipino teachers, especially amidst the pandemic. With the teachers representing a distinct context compared to other populations, their distress experiences and sources may differ considerably. This LPA will provide evidence of construct validity tailored to this sample’s pandemic-related psychological distress.
Methods
Participants
The sample consisted of 3032 teachers from public and private schools in Mindanao, Philippines, wherein 36.61% (n = 1110) were secondary education teachers and 63.39% (n = 1922) were tertiary Filipino teachers. There were 24.8% (n = 751) males and 75.2% (n = 2281) females whose ages ranged from 20 to 78 years, with an average age of 36.08 years (SD = 10.807). The participants had been teaching for less than one (1) year to a maximum of 49 years (M = 7.71 years; SD = 8.284).
Procedure
Participants were invited to answer a brief web-based anonymous survey. They were given a link that redirected them to a Google Forms platform. All respondents were given an explanation of the purpose of the study and their right to withdraw at any time. Indicated in the informed consent is the clause on voluntary participation and anonymity of their personal information. The respondents confirmed their participation by ticking “yes” if they agreed to answer the questionnaire. This study was approved by an ethics review committee of the Mindanao State University-Iligan Institute of Technology, Philippines.
Measures
A short self-report demographic questionnaire was used to collect demographic data requiring information, including age, gender, level of education, and years of teaching experience.
The Pandemic-Related Adversities Checklist
The Pandemic-related Adversities Checklist is a 14-item scale developed by the authors to assess distressing events experienced during the pandemic. Items are rated on a dichotomous scale, with 1 (yes) or 0 (no). Sample items include disruption of daily routine, lack of personal space in the family, the unpredictability of the situation, and negative news from social media. It has high internal consistency (α = .88). See Supplementary Table 1 for the frequency of distressing events encountered.
The Kessler Psychological Distress Scale
The Kessler Psychological Distress Scale (K-10; Kessler et al., 2002) is a 10-item self-report instrument. The K-10 was used to assess the psychological distress symptoms of the respondents during the past 30 days during the COVID-19 pandemic. Participants were asked to respond on a 5-point Likert scale ranging from 0 (none of the time) to 4 (all of the time). Sample items include “During the last 30 days, how often did you feel tired out for no good reason?” and “During the last 30 days, about how often did you feel nervous?” Higher scores on K-10 demonstrate higher psychological distress in the individual. Various studies have reported good psychometric properties of the K-10 (Fassaert et al., 2009; Pereira et al., 2019; Sampasa-Kanyinga et al., 2018). In this study, the coefficient omega for the best-fitting model’s factors—anxiety and depression were 0.90 and 0.92, respectively.
The Patient Health Questionnaire
The Patient Health Questionnaire (PHQ-9; Kroenke et al., 2001) is a 9-item self-report instrument used to assess depression symptoms over the last two weeks during the COVID-19 pandemic. Items were answered on a 4-point scale ranging from 0 (not at all) to 3 (nearly every day). Sample items include “Little interest or pleasure in doing things,” “Feeling down, depressed, or hopeless,” and “Trouble falling or staying asleep, or sleeping too much.” Research has found that the PHQ-9 has good psychometric properties that can be used for screening major depressive disorder (MDD; Kroenke et al., 2001; Sun et al., 2020). In the current study, Cronbach’s alpha was 0.92.
The Generalized Anxiety Disorder
The Generalized Anxiety Disorder-7 scale is (GAD-7; Spitzer et al., 2006) a 7-item self-report questionnaire used to assess how often a person experiences generalized anxiety disorder (GAD) symptoms over the last four weeks and during the COVID-19 pandemic. Items are rated on a 4-point scale, ranging from 0 (not at all) to 3 (nearly every day). Sample items include “Feeling nervous, anxious, or on edge,” “Not being able to stop or control worrying,” and “Worrying too much about different things.” Previous studies have shown the GAD-7 has good psychometric properties (e.g., Löwe et al., 2008; Zhong et al., 2015). In this study, Cronbach’s alpha was 0.94.
Data Analysis
All analyses were performed using Mplus version 7.11 (Muthén & Muthén, 2013). The analyses conducted in this study were two-fold. First, CFAs were conducted on the five a priori models of psychological distress (Table 1). Maximum-likelihood with robust standard errors (MLR) was used as the estimation method. Conventional guidelines were followed in evaluating the fit indices. Chi-square (χ2), comparative fit index (CFI), Tucker–Lewis index (TLI), and root mean square error of approximation (RMSEA) were used to assess the goodness of fit of the models. Models with CFI and TLI values of ≥0.90 were considered an adequate fit, while values of ≥0.95 were considered good. RMSEA values of ≤0.08 were interpreted as adequate fit, while values of ≤0.06 were considered excellent (Hu & Bentler, 1998). A scaled χ2 difference test was used to compare nested models following Satorra and Bentler’s (2001) algorithm. For non-nested models, comparisons were based on the differences in the model’s Bayesian information criterion (BIC). A difference of 10 BIC points indicates a 150:1 likelihood and very strong support that the model with the smaller BIC fits better, and a difference of 6–9 points signifies strong support (Raftery, 1995). Second, in establishing support for criterion-related validity, the best-fitted K-10 model was correlated with measures of scores of generalized anxiety and major depression.
The subsequent part analyzed the number of fundamental classes by LPA in view of the best-fitting model. LPA was used to categorize the respondents into classes. To determine the number of latent classes, indices of two- to five-class solutions were assessed, including the Akaike information criterion (AIC), BIC, sample-size adjusted BIC (SABIC), and the Lo-Mendell-Rubin adjusted likelihood ratio test (LMR-A-LRT). Latent class models with lower AIC, BIC, and SABIC demonstrate a good-fitting model. The LMR-A-LRT was utilized to compare the fit of two class models with a significant value, indicating that the target class solution fits better than the model with one fewer class. Entropy was additionally evaluated, which estimates how precisely people are gathered into their individual profiles, with values closer to one indicating a strong distinction of clusters.
Results
Descriptive Statistics
The mean score for K-10 was 11.61 (SD = 8.46). Using the cut-off score of ≥10 (Sampasa-Kanyinga et al., 2018), 52.27% (n = 1585) of teachers were likely to have psychological distress symptoms. Supplementary Table 2 presents the descriptive statistics per item (i.e., mean, SD, skewness, and kurtosis).
Construct-Related Validity
Results of Confirmatory Factor Analysis.
Note. χ2, chi-square statistic; df, degrees of freedom; CFI, comparative fit index; TLI, Tucker–Lewis index; RMSEA, root mean square error of approximation; CI, confidence interval; BIC, Bayesian information criterion; Best-fitting model is indicated in boldface. **p ≤ .01.
Standardized Factor Loadings of the Best-Fitted Model.
Note. All factor loadings are standardized and significant at p ≤ .001. N = 3032.
aCorrelated residuals of items 2 and 3; p ≤ .001.
bCorrelated residuals of items 5 and 6; p ≤ .001.
cCorrelated residuals of items 7 and 9; p ≤ .001.
A more detailed analysis of the average variance extracted (AVE) yielded a range from .597 to .682 for the anxiety factor and a range from .636 to .737 for the depression factor in the four 2-factor models (see Supplementary Table 4). The results showed that the items across different models are fairly higher than .50. These findings imply that the average variances of items can be accounted for well by each latent factor of the model more than the error in measurement items (Hair et al., 2022). The McDonald’s Omega coefficients ranged from .896 to .944 for the five models being investigated (see Supplementary Table 4). Overall, the five models examined show high reliability and the items, considering their intercorrelation, have high internal consistency.
Criterion-Related Validity
Correlation of K-10 Factors and General Score With MDD and GAD Measures.
Note. PD, psychological distress; **p ≤ .01; N = 3032.
Construct-Related Evidence of Validity: The LPA of Psychological Distress
Latent Profile Analysis Results of 10-Item Psychological Distress.
Note. AIC, Akaike information criterion; BIC, Bayesian information criterion; SABIC, sample-size adjusted BIC; LMR-A-LRT, Lo-Mendell-Rubin adjusted likelihood ratio test. Bold text corresponds to the model with the best fit. N = 3032.
Based on the properties of the classes, the four groups were labeled. These include the low psychological distress group (n = 1248, 41.16%), moderate psychological distress group (n = 1020, 33.64%), high psychological distress group (n = 550, 18.14%), and very high psychological distress group (n = 214, 7.06%). Participants in the low psychological distress group had an age range of 20–75 years old (M = 38.38; SD = 11.01) with scores ranging from 0 to 9 (M = 4.02; SD = 2.41). The moderate psychological distress group was between the ages of 21–78 years old (M = 35.42; SD = 10.42) with scores ranging from 7 to 19 (M = 11.95; SD = 2.55). The participants in the high psychological distress group range in age from 21 to 70 years old (M = 33.10; SD = 10.9) with scores ranging from 15 to 28 (M = 20.79; SD = 2.65). Finally, the very high psychological distress groups’ ages were 21–68 years old (M = 33.44; SD = 10.37) with scores ranging from 25 to 40 (M = 30.69; SD = 3.85). The four-class K-10 latent profile analysis model is illustrated in Figure 1. The lines in the graph depict the considerable variation in the level of psychological distress experienced by the four classes of teachers, provided that their responses were indeed influenced by their reflections on the COVID-19 pandemic. Four-class K-10 latent profile analysis model.
Discussion
The primary objective of this study was to critically evaluate the effectiveness of the repurposed K-10 scale in measuring psychological distress among Filipino teachers during the COVID-19 pandemic. This involved an assessment of the K-10’s factor structure through the examination of five competing models, as well as a thorough investigation into its criterion-related validity. Additionally, the study aimed to uncover distinct patterns of distress among the participants. These efforts provide valuable insights for practitioners, laying a foundation for the use of the K-10 scale as a reliable tool in identifying and addressing the multifaceted aspects of psychological distress in educational settings amidst challenging times.
Our results supported the two-factor model of the K-10, aligning with previous research (Brooks et al., 2006; Bu et al., 2017; Pereira et al., 2019; Sampasa-Kanyinga et al., 2018; Sunderland et al., 2012), and highlighting the scale’s effectiveness in assessing psychological distress. This validation, encompassing anxiety and depression, provides nuanced insights into the psychological distress experienced by educators. Notably, the model categorizes symptoms like fatigue and lack of energy as indicators of anxiety rather than depression (Brooks et al., 2006; Sunderland et al., 2012), aligning with research showing that persistent anxiety can lead to a loss of energy (Carmeli, 2013). The physiological mechanism behind this includes the constant release of adrenaline in states of high anxiety, leading to adrenal gland malfunction and subsequent fatigue (Lee, 2009; Wilson, 2014). Chronic stress can exhaust the adrenal glands, resulting in insufficient cortisol secretion and subsequent exhaustion (Carmeli, 2013). This biological insight supports the model’s categorization of “fatigue” and “lack of energy” as anxiety symptoms. The distinct identification of anxiety symptoms, such as fatigue and restlessness, and depressive symptoms, like hopelessness and sadness (Leung et al., 2022), enables practitioners to deploy more targeted interventions. By differentiating between anxiety and depression domains, mental health professionals can use screening tools based on this model (Brooks et al., 2006; Bu et al., 2017; Pereira et al., 2019; Sampasa-Kanyinga et al., 2018; Sunderland et al., 2012) to identify individuals needing specialized support. For instance, this distinction can guide the development of interventions focusing on stress management for anxiety, as opposed to exclusively depression-focused therapy (see Leung et al., 2022).
The criterion-related validity analysis further reinforced the utility of the K-10 scale in screening for anxiety and depression. Specifically, the scale’s anxiety and depression factors, as well as its general scores, showed significant correlations with standardized measures of MDD and GAD. These results align with previous research demonstrating the close relationship between psychological distress and prevalent mental disorders such as GAD and MDD (Donker et al., 2010; Spies et al., 2009; Valencia-Garcia et al., 2017; Vasiliadis et al., 2015). When psychological distress manifests as being emotionally upset, anxiety and depression emerge as key indicators (Lowe et al., 1999; Montgomery & McCrone, 2010; Schnur et al., 2008).
Building on this, the LPA further validated the K-10’s efficacy in assessing educators’ distress levels during the pandemic. The data delineated four distinct distress classes—low, moderate, high, and very high—reflecting the varying severity of symptoms related to the pandemic. While most teachers reported low levels of distress (n = 1248, 41.16%), a notable proportion of the sample experienced moderate to very high levels of distress. This four-class structure, which is in line with previous studies on distress typologies (Bryan et al., 2018; Fernández et al., 2022), underscores the diverse psychological impacts of the pandemic on teachers. The distinct and non-overlapping nature of these classes provides construct-related evidence supporting the K-10 scale as an adaptable measure for evaluating distress specific to the COVID-19 context among Filipino teachers.
Limitations and Research Implications
The findings of this study must be interpreted with due caution considering the following limitations. Being a cross-sectional design, it only provides a snapshot of distress and cannot determine changes over time. The use of self-report measures and retrospective recall can impact accuracy. The sample of Filipino teachers may limit generalizability to other populations with different pandemic experiences. Potential response biases exist due to voluntary participation and reliance on self-report without clinical validation. Factors like age, gender, location, and pre-existing conditions/life stressors were not considered. While latent profiles were identified, the analysis cannot determine causal relationships or rule out other potential solutions. Longitudinal research clinically validating profiles in larger, more diverse samples is needed. While this study provides preliminary validity evidence for the K-10 during the COVID-19 pandemic, more research is warranted.
Despite these limitations, this study has several contributions to the existing body of literature. To the authors’ knowledge, this is the first study to establish the validity of the K-10 scale as a measure of psychological distress among Filipino teachers in the context of the COVID-19 pandemic. This is relevant in view of the rising prevalence of psychological distress (Stapleton et al., 2020), especially among Filipino teachers (Mendoza et al., 2023; Mendoza & Dizon, 2023). Second, the findings add to the limited studies pertaining to the symptom structure of K-10 among Asians, particularly Filipinos. Third, the results extend the findings of other studies by correlating K-10’s anxiety factor, depression factor, and its general scores to GAD-7 and MDD measures, providing the model’s criterion validity. Finally, the four distinct profiles of teachers experiencing pandemic-related psychological distress highlight the implications for designing intervention programs appropriate to the level of distress the teachers are manifesting, specifically targeting both depression and anxiety symptoms as the two factors of distress.
Conclusion
This study examined the factor structure and latent profile of COVID-19 pandemic-anchored K-10 psychological distress scale. The results showed that the K-10 is best represented by anxiety and depression as its two factors and that it has a 4-class solution consisting of low, moderate, high, and very high symptom classes. These findings provide evidence for the validity of K-10 as a measure of pandemic-related psychological distress. This study highlights the importance of context in assessing psychological distress and the need for measures tailored to the specific challenges posed by specific major events (e.g., the COVID-19 pandemic). The results of this study can inform future research and intervention efforts aimed at addressing the challenges on the psychological well-being of Filipino teachers before, during, and even after the pandemic.
Supplemental Material
Supplemental Material - Repurposing the K-10: The Factor Structure and Latent Profile of COVID-19 Pandemic-Anchored Psychological Distress Symptoms Among Filipino Teachers
Supplemental Material for Repurposing the K-10: The Factor Structure and Latent Profile of COVID-19 Pandemic-Anchored Psychological Distress Symptoms Among Filipino Teachers by Imelu G. Mordeno, Queeneh T. Piape, Art Guilleane R. Roño, Wendel Ann C. Abejar, Michelle Anne L. Ferolino, and Norman B. Mendoza in Journal of Psychoeducational Assessment
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
