Abstract
The Brief Adjustment Scale–6 (BASE-6) was recently developed for measuring general psychological functioning within measurement-based care (MBC). The present study further evaluated psychometric properties, generalizability to race/ethnic populations, and clinical utility of the BASE-6. Three adult samples, Sample 1: online community participants (n = 394); Sample 2: college students (n = 249); Sample 3: outpatient clinic clients (n = 80), were included. The results demonstrated a high level of internal consistency, good test–retest reliability, and convergent validity in all samples. The unidimensional structure of BASE-6 was confirmed and factorial invariance was established across groups. Finally, the BASE-6 captured change over time by demonstrating a large effect size of pre–post treatment changes and significant linear change in multilevel growth modeling. These results support the BASE-6 as a reliable and valid measure regardless of race/ethnicity and can sensitively detect clinical change over the course of the treatment. Thus, the BASE-6 appears to accurately monitor overall psychological adjustment.
Keywords
Measurement-based care (MBC) is the systematic and collaborative evaluation of patient functioning data to inform behavioral health treatment. Core elements of patient functioning data typically measured within MBC are clinical symptoms and general psychological functioning (Lewis et al., 2019). Various measures for clinical symptomatology that have been developed and used for monitoring purposes and to track treatment effectiveness are reported to have sound psychometric properties such as the Patient Health Questionnaire 9 (PHQ-9; Kroenke et al., 2001) and Generalized Anxiety Disorder 7 (GAD-7; Spitzer et al., 2006). On the contrary, while there is a growing need for outcome measures of general psychological functioning and adjustment as a tool to capture common symptoms (Kraus et al., 2005), there is a lack of outcome measurements for general psychological functioning that are brief, valid, and free to use.
To help solve this problem, the self-report, six-item Brief Adjustment Scale–6 (BASE-6) was developed to meet the demand for a brief, feasible, comprehensive, and reliable measure of general psychological adjustment (Cruz et al., 2020). Although there are few previous studies that have evaluated the psychometric properties and appropriateness of the BASE-6 as a tool that can be utilized in the MBC context, further evaluations are still needed. Given the limitations in the previous studies and increased use of the BASE-6 in clinical and research settings, the primary objectives of the current study are to further investigate the psychometric properties, measurement invariance, and clinical utilities in the context of MBC.
Measurement-Based Care
MBC has demonstrated the ability to assess psychological symptoms, functioning and satisfaction with life, and clinical changes (Lambert et al., 2018). Specifically, research has shown that through the process of MBC, the accuracy of clinical judgment is approved and clinicians can objectively assess the client’s progress and avoid overestimation of their client’s improvement (Garland et al., 2003; Harding et al., 2011; Sapyta et al., 2005). In addition, it helps with early identification of failure, sudden gains, or changes that can, in turn, predict intervention outcomes and inform improvement efforts (Deschênes & Dugas, 2013; Fortney et al., 2017; Lambert et al., 2005; Shimokawa et al., 2010). Another advantage of MBC for mental and behavioral health services includes increased client engagement in the therapy process (Dowrick et al., 2009; Goldberg et al., 2020). Overall, standardized regular assessment and feedback to both clinicians and patients are related to improved outcomes and decreased dropouts (Carlier et al., 2012; Lambert et al., 2003; Levine et al., 2017).
Although there is a strong and growing evidence base for MBC, it is under-utilized (Hatfield & Ogles, 2004; Kazdin, 2008), with less than 20% of behavioral health practitioners engaging in MBC (Boswell et al., 2015; Zimmerman et al., 2008). The most cited primary barrier to MBC utilization is the time burden for both the clinician and patient (Aboraya, 2009; Hatfield & Ogles, 2007; Nasrallah, 2009). Clinicians already devote a significant amount of time to clinical and nonclinical tasks, and requiring clinicians to administer, score, review, and further discuss outcome measures’ results with their client increases their workload (Gleacher et al., 2016). In addition, clients may find it burdensome to fill out long questionnaires, and they may not be willing to complete them unless they are well informed of the confidentiality of data collection or further procedure regarding the results (Boswell et al., 2015; Lewis et al., 2019). In addition, in low-resource clinics, inadequate access to more expensive assessments may hinder MBC (Peterson & Fagan, 2017). Thus, for successful MBC implementation across mental health settings, there is a clear need for reliable, valid, generalizable, free, and brief measures of psychological functioning and clinical change.
Psychological Measures in the Context of MBC
The previously mentioned questionnaires such as the PHQ-9 and GAD-7 are designed to measure the symptoms of clients within the previous 2 weeks and have been proven to be useful for routinely monitoring treatment effects (Löwe et al., 2004; McMillan et al., 2010; Toussaint et al., 2020). Additional need for a measure of general functioning that can be commonly used for various clients is growing, as a measure that broadly assesses an individual’s functioning and distress in their daily life can be used for a wide range of clients. This can allow researchers and clinicians to have a common metric of treatment results, without being heavily affected by the client’s specific problems or diagnosis. Moreover, recent studies support the presence of a general psychopathology factor within various psychiatric disorders rather than viewing mental disorders as distinct and categorical conditions (Caspi et al., 2014). Such a measure would be of added value to MBC, especially considering that MBC has been proven to be effective for multiple disorders including depressive disorder (Guo et al., 2015; Kendrick et al., 2009; Lewis et al., 2015; Newnham et al., 2010), anxiety (Boer et al., 2019; Gual-Montolio et al., 2020), bipolar disorder (Cerimele et al., 2019), substance use (Simon et al., 2020), and schizophrenia (Dinakaran et al., 2020).
Thus far, the well-studied and validated Outcome Questionnaire (OQ-45.2; Lambert, 2015) has been mostly used in psychotherapy settings to assess symptom distress, interpersonal problems, and social role functioning, and to better serve the purpose of MBC (Overington & Ionita, 2012). However, practitioners report that the OQ-45.2 is long and complicated as it consists of 45 items, which hinders the completion and compliance from both clinicians and clients. As such, the OQ-45.2 does not serve the objective of low-burden regular administration (Miller et al., 2003). There are other possible measures to assess clients’ progress in terms of general adjustment, subjective evaluation of overall distress, or functioning such as the Outcome Rating Scale (ORS; Miller et al., 2003), but these measures only demonstrated moderate convergent validity with the OQ-45.2 (e.g., r = .59 between ORS total score and OQ-45.2 total score). In addition, the items of ORS evaluate well-being in four domains (i.e., individual, interpersonal, social, and overall), but do not provide information about common psychological symptoms including depressed mood or anxiety levels.
Due to the limitations of those measures, the primary aim of the development of the BASE-6 for MBC was to provide a brief measure that can be used to assess the overall distress and functioning of the adult-aged individual and offer information on an individual’s progress in psychological intervention via routine administration. There is only one study that shows acceptable psychometric properties and promise for MBC (Cruz et al., 2020), and internationally, the Turkish version of the BASE-6 was supported for its reliability, validity, factor structure, and measurement invariance across genders (Yildirim & Solmaz, 2021). In Cruz et al. (2020), the researchers evaluated the BASE-6 using three adult samples, including online participants, college students, and a clinical sample, by assessing reliability and validity through various analyses. The results showed that the BASE-6 has good internal consistency and test–retest reliability, and moderate to high convergent validity with the OQ-45.2 within a nonclinical sample and high convergent validity with the PHQ-9 and GAD-7 within a clinical sample. In addition, the nonclinical samples indicated that the weekly use of the BASE-6 is easier and more convenient than the OQ-45.2. Despite these promising results, there are some limitations in their initial study. First, the sample used by Cruz et al. (2020) was not representative, as the majority of the participants are White European American, which limits the generalizability of the findings, especially to individuals with diverse race/ethnic backgrounds. Second, Cruz et al. (2020) used the OQ-45.2 for nonclinical samples and the PHQ-9 and GAD-7 for a clinical sample to assess convergent validity. Even though they showed a large positive correlation between the BASE-6 and other measures, it would strengthen the degree of convergent validity by having another unified measure for both clinical and nonclinical samples. Finally, there is no research yet that has evaluated whether the BASE-6 can be used to capture the treatment-induced change in a clinical population over the course of the therapy.
The Present Study
Given the limitations in the previous study and increased use of the BASE-6 in clinical and research settings, further evaluations are needed for the psychometric properties, generalizability, and clinical utility of the BASE-6. Therefore, we set forth three main objectives in this study.
First, we examined the psychometric properties of the BASE-6 using a wide range of analyses to increase the evidence of reliability and validity with two nonclinical samples and one clinical sample. Regarding the reliability of the BASE-6, we computed the corrected item-total correlations, Cronbach’s α, McDonald’s ω, and test–retest reliability. Regarding the validity, the Depression Anxiety Stress Scale–21 (DASS-21; Lovibond & Lovibond, 1995) was used to assess the convergent validity as the previous study confirmed the BASE-6’s convergent validity with the PHQ-9, GAD-7, and OQ-45.2. It is known that the DASS-21 is a widely used measure that captures the most common mental health problems in clinical settings, and previous research showed that the three subscales of the DASS-21 (i.e., depression, anxiety, stress) ultimately constitute the factor “general psychological distress” (Henry & Crawford, 2005), which aligns with the BASE-6’s main evaluation objective. In addition, the DASS-21 was also validated as a tool that can be used as a routine outcome measure in clinical settings, as supported by the significant correlations with CGI (Clinical Global Impressions; Busner & Targum, 2007) and MHQ-14 (Mental Health Questionnaire–14; Ng et al., 2007; Ware & Sherbourne, 1992). Thus, these previous studies indicate the DASS-21 as an appropriate measure that can be used to evaluate the validity of the BASE-6. In addition, confirmatory factor analysis (CFA) was conducted to confirm the factor structure of the BASE-6 among clinical and nonclinical populations with the expectation to replicate the results from the previous study.
The second objective of the current study was to validate the BASE-6 across diverse race/ethnic groups and confirm its generalizability. Various research has emphasized the importance of a measurement invariance that yields comparable information regardless of race or ethnicity, which could be affected by the respondents’ attitudes, values, or language (Drasgow & Kanfer, 1985; Leong & Gupta, 2008; Okazaki & Sue, 2016). It is meaningful that these race/ethnic groups were equally included in this study, considering the underrepresentation of those individuals as research participants other than the White/non-Hispanic group, especially in the mental health field including assessment and measurement (Alegria et al., 2002; Snowden, 2001; Sue et al., 2012). The statistical methods used for the first objective were applied to the data of each race/ethnic group. In addition, we evaluated the measurement invariance across four different race/ethnic groups, including White/non-Hispanic, Black/African American, Asian/Asian American, and Latino/Hispanic. Testing measurement invariance (Marsh, 1994; Meredith, 1993) allows us to confirm whether the measures could be interpreted and perceived in the same way for various populations, as respondents’ attitudes, values, or language could affect the perception of measurements (van de Vijver & Leung, 1997). Considering the importance of addressing the measurement issues when evaluating the health status of each racial/ethnic group and conducting further cross-cultural research (Ramírez et al., 2005), it is critical to establish the generalizability of the BASE-6 to individuals with diverse ethnic backgrounds.
Third, we investigated the clinical utility of the BASE-6 by (a) examining the difference in the BASE-6 scores across clinical and nonclinical samples and (b) evaluating whether it demonstrates sensitivity and responsiveness to change over the course of the psychological intervention within the clinical sample. We investigated their pre- and post-treatment scores and its effect size as well as the participants’ overall treatment trajectories via longitudinal data to confirm whether the BASE-6 has acceptable sensitivity for detecting changes from session to session. We hypothesized that the BASE-6 scores would significantly differ between each session, indicating a change in the participants’ overall psychological adjustment during their psychological intervention that applied MBC.
In short, we seek to further confirm whether the BASE-6 is a useful tool for MBC in psychological and psychiatric settings by evaluating the psychometric properties of the BASE-6 with an additional measure of psychological functioning, across wider and more diverse samples, and by examining its sensitivity to the treatment-induced change.
Method
Participants and Procedure
For the present study, three samples of participants were recruited: two nonclinical samples (i.e., online community sample and online college student sample) and one clinical sample. The university’s institutional review board (IRB) approved data collection. The demographic information of each sample is presented in Table 1.
Demographic Characteristics.
Sample 1: Online Community Sample
Participants from the online community sample were recruited through the online survey platform Qualtrics (Qualtrics, Provo, UT). Participants were included if they were 18 years or older and if their ethnicity was equivalent to one of these categories: White/non-Hispanic, Black/African American, Asian/Asian American, or Latino/Hispanic. Participants were asked if they were undergoing psychotherapy or under the care of a psychiatrist (e.g., “Are you currently in psychotherapy or under the care of a psychiatrist?”) and if their answer was positive, they were automatically excluded from participation to avoid any bias in responses because of therapy experiences. After reviewing the IRB–approved consent form, participants completed surveys, the BASE-6 and DASS-21, one time. The final participants were 394 in total, aged 18 to 92 years (M = 43.76, SD = 18.25), and their race/ethnicity was equally distributed to four race/ethnic groups: White/non-Hispanic (24.9%), Black/African American (24.9%), Asian/Asian American (24.9%), and Latino/Hispanic (25.3%). Four ethnic groups were strategically recruited in similar numbers to achieve equal distributions across race/ethnic groups. More specific demographics of each race/ethnic group are presented in Table 1.
Sample 2: College Student Sample
A group of undergraduate students were recruited from a large public southeastern university. The university’s psychology subject pool was used to recruit college students, and individuals received course credit upon their completion of participation in the online questionnaire study. Participants were included if they were 18 years or older. The participants were provided a link and reviewed the IRB-approved consent form. Participants agreed to complete two self-report questionnaires, the BASE-6 and DASS-21, two times in total with a 1-week interval (M = 7.43, SD = 1.36, range = 4–10 days) between the first and second surveys. In this study, we chose a 1-week interval for test–retest reliability for several reasons as follows: (a) The current study followed the same interval as Cruz et al.’s (2020) study to allow comparison of the results, (b) similar outcome monitoring measures that are regularly used to psychotherapy clients with a purpose to routinely monitor the symptoms such as GAD-7 (Spitzer et al., 2006) also followed a 1-week interval, and (c) considering that outcome monitoring measures are recommended to be regularly used (i.e., weekly) in the MBC context, the participants of the current study were guided to complete the second survey after 1 week. Once the participants finished their first survey, they were contacted via email and received notifications to complete the same questionnaires one more time. In total, 306 participants were recruited and 18 participants were removed because of non-completion of the surveys. As an exclusion criterion, participants who were undergoing psychotherapy or medical therapy because of psychopathology (n = 42) or were under 18 years of age (n = 0) were excluded from the study. As a result, 249 participants were included in the final sample. The mean age of participants was 19.60 years (SD = 1.30; range = 18–23), majority of them were White/non-Hispanic (67.9%) and female (74.3%). Among the 249 participants, 151 students (60.6%) completed the retest within an appropriate period (i.e., about 1 week of an interval), and these students were used for test–retest analysis.
Sample 3: Outpatient Clinic Sample
Clinical participants’ data were collected from the outpatient clients receiving services at the community training clinic, where MBC was a routine and expected part of clinical care. A naturalistic observational approach was applied to the clinic sample by utilizing the retrospective data that have already been collected between the years 2015 and 2021 through their usual care. Participants were included only if they consented to the research project, called Standardized Clinical Outcome Research & Evaluation (SCORE; Cooper et al., 2021). Clients were asked to complete their outcome measures, including the BASE-6, using the Health Insurance Portability and Accountability Act of 1996 (HIPPA)–compliant measurement-feedback system (MFS) platform. Then the MFS scored and graphed the clients’ data, visually displaying it so that the client and clinician could review the results collaboratively. All therapists were trained to follow a standardized MBC protocol of the clinic, which included a standard battery of measurements at intake and discharge; selection, administration, and collaborative review of outcome measures; and how to tailor treatment measures to the client’s problems and needs. In this outpatient training clinic, the graduate student clinician provided evidence-based interventions to clients under the supervision of a licensed clinical faculty member and an advanced peer supervisor.
The client’s data were collected if the client was at least 18 years old at the time of consent and if the client completed the BASE-6 during the intake process. As this study was primarily focused on the BASE-6, data were excluded if there were any missing values. All participants in Sample 3 were clients who voluntarily participated after providing written or online informed consent forms. In total, 80 participants were included for the analyses. Participants’ ages ranged from 18 to 59 (M = 28.16, SD = 10.85) years and half of the sample was female (N = 40). The majority of participants identified as White/non-Hispanic (75.0%), followed by Black/African American (3.8%), Asian/Asian American (3.8%), Latino/Hispanic (1.3%), mixed ethnicity (1.3%), Other (11.3%), and not answered (3.8%). The data were then further filtered for specific analyses. For example, to confirm the convergent validity, participants were selected from the database if they had completed both the BASE-6 and DASS-21. For the multilevel growth modeling analysis, participants were selected from the database if they had completed the BASE-6 at least 16 times over the course of the treatment. This study chose the first 16 administrations for the analysis to reflect the typical duration of cognitive-behavioral therapy and to exclude clients who attended therapy much longer/shorter than the typical clients.
Measures
BASE-6
The BASE-6 is a self-report questionnaire evaluating general psychological adjustment. The measure comprises six items and was developed for both clinical and research purposes. Each item is scored on a 7-point Likert-type scale ranging from 1 to 7 (1 = not at all, 4 = somewhat, 7 = extremely), and the maximum total score is 42 points, obtained by summing the raw scores of the six items. The higher the score, the lower the general psychological adjustment. The respondents choose one of the seven statements that best define how they felt during that week. Specifically, among the six items, the first three items assess individuals’ perception of emotional distress (i.e., anger, anxiety, and depression), while the last three items assess related interference (i.e., self-esteem, interpersonal relationships, and occupational/academic performance). It is freely available to the public under a Creative Commons license, and it has been used as an MBC measure in various clinics to regularly track the changes of their client’s symptoms. It takes approximately 1 minute to complete the BASE-6 (see Supplemental Appendix).
DASS-21
The DASS-21 (Lovibond & Lovibond, 1995) is a questionnaire comprising 21 items assessing depression, anxiety, and stress symptoms. It is a short-form of the DASS-42, which is an original version with 42 items. Each item is scored on a 4-point Likert-type scale ranging from 0 to 3 (0 = not at all, 3 = nearly every day). Respondents select one of four statements that best applied to themselves over the past week. The scores for the Depression, Anxiety, and Stress subscales are calculated by summing the raw scores of the seven items within each subscale. The total score of the DASS-21 ranges from 0 to 126 as the sum of Items 1 to 21 is multiplied by 2. Specifically, the DASS-21 includes 21 items, and the score of each item ranges between 0 and 3. The total score can range from 0 to 126 as the sum of the scores of the 21 items is multiplied by 2, which is a total score minimum of 0 (i.e., score of 0 for all 21 items multiplied by 2 is 0) and a maximum of 126 (i.e., score of 3 for all 21 items multiplied by 2 is 126). Higher scores suggest more severe symptoms of depression, anxiety, and stress. A substantial body of literature indicates that the DASS-21 is reliable and valid in measuring depression, anxiety, and tension/stress in both clinical and nonclinical populations. The factor structure is stable, and its subscales have a good convergent and discriminant validity, and high internal consistency in adults and adolescents across different ethnic groups (Antony et al., 1998).
Data Analytic Plan
To assess the reliability of the BASE-6, we computed the corrected item-total correlations, Cronbach’s α (Bland & Altman, 1997), and McDonald’s ω (Beland et al., 2017). In addition, we computed the intraclass correlation coefficient (ICC) to evaluate the test–retest reliability. Correlation coefficients between the BASE-6 and DASS-21 total and subscale scores were used to evaluate the convergent validity of the BASE-6.
To investigate the factor structure of the BASE-6, all three clinical/nonclinical samples and four race/ethnic groups were analyzed with CFA. Based on the result of the previous study that supported the unidimensionality of the BASE-6 (Cruz et al., 2020), we fit the unidimensional model to each of the three clinical/nonclinical samples and each of the four race/ethnic groups. For parameter estimation, we used maximum-likelihood with Satorra–Bentler adjustment (referred to as MLSB or MLM hereafter; Satorra & Bentler, 1994) to estimate all models, as MLSB (or MLM) is a robust method that can accommodate data from non-normal distributions. All the fit indices were scaled accordingly (e.g., scaled χ2). To identify the model, the factor variance was fixed to one.
To examine measurement invariance across race/ethnicity groups, a series of multigroup models were fitted. Specifically, we fit the multigroup models at different levels of invariance, including configural, weak, strong, and strict invariance, as appropriate (Marsh, 1994; Meredith, 1993). At the level of weak invariance, we tested the invariance of factor loadings; at the level of strong invariance, we tested the invariance of factor variances; and finally, at the level of strict invariance, we tested the invariance of error variances.
To examine if the total and item scores of the BASE-6 are different between groups, analysis of variance (ANOVA) was conducted. In case of significant results from ANOVA, multiple comparisons with Scheffe’s test were employed as Scheffe’s test does not require the sample sizes to be equal across groups (Cohen, 2008).
Sensitivity to overall clinical change during psychotherapy was assessed by conducting a paired-sample t-test using the pre- and post-BASE-6 total scores. In addition, multilevel growth models (Raudenbush & Bryk, 2002) were fitted to evaluate the trajectories of the BASE-6 scores over time. In the multilevel growth model framework, all the BASE-6 scores were regarded as the Level 1 units, and the individuals were regarded as the Level 2 units, as the BASE-6 scores were nested within individuals. In particular, we compared two competing models. In Model A, we only modeled the linear trajectory, whereas in Model B, we also included the quadratic trajectory. A new variable, called TIME, was created as a Level 1 predictor in both Models A and B to capture the within-participant change in the BASE-6 total score. The TIME variable was coded from 0 (initial survey) to 15 (measured at the 16th session) to reflect the duration of total 16 sessions. For Model B, the squared TIME variable was created to capture the quadratic rate of change. At Level 2, the random effects of the participant’s initial status, the TIME variable, and the squared TIME variable were modeled to represent individual differences. For parameter estimation, full maximum-likelihood estimation method was used.
There were no missing values in Samples 1 and 2, because the “force response” function was used as a response requirement, and the participants were reminded if they missed any items. In Sample 3, any participants with missing values on either the BASE-6 or DASS-21 were excluded from the analyses, considering that this was the main focus of this study. In Sample 3, there were participants who did not fill out their educational level or race/ethnicity, but these participants were retained when reporting the demographic information.
The IBM SPSS Statistics 26 program was used for most of the analyses other than factor analyses and multilevel growth model analyses. All analyses regarding CFA were performed in SAS 9.4 using the CALIS procedure. Hierarchical linear and nonlinear modeling software (WHLM ver. 7.28f; Raudenbush et al., 2013) was used to fit multilevel growth models.
Results
Reliability
The internal consistency of the BASE-6 was high in Sample 1 (Cronbach’s α = .94; McDonald’s ω = .94), Sample 2 (Cronbach’s α = .88; McDonald’s ω = .89), and Sample 3 (Cronbach’s α = .87; McDonald’s ω = .87). Within Sample 1, all four racial/ethnic groups showed excellent internal consistency, evidenced by Cronbach’s α and McDonald’s ω values ranging from .93 to .96 (White/non-Hispanic group: α = .94, ω = .94; Black/African American group: α = .93, ω = .93; Asian/Asian American group: α = .96, ω = .96; Latino/Hispanic group: α = .94, ω = .94; see Table 2). Corrected item-total correlations ranged from .83 to .86 in Sample 1, from .62 to .80 in Sample 2, and from .60 to .82 in Sample 3, which demonstrates good internal consistency. Within Sample 1, all four racial/ethnic groups demonstrated good internal consistency as well (White/non-Hispanic group: r = .81–.90; Black/African American group: r = .75–.82; Asian/Asian American group: r = .82–.90; Latino/Hispanic group: r = .74–.84; see Table 2). Test–retest reliability was investigated with 151 college student participants from Sample 2 with approximately 1 week of interval (M = 7.43, SD = 1.36) between each completion. The ICC was .80 with the 95% confidence interval (CI) ranging from .73 to .86 and indicated good test–retest reliability.
Corrected Item-Total Correlations, Cronbach’s α, and McDonald’s ω Values.
Note. BASE-6 = Brief Adjustment Scale–6.
Convergent Validity
As shown in Table 3, in all samples, the BASE-6 total score was positively correlated with the DASS-21 total score (r = .77–.79, p < .01), depression (r = .66–.75, p < .01), anxiety (r = .61–.70, p < .01), and stress (r = .72–.76, p < .01) scores, indicating good convergent validity. Within Sample 1, all four racial/ethnic groups showed moderate to large correlation between the BASE-6 and DASS-21 total score (r = .65–.88, p < .01), depression (r = .61–.83, p < .01), anxiety (r = .61–.73, p < .01), and stress (r = .64–.87, p < .01) scores, indicating good convergent validity.
Correlation Coefficients of the BASE-6 Total Score With DASS-21 Scores.
Note. All correlations are significant with p-value < .01. BASE-6 = Brief Adjustment Scale–6; DASS-21 = Depression Anxiety Stress Scale–21.
Factor Structure
The commonly reported fit indices from the single-factor model fitted to the three clinical/nonclinical samples and four race/ethnic groups which are presented in Table 4. Based on the recommended cutoff values by Hu and Bentler (1998), most of the reported fit indices suggested an excellent fit of the model except the root mean square error of approximation (RMSEA) for Sample 3 (RMSEA = .08) and for the White/non-Hispanic group (RMSEA = .11). However, we decided to keep the single-factor model for Sample 3 and the White/non-Hispanic group for two reasons. First, the RMSEAs for Sample 3 and the White/non-Hispanic group were not exceedingly large. According to McCallum et al. (1996), the value of .08 for RMSEA was suggested as a mediocre fit, and could be acceptable. In fact, the RMSEA tends to be overestimated when the degree of freedom is low and the sample size is small (Curran et al., 2003; Kenny et al., 2015), which is the case in the current study. Second, the possible way to improve the fit is to add a second factor or error covariances, but these are not theoretically supported. The factor loadings and error variances were presented in Tables 5 and 6. Factor loadings were in the range from mostly above .7 to .9 across the clinical/nonclinical samples and race/ethnic groups. Overall, the unidimensional structure of BASE-6 was supported which means that all six items measure a single construct (i.e., general psychological adjustment) regardless of clinical/nonclinical status and race/ethnic group.
Model Fit Indices for the BASE-6 CFA Model From the Clinical/Nonclinical Samples and the Race/Ethnic Groups.
Note. BASE-6 = Brief Adjustment Scale–6; CFA = confirmatory factor analysis; p = p-value of χ2 statistic; SRMR = standardized root mean square residual; RMSEA = root mean square error of approximation; CI = confidence interval; CFI = comparative fit index; NNFI/TLI = non-normed fit index also called the TLI; TLI = Tucker–Lewis index.
BASE-6 CFA Result From the Clinical and Nonclinical Samples.
Note. BASE-6 = Brief Adjustment Scale–6; CFA = confirmatory factor analysis; Std = standardized estimate; UnStd = unstandardized estimate;SE = standard error of unstandardized estimate.
BASE-6 CFA Result From the Race/Ethnic Groups.
Note. BASE-6 = Brief Adjustment Scale–6; CFA = confirmatory factor analysis; Std = standardized estimate; UnStd = unstandardized estimate; SE = standard error of unstandardized estimate.
Measurement Invariance: Race/Ethnicity
After confirming the unidimensional structure of the BASE-6 for Sample 1, we examined the measurement invariance across the White/non-Hispanic, Black/African American, Asian/Asian American, and Latino/Hispanic groups. The fit indices in Table 7 suggested that the configural invariance can be established. To test weak, strong, and strict invariance, chi-square difference test and also the difference in comparative fit index (CFI) were used (Cheung & Rensvold, 2002). The results from these tests suggested weak and strong invariance across groups, which means that the factor loadings and factor variances are essentially the same between groups. However, strict invariance was not supported.
Results of Measurement Invariance Tests Across Race/Ethnicity Groups.
Note. CFI = comparative fit index; NNFI/TLI = non-normed fit index also called the TLI; TLI = Tucker–Lewis index; SRMR = standardized root mean square residual; RMSEA = root mean square error of approximation; CI = confidence interval.
Comparison of the BASE-6 Scores Across Samples
The results of one-way ANOVA showed significant differences in all items and total score except the Irritability item. Scheffe’s tests indicated that the mean BASE-6 total scores were higher in the clinical sample than in the community sample. In addition, among the six items of the BASE-6, the clinical sample showed higher scores in four items compared with the nonclinical samples; the irritability and social role interference items did not show higher scores. The college sample showed a significantly higher score on the social role interference item compared with the community sample. Mean scores and standard deviations of the BASE-6 for each sample are presented in Table 8.
BASE-6 Scores Across Samples.
Note. Values with different alphabets (a, b, c) indicate the Scheffe’s test results (adjusted p < .05 for the comparison). Same alphabets show that there is no statistically significant difference between the scores, while different alphabets imply significant differences among groups. BASE-6 = Brief Adjustment Scale–6.
Sensitivity to Clinical Change
Pre–Post Treatment Changes
The result of the paired-samples t-test using the pre- and post-treatment scores of the BASE-6 is presented in Table 9. The clients who participated in the therapy for at least more than 1 month were included in the analysis (n = 67). The BASE-6 total score difference between pre-treatment (M = 21.99, SD = 8.07) and post-treatment (M = 16.42, SD = 7.57) was statistically significant; t(66) = 4.95, p < .001, d = 1.22. The effect size for this analysis of the total score of the BASE-6 (d = 1.22) was found to exceed J. Cohen’s (1988) suggestion for a large effect (d = .80). Six items of the BASE-6 scores also showed a statistically significant decrease from pre-treatment to post-treatment, and most of the items showed moderate to large effect size (d > .50).
Paired Sample t-test for Pre–Post Treatment Scores and Cohen’s d Value (n = 67).
Sensitivity to Treatment Trajectories
The results of the linear trajectory model (Model A) showed that both the fixed effects and random effects were statistically significant (see Table 10). The significant fixed effects indicate that the averaged initial score and the linear rate of change in the BASE-6 over time is statistically significant. The negative coefficient of linear rate of change implies that the BASE-6 score decreased over time. In addition, the significant random effects imply that individual differences in initial status and the linear rate of change were statistically significant. The results of the quadratic trajectory model (Model B) also showed that the fixed effects of initial status and the linear rate of changes were statistically significant although the quadratic rate of change was not significant. The random effects of the initial status, and linear and quadratic rate of change were all statistically significant, which implies individual differences in initial status and change over time. Although the deviance and the badness-of-fit from Model B were slightly lower than in Model A, the fixed effect of the quadratic rate of change in Model B was not significant. Therefore, we can conclude that Model A, the linear trajectory model, is a better model.
Results of Fitting Multilevel Growth Model.
p < .05. **p < .01. ***p < .001.
Discussion
The present study aimed to (a) provide further evidence of the psychometric properties of the BASE-6, a measure of general psychological adjustment for MBC, through intense investigation of reliability and validity; (b) provide evidence of the generalizability of the BASE-6 for individuals with diverse racial and ethnic backgrounds; and (c) evaluate its clinical utility within the context of MBC. The findings of this study provided support for each of these study objectives.
Aim 1: Psychometric Properties of the BASE-6 in Clinical and Nonclinical Samples
First, the BASE-6 demonstrated excellent internal consistency for both the clinical and nonclinical samples by demonstrating high values of Cronbach’s α and McDonald’s ω. The results of item-total correlation analyses indicated that the correlation coefficient of each item score and the total score ranged from .72 to .91 in all three samples, and none of the six items showed unacceptable (r < .30) item-total correlations (Nunnally & Bernstein, 1994). Furthermore, the test–retest reliability obtained from this study also suggested good reliability, indicating the BASE-6’s adequate stability and consistency over a short period of time (range: 4–10 days). These test–retest reliability results are equivalent to that of other routine outcome monitoring measures such as the PHQ-9 and GAD-7 (Kroenke et al., 2001; Spitzer et al., 2006). These results indicate that the BASE-6 scores are consistent over time across items, and all items are measuring the same construct.
The correlational analyses of the BASE-6 and DASS-21 showed evidence of convergent validity by demonstrating moderate to large and positive correlations in all three samples, which are in accordance with the previous studies that explored the BASE-6’s convergent validity via DASS-21 (Cruz et al., 2020; Yildirim & Solmaz, 2021). These results show that the individual who is experiencing difficulties in overall psychological adjustment also experiences more symptoms related to depression, anxiety, and stress. Although the DASS-21 is a widely used and established measure that captures major psychological symptoms, there is a significant (e.g., depression, anxiety) but not direct overlap with the BASE-6 (e.g., relationships, work/school performance). Thus, these results provide correlational support to the capability of the BASE-6 to adequately reflect psychological distress. In addition, in all three samples, the total score of the BASE-6 showed stronger correlations with the total score of the DASS-21 than with the three subscales of the DASS-21, indicating that the BASE-6 is an appropriate measure that captures overall symptom distress and maladjustment. The results from CFA applied to both clinical and nonclinical samples supported the unidimensional structure of the BASE-6. The fit indices showed excellent to acceptable fit of the single factor model, and the large factor loadings indicated that the single factor of general psychological adjustment was highly correlated with all items. Therefore, the unidimensional structure of the BASE-6 was confirmed by the CFA results. The current findings are not only aligned with the previous studies by providing evidence of the same results on reliability and validity of the BASE-6, but also are in accordance with previous research via support for positive psychometric properties by utilizing further analysis (e.g., McDonald’s ω coefficient, item-total correlation) or measures (e.g., DASS-21 for convergent validity).
Aim 2: Generalizability of the BASE-6 Among Ethnic Groups and Measurement Invariance
As no previous studies had applied the BASE-6 to diverse race/ethnic groups, it was crucial to provide evidence for the generalizability of the BASE-6 across various racial/ethnic groups, and the second aim of this study was supported. Specifically, both Cronbach’s α and McDonald’s ω coefficient indicated strong internal consistency in various race/ethnic groups. Furthermore, convergent validity was also supported in all four ethnic groups by showing moderate to high positive correlations between the BASE-6 and DASS-21. The unidimensional structure was also confirmed by the CFA results for all four race/ethnic groups. It means that across different race/ethnic groups, all items of the BASE-6 measure only one factor—general psychological adjustment—rather than any other constructs. In terms of measurement invariance, the BASE-6 showed a good degree of invariance across four different race/ethnicity groups. The configural invariance supported that the same construct exists within each race/ethnicity group and each construct was measured by six items of the BASE-6. The invariance of factor loadings and factor variance showed that the construct was being measured in the same way between groups. Although the measurement error variances (i.e., residuals) were not invariant between groups, it does not prevent the effective use and interpretation of the BASE-6 across different race/ethnicity groups. In fact, it is the invariance of “factor loadings” that is the primary standard (or first step) for interpretation, while the fourth step (i.e., measurement invariance error) or even the third step (i.e., invariance of factor variances) has little practical implications for interpretation (Marsh, 1994). With this consideration, the current results indicate a stable unidimensional structure of the BASE-6 and comparable perceptions of the individual items across four different race/ethnic groups. As such, researchers can meaningfully compare the results of the BASE-6 among different racial/ethnic groups, and clinicians can assume that their clients from different backgrounds would still perceive each item of the BASE-6 in a similar manner.
Aim 3: Clinical Utility of the BASE-6
The results of our final aim indicate that the BASE-6 can (a) reliably discriminate between a clinical and nonclinical population and (b) sensitively capture longitudinal changes over the course of the therapy in clinical populations, which importantly, supports the clinical utility of this measurement. The first finding provides a basis for the BASE-6 as a screening measure that has sufficient sensitivity and specificity in predicting or discriminating clinical levels of psychological maladjustment. The mean score comparisons showed that the BASE-6 scores of the clinical sample were significantly higher compared with the nonclinical samples. Specifically, average total score and four item scores (i.e., anxious, depressed, interference: self, and interference: relationships) of the clinical sample were significantly higher than either the online community sample or college student sample (i.e., both nonclinical samples). The results of a previous study (Cruz et al., 2020) also showed that the clinical sample’s total score and item scores were significantly higher than the college student or online community sample.
There was an unexpected finding in which two items for the clinical sample did not show a higher score than the nonclinical samples, which are irritability and social role interference. Regarding the irritability item, all three samples did not show any difference in their scores. Regarding the social role interference item, the online college sample’s score was significantly higher than the online community sample, while there was no difference between the clinical sample and the online community sample. One explanation for this result is that it may be due to the effect of COVID-19 on the general population, including the online community sample and college student sample (Holman et al., 2020; Son et al., 2020). One study that utilized the BASE-6 for the college population during COVID-19 showed an even higher total score (i.e., 21.54) than the college student sample of this study (i.e., 19.64), which also implies the negative impact of the pandemic on psychological adjustment (Arslan et al., 2021). As the data were solely collected during the COVID-19 pandemic for nonclinical samples while the clinical sample’s data were collected before and during the pandemic, there is a possibility that the pandemic affected and increased overall mental health distress in these populations, in line with the growing research of pandemic’s negative influence on mental health (Horigian et al., 2021; Islam et al., 2021; Pierce et al., 2020; Serafini et al., 2020). Furthermore, another study that utilized the BASE-6 for adult psychiatric outpatients showed total scores between 22.71 and 26.38 during the COVID-19, which is higher than the total score of the clinical population in this study (Kablinger et al., 2022). These results imply the impact of the pandemic on clinical populations and suggest that the current study’s clinical sample’s scores could have been even higher if the data were solely collected during the pandemic, rather than before and throughout COVID-19. In addition, when comparing the overall BASE-6 scores with the Cruz et al. (2020) study, scores of all three samples were higher in this study. Nevertheless, it should be noted that the total score of the BASE-6, as well as the four items, indicated significantly higher scores in the clinical sample than nonclinical samples, supporting the idea that the BASE-6 total score and individual items offer clinical utility by allowing discrimination of individuals with significant clinical distress.
In addition, the second finding of longitudinal change identification was supported via two statistical analyses, the utilization of pre–post scores and further analysis with a multilevel growth model. The participants of the clinical sample were clients from the outpatient psychology training clinic, where standardized MBC was implemented throughout the multiple years of data collection. In this clinic, all clinicians went through training for a standardized MBC protocol, which required a standard battery of measures throughout therapy and a collaborative review and discussion of the outcome measures’ results with clients (Cooper et al., 2021). The clinical sample who participated in evidence-based treatment showed significant pre–post treatment changes assessed with the BASE-6, and importantly, the total score and each item score demonstrated moderate to large effect sizes (d = 0.64–1.27), which implies that even individual items of the BASE-6 can be reliably used when evaluating intervention effectiveness in both research and clinical settings. The current findings extend the previous research emphasizing that, in addition to the ability of the BASE-6’s differentiation of clinical and nonclinical populations, the BASE-6 also demonstrated its sensitivity to therapeutic changes when used with clinical populations.
The results from multilevel growth model analysis utilizing 16 sessions of longitudinal data further supported the BASE-6’s sensitivity in detecting treatment-induced changes over time. The findings imply the consistent identification of a treatment effect over the course of the intervention for the participants whose starting points (i.e., intake scores) were all significantly different. These results are meaningful in that this is the first longitudinal study proving support that the BASE-6 is a measure sensitive enough to detect changes in overall psychological functioning and adjustment over the course of treatment in a clinical population. Indeed, the BASE-6 could be reliably utilized each session in psychotherapy settings and would allow clinicians to meaningfully interpret the score changes between previous and current sessions, as it is initially supported in this study as a sensitive indicator of therapeutic changes.
Limitations and Future Directions
There were some limitations that should be noted. In the current study, there is a potential limitation regarding the recruitment setting and characteristics of the clinical sample. As this study utilized data of clinical samples from one setting, there was a lack of diversity; most of the clients were predominantly White/non-Hispanic (70.7%) due to the geographical characteristic of the community training clinic where the clinical sample data were collected. In addition, most of the client’s educational level was higher than “some college (86.2%),” which could affect the generalizability of these findings. Future research is recommended to replicate the previous findings with a more diverse and marginalized clinical population. In addition, while the community and college student samples were collected in a similar period when the COVID-19 pandemic was actively affecting an individual’s life (e.g., quarantine, virtual meeting/schooling; Kniffin et al., 2021; Murphy et al., 2020), clinical samples’ data were collected before and throughout the pandemic. These various conditions of data collection might have affected the BASE-6 scores of each group and overall results of the current study. Furthermore, in this study, test–retest reliability was only conducted with one of the nonclinical samples and not for the clinical population. Although participants of the clinical sample completed the BASE-6 weekly, these data were not utilized for the test–retest reliability considering the possible effect of the therapy on the BASE-6 score. In addition, the time interval for measuring test–retest reliability was 7 days, which may have allowed the participants’ second set of responses to be influenced by their first set of responses. As the purpose of the development of the BASE-6 was to utilize it as a routine outcome monitoring measurement that could be used in the clinical setting, stability and consistency of the BASE-6 scores should be further evaluated within the clinical sample and at longer intervals (e.g., 14 days). Utilizing the scores of the clinical sample while they are on a waitlist could be one of the options to collect data for a stronger study of test–retest reliability. Another potential limitation was related to discriminant validity. There is an inherent difficulty in examining the discriminant validity of the BASE-6 in that it is conceptually and empirically difficult to separate anxiety, depression, and stress from general psychological functioning and distress. As a result, this study could not investigate whether specific concepts or measures of constructs that are not meant to be related are, in fact, unrelated to the BASE-6. In addition, there is a potential limitation for convergent validity, as the DASS-21 may not fully reflect the “interference” areas as the three items of the BASE-6 (i.e., interference in self, relationships, and social role), considering that the primary areas that the DASS-21 measures are focused on psychopathology itself (i.e., depression, anxiety, and stress). As the correlational results between the BASE-6 and DASS-21 showed a moderate to large correlation, these results reflect that psychological adjustment that is measured via the BASE-6 has a strong association with psychopathology but does not fully overlap. Finally, the clinical cutoff value was not examined in this study due to the comparatively small number of individuals in the clinical sample compared with the nonclinical samples.
In future research, with larger clinical samples, identifying clinical cutoffs on the BASE-6 is needed to facilitate the use of the BASE-6 in both clinical and research settings by indicating individuals with clinically significant distress. Furthermore, adding clinical utility by comparing the BASE-6 scores with real-world evaluations would increase the clinical usefulness and validity of the measure. For example, the BASE-6 scores could be compared with the clinician/patient’s own evaluations, such as CGI-S/PGI-S (Clinician/Patient Global Impression of Severity; Guy, 1976) and/or CGI-I/PGI-I (Clinician/Patient Global Impression of Improvement; Guy, 1976), or with the treatment outcomes (e.g., number of sessions needed to attain treatment goals, whether the treatment goals were attained). Another suggestion for future research is evaluating the BASE-6’s appropriateness and psychometric properties within the adolescent population. Although there are various self-report measures that are well validated for youth, such as the Child Behavior Checklist (CBCL; Achenbach, 1991) and Symptom Checklist–90–Revised (SCL-90-R; Derogatis, 1977, 1994) or parent-reported measures such as Pediatric Symptom Checklist (PSC; Gardner et al., 1999), there could be still barriers to implement those measures into MBC considering their length (15+ items) and cost (Aboraya et al., 2018). Considering the limited implementation and research of MBC for youths compared with research with adults (Bickman et al., 2011), investigating and validating appropriate outcome measurement that encompasses overall psychological distress and adjustment is recommended. Currently, no previous studies have evaluated the reliability and validity of the BASE-6 with the youth population. As this measure is simple, brief, easily understood and administered, it has the potential value to be utilized as an outcome measure in MBC therapy settings with adolescents.
Implications and Conclusion
Despite the aforementioned limitations, the current study has several implications and strengths. It should be noted that this study was the first validation of the BASE-6 that confirmed its generalizability with diverse race/ethnic populations when evaluating overall psychological adjustment. Findings from the present study show that the BASE-6 has excellent reliability and validity in both clinical and nonclinical samples as well as in diverse minority groups. Importantly, results of invariance testing with four different race/ethnic groups established good invariant unidimensional factor structure across individuals of White/non-Hispanic, Black/African American, Asian/Asian American, and Latino/Hispanic. Furthermore, this study has implications for MBC research and clinical practice, as the BASE-6 reliably captured therapeutic changes every session in the clinical population. In this study, the BASE-6 was administered routinely and used in the context of MBC, so the primary components of MBC, such as collaborative evaluation of client functioning data, were thoroughly incorporated with the clinical sample. In addition, the results of the measures were discussed within the session and informed clinicians’ clinical decision making (Cooper et al., 2021). As such, the BASE-6 served the purpose of MBC well over the course of treatment and established its clinical utility by differentiating clinical and nonclinical samples and demonstrating its sensitivity to changes in various ways. These results further support the BASE-6 as a reliable, valid, brief, and no-cost tool that can be used as a routine outcome monitoring measure for psychological adjustment and distress, and suggests that incorporating the use of the BASE-6 in treatment settings, as part of MBC, could offer feasible and valuable information for both clinical practice and research.
Supplemental Material
sj-docx-1-asm-10.1177_10731911221115144 – Supplemental material for Brief Adjustment Scale–6 for Measurement-Based Care: Psychometric Properties, Measurement Invariance, and Clinical Utility
Supplemental material, sj-docx-1-asm-10.1177_10731911221115144 for Brief Adjustment Scale–6 for Measurement-Based Care: Psychometric Properties, Measurement Invariance, and Clinical Utility by Hayoung Ko, Jaehyun Shin and Lee D. Cooper in Assessment
Footnotes
Acknowledgements
We are appreciative of the support provided by the Virginia Tech Psychological Services Center. We would also like to thank Dr. Fei Gu for providing consultation on measurement invariance analysis and factor analysis. The authors declare that they have not published, posted, or submitted any related papers or presentations from this same study.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethical approval and informed consent
Informed consent was obtained from all individual participants included in the study. Procedures were approved through the Virginia Polytechnic Institute & State University IRB # 20-345 and # 15-083.
Supplemental Material
Supplemental material for this article is available online.