Abstract
Objectives: The Brief Symptom Inventory (BSI) is widely used in juvenile justice settings; however, little is known regarding its factor structure in antisocial youth. The authors evaluated the BSI factor structure in a state residential treatment population. Methods: 707 adolescents completed the BSI. Exploratory and confirmatory factor analyses were used to assess factor structure. Intergroup variability was examined using multiple-group structural equation modeling. Results: Findings supported a 6-factor, 25-item model explaining 49.5%of sample variance. The derived structure differed from prior findings with adult psychiatric patients by including a suicidal ideation latent variable and excluding several developmentally inappropriate factors. Conclusion: There may be problems associated with indiscriminant application of the original BSI factor model to juvenile justice populations.
Many adolescents within the juvenile justice system have psychiatric disorders that are undiagnosed and/or untreated. Thus, for these youth, involvement with the juvenile justice system may represent their first opportunity to access and benefit from mental health screening and treatment. Youth remanded to correctional facilities, residential treatment centers, and community-based programs consistently evidence higher rates of mental health disorders than their general population counterparts (Teplin, Abram, McClelland, & Dulcan, 2002; Wasserman, McReynolds, Ko, Katz, & Carpenter, 2005; Wasserman, McReynolds, Lucas, Fisher, & Santos, 2002). Approximately 70% of youth enter detention with a diagnosable psychiatric disorder and 20% of these youth have serious mental health disorders (Skowyra & Cocozza, 2006). Also, youth within the juvenile justice system are more likely to be minorities, to be of low socioeconomic status, to have limited institutional education, and to lack health insurance (Abram, Paskar, Washburn, & Teplin, 2008; Feld, 1991; Heflinger, Chatman, & Saunders, 2006; Moffit & Slade, 1997). Serving the most vulnerable of our nation’s youth, the juvenile justice system has a critical societal obligation to accurately identify and effectively treat youth who are displaying serious psychiatric symptoms.
An important predictor of the quality of mental health treatment provided to juvenile justice system-involved adolescents is the quality of the screening process designed to detect signs and symptoms of these disorders (Skowyra & Cocozza, 2006). Over the past two decades, a number of mental health screening tools have been developed to assess mental health problems of delinquent youth (Grisso, Vincent, & Seagrave, 2005). However, most of these screening instruments have not been subjected to adequate scrutiny vis-à-vis their psychometric properties and clinical utility in a range of contexts.
The Brief Symptom Inventory (BSI) is a screening and assessment instrument commonly used in a variety of adult and adolescent inpatient, residential, and outpatient settings. The BSI is a 53-item, self-report index of psychopathology that measures nine primary symptom dimensions that can be summed to reflect three global indices of distress (Derogatis, 1993). The Global Severity Index (GSI) measures overall current distress in relation to psychiatric symptoms. The Positive Symptom Distress Index (PSDI) measures symptom severity, whereas the Positive Symptom Total (PST) reflects the number of self-reported symptoms. For each item, respondents indicate to what extent they were “bothered or disturbed” by each listed symptom “over the past 7 days including today.” Responses are given on a 5-point Likert-type scale ranging from 0 (not at all) to 4 (extremely). Prior studies have confirmed the reliability and validity of the BSI, its subscales, and its relative diagnostic accuracy compared to similar instruments using large samples of mentally ill adult respondents representing diverse demographic characteristics (Boulet & Boss, 1991; Deragotis & Melisaratos, 1983; Hayes, 1997). Though widely used in juvenile justice settings (e.g., Roberts, 2004; Ryan & Redding, 2004), most psychometric evaluations of the BSI have been conducted with adult psychiatric populations. More data are needed specific to juvenile justice populations bearing on the usefulness of the BSI as a measure of current psychiatric symptoms.
The aims of this study were to (a) use exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) to determine the factor structure of the BSI in a state population of juvenile offenders, (b) compare the derived factor structure to the originally identified BSI factor structure, and (c) examine differences between subgroups of delinquent youth with regard to the BSI factor structure.
Hypothetical Structure
Deragotis and Melisaratos (1983) used principal components analysis (PCA) with samples of psychiatric inpatients and outpatients to identify nine primary BSI factors: somatization, obsessive–compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism. The somatization subscale assesses psychological distress caused by perceived bodily dysfunction. The obsessive–compulsive subscale is comprised of items assessing distress related to obsessive thoughts and compulsive behaviors. The interpersonal-sensitivity subscale assesses feelings of inadequacy and inferiority in relation to others. The phobic anxiety subscale assesses fears of open or crowded spaces, whereas the paranoid ideation subscale assesses the tendency to be suspicious of or to distrust other people. The psychoticism subscale assesses symptoms of psychosis such as feeling that others control your thoughts or that something is wrong with your mind.
In addition to the mismatch between the adult psychiatric populations for which the BSI was originally intended and on which basis the factor structure was identified and its current usage in juvenile justice settings, the methods by which the original structure was derived are problematic. The PCA does not partition shared variance within the model from the unique variance and error variance of each variable during factor extraction (Costello & Osborne, 2005; Floyd & Widaman, 1995). Subsequent researchers have applied true factor analytic methods to the BSI (Hoe & Brekke, 2008, 2009); however, none of these investigators have used these methods to test the factor structure of the BSI within a juvenile justice population.
Method
Study Sample
The data utilized for this study were collected for a previous investigation that focused on inhalant use among delinquent adolescents (see Howard, Balster, Cottler, Wu, & Vaughn, 2008). Participants were recruited from all Missouri Division of Youth Services (DYS) facilities (N = 27) that provided residential rehabilitation services within the state. All residents of each facility were eligible to participate and were recruited at the time of data collection. Interviewing took place over a 3-month period in 2004. Each interview required 30–90 min.
In total, 707 of the 740 (95.5%) eligible DYS adolescents participated in the study and provided useful data. A total of 12 residents were unavailable at the time of the interviews due to in-progress transfers between facilities. Five interviews were discontinued due to serious psychiatric problems displayed by youth. Missing data across more than two analytic variables accounted for the remaining 16 participants who were excluded from analyses. In general, missing data for most variables was <1%, and in no case was >5%. Each participant signed an informed assent form, which detailed information about the study and clarified that he or she was not required to participate and could discontinue the interview at any time. As the legal guardian of all youth receiving care, the Missouri DYS provided formal permission for participation.
Residents were compensated $10 for the completion of the interview. The Missouri DYS Institutional Review Board (IRB), Washington University Human Studies Committee IRB, and federal Office of Human Research Protections approved the informed consent and study protocols and the study was granted a Certificate of Confidentiality by the National Institute on Drug Abuse. Although potential study participants were assured they did not have to participate in the study and could discontinue their participation at any time, it is possible that some youth felt obligated or otherwise pressured to participate. Such youth and other participants may also be inclined to present themselves favorably (i.e., evidence social desirability bias) despite repeated assurances that their responses are and will remain absolutely confidential.
A total of 15 graduate students conducted study interviews. To insure data accuracy, each interviewer was required to complete intensive 1-day training and was assisted by an onsite interview editor.
Measures
Demographic factors including gender, race, and geographical location of family residence (i.e., urban, suburban, small town, and rural) were recorded for each participant. For the analyses reported below, the variables of race and location were dummy-coded to identify participants as either White or non-White (i.e., Black, Latino, and other Race) and living in either urban or nonurban settings (i.e., rural and suburban), respectively. Though race and location are evaluated as dichotomous groups, for comparison purposes, the mean score for each individual classification variable are displayed.
Data Analysis
Descriptive, reliability, and EFA were conducted using the Statistical Package for the Social Sciences (SPSS) 15.0 (SPSS, 2006). The CFA and multiple-group structural equation modeling components of the analysis were conducted with Amos Software Version 12. Throughout this report, the PCA factor structure of the BSI obtained by Deragotis and Melisaratos (1983) is considered the original model. The model we obtained by applying CFA, a true factor analytic method, to the BSI is considered the derived model.
Sample Characteristics
Participants were predominantly male (N = 615, 87.0% of total sample), generally originating from suburban or small town settings (N = 430, 60.3% of total sample). The average age of participants was 15.5 (SD = 1.2) and the racial distribution was approximately 55.4% Caucasian, 33.0% African American, and 3.9% Latino/Latina. More than half (51.4%) of the sample had a previously physician-diagnosed mental illness. For a complete sample description see Howard et al. (2008).
Exploratory Factor Analysis
The EFA is a statistical technique used to estimate how many latent variables are necessary to determine the relationships between a set of items (Pett, Lackey, & Sullivan, 2003). Variables were extracted with principle axis factoring and Oblimin rotation was utilized. The number of factors was determined using the eigenvalue rule (Kaiser, 1960), which recommends including elements with values >1.0, and by evaluation of the Scree plot (Cattell, 1966). Cronbach’s α was used to assess internal consistency reliability for sets of items associated with each derived factor.
Confirmatory Factor Analysis
The CFA is a structural equation modeling technique used to test measurement models with a priori specification of relationships between latent and observed variables (Kline, 2011). The CFA was conducted using asymptompically distribution-free estimator procedures. Fit indices used in the study included root mean square of error approximation (RMSEA), the Tucker-Lewis index (TLI), and the goodness-of-fit index (GFI). The fit criteria used were a RMSEA between 0.05 and 0.08 with a confidence interval ≤0.10 upper bound, TLI ≥ 0.90, and GFI ≥ 0.90 (Kline, 2005).
Multiple-Group Analysis
Two multiple-group structural equation modeling analyses were conducted to determine if the hypothesized structure existed, when accounting for racial differences and urban versus rural location of residence. Both analyses were conducted using the generalized least squares extraction method. In separate analyses, the best-fit factor structure identified with the CFA was tested for White youths compared with non-White youths and for youth living in an urban setting prior to treatment compared with those from rural/suburban environments. Utilizing Amos, data were partitioned by comparison groups and evaluated for adequate fit within the derived factor structure. When adequate fit was achieved, individual loadings onto latent factors were constrained consecutively following the procedure recommended by Byrne (2010). The overall fit of the model with the constrained loading/loadings was tested against the full model with the tested loadings freely estimated. A significant difference between the chi-squared of each model identified which element of the BSI elicited significantly different responses.
Results
Exploratory Factor Analysis
Results of the EFA are displayed in Table 1 . Six factors were extracted from the 53 BSI items; these factors accounted for 49.53% of the variance in the matrix and all factors had eigenvalues greater than 1. The preliminary analysis involved the visual inspection of factor loadings. Items with large loadings across multiple factors (i.e., values >.40) were excluded. The cutoff for minimum loading was .32, based on widely accepted standards (Tabachnick & Fidell, 1996). According to these standards, 8 of the 25 items included in the derived 6-factor structure had loadings that were considered excellent (i.e., .71 and higher) and four had loadings considered very good (i.e., .63–.71). All subscale internal consistency reliabilities approached or exceeded the acceptable minimum value of .70 (Nunnaly, 1978).
Exploratory Factor Analysis Factor Loadings for 25 Brief Symptom Inventory Items Using Asymptotically Distribution-Free Extraction Method and Oblimin Rotation
The naming of the derived factors was based on the original factors defined by Deragotis and Melisaratos (1983). In instances where the majority of items included in the new factor matched those comprising the originally identified factor, the original factor label was kept. These factors included phobic anxiety, depression, obsessive–compulsive, hostility, and somatization. In one case, a unique factor was identified: suicidal/death ideation.
Confirmatory Factor Analysis
All factor loadings were significant and exceeded .40. The goodness-of-fit indices of the 6-factor model were acceptable (χ2 = 647.68, df = 260, RMSEA = .03, CFI = .75, GFI = .93), suggesting that the revised structure adequately explained covariances among item scores and covariances between latent variables. A RMSEA value <.05 indicates a “close fit” between the covariance structure suggested with the factor model and the relationships within the population data (Kline, 2005). Fit indices including the CFI and GFI are considered adequate if >.90 (Bentler, 1990). Model respecification was attempted to improve the model fit statistics; however, overall model fit could not be maintained without compromising theoretical consistency with previous research on the psychological traits of antisocial youth.
The results of the standardized regression estimates and squared multiple correlation values for the items associated with the six factors are displayed in Table 2 . All regression weights were significant and have appropriate magnitude. Approximately 88% of the associated squared multiple correlations were above the acceptable range (i.e., values >.32).
Standardized Regression Estimates and Squared Multiple Correlations from Confirmatory Factor Analysis of 25 Brief Symptom Items Included Under Six Factors Using General Least Squares Extraction Method
Note. Std. = standardized; Sig. = signifance level; SMC = squared multiple correlation.
*** p < .001.
Multiple Group Analysis
A model in which factor loadings for “feeling afraid in open spaces or on the streets” (Δχ2 = 4.1, df = 1), “suddenly feeling scared for no reason” (Δχ2 = 5.8, df = 1), “feeling nervous when you are left alone” (Δχ2 = 7.6, df = 1), and “difficulty making decisions” (Δχ2 = 4.1, df = 1) were freely estimated in urban and nonurban youth fit significantly better than the constrained model.
A model in which factor loadings for “feeling lonely” (Δχ2 = 7.2, df = 1), “faintness or dizziness” (Δχ2 = 4.5, df = 1), and “thoughts of ending your life” (Δχ2 = 10.0, df = 1) were freely estimated in White and non-White youth fit significantly better than the constrained model.
Mean Score Comparison
Table 3 presents the mean scores of the factors for the original and derived models by gender, race, and rural versus urban place of residence. As compared with the original structure, the factor structure of the derived model identified increased presence of depressive and obsessive–compulsive thoughts and behaviors within the sample population. The other three latent variables present in the original and updated factors solutions (though with somewhat different but overlapping items), phobic anxiety, hostility, and somatization, maintained similar size and variability of magnitude across demographic groups.
Mean Raw Item Scores for Original and Newly Identified BSI Factor Structure by Place of Residence, Gender, and Ethnicity
Note. M = mean; SD = standard deviation; Comp. = compulsive; Sens. = sensitivity; BSI = Brief Symptom Inventory.
* p < .05.
** p < .01.
*** p < .001.
Additionally, Table 3 displays the results of significance testing of BSI scores between demographic groups. Being male, residing in an urban area, and being of Caucasian race were used as reference groups in analysis of variance (ANOVA) modeling. Within the derived factor structure, individuals from urban settings were significantly less likely to experience distress from obsessive–compulsive symptoms than nonurban youth. Significant gender differences were present across the phobic anxiety (Cohen’s d = .21), depression (Cohen’s d = .40), obsessive–compulsive (Cohen’s d = .30), and somatization factors (Cohen’s d = .24). For each of these measures, girls were more likely to report symptoms than boys. White adolescents were less likely to display distress due to symptoms of hostility than non-Whites (Cohen’s d = .11–.52); however, White youth were significantly more likely to have symptoms of depression (Cohen’s d = .27-.30) and obsessive–compulsive traits than non-White youth (Cohen’s d =.35–.38).
Discussion and Applications to Social Work
Findings of the present study suggest that when applied to a juvenile justice population the BSI may be appropriate for the assessment of some, but not all, of the psychiatric syndromes it was originally designed to assess. Associations between BSI items that comprise the phobic anxiety, depression, obsessive–compulsive, hostility, and summarization factors differed slightly from those of the original scale construction (Deragotis & Melisaratos, 1983). A new BSI factor assessing suicidal/death ideation was identified. Tables 4 and 5 provide a full comparison between the structure derived in the current study and the structure of the original formulation.
Factors and Constituent Items of Original BSI Factor Structure
Note. BSI = Brief Symptom Inventory.
Factors and Constituent Items of Derived BSI Factor Structure
Note. BSI = Brief Symptom Inventory.
The findings of this study on mental health symptomatology are consistent with current research on adolescents in detention. Suicidal ideation and behavior is a significant concern among youth likely to have contact with the juvenile justice system (Abram, Choe, Washburn, Teplin, King, & Dulcan, 2008). The BSI items, which this analysis supports as making up an independent factor of suicidal ideation, were previously aspects of the depression factor. The other significant departure from the original factor structure was the exclusion of the psychoticism, anxiety, and paranoid ideation factors. This is likely a result of juvenile detention centers having greater heterogeneity of mental health diagnoses among residents than would be expected of adult psychiatric populations on which the BSI was originally tested (Derogatis, 1993). It may also be the case that the youth in this study, aged 13–17, had not yet reached the age at risk for the development of psychotic and paranoid disorders and related symptoms. The derived factor model identified in this study reduces the BSI to the 25 most appropriate items to assess 6 key common psychiatric symptom domains affecting adolescents served by the juvenile justice system. Tailoring the BSI to this specific population improves the ease with which need can be identified and subsequently treated.
The reduction of factors and related BSI items as compared to the original structure is due to the use of more precise factor analysis methodology in addition to the characteristics of the study population. The results of the multiple group analyses suggest that the expression of psychiatric disorders may differ in juvenile justice populations depending on race and geographic area of family residence. By constraining individual components of the derived configuration, estimating coefficients, and performing significance testing, variations due to demographics when applying the BSI in juvenile justice settings have been identified. With regard to place of residence, phobic anxiety displayed in youth living in urban areas more likely to be driven by fears of being on the street or open space, being alone, and unexplained fear than their nonurban peers. Researchers have found fear of criminal victimization, which disproportionately affects those living in urban environments, to be an important factor in adolescent anxiety, though other mechanisms may explain this difference (National Institutes of Mental Health, 2006). The variable significance of BSI items between White and non-White youth were identified as individual items within the depression, somatization, and suicidal ideation factors. The overall BSI structure appears consistent across races; however, the monolithic minority characterization reduced the ability to specify differences between individual ethnic groups. As with other health assessment tools, the derived reduced-form BSI is designed to accurately identify areas for more extensive follow-up. In the context of the overall model fit, these intergroup differences do not compromise the 25-item BSI’s potential effectiveness. Practitioners are advised to take potential differences in item interpretation and individual experiences into consideration, when administering and interpreting the BSI.
The derived 25-item BSI structure more accurately assesses the conditions most often faced by antisocial adolescents. The comparison of mean scores produced evidence that the derived factor model is superior in measuring depressive and obsessive–compulsive thoughts and behaviors within the Missouri DYS population. Unlike the factor model based on adult psychiatric population data on which the BSI was originally tested, the derived obsessive–compulsive variable does not include “feeling blocked getting things done” or “having to check or double check what you do.” The primary difference between the depression latent variable of each model is the derived model’s isolation of suicidal ideation onto an independent factor. The derived BSI model is an excellent clinical tool for screening juveniles for suicide risk. Recent studies have confirmed the importance of intake screening for suicide risk to reduce serious suicide attempts (Gallagher & Dobrin, 2005); despite the majority of juvenile detention facilities failing to effectively screen new residents (Wasserman et al., 2003). This research has provided practitioners a tool to effectively diagnose and treat the most vulnerable of youth populations.
A primary limitation of the current study was the choice estimators in the factor analysis. The decision was based on the functionality of the Amos software. Whereas it was appropriate to discard the default maximum likelihood estimator due to the ordinal nature of the BSI response data (Bollen, 1989), a weighted least squares estimator adjusted for means and variances has received support in recent literature on the issue (Flora & Curran, 2004). Future research should test outcomes between the available software programs and estimators.
Footnotes
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: grants DA15929, DA15556, and DA21405 (M.O. Howard, Principal Investigator) from the National Institute on Drug Abuse.