Abstract
Trauma exposure is overrepresented in incarcerated male populations and is linked to psychiatric morbidity, particularly posttraumatic stress disorder (PTSD). This study tests the feasibility, reliability, and validity of using computer-administered interviewing (CAI) versus orally administered interviewing (OAI) to screen for PTSD among incarcerated men. A 2 × 2 factorial design was used to randomly assign 592 incarcerated men to screening modality. Findings indicate that computer screening was feasible. Compared with OAI, CAI produced equally reliable screening information on PTSD symptoms, with test–retest intraclass correlations for the PTSD Checklist (PCL) total score ranging from .774 to .817, and the Clinician-Administered PTSD scale and PCL scores were significantly correlated for OAI and CAI. These findings indicate that data on PTSD symptoms can be reliably and validly obtained from CAI technology, increasing the efficiency by which incarcerated populations can be screened for PTSD, and those at risk can be identified for treatment.
Keywords
Trauma exposure is overrepresented in incarcerated male populations (Wolff, Huening, Shi, & Frueh, 2014). Surveys of trauma exposure among incarcerated men have found rates of trauma exposure ranging from 62.4% to 100% (Gibson et al., 1999; Saxon et al., 2001; Wolff et al., 2014; Wolff & Shi, 2009; Wolff, Shi, Blitz, & Siegel, 2007), compared with rates of 43% to 92% for community-based male populations (Breslau, 2009; Teplin, McClelland, Abram, & Weiner, 2005). Moreover, incarcerated men, compared with men residing in the community, are more likely to report experiencing physical and sexual assaultive violence. For example, the lifetime rate of assaultive violence for a community sample of men is estimated at 43.3% (Breslau, Chilcoat, Kessler, Peterson, & Lucia, 1999), less than half the rate of 96.5% estimated for incarcerated men (Wolff et al., 2014). The difference in lifetime rates of sexual trauma between male community and incarcerated populations is even more striking. The lifetime prevalence of sexual assault reported by incarcerated men is estimated at 15% to 16% (Saxon et al., 2001; Wolff et al., 2014), compared with rates of 1% to 3% for adult male populations (Breslau et al., 1999; Lukaschek et al., 2013).
Trauma exposure is linked to psychiatric morbidity, particularly posttraumatic stress disorder (PTSD). Based on community samples, 5% of men have experienced PTSD, representing approximately 7% to 14% of those who have experienced a severe traumatic event (Breslau, 2009). By contrast, for incarcerated men, roughly 33% have severe PTSD symptoms, increasing to 60% having moderate to severe PTSD symptoms (Wolff et al., 2014). Depending on type of trauma exposure, 30% to 60% of incarcerated men exposed to a physically violent traumatic event manifest moderate to severe symptoms of PTSD, increasing to 43% to 75% for those exposed to sexual assault. Rates are higher yet for incarcerated men with other co-occurring mental disorders (Wolff et al., 2014).
The high rates of trauma and PTSD among incarcerated men are less surprising than the lack of screening and treatment for these conditions in incarcerated settings. To date, most of the expanding but limited attention on trauma and PTSD in corrections has focused on women (Federal Partners Committee on Women and Trauma, 2013), even though men comprise 93% of the incarcerated population (Carson & Sabol, 2012). Only a minority of male inmates receive treatment for behavioral health disorders while incarcerated (Ditton, 1999; Peters & Matthews, 2003). Focusing on the male population is critical to the rehabilitation effort of modern corrections, especially in light of the high correlation among trauma, behavioral health problems, and criminality (Wolff & Shi, 2009).
Background
Developing cost-effective screening strategies is essential for population-wide diffusion. Screening must impose minimal fiscal and staff burden to be adopted by departments of corrections that are facing pressures to lower costs. For this reason, we explored the feasibility, reliability, and validity of computer-administered screening for PTSD symptoms among incarcerated men. This modality of screening has been tested using cognitive and non-cognitive survey and screening instruments in health clinics, mental health clinics, and educational settings with psychiatric, low-income, minority, and low computer literacy populations (Campbell et al., 1999; Richman, Kiesler, Weisband, & Drasgow, 1999). There has been no study to our knowledge testing this modality among incarcerated populations, although they share many of the characteristics of populations used in previous studies (e.g., HIV-positive individuals, injecting drug users, people with serious mental illnesses, substance users).
Advantages most frequently identified for computer-administered interviewing (CAI) focus on efficiency and validity. In terms of efficiency, because CAI is administered by computer, minimal staff and training are needed to administer or score the instrument, although there are immediate start-up costs for software and perhaps computers if none are available. Agencies with information management systems may, with minimal investment, integrate CAI with other databases, such as inmate clinical records, to improve case management and treatment planning. Efficiency is also enhanced by the conservation of paper and data handling (e.g., transcription and data entry).
In terms of validity, self-report data collected by CAI may be more complete and accurate, increasing the quality of the data. Administratively, data quality is improved in part because non-response bias is nearly eliminated by computer formatting and sequencing; in part because random and non-random interviewer-bias is eliminated with standardization of question formatting; and in part because with self-pacing, users have more time to reflect as needed prior to answering questions (Chang & Krosnick, 2009; Nicholls, Baker, & Martin, 1997). The accuracy of self-report data is also expected to improve if CAI is perceived as being a more private mode for reporting behaviors that are socially undesirable or stigmatizing (e.g., sexual dysfunction, illegal behavior, risky sexual behavior, sexual or physical abuse, abortion). With CAI, respondents are less motivated to answer questions in more socially desirable or undesirable ways because there is no social pressure or context to motivate distortion. Intentional impression management is minimized to the extent that respondents feel more anonymous (i.e., no one is directly observing and perhaps judging their responses) when answering questions administered by computer. Indeed, there is an extensive literature showing a negative association between anonymity and social desirability distortion, and a positive association between anonymity and self-disclosure (Richman et al., 1999).
Validity, however, may be decreased with computer administration if respondents feel threatened by having identifiable information about them stored in a computer (known as the “big brother syndrome”; Rosenfeld, Booth-Kewley, Edwards, & Thomas, 1996) or misunderstand or misinterpret questions that could be clarified by an interviewer. Clarification of questions by interviewers, however, introduces the possibility of interviewer-bias as interviewers may differ in their explanation of survey questions or provide inappropriate cues in their explanations that could distort responses (Fowler, 1990).
The evidence on the relative quality of data reported by CAI and traditional formats (TF; for example, paper-and-pencil, face-to-face interviews, oral administration) is mixed and depends chiefly on the type of data being collected. A meta-analysis of 28 studies comparing CAI and face-to-face interviewing (FFI) found that there was less social desirability distortion with CAI than with FFI when the survey probed highly sensitive personal information (Richman et al., 1999). In general, respondents tend to more completely and accurately report stigmatizing behaviors (i.e., behaviors that are shaming or embarrassing) with CAI compared with FFI, while the opposite holds for reporting psychological distress (DeLeeuw, Hox, & Kef, 2003; Ghamen, Hutton, Zenilman, Zimba, & Erbelding, 2005; Newman et al., 2002; Nicholls et al., 1997; Richman et al., 1999). Newman et al. (2002) argue that the “impersonal” nature of computers enhances the reporting of information that is stigmatizing but hinders the reporting of information that requires human engagement, such as feelings of depression, where empathy may be required to fully elicit information on feelings. The presence of an interviewer in a face-to-face condition may distort responses: While orally administering survey questions, social expectations may be created directly through subtle or nuanced voice intonation, hesitations, gestures, and facial expressions; indirectly through appearance and body language; as well as by the sense of being directly observed while responding to orally administered questions, reducing privacy (Richman et al., 1999; Sudman & Bradburn, 1974). Overall, research evidence suggests that CAI is a valid and reliable method for collecting sensitive survey and behavioral data, but not data on psychological distress, especially for respondents who are comfortable interacting with computers (Metzger et al., 2000).
Several instruments are available to screen for PTSD symptoms (National Center for PTSD, 2014), with the PTSD Checklist (PCL) being one of the most commonly used screens in studies with samples exposed to motor vehicle accidents, sexual assault, sexual harassment, cancer, natural disasters, and combat (Blanchard, Jones-Alexander, Buckley, & Forneris, 1996; Hoge et al., 2004; Krause, Kaltman, Goodman, & Dutton, 2007; Palmieri & Fitzgerald, 2005; Palmieri, Weathers, Difede, & King, 2007; Shelby, Golden-Kreutz, & Andersen, 2005). It also has been used in studies of trauma and PTSD among people with serious mental illnesses (Mueser et al., 2001; Wolford et al., 2008) and incarcerated veterans (Saxon et al., 2001). The PCL—a 17-item self-report measure of PTSD symptoms based on Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association [APA], 1994) criteria—uses a 5-point Likert-type scale format generating a score ranging from 17 to 85, with higher scores indicating greater symptom severity, to assess presence of current PTSD symptoms (Wilkins, Lang, & Norman, 2011). The psychometric properties of the PCL are robust for internal consistency, test–retest reliability, and convergent validity (Blanchard et al., 1996; Keen, Kutter, Niles, & Krinsley, 2008). Comparative studies of PCL administered by computer and other modalities (e.g., face-to-face interview, paper-and-pencil self-administration) have demonstrated psychometric equivalence (Campbell et al., 1999; Wolford et al., 2008).
No studies have assessed the feasibility, reliability, and validity of using CAI to screen for psychiatric disorders among people who are incarcerated. Standard screening practice for psychiatric conditions in prisons is the FFI conducted by correctional mental health care staff. For this reason, we compare the relative psychometric performance of computer versus oral administration of screening for PTSD among incarcerated men. While the literature suggests that CAI is positively perceived by respondents and yields data quality that is equal to or better than interviewer-involved administration when eliciting information about stigmatizing behavior, it is unclear whether these findings will apply to screening for PTSD among incarcerated men for several reasons. First, incarcerated people do not have access to computers and, for those incarcerated since the 1980s, they have had minimal or no experience using a computer with a mouse device. As such, incarcerated men, particularly older men, may be uncomfortable with computer administration, reducing their willingness to participate in computerized screening. Feelings of discomfort with the technology may also trigger suspicion if computer technology is seen as threatening safety. Not knowing where the information goes and how it will be used after being entered into the computer can cause some people to be distrustful of the alleged privacy and anonymity expected with computer administration (Rosenfeld et al., 1996). Incarcerated people, by virtue of their custody status and environment, are reflexively suspicious, which may make them less candid when responding to the computer-administered survey questions.
Second, CAI may not perform optimally because the PCL instrument is tapping into behaviors that are stigmatizing (e.g., sexual trauma, childhood abuse) and feelings that are distressful. The literature suggests that CAI outperforms interviewer-involved administration when questions probe stigmatizing behaviors but interviewer-involved administration outperforms CAI when questions probe psychological distress. Which effect dominates in the administration of the PCL will depend on how incarcerated men respond to questions that probe both stigmatizing behaviors and psychological distress. To the extent that incarcerated men are more inclined to hide their feelings to protect their vulnerability, they may be more comfortable revealing their feelings and experiences to an impersonal computer rather than in the presence of an interviewer who may arouse evaluation apprehension.
Current Focus
The purpose of this study is to test the feasibility, reliability, and validity of using CAI versus orally administered interview (OAI) to screen for PTSD among incarcerated men. Feasibility is tested by the ability to recruit incarcerated men to complete a computer-administered survey. Test–retest reliability is determined using a 2 × 2 factorial design with random assignment to one of four administration conditions: (a) CAI and CAI, (b) CAI and OAI, (c) OAI and CAI, and (d) OAI and OAI. Validity is assessed by comparing PCL scores on symptom severity with the Clinician-Administered PTSD Scale (CAPS), which is widely acknowledged as the gold-standard measure for PTSD diagnosis (Blake et al., 1990; Weathers & Litz, 1994).
Method
This study screened for PTSD among male residents housed at a high security prison operated by the Pennsylvania Department of Corrections from March to June 2012. The primary focus was to compare CAI screening with OAI screening for PTSD symptoms. The protocols for recruitment and interviewing were approved by the appropriate institutional review boards. All participants signed informed consent forms after the conditions of participation (including confidentiality, duty to inform, privacy, risks, benefits, and right to withdraw or refuse to answer questions) were reviewed with them by research staff. Participants received a calendar for completing the first interview and a reentry manual customized to Pennsylvania for completing the second interview.
Participants
Residents eligible for the screening were 18 years or older and had at least 10 months remaining on their mandatory minimum sentence to be completed at the host facility (to ensure sufficient time to complete the parent study prior to release). Excluded were residents with active psychosis or organic brain impairment (limiting the ability to give informed consent), or currently on or had been on suicide watch in the past 3 months. According to prison administrative records, of the estimated 4,000 residents, 1,887 were eligible for the study. Half of these men were randomly invited to be screened, and 592 (63%) gave written consent and participated in the screening interviews. Those who declined mentioned several reasons for not participating, including not being ready to address trauma issues, expecting to be released or transferred, or scheduling conflicts with other required programs. Of the 592 screened participants, 61 were ineligible for the reliability and validity analysis because they did not meet the inclusion criteria (retest was completed outside the 14-day evaluation period, n = 57; missing retest, n = 2; required reading assistance on the computer, n = 2). Of the 531 eligible cases, the first 100 participants for each modality (CAI–OAI, OAI–CAI, CAI–CAI, OAI–OAI) were selected for analysis to ensure balanced group sizes. The OAI–OAI group only had 96 eligible cases, limiting the analysis to 100 per modality.
Design
The study was conducted from March through June 2012. A 2 × 2 factorial design (inclusive of parallel and crossover interview assignments) was used to randomly assign participants to screening modality and order of modality screening for the test and retest sessions. Therefore, at the initial interview, participants were randomly assigned to one of four test–retest interview conditions: OAI and OAI, n = 96; CAI and CAI, n = 100; CAI and OAI, n = 100; and OAI and CAI, n = 100. Once assigned a number that indicated modality assignment, participants sat at a laptop computer and completed questions about criminal history and demographic characteristics, followed by a survey on trauma history. After completing these questions, and depending on their modality assignment, participants either continued to complete the screening instruments on the computer or were relocated to another room where the screening instruments were orally administered by experienced interviewers. At the second (retest) interview, conducted within 14 days (M = 6.6 days), the screening instruments were administered again in accordance with group assignment. For the validity part of the study, a clinical interview (CAPS) was conducted; participants were interviewed face-to-face by interviewers for approximately 1.5 hr. CAPS interviews were conducted within 7 to 14 days of the screening retest PCL for a group of randomly selected participants who screened positive (n = 105) or negative (n = 36) for PTSD.
Administration Modality: Computer Interview Administration
Questionnaire Development System software and computer-assisted self-interviewing (CASI) technology were used to administer the surveys by laptop computers with mouse devices. The surveys were available in English. Two research assistants were in the room with 25 computer stations. One research assistant issued a unique code and recorded date of participation, while the other assistant logged the participant into the computer by a twice-entered code unique to the participant and provided guidance on how to use the computer. Research staff was available throughout the session to provide assistance with the computer as needed and to save the survey and clean the work station at the end of the interview. The research staff did not interpret survey questions or watch the person respond to specific survey questions. Question responses included “refuse to answer” and “not applicable.” Each question was provided on a single screen, and participants could move backward and forward among questions.
Administration Modality: Clinician Interview Administration
Screening interviews were orally administered by seven master’s-level, clinically trained social workers or psychologists and one bachelor’s-level researcher with 3 years of experience administering psychological instruments. The interviewers were trained and supervised by doctoral-level researchers with experience administering the instruments in clinical and research settings. Interviews were conducted in private rooms. In the reliability phase, interviewers read the same questions appearing in the computerized version of the screening instruments and in the same order. They did not interpret or clarify the questions. Questions were read verbatim. The participant recorded the appropriate response on the paper questionnaire after the question was read to him in the presence of the person reading the questions to them. Scales for each question were presented on large laminated cards that were placed in front of the participant. Participants could refuse to answer a question or indicate that the question was not applicable to them. Completing the PTSD screening instrument took approximately 5 to 10 min. In the validity phase, interviewers conducted a clinical interview where questions were asked and probed by scripted follow-up questions to elicit information about levels of distress, types of feelings, duration of affect states, and so forth. These interviews lasted 60 to 120 min. Participants could refuse to answer a question or indicate that it did not apply to them.
Measures
The first screening interview included background questions on criminal history, demographics, and trauma history, followed by the screening instruments for PTSD symptoms and substance abuse problems (the screening for substance abuse problems is beyond the scope of this article). Background questions focused on incarceration history, followed by demographic characteristics including education, race/ethnicity, age, citizenship, veteran status, marital status, number of minor children, and employment history. Participants also completed the Trauma History Questionnaire (THQ) to assess trauma history (Green, 1996) and the PTSD Checklist-Civilian version (PCL-C; Wilkins et al., 2011). The PCL-C was completed by those who reported at least one traumatic event on the THQ (only two participants reported no traumatic events in their lifetimes).
At the second interview, the CAPS and Structured Clinical Interview for DSM-IV (APA, 1994)–Non-Patient Version with Psychotic Screen (SCID-NP; Blake et al., 1990; First, Spitzer, Gibbon, & Williams, 2002; Weathers & Litz, 1994) were used to assess psychiatric disorders. The CAPS, the gold-standard measure for the diagnosis of PTSD, was used to diagnose lifetime and current full or sub-threshold PTSD (Blake et al., 1990; Weathers & Litz, 1994). The CAPS diagnosis of PTSD is based on the DSM-IV (APA, 1994) criteria. Diagnostically, full PTSD was defined as follows: a qualifying traumatic event, one re-experiencing symptom, three avoidance symptoms, and two arousal symptoms with the presence of co-occurring significant distress or impairment in functioning. Sub-threshold PTSD classification required a qualifying traumatic event, one re-experiencing symptom, and either three avoidance or two arousal symptoms with the presence of co-occurring significant distress or impairment in functioning (Grubaugh, Elhai, Cusack, Wells, & Frueh, 2007). Interviews were conducted in private rooms. All the clinical measures are commonly used to assess traumatized populations (Elhai, Gray, Kashdan, & Franklin, 2005) and were scored in standard fashion.
Data Analysis
Proportions and means (with standard deviations) were computed to describe demographic and background characteristics. The significance level used to test differences was .05, with 95% confidence intervals (CIs). We used Proc means, freq, t test, corr, and logistic in SAS 9.2 and reliability analysis in SPSS Version 21 for data analyses.
Test–retest reliability for the PCL was assessed by intraclass correlation coefficients (ICCs) using procedures described by McGraw and Wong (1996) with 95% CIs. The intraclass correlations were calculated using a two-way mixed reliability model; participants were assessed at two points in time (by randomly selected raters and/or the computer). Reliability was evaluated using the following classifications: strong (r = .80 or above), moderate (r = .50-.79), and weak (r = lower than .50; Devore & Peck, 1993). These test–retest reliability analyses of the PCL were conducted for the total score and for the three PTSD cluster scores. Order effects were assessed by repeated-measures ANOVA, with test modality (OAI vs. CAI) and retest modality (OAI vs. CAI) as between-subjects factors and time as the repeated measure (test vs. retest). The Pearson correlation coefficient (r) was used as an index of effect size, with a value greater than .5 indicating a large effect (Cohen, 1988).
The CAPS was used to assess the criterion validity of PTSD screening. Pearson’s correlation coefficients were computed between the PCL total score and both the CAPS total score and the CAPS intensity sub-score. The conceptual reason for the comparison with the CAPS intensity sub-score was that it more closely matched the construct assessed by the PCL (Weathers, Keane, & Davidson, 2001; Weathers, Ruscio, & Keane, 1999). In addition, logistic regression was used with current CAPS diagnosis as the dependent variable (1 = full or sub-threshold PTSD, 0 = no PTSD) and PCL-C score as the independent variable. We assessed how the PCL scores related to CAPS diagnosis and the corresponding classification rates. Sensitivity, specificity, positive predicted value, negative predicted value, and the proportion of participants correctly classified were calculated to find the optimal cut-point for the PCL-C score for determining sufficient symptom severity that PTSD is likely for this population. Receiver operating characteristics (ROC) analysis was performed to assess the diagnostic accuracy of the PCL-C score against the CAPS compared with no discrimination (diagonal line).
Results
On average, the study sample of 396 incarcerated men was 43 years old, African American, at least high school graduates or equivalent, and non-Veterans. Most were serving time for a violent offense and had served on average 15 years in prison since turning 18 (see Table 1). The sub-samples randomly assigned to the different combinations of survey modality did not differ statistically in their demographic characteristics with four exceptions: The OAI–CAI sample was several years older, more likely to be college educated, and spent more time incarcerated since age 18, while the CAI–CAI sample was more likely to have some college, compared with the OAI–OAI sample.
Demographic and Criminal History Characteristics of Incarcerated Men.
Note. CAI = computer-administered interviewing; OAI = orally administered interviewing; PTSD = posttraumatic stress disorder; PCL-C = PTSD Checklist–Civilian version.
Modality assignment refers to the administration of the PCL-C by computer (CAI) or interviewer (OAI) and the order of administration, first or second. For example, CAI–CAI indicates computer-administered interviewing of the PCL-C for the first and second test, whereas CAI–OAI indicates computer administration of the PCL-C first and oral administration by interviewer second.
Sample percentages based on 1% to 2% of missing data.
Sample percentages based on 4% of missing data.
p < .05 comparing CAI–CAI, CAI–OAI, and OAI–CAI to OAI–OAI using t test or chi-square test.
Feasibility
All participants completed at least the baseline portion of the survey on the computer. Less than 1% of participants expressed difficulty reading the survey on the screen. In these cases, a research assistant read the questions on the screen to the participants so they could complete the CASI surveys (these subjects were not included in the reliability or validity analysis). There was no problem with computer literacy even among participants older than 50. Within several minutes of instruction, they were able to maneuver the mouse without difficulty, with one notable exception. One elderly participant (older than 75) was unable to maneuver the mouse and was assisted by research staff. Overall, participants were intrigued by the computers and were eager to use them to answer survey questions.
Reliability
Each participant was screened twice to examine the test–retest reliability of the PCL-C total score and each sub-scale by modality combination. The ICC for the PCL-C total score for the four modality combinations ranged from .774 to .817, indicating moderate to strong reliability for each modality combination (Table 2). ICCs for the sub-scales (PTSD clusters) were similar, varying between .627 and .805. The 95% CIs overlapped substantially for the PCL-C total score and the sub-scale scores among the four modality combinations. Thus, order and modality of administration of the PCL-C did not affect test–retest reliability as assessed by the ICCs. Pearson’s correlation coefficients were calculated to assess the effect size of the associations shown in Table 2 (Cohen, 1988). The Pearson correlation coefficients for the PCL-C sub-scales ranged from .64 to .81 for the sub-scales and .77 to .82 for the total scores, indicating a large effect size (data are available from the first author upon request).
Test–Retest Reliability Based on the Intraclass Correlation Coefficients and Their Respective 95% CIs for PCL-C Subscales and Total Score by Modality Assignment.
Note. CI = confidence interval; PCL-C = Posttraumatic Stress Disorder Checklist–Civilian version; CAI = computer-administered interviewing; OAI = orally administered interviewing.
Modality assignment refers to the administration of the PCL-C by computer (CAI) or interviewer (OAI) and the order of administration, first or second. For example, CAI–CAI indicates computer-administered interviewing of the PCL-C for the first and second test, whereas CAI–OAI indicates computer administration of the PCL-C first and oral administration by interviewer second.
We also examined order and modality effects for the PCL-C total score based on repeated-measures ANOVA of group mean scores. The between-subjects design was 2 × 2 factorial, with one factor being the modality for the test session (OAI vs. CAI) and the second factor being the modality for the retest session. Time (test vs. retest) was the repeated measure (within-subjects factor). Figure 1 shows the mean PCL-C total score at test and retest for the four modality combinations. Only two effects were significant at p < .05. The significant main effect for time indicated that PCL-C scores were lower at retest than at test, but this effect was modified by a significant time by retest modality interaction. Essentially, if the retest modality was OAI, there was a drop in mean PCL-C total score between test and retest, but if the retest modality was CAI, there was no difference between mean test and retest scores, F(1, 392) = 9.78, p = .002. The sample size provided ample statistical power for these analyses, and we note that the differences in mean PCL-C total scores were less than the minimum for clinical significance, which is 10 to 20 points (Monson et al., 2008; National Center for PTSD, 2012).
Mean PCL-C Total Scores at Test and Retest by Modality Assignment
Validity
Validity analysis compared total scores on the PCL-C administered closest to the date of the CAPS interview, with the CAPS diagnosis of PTSD used as the criterion measure (yes = full or sub-threshold PTSD, no = no PTSD). The CAPS was conducted within 14 days of the PCL-C (a reasonable time frame for a correctional mental health service system). Overall, for the CAI group (n = 70), scores on the PCL-C ranged from 17 to 74 (M = 44.2, SD = 14.7), compared with a range of 20 to 79 (M = 48.1, SD = 14.1) for the OAI group (n = 71). The mean PCL-C total scores were not significantly different between the two modality groups, t(139) = −1.61, p = .109.
Whether CAI and OAI yield equally valid scores was examined in two ways. First, Pearson’s correlations were calculated for the total score and intensity sub-score of CAPS vs. total PCL-C scores by administration modality. The total score for the CAPS was significantly correlated with PCL-C total scores for the CAI (N = 70, r = .570; p < .0001) and OAI (N = 71, r = .655; p < .0001) conditions. Similarly, the intensity sub-scores for the CAPS were significantly correlated with PCL-C total scores for the CAI (N = 70, r = .537; p < .0001) and OAI (N = 71, r = .634; p < .0001) conditions.
Second, logistic regression and ROC analysis were used to assess the effect of PCL assessment modality on the association with CAPS diagnosis. Logistic regression findings showed that, for the CAI group, the association between the PCL-C score and CAPS diagnosis of PTSD was significant, χ2(1, 70) = 22.05, p < .0001; r2 = .2749, rescaled r2 = .3669. The regression coefficient for the CAI-PCL-C was .093 (SE = 0.02), with an odds ratio of 1.10, 95% CI [1.05, 1.15]. For CAI sample, 80.8% of the 70 participants were classified correctly based on their PCL-C total score. For the OAI group, the PCL-C was also significantly associated with the CAPS PTSD diagnoses, χ2(1, 71) = 15.57, p < .0001; r2 = .1969, rescaled r2 = .2693. The regression coefficient for the OAI-PCL-C was .080 (SE = 0.02), with an odds ratio of 1.08, 95% CI [1.04, 1.13]. For OAI sample, 74.7% of the 71 participants were classified correctly based on their PCL-C total score.
The ROC curves for the CAI and OAI are shown in Figures 2 and 3, respectively. The ROC curves are above the diagonal center line (i.e., the line of no discrimination), indicating good classification discrimination for both CAI (area under the curve [AUC] = 0.81, SE = 0.06, p < .0001) and OAI (AUC = 0.75, SE = 0.06, p < .0001). Table 3 shows the sensitivity and specificity rates, plus the percent PTSD accurately identified (i.e., overall diagnostic efficiency or ODE), positive predicted value, and negative predicted value, by cut-point on the PCL-C for diagnosing PTSD based on the CAPS current diagnosis. For the CAI group, 80% of participants with a CAPS diagnosis of PTSD would be validly identified with a cut-point of 43 (based on the ODE) on the PCL-C, with a sensitivity (proportion of true PTSD diagnoses) of 0.89 and a specificity (proportion of true-negative diagnoses) of 0.70. At that cut-point based on the optimal ODE, the positive predicted value (probability that a person has PTSD when PCL-C is at or above a cut-point) is 0.77, and the negative predicted value (probability that a person does not have PTSD when the PCL-C is at or below the cut-point) is 0.85. For the OAI group, the optimal cut-point score is 37 on PCL-C, at which 79% of participants with a CAPS diagnosis of PTSD would be correctly identified, with a sensitivity of 0.96, specificity of 0.50, positive predicted value of 0.77, and negative predicted value of 0.87.
ROC Curve for the CAI-PCL-C to the CAPS Diagnosis of PTSD
ROC Curve for the OAI-PCL-C to the CAPS Diagnosis of PTSD
Comparative Cut-Points and Contingency Estimates by Time Between Screen and CAPS Interview and CAPS Criterion.
Note. CAPS = Clinician-Administered PTSD Scale; PCL-C = Posttraumatic Stress Disorder Checklist–Civilian version; PTSD = posttraumatic stress disorder; PPV = positive predicted value; NPV = negative predicted value; CAI = computer-administered interviewing; OAI = orally administered interviewing.
CAI denotes computer administration of the interview.
OAI denotes oral administration of the interview by an interviewer.
Additional analyses were conducted using the CAPS criterion of full PTSD only and a 0-to-7-day period between PCL-C screen and CAPS interview (see Table 3). The validity findings were consistently robust and equivalent when the CAPS criterion was set to full PTSD only. The optimal cut-point for the PCL-C, however, increased for OAI samples. Changing the time period between screening and CAPS interview from 0 to 14 days to 0 to 7 days improved the accuracy of PTSD identified while lowering the optimal cut-point to 39 for CAI but raising the cut-point for OAI to 41 when the CAPS criterion includes full and sub-threshold PTSD. For the OAI group, the optimal cut-point increased significantly for CAPS criterion of full PTSD to 56, which is likely a result of the relatively small sample size.
Discussion
This study explored the psychometric equivalence of the PCL-C administered by computer versus orally by an interviewer to determine whether the computer-administered PCL-C could be used as a useful, feasible, reliable, and valid substitute for person-administered screening to identify PTSD among incarcerated men. The efficiency gains of computer-administered screening enhance its administrative appeal, but only if clinical effectiveness is demonstrated. The goal of screening is not to efficiently screen for a treatable condition, but rather, to maximize the number of people who are accurately identified as needing treatment for a particular condition, such as PTSD. For this reason, we explored whether computer-administered screening was feasible (i.e., incarcerated men were willing and able to respond by computer), reliable (i.e., both methods yielded equivalent PCL-C scores), and valid (i.e., both modalities equally differentiated between diagnostic groups: PTSD and no PTSD). Findings presented herein provide general support for the feasibility of computer-administered screening for PTSD and for the psychometric equivalence of computer- and person-administered PCL-C for incarcerated men.
Overall, we found that incarcerated men, independent of age, were willing and able to report information about their PTSD symptoms that were distressing to both a computer and a clinician, and their reporting was equally complete and yielded similar total and sub-scale scores regarding the severity of their symptoms. There was no evidence of differences in intentional impression management between the modalities for eliciting information. Also, participants did not report suspicion about information entered into the computer or reported to clinicians, which may reflect in part the assurances of privacy and confidentiality that they received during the human subjects consent process, and in part their readiness to reveal distress that had been historically concealed.
In terms of the temporal stability of the ratings on the PCL-C, both modalities yielded high and nearly identical values for test–retest reliability. Moreover, analysis of order effects showed that mean differences were few across the modality conditions, and none of them approached clinical significance. The only significant modality difference among the scores was lower retest means if PCL-C was orally administered, compared with computer-administered. We conclude that either modality can be expected to yield nearly identical mean PCL-C total and sub-scale scores.
Whether the modalities are comparable in terms of discriminating between diagnostic groups with and without PTSD was explored in several ways, and all yielded the same general conclusion: Both modalities produce adequate diagnostic prediction. The difference between modalities relates less to their diagnostic accuracy and more to the optimal cut-point to determine a true positive diagnosis of PTSD. In general, the optimal cut-point varied by screening modality, definition of PTSD (i.e., full PTSD only, full and sub-threshold PTSD), and time lapse between screening and clinical interview (i.e., 0-7 days, 0-14 days). In general, the optimal cut-point for computer-administered screening ranged from 43 (either sub-threshold and/or full PTSD and 0-14 days) to 39 (either sub-threshold and/or full PTSD and 0-7 days), whereas the range for face-to-face screening was 56 (full PTSD and 0-7 days) to 37 (full and sub-threshold PTSD and 0-14 days).
The optimal cut-point estimated for civilian populations is 44 (Blanchard et al., 1996; Ruggiero, Del Ben, Scotti, & Rabalais, 2003), but it varies by population characteristics (Keen et al., 2008), with PCL cut-points recommended as low as 28 to 30 for female veterans in a primary care setting (Lang, Laffaye, Satz, Dresselhaus, & Stein, 2003) and 38 to 44 for women in residential treatment for substance abuse (Harrington & Newman, 2007). Ultimately, the “optimal” cut-point for a population depends on whether the objective of screening is to maximize the number of true cases (i.e., high sensitivity) or minimize the number of false positives (i.e., high specificity). Given the high correlation among PTSD, substance abuse, and criminality in correctional populations (Wolff & Shi, 2009) and the implications for public health and public safety, we would argue for a cut-point that includes full and sub-threshold PTSD, suggesting a cut-point of 43 (0.892 sensitivity) or 39 (0.955 sensitivity) for computer-administered PCL-C screening depending on the time between screening and clinical interview.
Our findings are consistent with those reported by Wolford et al. (2008), in which they tested equivalence between computer- and interviewer-administered screening for PTSD among patients with severe mental illnesses. They demonstrated that computer-administered screening for PTSD symptoms yielded equally reliable and valid data compared with interviewer-administered screening, and had the added advantage of producing data less expensively and faster. Similarly, patients did not have difficulty using the computer.
As with all studies, there are limitations that must be considered. First, our sample, while of robust size, was limited to men at a single maximum-security prison. Rates of trauma exposure in maximum-security prisons may not be representative of prisons with lower security levels. We explored this issue with trauma exposure data drawn from incarcerated men residing in 13 prisons within a single state system. Comparing three prison populations, one with low representation of violent crime convictions (39% of population had a violent conviction, n = 830) and two with high representation of violent convictions (63% and 85% had violent convictions, n = 987), we found no significant differences in rates of physical or sexual victimization during childhood or adulthood (prior to incarceration) between the prisons (data available from first author upon request). While this issue merits more careful investigation, in the absence of evidence to the contrary, screening strategies for PTSD should not vary by prison security level.
Sample bias is also possible. The study sample volunteered to be screened for trauma history, which may have introduced recruitment bias. Our sample may be more representative of incarcerated men who are ready to acknowledge their PTSD distress and to seek treatment for this distress, and as such, they may not be representative of the full population of men housed at that particular maximum-security prison, which in turn may not be representative of all prisons in the state system. As with all studies of this type, sample bias challenges generalizability but only in ways that temper the rigidity in which these findings are interpreted and applied. At most, our study suggests that there is psychometric equivalence between computer- and orally administered screening for PTSD using the PCL-C. This result encourages the implementation of computer-administered screening to efficiently and effectively identify incarcerated men who would benefit from PTSD treatment. Depending on the degree of risk aversion (e.g., not identifying a true case), prison systems may choose to lower cut-points to increase the proportion of participants correctly diagnosed and treated.
Second, this study recruited incarcerated men who could speak and read English. Our findings do not extend to incarcerated women or incarcerated people who are not literate in English. An important next step would be to explore the validity and reliability of computer-based screening compared with orally administered screening in non-English-speaking and female incarcerated samples. Third, the research staff who conducted the CAPS and PCL-C interviews were highly trained and closely monitored for fidelity and interpersonal civility. Their training and style of interaction with participants may not be representative of department of corrections clinical staff and, as a consequence, may have improved performance of the interviewers (and the quality of the data reported) compared with the typical clinical screener in the correctional system. If correctional staff is perceived as less trustworthy or is less skilled at eliciting psychiatric distress, computer-administered may be superior to face-to-face screening. The strengths of our study, however, offset these limitations to generalizability and suggest cautious application.
Conclusion
In conclusion, our study indicates that incarcerated men are willing and able to reveal information about their PTSD symptoms whether engaged by a computer or a clinician, and the information reported is equally complete and valid. Computer-administered screening for PTSD using the PCL-C has the potential to reliably and accurately identify incarcerated men who would benefit from evidence-based treatment for PTSD. Using a wide net to identify true positives has merit because of the high prevalence of trauma exposure among incarcerated men and the high correlation among PTSD, substance use, and criminality. If the goal is to prepare incarcerated men for the community, using screening methods for PTSD that are efficient, reliable, and valid merit adoption in an effort to protect the public and improve the welfare of those who are incarcerated.
Footnotes
Acknowledgements
The authors acknowledge the constructive and invaluable feedback from three anonymous reviewers and the unstinting support and assistance from the professional staff of Graterford State Correctional Institution and the Pennsylvania Department of Corrections.