Abstract
Forensic evaluators may be assisted by comparing their use of instruments with that of their peers. This article reports the results of a 2017 survey of instrument use by forensic evaluators carrying out sexual recidivism risk assessments. Results are compared with a similar survey carried out in 2013. Analysis focuses primarily on adoption of more recently developed instruments and norms, and on assessment of criminogenic needs and protective factors, and secondarily, on exploring factors related to differences in evaluator practice. Findings indicate that most evaluators have now adopted modern actuarial instruments, with the Static-99R and Static-2002R being the most commonly used. Assessment of criminogenic needs is now common, with the STABLE-2007 being the most frequently used instrument. Evaluators are also increasingly likely to consider protective factors. While a majority of evaluators uses actuarial instruments, a substantial minority employs Structured Professional Judgment (SPJ) instruments. Few factors discriminated patterns of instrument use.
It is widely accepted that the practice of risk assessment should be informed by research. Professionals who wish their practice to be empirically based will need to periodically revise their methodologies in light of new research findings. However, research related to sexual risk assessment evolves rapidly. In the past decade, for example, research related to the Static-99 has led evaluators to consider age as a protective factor and to begin using the Static-99R (Helmus, Thornton, Hanson, & Babchishin, 2012), and to consider factors external to the Static-99/99R that are empirically related to sexual risk (e.g., criminogenic needs, treatment change, protective factors). This has led to increased use of instruments designed to measure these factors, such as the STABLE-2007 (Hanson, Harris, Scott, & Helmus, 2007) and the Violence Risk Scale–Sex Offense version (VRS-SO; Olver, Wong, Nicholaichuk, & Gordon, 2007). Existing instruments and manuals have been revised and updated requiring users to obtain additional training. For example, The Minnesota Sex Offender Screening Tool–3 (MnSOST-3; Duwe & Freske, 2012) debuted in 2012, with the fourth edition following in the summer of 2017 (MnSOST-4; Duwe, 2017).
Adopting new instruments into existing methodologies can be difficult because learning to score and interpret instruments takes time and effort. This can be potentially costly and time-consuming for some professionals, especially those whose employers may have limited financial resources and those in private-practice settings who are paid per diem. Furthermore, evaluators’ choice in instruments may be constrained by their employment setting (e.g., an employment contract allows only certain risk tools) as well as potential legal standards for admissibility if the results will be used in a court setting (e.g., Daubert; Frye). Given that evaluators are likely to modify their risk assessment practices at differing rates, surveys are useful in helping them monitor how their practices are similar or dissimilar to overall trends in the field, so they become aware of when their methodologies have become markedly inconsistent with others and need to be modernized.
There have been a number of surveys of evaluator practices in recent years. However, many surveys are focused on instruments used to assess general violence (Pham et al., 2016; Singh et al., 2014; Singh, Grann, & Fazel, 2011) or focus only on highly specific populations such as those who are civilly committed pursuant to Sexually Violent/Dangerous Person (SVP) and Dangerous Offender laws (Blais & Forth, 2014; Chevalier, Boccaccini, Murrie, & Varela, 2014; Doyle, Ogloff, & Thomas, 2011; Schneider et al., 2017). Other important surveys now reflect practices that have since greatly changed (McGrath, Cumming, Burchard, Zeoli, & Ellerby, 2010).
Neal and Grisso (2014) provided a thoughtful survey, which asked participants to answer questions about the two most recent forensic evaluations they had completed. They received responses on a variety of forensic evaluations (e.g., Competence to Stand Trial; Criminal Responsibility), as well as information related to 91 sexual recidivism risk assessments. While informative, the results collapsed reports of usage for the Static-99, Static-99R, Static-2002, and Static-2002R into one broad category called “Static.” As such, it does not indicate whether evaluators are using the most current version of instruments and, thus, staying current with the research. Also, given that their survey was designed to capture many different types of forensic evaluations, professional organizations specializing in individuals who have engaged in sexual offenses were not solicited for responses. Overall, their results indicate that the majority of the sexual risk assessments utilized an actuarial measure (82.4%) whereas far fewer utilized a Structured Professional Judgment (SPJ) measure (30.8%). Interestingly, actuarial measures are frequently used in other types of risk assessments. For example, although diagnostic assessments were frequently used for aid in sentencing evaluations, versions of the Level of Service Need Inventory (e.g., LSI/CMI; Andrews, Bonta, & Wormith, 2004) were utilized in about 18.7% of the reports whereas very few included an SPJ measure. Note that these percentages are approximations based on the available data provided in the article by Neal and Grisso (2014).
In 2013, the authors of the current article sought to develop a survey to determine which instruments were used for sexual recidivism risk assessments and to solicit responses from evaluators who were members of the Association for the Treatment of Sexual Abusers (ATSA; Kelley, Barahal, Thornton, & Ambroziak, 2017). Unfortunately, there were notable limitations. First, because the sample was primarily drawn from ATSA members who responded to the listserv, the participants may not have been a good representation of what most evaluators completing sexual recidivism risk assessments were doing. Second, while the survey was conducted in 2013, the results were not formally published until January 2017. Within that lag, there were many notable research advances that could have had an impact on risk assessment practices. These include the publication of the 2015 Static-99R norms as well as increased recommendations to consider risk factors external to the Static-99R (Hanson, Thornton, Helmus, & Babchishin, 2016). Furthermore, an online publication of the ATSA Practice Guidelines for the Assessment, Treatment, and Supervision of Individuals with Intellectual Disabilities and Problematic Sexual Behaviors was released in 2014 (Blasingame, Boer, Guidry, Haaven, & Wilson, 2014). These practice guidelines recommended the use of certain instruments (e.g., Assessment of Risk and Manageability for Individuals with Developmental and Intellectual Limitations who Offend Sexually [ARMIDILO-S]; Boer, Tough, & Haaven, 2004) with this population. Third, the 2013 survey did not ask questions we became particularly interested in, such as how evaluators consider protective factors, choose norms, and whether there are variables that may influence their choice of instruments. The current survey is an attempt to address these limitations and to obtain samples from multiple forensic professional organizations to better capture the larger practice of forensic sexual recidivism risk assessments.
The current survey was intended to address the following questions:
To what extent have professionals conducting forensic sexual recidivism risk assessments adopted more recently developed static actuarial risk instruments?
To what extent have professionals conducting forensic sexual recidivism risk assessments incorporated criminogenic needs instruments into their practice?
Have instruments used in forensic sexual recidivism risk assessment changed since 2013?
Are forensic sexual recidivism risk assessment practices changing with empirical advances? For example, are evaluators using the most current norms?
What influences evaluators’ choice of instruments?
For the purposes of the current study, we chose to focus on participants who complete sexual risk assessments on adults for the court as part of their primary or secondary employment. Professionals conducting forensic sexual risk assessments need to select instruments that meet a high standard to be admitted into evidence in a court proceeding in their jurisdiction (e.g., the procedure is published, contains a manual and a known error rate, demonstrates reliability and validity, and is generally accepted). Furthermore, the potential impact of using improper measures within a treatment context is going to affect the client to some degree but much less so than a forensic evaluator who chooses improper methodology, which leads the court to make decisions about the deprivation of liberty or prematurely releases individuals who are at high risk. There were no other restrictions in the current study.
Method
We report how we arrived at our sample size, all data exclusions, all manipulations, and all measures in the study.
Participants
Participants were recruited to complete an online survey through invitations on the ATSA Listserv and through e-mails sent to all members of the American Psychology and Law Society (AP-LS; Division 41 of the American Psychology Association) and the Sex Offender Civil Commitment Programs Network (SOCCPN). The online survey was also sent to a senior member of the International Association for the Treatment of Sexual Offenders (IATSO), requesting that it be distributed to IATSO members. However, unlike the formal access the authors obtained to disseminate the survey to the other three professional organizations, the attempt to solicit responses from IATSO members was informal, and it is unclear to what degree the survey reached its membership. As the survey was available through accessing a link within e-mails, it is possible that the survey was forwarded to professionals who are not members of any of the four organizations. These professional organizations were chosen because it was thought the members would have specific training and experience in completing forensic risk assessments with adults who have been charged with a sexual offense. Within the survey, respondents were also asked about membership in other professional organizations, which also had a focus on forensic risk assessment (e.g., American Academy of Psychiatry and the Law [AAPL]).
Survey responses were collected between March 2017 and June 2017. Data collection was ended when professionals ceased responding to the survey after two formal attempts of solicitation. In total, 145 participants responded to the survey. However, eight respondents identified that they only work with children or adolescents, and another 18 respondents reported that they do not complete risk assessments for the court. As such, these respondents were excluded from further analysis as the purpose of the study was to examine how forensic professionals assess sexual recidivism risk of adult males as opposed to only assessing treatment needs. This left a total of 119 participants. Although the entire sample indicated that they complete risk assessments for adults, this was not necessarily what they identified as their primary role. Almost three quarters (74.0%) identified themselves as evaluators and 19.3% as treatment providers. The rest of the sample fell into other professional categories including researchers, administrators, and community supervision agents. The majority of the sample had a doctoral degree (78.2%) followed by master’s level qualifications (15.1%). No professionals with medical degrees responded to the survey. The range of experience completing risk assessments among participants was quite large, ranging from a half year to 40 years. The average was 13.2 (SD = 9.3) years of experience (median years of experience = 12). The sample largely represented professionals from the United States of America (88.2%) with few respondents from Canada (7.6%) and elsewhere 1 (4.2%). Within the United States, 28.6% of the respondents reported working in two or more states. Overall, all geographical regions of the United States were well represented with participants working in the Northeast (36.1%), Midwest (47.1%), South (25.2%), and West (30.3%).
Participants could identify themselves as having more than one professional membership. There was equal representation from both ATSA (65.5%) and AP-LS (64.7%). SOCCPN and IATSO were less well represented (10.9% and 3.4%, respectively). Four individuals (3.4%) indicated having no membership in a forensic professional organization. Many of the participants identified being in two or more forensic professional organizations (39.5%). All but one who identified themselves as being involved in SOCCPN also identified themselves as having at least one other professional membership. The number of individuals who identified as being only involved in AP-LS (29.4%) was similar to the number who was only involved in ATSA (26.9%).
The majority of the participants worked in either a public-sector position (federal or state; 31.9%) or by themselves in private practice (35.3%). Other less common settings included group private practice, colleges or universities, and private institutions. Most participants worked with clients who were in the process of going through the court system (67.2%). Respondents also identified working with clients on probation or parole (40.3%), in a prison setting (31.1%), or in an outpatient setting (21.8%). Half of the respondents reported working with a civilly committed population pursuant to their state’s SVP statute (49.6%).
Participants reported keeping up-to-date with research and training in a variety of ways, which is depicted in Table 1. Overall, almost three quarters reported using at least four of the different ways to remain current on research.
How Participants Stay Aware of New Research and Training.
Note. Participants (N = 119) could choose as many activities as were relevant (Range = 1-9; Median = 5).
Most participants reported having independence in being able to choose their own risk assessment instruments. However, 23 participants reported that their selection had a large degree of oversight by the institution they worked for and, of this group, four participants reported that the institution chooses their assessment methodology, and the choice is nonnegotiable. The degree to which participants identified having independence in their instrument choices is further described in Table 2.
The Level of Independence and Flexibility in Participants’ Assessment Methods.
Note. N = 119. Employer includes institutions and contracts with other agencies.
Measure and Procedure
The survey was developed and implemented via SurveyGizmo, an online survey program. Invitations and requests to complete the surveys were sent via e-mail and included a link to SurveyGizmo. Embedded within the e-mail were relevant information about the survey and informed consent. All participants remained anonymous. A copy of the survey questions is available in the Online Supplemental Material. Prior approval for this study was obtained through the Institutional Review Board at Sand Ridge Secure Treatment Center, and formal permission to utilize the professional organization’s listserv or e-mail distribution list was obtained from ATSA, AP-LS, and SOCCPN.
Static actuarial scales were categorized as “Old” or “New” according to the degree to which they incorporated updated norms and provide scoring that better accounts for the protective effect of advanced age. The following static actuarial scales were categorized as Old: Rapid Risk Assessment for Sexual Recidivism Risk (RRASOR; Hanson, 1997); Static-99 (Hanson & Thornton, 2000); Static-2002 (Hanson & Thornton, 2003); Minnesota Sex Offender Screening Tool–Revised (Mn-SOST-R; Epperson, Kaul, Huot, Goldman, & Alexander, 2003); and Risk Matrix 2000 (RM2000; Thornton et al., 2003). The following scales were categorized as New: Static-99R (Helmus, Thornton, Hanson, & Babchishin, 2012); Static-2002R (Helmus et al., 2012); Mn-SOST-3; VRS-SO (static factors); and the Multisample Age-Stratified Table of Sexual Recidivism (MATS-1; Wollert, Cramer, Waggoner, Skelton, & Vess, 2010).
Scales were categorized as measuring criminogenic needs if they are intended to measure factors that are targeted within treatment and empirically related to the prediction of sexual re-offense risk. Criminogenic needs instruments fell into two categories: Actuarial measures of criminogenic needs and Structured Professional Judgment (SPJ) measures. Actuarial measures refer to instruments in which items are selected based on statistical research; there are firm coding rules for each of the items; and items are given numerical scores that are summed for a total score, which can be used to compare with existing norms. Examples of actuarial measures of criminogenic needs include the STABLE-2007 and the VRS-SO (dynamic factors). In contrast, SPJ measures contain items that are based on theory, literature review, and/or professional consensus. The coding rules for items provide a way to anchor ratings but scoring is conceptual rather than mechanical. The professional makes decisions about what weight to assign to items based on the clinical features of the particular case, and the final risk judgment is based on integrating the ratings with a case conceptualization of the individual’s risk and risk relevant scenarios. Examples of SPJ measures include the Sexual Violence Risk–20 (SVR-20; Boer, Hart, Kropp, & Webster, 1997) and Risk for Sexual Violence Protocol (RSVP; Hart et al., 2003).
Initial data analyses were completed for a prior conference presentation and before the database had been fully cleaned and organized (Kelley, 2017). The initial analyses were insufficient to fully answer all of the preplanned research questions for the current study. In preparing the current manuscript, none of the initial data that were generated were used. Rather, once the database was in a satisfactory form, we developed planned analyses to answer all of the research questions and to demonstrate an adequate level of consistency throughout. Although we had some idea of what to expect from prior analyses, we maintained the planned analyses (which sometimes differed from what appeared in the conference presentations) regardless of whether this changed the results of our analysis.
Results
Instrument Use
Static instruments
Participants were asked which instruments they used in the past year to measure static risk factors as well as which instruments they use routinely. They could select as many static risk instruments as was relevant to their methodology. The results can be found in Table 3.
Use of Static Risk Instruments.
Note. Participants (N = 119) could choose as many instruments as were relevant. CPORT = Child Pornography Offender Risk Tool (Seto & Eke, 2015); MATS-1 = Multisample Age-Stratified Table of Sexual Recidivism (Wollert, Cramer, Waggoner, Skelton, & Vess, 2010); MnSOST-R = Minnesota Sex Offender Screening Tool–Revised (Epperson, Kaul, Huot, Goldman, & Alexander, 2003); MnSOST-3 = Minnesota Sex Offender Screening Tool–3rd Edition (Duwe & Freske, 2012); RRASOR = Rapid Risk Assessment for Sexual Recidivism Risk (Hanson, 1997); RM2000 = Risk Matrix 2000 (Thornton et al., 2003); SVR-20 = Sexual Violence Risk–20 (Boer, Hart, Kropp, & Webster, 1997); VRS-SO = Violence Risk Scale–Sex Offense version (Olver, Wong, Nicholaichuk, & Gordon, 2007).
Overall, the Static-99R was more commonly used than any other static risk instrument with the majority of the participants having endorsed using it both in the past year and using it routinely. Individuals endorsed using the Static-2002R and SVR-20 in the past year to a lesser extent, although it is notable that routine usage was less frequent, suggesting that these scales were more inconsistently used. Some scales such as the CPORT were used to a greater extent in the past year than routinely. This makes sense given that certain instruments would only be utilized for certain referrals. For example, the CPORT would only be utilized when the referral question is related to Internet offending. Interestingly, slightly more participants reported using the Static-99R routinely than they reported use of this instrument in the past year. This may be the result of error when completing the survey, or it may reflect one or two participants who recently began using the Static-99R. The majority of the sample (84.9%) reported that they routinely utilize a New static actuarial instrument (i.e., Static-99R, Static-2002R, VRS-SO Static, Mn-SOST-3, and MATS-1). Fewer participants (16.0%) reported routinely utilizing an Old static actuarial instrument (i.e., Static-99, Static-2002, Mn-SOST-R, RRASOR, and RM2000).
Participants were asked about the use of multiple static actuarial risk instruments within the same assessment report. In total, 33 participants (27.7%) reported using multiple static risk instruments within one assessment. The majority of these participants (23 of the 33 participants) use both the Static-99R and Static-2002R. It is less common for participants to combine an Old static risk instrument (i.e., RRASOR or Static-99) with a New static risk instrument (i.e., Static-99R), although nine individuals out of the 33 reported doing so. Overall, the majority of participants (58.0%) reported using only one static risk instrument in their evaluations, and participants infrequently used three or four static risk instruments within the same evaluation (5.1%).
The current survey was distributed just 4 months after the new 2016 Static-99R coding manual was released (November 2016; Phenix et al., 2016). Most of the participants (71.4%) reported that they had already begun using the 2016 Static-99R coding manual. The majority of the participants (63.0%) also endorsed using the updated 2015 Static-99R norms (Hanson et al., 2016; Phenix, Helmus, & Hanson, 2015). A smaller group (23.5%) reported using the 2009 Static-99R norms that were last made available in the 2012 Evaluator’s Workbook (Phenix, Helmus, & Hanson, 2012), and a few (5.9%) reported using the original Static-99 norms that accompanied the initial Static-99 publication (Hanson & Thornton, 2000). Eight participants (6.7%) reported using local norms, which were identified as being from New York, Florida, Washington, and California. A few participants did not specify the source of their local norms. Of these eight participants who used local norms, six identified being in private practice and two identified working for a federal or state facility. Six identified completing SVP evaluations, and two identified working with a non-SVP population. Overall, the majority of the participants (67.2%) reported using only one normative source related to the Static-99/99R. Two participants (1.7%) reported using three normative sources, and one participant (0.8%) reported using five normative sources.
The 2016 Static-99R Evaluators’ Workbook currently has two reference groups evaluators can select when utilizing the 2015 norms (available at http://static99.org/pdfdocs/Evaluators_Workbook_2016-10-19.pdf). These include the Routine/Complete group and the High Risk/Needs (HR/N) group. Participants were asked how they select a reference group for a given case. While 29.4% reported using a “matching” method, they appear to be doing so in different ways. Some participants identified using historical selection factors (e.g., administrative screening decisions) to match cases to the reference group (11.8%), whereas others used clinical judgment to determine the density of criminogenic needs to match cases to the reference group (17.6%). A quarter of the participants (25.2%) reported that they only use the Routine/Complete group, and 21.0% reported using an actuarial measure of criminogenic needs to choose the reference group.
Criminogenic needs
Participants were asked which instruments they used to measure criminogenic needs in the past year as well as which instruments they used routinely. They could select as many criminogenic needs instruments as was relevant to their methodology. The results can be found in Table 4.
Use of Criminogenic Needs Instruments.
Note. Participants (N = 119) could choose as many activities as were relevant. ARMIDILO-S = Assessment of Risk and Manageability for Individuals with Developmental and Intellectual Limitations who Offend Sexually (Boer, Tough, & Haaven, 2004); MIDSA = Multidimensional Inventory of Development, Sex, and Aggression (MIDSA, 2011); RSVP = Risk for Sexual Violence Protocol (Hart et al., 2003); SARN = Structured Assessment of Risk and Need (S. Webster et al., 2006); SOTIPS = Sexual Offender Treatment Intervention and Progress Scale (McGrath, Lasher, & Cumming, 2012); SRA-FV = Structured Risk Assessment–Forensic Version (Thornton & Knight, 2015); SVR-20 = Sexual Violence Risk–20 (Boer, Hart, Kropp, & Webster, 1997); VRS-SO = Violence Risk Scale–Sex Offense version (Olver, Wong, Nicholaichuk, & Gordon, 2007).
For all of the instruments, individuals tended to report use in the past year at a higher rate than routine use, suggesting that participants may be using certain instruments only occasionally. Overall, the STABLE-2007 was used by half of the participants in the past year and 42% of the participants on a routine basis, making it the most frequently used instrument. The SVR-20 was the second most frequently used instrument, followed by the VRS-SO and RSVP, both of which were used at a similar rate. The majority of the overall sample reported using some measure of criminogenic needs (71.4%), and most reported using an actuarial measure of criminogenic needs (60.5%). Only 15.1% of participants reported using more than one criminogenic needs instrument in the same evaluation, and it was even less common to incorporate an actuarial measure and an SPJ measure (7.6%).
To measure changes over time in the use of criminogenic needs instruments, participants were asked to identify what they were using in 2013 in addition to current usage choices. Response choices included options to indicate that they were not completing risk assessments in 2013 or could not recall which measures they utilized in 2013. A total of 18 participants reported that they were not completing risk assessments in 2013, and five individuals could not recall which measures they were using in 2013. These 23 individuals were excluded from this specific analysis, leaving 96 participants. The results are provided in Table 5.
Comparison of Use of Criminogenic Needs Instruments Across Time.
Note. Participants (n = 96) could choose as many instruments as were relevant. Excludes 23 participants who were not completing risk assessments in 2013 or who could not recall which instruments they were using in 2013. ARMIDILO-S = Assessment of Risk and Manageability for Individuals with Developmental and Intellectual Limitations who Offend Sexually (Boer, Tough, & Haaven, 2004); MIDSA = Multidimensional Inventory of Development, Sex, and Aggression (MIDSA, 2011); RSVP = Risk for Sexual Violence Protocol (Hart et al., 2003); SARN = Structured Assessment of Risk and Need (S. Webster et al., 2006); SOTIPS = Sexual Offender Treatment Intervention and Progress Scale (McGrath, Lasher, & Cumming, 2012); SRA-FV = Structured Risk Assessment–Forensic Version (Thornton & Knight, 2015); SVR-20 = Sexual Violence Risk–20 (Boer, Hart, Kropp, & Webster, 1997); VRS-SO = Violence Risk Scale–Sex Offense version (Olver, Wong, Nicholaichuk, & Gordon, 2007). Other category included Estimate of Risk of Adolescent Sexual Offense Recidivism (ERASOR; Worling & Curwen, 2001); Hare Psychopathy Checklist–Revised (PCL-R; Hare, 2003); Structured Risk Assessment–Forensic Version–Light (SRA-FVL; Thornton & Knight, 2015); Level of Service/Case Management Inventory (LS/CMI; Andrews, Bonta, & Wormith, 2004); Inventory of Offender Risk, Needs, and Strengths (Miller, 2006); Sexual Adjustment Inventory (Davignon, 2000); Juvenile Sex Offender Assessment Protocol–2 (J-SOAP-II; Prentky & Righthand, 2003); and use of risk factors derived from the Mann, Hanson, and Thornton (2010) study.
Overall, there has been an increase in the number of participants using the VRS-SO and a slight decrease in the number of individuals using the SVR-20 and the STABLE-2007. There have been no changes in how frequently participants use the ARMIDILO-S.
For those individuals who reported that they do not routinely use a criminogenic needs instrument (n = 26), their reasons for this decision were further explored. They could select multiple reasons, which included problems with the research, problems with the existing norms, that the measures are too time-consuming, and that the participants did not have adequate training. The most common reasons included concerns that the research was insufficient to support its use (n = 25) and concerns about the adequacy of the norms related to the instruments (n = 18).
Measuring treatment gains
Participants who identified that treatment progress was relevant to their assessment were asked which instruments they use. In total, 29 participants (24.4%) reported that measuring treatment gains was not relevant to their assessments. Consequently, these individuals were excluded from this specific analysis, leaving 90 participants. Of the 90 remaining participants, a third reported they do not use any instrument to measure treatment progress (33.3%). For those who did select a measure, the majority use the STABLE-2007 (30.0%). Other measures used included the SOTIPS (14.4%) and the VRS-SO (13.3%). The rest of the measures were used infrequently and ranged from 1.1% to 8.9%. The entire sample (N = 119) was also asked which of the criminogenic needs instruments has the best research support to measure a reduction of sexual recidivism risk as a result of treatment gains. A little more than a third of the sample (36.1%) reported that none of the instruments had any research support to measure sexual recidivism risk as a result of treatment gains. Of those who selected an instrument, most were split between the STABLE-2007 (19.3%) and the VRS-SO (18.5%). The SOTIPS was selected by 8.4% of the participants. Other measures were infrequently selected and ranged from 0.84% to 3.4%. Of the participants who use the VRS-SO to measure treatment change, 75.0% believe the VRS-SO has the best research support whereas 66.7% of those using the STABLE-2007 to measure treatment change believe the STABLE-2007 has the best research support. In contrast, 30.8% of those using the SOTIPS to measure treatment change believe it has the best research support; more SOTIPS users identified the STABLE-2007 as having the best research support (38.5%).
Protective factors
While some participants (10.1%) identified using the Structured Assessment of Protective Factors (SAPROF; de Vogel, de Ruiter, Bouman, & de Vries Robbé, 2012) to measure protective factors in their assessments, other instruments specifically for adults and designed to incorporate an assessment of protective factors into risk assessments were infrequently used (e.g., Short-Term Assessment of Risk and Treatability [START] and Inventory of Offender Risk, Needs, and Strengths [IORNS]; Miller, 2006; C. D. Webster, Nicholls, Martin, Desmarais, & Brink, 2006). Of note, more than half (58.8%) of the participants reported completing empirically guided assessments of protective factors in the absence of a formal instrument. Again, participants were asked about what protective factors instruments they were using in 2013, if applicable. A total of 19 participants reported they were not completing risk assessments in 2013, and another 19 could not recall. As such, these individuals were excluded from analysis, leaving 81 participants. There was no notable increase in SAPROF users between 2013 and 2017 (n = +2). However, what was notable was the number of participants who now consider protective factors in their evaluations. Only 40.7% of the participants reported that they were considering protective factors in their evaluations in 2013; however, now, 83.9% consider protective factors. This is a 43.2% change between 2013 and 2017.
What Influences Evaluators’ Choice of Instruments?
Overall, few variables were significantly associated with the measures used by evaluators. Evaluators with doctoral degrees were more likely than other evaluators to assess protective factors (odds ratio [OR] = 4.22), and the New static actuarial instruments were more likely to be used by ATSA members (89.7%, 70/78) than non-ATSA members (75.6%, 31/41). ATSA membership was also positively associated with using an actuarial measure of criminogenic needs (OR = 2.46). To explore whether there was an association between instrument use among individuals who were involved in training/research activities that require a more active role (e.g., presenting at conference, completing research for publication, being a peer reviewer for a journal, and serving on the editorial board for a journal), we separated participants into active (n = 58) and passive (n = 61) training/research activities. There was no significant association between individuals with active or passive involvement in training/research activities and their routine use of instruments (see Table 6). There was also no significant association in how they select Static-99R reference groups (Cramer’s V = .11, p = .597), whether they only used Routine Norms, whether they reported absolute recidivism rates, and how they measure treatment gains (Cramer’s V = .13, p = .366).
Individual Characteristics and Risk Assessment Practices.
Note. OR = odds ratio; CI = confidence interval. Old Static Actuarial includes Rapid Risk Assessment for Sexual Recidivism Risk (RRASOR; Hanson, 1997), Static-99 (Hanson & Thornton, 2000), Static-2002 (Hanson & Thornton, 2003), Minnesota Sex Offender Screening Tool–Revised (Mn-SOST-R; Epperson, Kaul, Huot, Goldman, & Alexander, 2003), and Risk Matrix 2000 (RM2000; Thornton et al., 2003). New Static Actuarial includes Static-99R (Helmus, Thornton, Hanson, & Babchishin, 2012), Static-2002R (Helmus et al., 2012), Mn-SOST-3 (Duwe & Freske, 2012), Violence Risk Scale–Sex Offense version (VRS-SO; Olver, Wong, Nicholaichuk, & Gordon, 2007), and Multisample Age-Stratified Table of Sexual Recidivism (MATS-1; Wollert, Cramer, Waggoner, Skelton, & Vess, 2010).
Excludes n = 29 participants for whom norms selection was not applicable or other methods were specified.
p < .05. **p < .01.
Participants were asked to indicate which ways they kept up on research and training by selecting all that apply from a list of 10 options (see Table 1). The number of different kinds of training/research activities was counted, this total was treated as a continuous variable, and its relationship to different aspects of evaluator behavior described with b coefficients and odds ratios from logistic regression equations. As Table 7 shows, there were some trends that were significant at the 0.10 level and one that reached conventional standards for statistical significance. Overall, for those who engaged in more training/research activities, the odds of assessing criminogenic needs, using actuarial measures of criminogenic needs, assessing protective factors, and reporting absolute recidivism rates increased, while the odds of only using Routine Norms decreased. Importantly, for the most part, the confidence intervals indicate that the odds ratios may not be reliable.
Predictive Value of Years of Experience and Number of Training Activities.
Note. OR = odds ratio; CI = confidence interval; SPJ = Structured Professional Judgment. Old Static Actuarial includes Rapid Risk Assessment for Sexual Recidivism Risk (RRASOR; Hanson, 1997), Static-99 (Hanson & Thornton, 2000), Static-2002 (Hanson & Thornton, 2003), Minnesota Sex Offender Screening Tool–Revised (Mn-SOST-R; Epperson, Kaul, Huot, Goldman, & Alexander, 2003), and Risk Matrix 2000 (RM2000; Thornton et al., 2003). New Static Actuarial includes Static-99R (Helmus, Thornton, Hanson, & Babchishin, 2012), Static-2002R (Helmus et al., 2012), Mn-SOST-3 (Duwe & Freske, 2012), Violence Risk Scale–Sex Offense version (VRS-SO; Olver, Wong, Nicholaichuk, & Gordon, 2007), and the Multisample Age-Stratified Table of Sexual Recidivism (MATS-1; Wollert, Cramer, Waggoner, Skelton, & Vess, 2010).
Excludes n = 29 participants for whom norms selection was not applicable or other methods were specified.
p < .05.
Years of experience was treated as a continuous variable and its relationship to different aspects of evaluator behavior described with b coefficients and odds ratios from logistic regression equations. As Table 7 shows, years of experience had little effect with the only statistically significant relationship being an increased odds of only using Routine norms with additional years of experience (b = .058, p = .018, OR = 1.06 [1.01-1.11]).
To determine what might account for differences in risk assessment methodology, the participants’ ability to independently select measures as opposed to having such decisions being imposed upon them by their employer was categorized into Low versus High. As noted in Table 8, there were no statistically significant associations with whether participants had freedom in choice of methodology and use of most instruments. There was also no significant association found for how participants select reference groups (Cramer’s V = .15, p = .380), use of only Routine Norms, use of absolute recidivism rates, and how they measure treatment gains (Cramer’s V = .09, p = .614). As noted in Table 8, the odds of using an SPJ instrument significantly increased when participants reported having freedom in choice of methodology. However, this result should be interpreted cautiously given the large confidence interval. Furthermore, of those with methodological freedom, the vast majority reported not using an SPJ measure (66.3%) compared with those that did report using an SPJ measure (33.7%). The odds of considering protective factors in assessments also significantly increased for those participants who reported having freedom in their methodology.
Employment Arrangements and Risk Assessment Practices.
Note. OR = odds ratio; CI = confidence interval; SVP = Sexually Violent/Dangerous Person; SPJ = Structured Professional Judgment. Old Static Actuarial includes Rapid Risk Assessment for Sexual Recidivism Risk (RRASOR; Hanson, 1997), Static-99 (Hanson & Thornton, 2000), Static-2002 (Hanson & Thornton, 2003), Minnesota Sex Offender Screening Tool–Revised (Mn-SOST-R; Epperson, Kaul, Huot, Goldman, & Alexander, 2003), and Risk Matrix 2000 (RM2000; Thornton et al., 2003). New Static Actuarial includes Static-99R (Helmus, Thornton, Hanson, & Babchishin, 2012), Static-2002R (Helmus et al., 2012), Mn-SOST-3 (Duwe & Freske, 2012), Violence Risk Scale–Sex Offense version (VRS-SO; Olver, Wong, Nicholaichuk, & Gordon, 2007), and Multisample Age-Stratified Table of Sexual Recidivism (MATS-1; Wollert, Cramer, Waggoner, Skelton, & Vess, 2010).
Excludes n = 29 participants for whom norms selection was not applicable or other methods were specified.
p < .05. **p < .01. ***p < .001.
There were no significant associations found for participants who identified being in private practice as opposed to a public-sector setting and routine use of instruments, whether they consider protective factors, how they report absolute recidivism rates, and how they measure treatment gains (Cramer’s V = .08, p = .662). However, there was a significant association found for those in private practice and how they select the Static-99R reference group (Cramer’s V = .28, p = .027). As noted in Table 8, the odds of only using Routine Norms was 3.45 times higher for those in private practice than those who work in public-sector positions.
SVP evaluators
There were 59 participants who identified their job duties as completing SVP evaluations. For this subgroup, the Static-99R and the STABLE-2007 were the most popular instruments used (88.1% and 33.9%, respectively). However, unlike the larger sample, the second most frequently used measure of criminogenic needs by SVP evaluators was the VRS-SO (18.6%). Compared with the overall sample, a slightly larger percentage of SVP evaluators reported not using any instruments to measure criminogenic needs (37.3%). The SAPROF was utilized by SVP evaluators at a similar rate as the larger sample (10.2%), and most of the SVP evaluators reported using an empirically guided approach to identify relevant protective factors (67.8%).
As noted in Table 8, SVP evaluators were significantly less likely to report routine use of an Old static actuarial instrument. The odds of routinely using an Old static actuarial instrument were 4.58 times higher for non-SVP evaluators than for those who did SVP evaluations. However, there was no significant association for routine use of a New static actuarial instrument. SVP evaluators were significantly less likely to routinely utilize any criminogenic needs instrument, χ2(1) = 4.36, p = .04. The odds of routinely using any criminogenic needs instrument were 2.38 times higher for non-SVP than SVP evaluators. There was no significant association between the type of evaluator (non-SVP vs. SVP) and whether they routinely used actuarial measures of criminogenic needs and routine use of SPJ instruments. More often than not, both types of evaluators routinely utilized actuarial measures of criminogenic needs (SVP evaluators = 54.2%; non-SVP evaluators = 66.7%). There was no difference in how SVP and non-SVP evaluators select the Static-99R reference group (Cramer’s V = .15, p = .381) or measure treatment gains (Cramer’s V = .05, p = .882). SVP evaluators were significantly more likely to report absolute recidivism rates associated with the Static-99/99R in their reports (66.1%) compared with non-SVP evaluators (30.0%; χ2(1) = 15.54, p < .001). The odds of reporting absolute recidivism rates were 4.55 times higher for SVP than non-SVP evaluators.
Because previous analysis found a relationship between those in private-practice settings and only using the Routine/Complete sample, we ran similar analysis with SVP evaluators. We hypothesized that with a high-risk population, we would not find a similar result. In fact, we found a significant association similar to the overall sample (Cramer’s V = .37, p = .020), although cell sizes become quite small. SVP evaluators in private-practice settings were significantly more likely to only use the Routine/Complete sample (55.0%) whereas evaluators in federal/state institutions were more likely to use an actuarial measure of criminogenic needs (47.4%) when choosing the Static-99R reference group.
Discussion
The primary aim of the current survey was to answer the following questions: whether professionals are adopting newer static risk instruments, whether they are using measures of criminogenic needs, whether instrument choices have changed since 2013, and whether changes appear to be consistent with new research. In addition, we wanted to explore factors that might be influencing evaluators’ choice of instruments.
The Static-99R and the STABLE-2007 continue to be the dominant instruments in sexual recidivism risk assessments. The vast majority of the sample is using a New static actuarial instrument, and those who are using the Static-99R have updated their practices to incorporate the new norms and coding manual. While ATSA members are more frequently utilizing a New static actuarial measure, the majority of professionals, whether ATSA members or not, are also doing so. Furthermore, there was no difference between ATSA and non-ATSA members in their use of Old static instruments, and this practice was rare regardless of professional membership.
About a quarter of the sample is using more than one static actuarial instrument within the same assessment report, most commonly consistently employing both the Static-99R and Static-2002R. It is uncommon for participants to use a combination of Old and New static actuarial instruments, and the majority reported using only one static risk instrument in their assessments. It was also less common to combine multiple measures of criminogenic needs instruments in the same evaluation. Rather, it appears the majority of the participants’ methodologies largely include a New static actuarial instrument, a measure of criminogenic needs, and a qualitative description of protective factors. This practice is consistent with previous recommendations made by Neal and Grisso (2014) in which multiple measures can be useful provided they demonstrate good reliability and validity. The decision to use multiple tools, however, requires balancing the incremental information from any additional tools with the costs of administration and the lack of professional consensus on how the results of different risk tools should be integrated into an overall assessment of risk.
Although used at a much lower rate, it is concerning that Old static actuarial instruments are still being employed. A total of 8% of the sample reported using the RRASOR and the Static-99, and 5% reported using the MnSOST-R. These instruments have either been replaced by newer versions with modern norms or, in the case of the RRASOR, the developers have specifically opined that they should no longer be used: see http://www.static99.org/pdfdocs/Legal-Thorton-Hanson_Affidavits(2016).pdf (Thornton & Hanson, 2016). The use and reporting of sexual recidivism rates from these older measures will lead to overestimations of risk, especially for older offenders. At minimum, this can be confusing to the court. At worst, it can be deliberately misleading.
SPJ instruments, such as the SVR-20, are used by approximately a quarter of the sample, although not routinely. This may reflect preferences by some participants to use SPJ measures for certain types of cases that may otherwise not be well represented in the existing norms for actuarial instruments (e.g., using the SVR-20 for individuals with major mental illness). Overall, almost three quarters of the sample reported using a criminogenic needs measure, and more than half chose one that is actuarial. Despite the online publication of the 2014 ATSA guidelines for assessing individuals with intellectual disabilities, there have been no changes in the use of the ARMIDILO-S. This finding may be related to the lack of new research on additional samples demonstrating its reliability and predictive validity, as the last research publication was in 2013 (Lofthouse et al., 2013).
There is increased use of the VRS-SO to measure pretreatment dynamic risk factors and treatment change when compared with participants’ report of their practice in 2013 as well as the previous survey (Kelley et al., 2017). This increased popularity makes sense in light of the additional research publications associated with this instrument. Since 2013, there have been approximately 10 additional research articles published in peer reviewed journals. The VRS-SO has been validated in Canada, New Zealand, and Austria as well as with unique samples, including individuals diagnosed with Pedophilic Disorder, individuals with high scores on the Psychopathy Checklist–Revised (Hare, 2003), and with individuals who identify as being Aboriginals in Canada. A list of VRS-SO references can be obtained from the first author. Perhaps surprisingly, more participants were using the STABLE-2007 and SOTIPS to measure treatment gains than the VRS-SO. While the STABLE-2007 has a good deal of empirical support as a tool for measuring changeable factors at the beginning of treatment, there are no published studies demonstrating its ability to measure the reduction of sexual recidivism as a direct result of treatment change. The SOTIPS demonstrates promise but, as of yet, there is only one published study (McGrath, Lasher, & Cumming, 2012). The VRS-SO has the most research publications demonstrating its ability to measure a reduction of sexual recidivism as a direct result of treatment change (Olver et al., 2018). Most participants who utilize either the VRS-SO or the STABLE-2007 to measure a reduction of sexual recidivism risk as a result of treatment change identified their respective measure as having the most empirical support. The SOTIPS was an exception in that almost a third of its users identified the STABLE-2007 as having the most research support.
There was some skepticism with regard to the research related to criminogenic needs instruments overall. In total, 22% of evaluators do not use these instruments routinely due to concerns about the research and the existing norms. One third of the participants who need to measure treatment gains over time do not use a formal measure, and another third believe that none of the existing tools can measure a reduction of sexual re-offense risk as a result of treatment gains. Again, this finding was surprising in light of the existing research (McGrath et al., 2012; Olver et al., 2018). It is also notable given the research that suggests that relying on the Static-99/99R alone for individuals who have demonstrated moderate to large treatment gains would lead to an overestimation of their risk (McGrath et al., 2012; Olver & Wong, 2011). Evaluators’ threshold for incorporating new measures into existing methodologies will understandably vary. A good standard of practice may involve making a priori determinations of what one would need to see (i.e., research or norms) before incorporating a new method or instrument, and then staying informed of research advances. This may help to mitigate the effect of potential competing external forces (e.g., costs related to time and money) as well as the potential to overlook important areas of research (Zapf & Dror, 2017). It is also important for evaluators to be aware of professional guidelines in their field of practice. For example, Section 6.08 of the 2014 ATSA Adult Practice Guidelines reads, “Members appreciate that recidivism risk is not static and may change as a result of interventions, client actions, or other circumstances and, therefore, members conducting risk assessments employ research-supported methods assessing dynamic risk factors as warranted over time.” Indeed, research supports that risk is not static and that estimates of sexual risk using the Static-99R alone will become inaccurate over time (Hanson, Harris, Letourneau, Helmus, & Thornton, 2018) while pretreatment dynamic risk factors and measures of treatment change have been demonstrated to have significant incremental predictive validity to the Static-99R (Olver et al., 2018).
Relative to other professionals, psychologists were more likely to routinely consider protective factors in their risk assessments, and this component of risk assessment saw a notable increase since 2013. However, there was no real increase in SAPROF usage. This seems to reflect a greater awareness of protective factors in general rather than enthusiasm for any particular instrument.
An interesting finding emerged between employment settings and how participants select the Static-99R reference group. Those in federal/state institutions primarily reported using an actuarial measure of criminogenic needs whereas those in private practice primarily used only the Routine/Complete group. This finding held up even when we limited the analysis to those evaluators completing assessments on presumably high-risk individuals (SVP). This may reflect two issues. First, there is increased uncertainty about existing recidivism rates associated with the Static-99R norms as well as for high-risk samples. Indeed, some findings suggest that base rates for high-risk samples may be lower than expected (e.g., see DeClue & Rice, 2016), although studies on base rates are often compromised by methodological difficulties or fail to account for important variables (e.g., time at risk; suppression of risk due to community supervision; treatment effect). Second, consistent with findings from Chevalier et al. (2014), it is possible that there may be an effect of adversarial allegiance. Forensic evaluators in private practice may receive the vast majority of their referrals from defense attorneys as opposed to state agencies, which usually employ their own evaluators. Interestingly, there was a 6% increase in the odds of using only Routine Norms for each additional year of experience. Thus, the findings related to only using Routine Norms may indicate a higher level of skepticism among more experienced professionals as well as increased vulnerability to allegiance effects.
More evaluators reported having moved to using the 2016 Static-99R coding manual than having moved to using the 2015 Static-99R norms. A quarter of the sample continued to utilize the 2009 norms, which was last available in the 2012 Evaluator’s Workbook. It appears that evaluators felt more obligations to use the most recent scoring instructions than they did in employing the most recently released norms. Two SVP evaluators reported using the original 2000 norms and 10 reported using the 2009 norms. This may have to do with controversy or dissatisfaction with some of the decisions with the development of the 2015 norms. Some concerns about the 2015 norms have primarily focused on changes to the HR/N group (i.e., Bridgewater sample removed resulting in fewer overall cases and no U.S. sample; Elwood, Kelley, & Mundt, 2017). However, a larger group of participants who reported using the original 2000 (n = 5) and 2009 norms (n = 18) does not work with an SVP population. This may reflect that there was more advertising associated with the 2016 coding manual than when the 2015 norms were released. For instance, shortly before the survey was released, the yearly ATSA conference offered a full-day training session on the new manual, and webinar trainings were being heavily advertised through e-mail notifications. Furthermore, it is possible that there are fewer demands of some employment settings to utilize updated norms and/or ensure that evaluators are aware of new norms.
The majority of evaluators using local norms were SVP evaluators (six out of the eight participants). Developing local norms can be advantageous, but it is an arduous task that requires the ability to control for confounding variables (e.g., time at risk; reoffending in other jurisdictions). Hanson et al. (2016) recommended that local norms include about 1,000 cases with 100 sexual recidivists to obtain stable logistic regression estimates. The norms that the SVP evaluators identified using were based on individual states’ review of sexual recidivism and not all have been subjected to a peer review process. For example, the Florida sample (Montaldi, 2015) is based on 710 respondents who were referred for SVP commitment but then released with a reported sexual recidivism base rate of 10% (less when considering only felonies). The Washington sample (Milloy, 2007) is based on 135 respondents who were screened as meeting criteria for SVP commitment but for whom no petitions for commitment were filed. Of these, 23% were identified as being reconvicted for a new felony sex offense. It is assumed that the California sample the evaluators are using is based on a written memorandum in response to a court order (Padilla, 2006), and it has never been formally published. This memorandum provides information on 93 individuals who had been released from Atascadero State Hospital without having completed sex offense-specific treatment and which had reliable follow-up information. Of these individuals, 6% were reported to have sexually recidivated. Another California study (Lee, Restrepo, Satariano, & Hanson, 2016) is based on a routine sample of individuals with sex offense convictions who have been released from prison on either probation or parole, and the study recommended that local norms were not needed at this time. The nature of the New York sample is unclear. Overall, with the exception of Lee et al. (2016), none of these studies had around 100 recidivists nor did they offer risk estimates for each Static-99R score generated from logistic regression. As such, none of these norms appears to be a credible replacement for the 2015 Static-99R norms.
Limitations and Future Directions
We found patterns of assessment practices that may suggest adversarial allegiance effects. This included the aforementioned use of only the Routine/Complete sample among those in private-practice settings but also other practices (i.e., use of the original Static-99 norms published in 2000). We are unable to determine whether this is related to adversarial allegiance or simply the result of well-reasoned practice. For example, the Routine/Complete norms have substantially more samples and cases than the High Risk/Needs norms, and Hanson et al. (2016) indicated that this reference group would be appropriate for the majority of risk assessments. Furthermore, use of the original Static-99 (2000) norms may not be related as much to adversarial allegiance as it is to a lack of awareness of current research (i.e., the original norms result in extreme overestimations of risk and are currently not recommended by the instrument’s authors); see http://www.static99.org/pdfdocs/Legal-Thorton-Hanson_Affidavits(2016).pdf (Thornton & Hanson, 2016). The exploratory analyses about what influences risk assessment practices should be interpreted with caution given some of the confidence intervals and cell sizes. It would have been helpful to have asked participants what legal questions they routinely answer in their risk assessments and how they are primarily hired to complete these assessments (i.e., are they state evaluators or are they predominately hired by the prosecution and/or defense).
The results of the current survey largely represent practices in the United States, as few participants from other countries responded to the survey. This was the result of having primarily sampled professional organizations from the United States. Although ATSA considers itself an international professional organization, those who responded to the survey were predominately from the United States. Future surveys should systematically identify forensic professional organizations in differing countries and attempt to obtain formal access to utilize their e-mail distribution list or listserv. We received more responses when professional organizations were approached formally rather than informally.
A surprisingly large number of respondents to the current survey were actively involved in research (e.g., presenting at conferences; completing research; publishing articles; volunteering as a peer reviewer for a journal; journal editorial board member). This may reflect the self-selection process that is inherent in online surveys. Specifically, those participants who are active consumers of research related to risk assessment, and/or themselves involved in such research, may be more interested in completing a survey focusing on this topic area. It is not clear, however, whether the general membership of ATSA and AP-LS has a similar level of research activity or whether the current results may not accurately reflect the larger memberships’ practices. Future studies should attempt to better capture members who may not be part of a listserv or responding to e-mails through their professional organization.
It is notable that approximately half of the current sample identified themselves as SVP evaluators, which could potentially contribute to responses that overestimate adherence with emerging research on sexual risk scales. When participants were asked to identify which populations they work with, they were able to select all that might apply. Of the 59 who indicated they work with SVP clients, 38 (64.4%) also reported working with non-SVP populations (e.g., clients who are incarcerated, outpatient, and on probation/parole). This suggests that the majority of SVP evaluators are responsible for a diverse array of sexual risk assessments, and it is probably less common for professionals to only complete one type of risk assessment. Therefore, while it is important to highlight the large proportion of SVP evaluators in the sample, it is likely that this reflects the general practices in the United States.
Supplemental Material
Supplementary_Material_-_Survey – Supplemental material for How Do Professionals Assess Sexual Recidivism Risk? An Updated Survey of Practices
Supplemental material, Supplementary_Material_-_Survey for How Do Professionals Assess Sexual Recidivism Risk? An Updated Survey of Practices by Sharon M. Kelley, Gina Ambroziak, David Thornton and Robert M. Barahal in Sexual Abuse: A Journal of Research and Treatment
Footnotes
Acknowledgements
The authors would like to thank the reviewers for their thoughtful comments and suggestions on this manuscript.
Authors’ Note
The authors take responsibility for the integrity of these data, the accuracy of the data analyses, and have made every effort to avoid inflating statistically significant results. Aside from general research support provided by the Sand Ridge Secure Treatment Center, there was no additional funding support. The third author is a co-developer of several of the scales examined in this study but does not receive any royalties. The first and third authors provide trainings for one or more of the instruments and occasionally receive payment for these trainings. The views expressed are those of the authors and not necessarily those of the Wisconsin Department of Health Services.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Notes
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
