Assessment of HiTOP Constructs Across the Population: A Commentary on the HiTOP Measure Development Project

Lingering Assumptions

It is obvious that the Measure Development Workgroup sought to be overinclusive in identifying constructs within each spectrum and defining them carefully. Many of the subgroups worked collaboratively among themselves and with outside experts to ensure that they were capturing both historical and current concepts that help describe psychopathology, at least in Western psychiatric terms. For example, the thought disorder workgroup conducted an exhaustive review of existing measures to ensure inclusion of a large number of relevant constructs, and the somatoform/eating disorders workgroup invited international experts to comment on the identified constructs and to add to these.

Despite this exhaustive process, there was clearly a tendency to rely on current diagnostic taxonomies, such as DSM and ICD categorical diagnoses and criteria, to develop the initial constructs and item content. This is practically unavoidable. Members of the workgroups (and the majority of mental health professionals in the Western world) have been trained and conduct research and practice based on the diagnostic systems in place, our research literature is steeped in such conceptualizations, and our common language of psychopathology derive from them. Furthermore, the HiTOP hierarchy includes a level of “syndromes or disorders” that is populated primarily by existing diagnostic categories. The developers did incorporate constructs and data coming from the personality literature and trait-based conceptualizations, although there was little discussion about disentangling “symptoms” from “trait” constructs, which may be important for clinical practice (DeYoung et al., 2020). In any case, one problem with relying heavily on existing conceptualizations is that the assumptions inherent in them are brought forth in the items and constructs identified by the workgroups. As noted by Poland (2015), one broad assumption that pervades our diagnostic system and clinical assessments is that of “individualism,” in that it is presumed that “all mental disorders are constituted by bodily states and processes (viz., behavioral, psychological, or biological dysfunctions and the symptomatic patterns they produce) within the individual who has the disorder” (Poland, 2015, p. 28). In psychology, our measures tend to neglect the complex interplay of individual and external forces (physical environment, social context) in defining behavior, symptoms, and traits (Shim, 2021). It is often left up to clinicians, some who do not have the adequate knowledge and background, to infer the relational, contextual, or sociocultural basis for symptom presentation, a process of clinical judgment which can introduce biases in diagnoses and treatment planning (Asonye et al., 2020; Garb, 1997).

One recommendation from the present commentary is for the HiTOP measure development workgroups to evaluate the ways in which the procedures being followed may further embed the HiTOP measures in the current nosology and force certain factor structures that reify existing diagnostic systems. As reported in the papers, the large number of constructs and items identified in most workgroups required the use of a rational approach to evaluate the homogeneous item composites and to reduce the item pool. This means that structural analyses could not be conducted with all items across all symptom components or maladaptive traits, except in the case of the somatoform workgroup. Instead, item-level exploratory factor analyses were conducted within broader domains, but not across them. This strategy reduced the ability to identify or further consider items with less intuitive relevance for persons embedded in majority culture or that were less consistent with assumptions in the field. Moving forward, Phase 2 would benefit from larger sample sizes to ensure that structural analyses could be conducted with items at all levels of the hierarchy. Furthermore, the workgroups may want to continue to identify relevant symptom/trait components, and novel definitions, making sure to borrow from literatures outside of clinical psychology and psychiatry, including community psychology (e.g., Prilleltensky, 2008), sociology (Aneshensel et al., 2013), anthropology (Drake, 2015), and other social and human sciences that are less anchored to psychiatric nosology or dominant psychological theories.

Considering Symptom Presentation and Psychosocial Functioning

Clearly, a key consideration in developing measures of clinical constructs is the adequate representation of items across the spectrum of severity and dysfunction (Carter et al., 2018; Samuel & Widiger, 2011). The ability to do so has implications for adequately capturing the comorbidity and shared variance, which is a key goal of dimensional modeling. For example, Helle et al. (2020) found that persons meeting a higher number of criteria for alcohol use disorder showed more comorbidity with other disorders and higher severity presentations of comorbid disorders. The workgroups described their efforts in this regard. For example, item reduction was conducted cautiously, with an eye toward ensuring broad content coverage. Some of the workgroups also made an effort to recruit large samples of participants with varying levels of psychopathology. For example, the internalizing workgroup used an iterative process to refine item content across large-sized development and validation samples, with a clear focus on recruiting persons with history of mental health treatment. The thought disorder workgroup included a large sample of college student participants and a somewhat smaller sample of individuals with a diagnosis of a serious mental illness (e.g., schizophrenia, schizoaffective, bipolar disorders). The latter is important in ensuring that items at the higher end of severity (e.g., more difficult items) would still be represented in the scales, ensuring the clinical utility of these measures across settings.

It is also true, though, that most of the data were collected from convenience samples with presumably low base rates of disorder and severe symptoms. Little information was obtained from participants on psychosocial functioning, which is distinguishable from symptom severity and shows distinct implications for treatment response and long-term prognosis (Rossi et al., 2017). The underrepresentation of individuals at the higher ends of symptom severity is indicated by low endorsements of certain items reflecting rarer and more severe manifestations of disorder, although I understand that many of these items will be carried over to Phase 2 and tested in clinical or patient samples. The lack of information on psychosocial functioning perpetuates the little attention paid to the real-life consequences of symptom presentation and the potential divergence of symptoms and functioning.

I would encourage the workgroups to consider another round of data collection, even prior to Phase 2, to gather data on participants coming from various settings and presenting with different levels of symptoms and psychosocial functioning. There is the concern that many items removed in Phase 1, which will not be revisited during Phase 2, could play a more important role in distinguishing persons across presentation of symptoms and/or social and occupational functioning. Not adequately capturing severity and a range of symptoms within individuals threatens to misrepresent spectra or superspectra in persons at the higher level of severity or number of symptoms (e.g., Helle et al., 2020).

Incorporating Multicultural Perspectives

In the past decade or more, American psychiatric and clinical psychology communities have experienced a reckoning of sorts, with the acknowledgment that despite the development of successful treatment approaches, we have not made much of a dent in reducing the burden of mental illness (McNally, 2011; Shoham et al., 2014). This bitter realization is accompanied by data on mental health disparities in the United States and other developed nations, showing that racial, ethnic, and other minorities and persons living in rural areas are less likely to access or receive formal mental health services (Constantine et al., 2004; U.S. Surgeon General, 2001). These concerns are relevant to the HiTOP consortium’s work, including the Measure Development Workgroup, since mental health disparities do not only result from lack of treatment but also from how we define and apply diagnoses (Perreira et al., 2005). For example, although African Americans are disproportionately diagnosed with psychotic disorders, there is evidence that racial biases may explain the disparities (Asonye et al., 2020; Garb, 1997). Even when using measures less reliant on clinical judgment, we define wellness and pathology according to the idioms and norms of Western or White-majority cultures, which can result in pathologizing culturally normative behaviors in persons from other ethnic, racial, or cultural groups (Ryder et al., 2008). Furthermore, root causes of mental health disparities often lay in historical inequities and structural racism, which must lead one to question the fit of explanatory models that primarily frame psychopathology in terms of biological, individual, or cultural factors (Shim, 2021).

Existing work supports factorial invariance across ethnic groups of HiTOP dimensions, at least as typified by DSM disorder symptom counts (He & Li, 2021), and describes some benefits of transdiagnostic approaches like HiTOP to examining mental health disparities (Eaton, 2020). Nonetheless, I encourage the workgroups to ensure that the HiTOP measures will be useful in assessing manifestations of psychopathology across various demographic groups, especially in terms of diversity in ethnicity, race, gender, sexual orientation, and other identities. First, the papers in this issue failed to acknowledge the challenge of mental health disparities and how an alternative to existing nosology could address existing problems. Furthermore, there was little to no discussion of multicultural assessment and diversity in manifestations of psychopathology (Sue, 1996), no seeming consideration of ethnic or cultural issues as part of the process of writing or evaluating item content, nor testing of measurement invariance across key groups. Given the large demographic shifts occurring across many Western nations (e.g., growing representation of multiracial individuals; Noe-Bustamante et al., 2020), the potential exists that the constructs extracted do not fully capture symptom dimensions across various groups often underrepresented in psychopathology and instrument development research (Sue, 1996). To challenge our perspectives more deeply, the developers are encouraged to invite experts in multicultural assessment to join the various workgroups, as well as review work by cultural psychologists, diversity scientists, and critical race theorists, including critical psychologists (e.g., Akomolafe, 2012) to inform measure development.

Second, the set of articles in this issue varied in the extent to which ethnic and racial minorities were recruited, and the large majority of participants across workgroups were White/European descent. Two articles failed to report Hispanic ethnicity in U.S.-based samples (even though Hispanic ethnic groups represent the largest minority in that country), and one article did not report any race/ethnicity information at all. The exceptions to this include the two university samples in Cicero et al. (this issue), which overrepresented persons identifying as Asian descent in the United States, and the proportional representation of Maori and Pacific Islanders in New Zealand in Sellbom et al. (this issue). None of the articles reported other marginalized identities that are often encountered in mental health settings (e.g., sexual and gender minorities, immigrant status) or that relate to mental health disparities (e.g., urban/rural). Of course, there are complexities in how to measure racial/ethnic identities, intersectional identities, and the heterogeneity within subgroups (Waters, 2000); however, this should not be an impediment to attempting to obtain this information from participants.

Concerns have been raised about using instruments normed on one group to make clinical decisions for other groups, increasing the potential for measurement bias and discrimination (Perreira et al., 2005; Sue, 1996). The set of articles in this issue did acknowledge as a limitation the lack of diversity in the samples and that Phase 2 would involve an attempt to better represent different demographic subgroups. However, there is reason for concern that this attempt did not seem to occur during Phase 1 of measurement development, such that items were developed, trimmed, or retained during this phase. It is possible that the items removed, or those not even considered, could be most discriminating for some non-White ethnic minority groups or for persons at the intersection of multiple identities (Perreira et al., 2005). The behaviors or thoughts (e.g., symptom components/traits) that typify a construct in one culture may not do so in another culture (Ryder et al., 2008). For these reasons, I would encourage the workgroups to revisit item content and definitions of constructs with a multicultural and intersectional lens.

Conclusion

The HiTOP consortium, including the Measure Development Workgroup, has gathered an impressive group of scholars to accomplish the hefty goals of creating a new empirically-based classification system, developing and validating a unified set of measures to advance the research, and communicating with stakeholders to ensure clinical utility. I want to recognize that they have done a lot already, and the scope of the work must be limited to some extent. I hope this commentary, however, challenges some basic assumptions and helps increase self-reflection in the process, with an eye toward reducing bias and increasing inclusivity. It is a challenge to “carve nature at its joints,” especially when we are trying to apply human concepts that are highly constructed by social and contextual factors. An acknowledgment of the hegemony of Western conceptions of illness and knowledge of how biases play a role in assessing and interpreting behaviors across cultures should be incorporated in the attempt to deliver a new system of classification that can meet the needs of an ever more global and diverse world.