Abstract
Dismantling racism in American psychology has recently emerged as a critical goal of the field accompanied by actionable steps being developed in training, research, and practice. To contribute to this significant undertaking, we offer an exploration of the impact of racism on psychological research methods and practices using the ABC model. First, antecedents to the present manifestation of the problem are identified as an advancement of research topics that supported a hereditary hierarchy based on race, the normalization of White participants as standard in comparative studies, and the use of biased measures to investigate various constructs, particularly as it relates to psychological testing. Second, sampling bias is discussed as a systemic behavior that produces racism in psychological research methods; specifically, issues of omission and commission are described along with the use of WEIRD samples and the role of implicit bias in the limited production of knowledge on people of color. Third, the antecedents and behaviors give way to consequences that include threats to cultural validity, unequal treatment and health disparities, and epistemic oppression in the field. We end with recommendations for avoiding a continuation of the field’s racist legacy in present-day psychology.
In order to dismantle systemic racism in American psychological science, it is critical to address underlying factors in both our theories/models (see article by Teo, this issue) and our research methods and practices. To help in the specific analysis of research methods, we have adopted the ABC model where we will begin with a historical review (Antecedents of the Problem) and an analysis of current practices and policies (Behaviors that are problematic) that have created the negative impact (Consequences that have been harmful) that led to the APA apology (American Psychological Association, 2021a). To promote a more equitable Diversity Psychological Science, we will end with recommendations for change within the domain of research methods underlying our science.
In the United States, there is often a simplistic understanding of racism as prejudice, in fact, many inaccurately use the terms interchangeably. However, despite various conceptualizations of the term “racism,” the definition that most comprehensively captures the countless ways in which racism has manifested throughout history is one in which Wellman (1993), as cited in Tatum (2017, p. 87), indicates that racism is a “system of advantages based on race.” In the United States, this system was systematically established by the nation’s forefathers who utilized pseudo-scientific means to establish a racial hierarchy that placed Whites at an advantage and people of color at a disadvantage. This definition underscores that racism goes beyond prejudice—“a preconceived judgment of opinion, usually based on limited information” (Tatum, 2017, p. 85) and even beyond social power that yield policies that perpetuate racial bias. Instead, it is a more ubiquitous and insidious phenomenon that involves cultural messages, institutional practices and beliefs, and individual actions (Tatum, 2017).
As highlighted by this definition, racism operates on multiple levels (i.e., institutional, personally mediated, and internalized) and may be perpetuated in different forms (i.e., active, passive, and cultural). According to Jones (2000), institutional racism is characterized as differential access to resources and opportunities by race, personally mediated racism is marked by prejudice and discrimination (i.e., differential actions toward others based on race), and internalized racism is the acceptance of negative messages about self by members of the disadvantaged race. At the personally mediated level, racism may be active (i.e., blatant, intentional acts of racial bigotry, and discrimination) or passive (i.e., subtle, exclusionary, and acceptance of current social norms) (Tatum, 2017). Furthermore, racism is also perpetuated onto society at large through cultural racism—“the cultural images and messages that affirm the assumed superiority of Whites and the assumed inferiority of people of color” (Tatum, 2017, p. 86).
Ultimately, racism is by nature structural (i.e., referring to the totality of ways in which societies foster racial discrimination through mutually reinforcing systems of housing, education, employment, earnings, benefits, credit, media, healthcare, and criminal justice; Bailey et al., 2017) and systemic (i.e., embedded in society such that social, political, economic, cultural, and psychological rewards are allocated along racial lines; Bonilla-Silva, 2021). Therefore, it has far-reaching societal impacts, which makes it critical for scientists and activists across all domains of society to take part in dismantling racism. This article addresses a subset of one such domain: unearthing and dismantling racism in research methodology and practices in the field of psychology.
Antecedents
Antecedents consist of beliefs in human hierarchy (e.g., colonialism) that have served to produce racism and prejudicial research processes. The U.S. institutionalization of the field of psychology has been intertwined with systemic racism since its inception. Psychology was established as a field between 1850 and 1900, during the time of the Civil War, the Indian Appropriations Act, the Chinese Exclusion Act, and the Supreme Court’s ratification of segregation laws (Cummings & Cummings, 2021). The extreme views of the times demanded either clear support for racist practices that would advance racism for generations or a staunch denouncement of the oft-normalized racist approaches that corrupted every major institution birthed in the United States during this time. At its foundation, psychology chose the former, subscribing to evolutionary theory coupled with a belief in human hierarchy based on race. At a pivotal time in history, when the field could have originated from a significant need for advocacy for communities of color, those at its forefront, primarily White men, chose to support White superiority instead. In doing so, the field’s earliest contributions focused on individual differences across race as a mode to give credence to racist laws and policies (Cummings & Cummings, 2021), whose far-reaching effects are still felt today.
The fundamental belief in a hereditary human hierarchy across racial groups collectively held throughout the field produced racist and prejudicial methods and processes that still impact research methods and practices in psychology today. Amidst the racial unrest spurred by the murder of George Floyd in 2020 and the simultaneous disproportionate impacts of the COVID-19 pandemic on communities of color, APA initiated a series of efforts aimed at dismantling racism. Among these was a Historical Chronology (Cummings & Cummings, 2021), which documented psychology’s contributions to the widespread U.S. belief in racial hierarchy and the perpetuation of inequality for people of color in the United States. This examination offers historical examples of ways in which psychologists have engaged in scientific racism—“the use of scientific concepts and data to create and justify ideas of an enduring biologically based [racial] hierarchy” (Winston, 2020, p. 2). These antecedents to present-day psychological racism, particularly those that coupled racist ideologies with psychology to produce racist research methodologies are explored in this section.
Seminal research in psychology led by Francis Galton, as early as 1869, centered on individual differences and psychometrics. Flawed research methodologies enabled Galton to rank the “intellectual standard” of Blacks as two grades below that of Whites (Galton, 1869). Galton continued this work steeped in pseudo-scientific evidence, purposed in advancing the United States’s race-based hierarchy, for decades, and later emerged as a pioneer for the eugenics movement. In fact, he is credited with introducing the word “eugenics,” defined as “a science of improving ‘racial stock’” (Cummings & Cummings, 2021). Galton’s work significantly influenced the first generation of psychologists, who studied topics that were inherently biased and well-positioned to advance a racist agenda. Much of this work was published in Mankind Quarterly, the official journal for the International Association for the Advancement of Ethnology and Eugenics (Schaffer, 2007).
At the foundation of scientific inquiry are the constructs that are studied as they ultimately guide the knowledge yielded for public consumption. A study that comparatively investigated reaction times concluded that Black participants and First Peoples had faster reaction times than their White counterparts, which was indicative of a primitive reflex that was inferior to the evolved intelligence of White participants (Bache, 1895). Similarly, another study examining differences in memory across Black and White children found that Black children outperformed White children on a memory task; the outperformance was attributed to Black children having more “primitive brains” and being deficient in reasoning (Stetson, 1897). Some psychologists even outlined ways in which psychology can focus on specific topics to advance racism; for example, McDougall (1914) developed a plan by which experimental psychology can advance the goals of eugenics by studying the hereditary basis of mental qualities. To further highlight the field’s commitment to research topics that emphasize individual differences across racial groups as a mode of marking Black Americans as degenerate, APA’s first financial commitment beyond its own administration went toward a Committee on Physical and Mental Testing (Baldwin et al., 1898).
In addition to the strategic selection of research topics that would lend themselves to advancing racism, the samples selected were also systematically racist. Comparative studies intentionally selected people of color, who were already viewed as inferior, for comparisons to Whites, who were seen as normative. First Peoples and Black participants were compared to Whites (Bache, 1895), Black children were compared to White children (Stetson, 1897), and First Peoples were described as childlike such that adults were identified as more similar to White children and adolescents (Hall, 1904). Furthermore, noncomparative, deficit-oriented studies strategically selected minoritized racial groups to investigate various negative outcomes. Psychologists focused on Black students when studying school underperformance (Mayo, 1913) or immigrants, Blacks, and First Peoples when seeking to identify socially or mentally unfit people for segregation and sterilization (Kevles, 1998; Laughlin, 1914).
Beyond the selection of research topics and appropriate samples, suitable measures and procedures must be identified before beginning data collection. Psychological testing, which is perhaps psychology’s most significant contribution to scientific racism, is an area marked by biased measures. For example, in 1917, psychological tests were administered to almost two million soldiers in World War I; however, the tests used were comprised of culturally biased test questions and examination procedures, which led to a reinforcement of the racial hierarchy with White Americans at the top (with the highest IQ scores) and Black Americans at the bottom (with the lowest scores) (Kevles, 1968; Rury, 1988; Yerkes, 1921). These culturally biased tests formed the basis of present-day Intelligence Tests and Scholastic Aptitude Tests (SATs), which reflect tremendous racial gaps in scores across race. The first administration of the SATs, a decade later in 1927, stemmed from WWI’s military psychologists’ testing program (Cummings & Cummings, 2021). Similarly, the Graduate Record Examination (GRE) was published in 1950, ridden with the same concerns of cultural bias inherent in psychological tests (Eells et al., 1951; Holliday & Holmes, 2003). Psychological assessments were pivotal in the normalization of White culture as tests were designed by White psychologists and standardized on White populations before being administered to people of color.
As Black psychologists began to enter the scene in the 1920s and 1930s, testing procedures came under scrutiny in studies that yielded results indicating the role of environment in the outcomes of intelligence testing (Beckham, 1933; Long, 1935) and the role of White examiners on the test scores of Black test takers (Canady, 1936). Although their work was largely ignored by mainstream psychology, concerns about cultural and racial bias in psychological testing remained continuous (Canady, 1943) and are still not fully addressed in present-day psychology (Cummings & Cummings, 2021). Instead, APA established a Committee on Test Standards in 1950 focused on technical recommendations for improving test administration and design, which failed to sufficiently address race, ethnicity, and culture (Holliday & Holmes, 2003; Jackson, 1975; Simpkins & Raphael, 1970). In 1966, newly published test standards primarily addressed test validity, leaving calls to address discriminatory use of tests (that met the validity guidelines) largely unanswered (Plotkin, 1972). These concerns were further voiced when the Association of Black Psychologists was formed in 1968 calling for a suspension of testing, referring to these tests as culturally biased, racist, and unfair (Williams & Mitchell, 1978). In response, APA formed the Ad Hoc Committee on Educational Uses of Tests with Disadvantaged Students and found that issues of bias were rooted in the misuse and misinterpretation of tests and not the test themselves (Cummings & Cummings, 2021). While interpretation is indeed a part of the problem, there was a clear failure to acknowledge the wholistic nature of the problem. Therefore, concerns about psychological testing continued and even became the subject of legal arguments. Revisions to testing standards have continued throughout the life of the field with the sixth set of standards, spanning a 30-year period, appearing in 2014. This most recent revision considers fairness alongside reliability and validity (Worrell & Roberson, 2016).
Ultimately, the selection of topics that advanced a hereditary race-based hierarchy, strategically studying racially minoritized samples against White participants as the standard, and the utilization of racially and culturally biased measures deliberately supported scientific racism and the disenfranchisement of people of color throughout the history of the field. Furthermore, the biased procedures and interpretation of tests and other research findings throughout the history of the field also lent to advancing White superiority and disadvantaging people of color. These racist methodologies led to severe consequences for people of color, including but not limited to segregation and sterilization. Throughout the history of the field, there has been an ongoing critique of research methods in psychology that have harmed countless members of racially marginalized groups. However, unfortunately, dominant psychology has exhibited numerous “critical moments of silence,” described by Cummings and Cummings (2021) as “moments when the field could have spoken or acted on behalf of communities of color but failed to do so” (p. 2). The current efforts to address these injustices mark important attempts to ensure that the “legacy of the past” does not continue to resurface in present-day psychology.
Behaviors
Using the ABC model, B refers to scientific behaviors that existed in the past, continue to exist in the present and have caused harm to communities of color. One of the systemic behavioral problems in research methods is concerned with sampling bias or “whom we study” and therefore “whose problems get addressed in science.” As far back as 1975, Robert Guthrie, a historian of Black Psychology, had already noted this racial bias problem in Psychology with his aptly titled book Even the Rat was White (1976). In this volume, Guthrie (1976) provided a critical review of the history of American psychology that represented both errors of omission and commission (Marsella & Leong, 1995). The omission included a dearth of studies on racial and ethnic minority groups in our developing psychological knowledge base. Errors of commission included a biased readiness to obtain and accept findings that showcased the negative aspects African Americans experience (e.g., readiness to accept the lower intelligence of African Americans without examining the biases in our tests). These sampling biases created an abundance of knowledge for promoting White health and mental health. At the same time, limited knowledge was being developed for African Americans and other racial groups, which then served as the foundation of the Unequal Treatment findings with regards to racial and ethnic health disparities (Institute of Medicine US, 2003).
The history of social exclusion documented in Even the Rat was White represents the primary driver in current Diversity Equity and Inclusion (DEI) initiatives within our universities and institutions. It is ironic that the current backlash against such DEI initiatives would have us roll back progress with the message that “Non-White rats do not need to be studied.” The biased focus on the White privileged class continues, but with a recent shift from errors of omission to errors of commission. The historical lack of inclusion has now changed to active exclusion with book bans and defunding of DEI initiatives.
More recently, the racial and ethnocentric sampling bias in Psychology was systematically examined by Henrich et al. (2010) who found that the majority of psychological research (96%) was conducted on Western, Educated, Industrialized, Rich, and Democratic (WEIRD) samples, which represent only 12% of the world’s population. They reviewed research in a number of domains (e.g., cooperation, categorization and inferential induction, reasoning styles, and self-concepts and related motivations) and concluded that WEIRD societies “are among the least representative populations one could find for generalizing about humans” (p. 61). One would never accept a series of studies on men to be representative studies of gender differences. Yet, for decades we have automatically accepted studies of WEIRD samples as representative of our understanding of human psychology.
This consistent pattern of sampling bias creates a problem of limited population validity in psychological science. According to Bracht and Glass (1968), population validity is a form of external validity. It is concerned with whether the study results from a specific sample can be generalized to a larger target group of interest (APA Dictionary, 2023). The sample needs to be representative of this target group if there is to be population validity. A study of U.S. college students will lack population validity when applied to Chinese college students. Similarly, a study of White European American workers’ job satisfaction will lack population validity for African American workers. This tendency to ignore issues of population validity has a long history in American Psychology. Indeed, the common dictum that “Psychology is the science of the White College sophomore” (e.g., see Graham, 1992) captures the problem just as well as the title of Guthrie's (1976) history book, Even the Rat was White.
The logical question is where does this sampling bias come from and why have we ignored it? In Blindspot: Hidden Biases of Good People (2013), Banaji and Greenwald summarize their program of research on implicit bias, which provides a reasonable theoretical framework for understanding scientific racism in terms of sampling bias. They define implicit biases as unconscious biases toward others that often results in prejudice and discriminatory behaviors. Based on their Implicit Association Test (IAT), Banaji and Greenwald demonstrated through a series of studies that we are victims of these hidden biases with a majority of Whites having positive biases toward Whites and negative biases toward Blacks. Although there are questions regarding the strength of the relationships between these hidden biases and actual discriminatory behaviors (Blanton et al., 2009), the existence of these biases helps explain some of the racism revealed in the history of American Psychology (Cummings & Cummings, 2021).
Whether as journal editors, grant review panelists, or dissertations advisors, our intentions to do good science may be strong but Banaji and Greenwald’s (2013) research shows that despite good intentions, our unconscious biases may skew our judgments and decisions negatively toward minority scientists and their research methods and paradigms.
Mainstream psychological science has continuously struggled to publish research that highlights the importance of race and recruit racially diverse populations (Roberts et al., 2020). DeJesus and colleagues (2019) reviewed 1,149 articles published in 2015–2016 in 11 psychology journals and found that 73% of those did not include the race of their participants. Roberts and colleagues (2020) reviewed 26,000 published empirical articles between 1974 and 2018 in the most prestigious cognitive, developmental, and social psychology journals and noted several racial inequalities in psychological research. First, they noted that there were very few articles that highlighted race between 1970s and 2010s. Specifically, only 1% of cognitive psychology publications highlighted race compared to 5% in social psychology and 8% in developmental psychology. Roberts et al. (2020) also found that White editors were less likely to publish research focusing on race and 83% of editors were White. Finally, research focusing on race in these top-tier journals was published by mostly White authors (63%). In turn, White authors were more likely to recruit White participants than participants of color in their samples. Roberts et al. (2020) proposed that this difference may be related to higher investment of psychologists of color to communities of color and recognition of the importance of racial diversity in recruiting participants. Roberts et al. (2020) described the lack of racial diversity in psychology publications as both “biased and impractical” (p. 1303). It perpetuates systemic inequality as authors of color who conduct race-related research are more likely to publish in specialty instead of top-tier mainstream psychology journals and their work may be devalued by their peers and institutions.
In 2022, Steven Roberts was invited to participate in a “science debate on diversity” (p. 3) organized by Klaus Fiedler, former editor-in-chief of Perspectives on Psychological Science (Roberts, 2022). In setting up the debate, Fiedler revealed a strong motivation to showcase criticisms of the Roberts et al. (2020) article and did not allow Roberts the opportunity to address them fairly. Fiedler was asked to resign by the Board of the Association of Psychological Science due to violations of editorial practices and standards resulting in unfair treatment of Roberts. This incident demonstrates behaviors in the field of mainstream psychology that are consistent with experiences of marginalization and silencing of scholars of color. There is a pressing need for psychology scientists to understand the role of race and racism in the field in order to change how research is conducted, reviewed, and disseminated (Buchanan et al., 2021; Roberts, 2022).
Consequences
The antecedents and behaviors described in the previous sections have consequences that cause harm and racial disparities. In this section, we continue to connect behaviors with consequences. This section will focus on various threats to cultural validity (Leong & Kalibatseva, 2016), unequal treatment and health disparities, and epistemic oppression within psychological science. These consequences are the direct or indirect effects of mainstream psychological science’s neglect or oversight regarding of cultural, racial, ethnic groups.
Consequences of the Lack of Cultural Validity
One form of population validity is cultural validity, which is concerned with the degree of validity of our models, methods, and measures for culturally different populations (see Sue et al., 2022). A major consequence of the sampling bias is the development of theories and models, which are not valid for cultural, racial, and ethnic groups that do not represent the dominant White group.
The development and use of psychometrically sound tests and measures is one of Psychology’s major contributions to science. From IQ tests to neuropsychological tests, Psychology has contributed significantly to our diagnosis, assessment, and treatment of various psychological problems. According to Leong and Kalibatseva (2016), historically there has been a tendency in assessment and clinical diagnosis to neglect the role of cultural differences on psychopathology. They argued that cultural validity is an important corollary to psychometric validities (e.g., face, construct, predictive, and concurrent). The concept of cultural validity refers to the effectiveness of a measure or the accuracy of a clinical assessment to address the existence and importance of essential cultural factors. Such cultural factors may include values, beliefs, experiences, communication patterns, and epistemologies inherent to the clients’ cultural backgrounds (Solano-Flores & Nelson-Barber, 2001). The lack of cultural validity in clinical diagnosis and assessment may produce incorrect diagnoses and ineffective treatments for culturally diverse populations as a consequence. Inappropriate over-diagnosis could unnecessarily stigmatize and institutionalize racial and ethnic minorities due to lack of cultural validity in diagnoses. Hence, the neglect of cultural validity has led to the creation and application of biased tests and assessment tools with culturally different populations that introduce significant harm to communities of color.
Threats to Cultural Validity
Leong and Kalibatseva’s (2016) review of cultural validity found that the interpretation of assessment data, the accuracy of clinical diagnosis, and the outcome of psychotherapy with culturally diverse populations can be influenced by many factors. With an unrecognized ethnocentrism, much of clinical diagnosis in the United States is often conducted from a universalist perspective, which assumes that all people, regardless of race, ethnicity, or culture, develop along uniform psychological dimensions (Canino & Alegría, 2008; Malgady, 1996). This assumption of uniformity has been applied to racial and ethnic minority clients in many treatment-as-usual situations. Leong and Kalibatseva (2016) argued that this cultural uniformity assumption prevents clinicians from recognizing and attending to important cultural differences that may moderate the diagnostic process for culturally diverse clients.
Leong and Kalibatseva (2016) adopted Campbell and Stanley’s (1966) concept of threats to validity in experiments to propose that the lack of cultural validity in clinical assessment and diagnosis can also be conceptualized in terms of multiple threats to validity. According to Leong and Kalibatseva (2016), “these threats to cultural validity in clinical assessment are largely due to a failure to recognize, or a tendency to minimize, cultural factors in clinical assessment and diagnosis. They pointed out that these factors include but are not limited to: (1) pathoplasticity of psychological disorders; (2) cultural factors influencing symptom expression; (3) therapist bias in clinical judgment; (4) language capability of the client; and (5) inappropriate use of diagnostic and personality tests” (Leong & Kalibatseva, 2016, p. 59).
Pathoplasticity of Psychological Disorders
Leong and Kalibatseva (2016) identified Westermeyer’s (1985) concept of pathoplasticity of psychological disorders as the first threat to cultural validity. According to Westermeyer (1985), pathoplasticity refers to the variability in symptoms, course, outcome, and distribution of mental disorders among various cultural groups. In his model, Westermeyer (1985) delineated three different examples of pathoplasticity: (1) different content, severity, or frequency of symptoms for a disorder (e.g., schizophrenia) based on culture; (2) different rates of non-psychotic disorders (e.g., mood and anxiety) based on culture; and (3) different outcomes for disorders (e.g., schizophrenia outcomes are better in developing vs. developed countries) due to sociocultural factors (e.g., more stable social networks and integration in the community). Another example of pathoplasticity is that 87% of Chinese outpatients diagnosed with neurasthenia met criteria for major depressive disorder suggesting that somatic symptoms may mask depression (Kleinman, 1982).
Cultural Factors Influencing Symptom Expression
A second threat to cultural validity in clinical diagnosis noted by Leong and Kalibatseva (2016) concerns the influence of the clients’ cultural background on their symptom expression. Many studies have found significant cultural differences in emotion regulation and expression (Essau et al., 2017). Leong and Kalibatseva (2016) provided an example of how culture interacted with situational constraints to influence the manifestation of symptoms among Asian Americans. In Hawaii, Katz and colleagues (1969) studied the descriptions of Japanese and White patients with acute schizophrenia before hospital admission and during their stay in the hospital. Before hospitalization, as represented in the private settings of their homes, Japanese patients were described as more socially obstreperous, nervous, and hyperactive than White patients. Once these patients entered the public setting of the hospital, their behaviors changed. White patients were rated as more excited and disturbed than Japanese patients in the public hospital setting. This change suggests that symptom expression for Japanese may be influenced by face concerns when in public versus private spaces. When diagnosing Japanese Americans, clinicians may need to sample their behaviors broadly rather than base the diagnosis solely on observations and information obtained only in the diagnostic interview, which often occurs in the public settings of clinics and hospitals. Failure to recognize that a client’s cultural background may mediate when and how symptoms are expressed is a threat to accurate diagnosis (with misdiagnosis as a consequence).
Therapist Bias in Clinical Judgment
Therapist bias is the third threat to cultural validity in clinical diagnosis and refers to therapist’s racial bias against clients’ racial, ethnic, and cultural backgrounds (Constantine, 2007; Leong & Kalibatseva, 2016; Rosenberger & Hayes, 2002). Such biases could be conceptualized as culture-based countertransference where the skin color or accent of a client may elicit certain negative therapist reactions, which may then adversely affect the services provided. For instance, Dictado and Torres-Harding (2023) found that therapy trainees who had strong negative countertransference to racial and sexual minority clients and who endorsed colorblindness were more likely to over-pathologize their clients’ symptoms and to distrust their clients’ reports of microaggressions and mistreatment. Overall, it appears that cultural or ethnic differences may affect therapist clinical judgment and assessment, such that a therapist may over-pathologize a culturally different client (Whaley, 1997).
Language Capability of the Client
In Leong and Kalibatseva’s (2016) model, language capability of the client is the fourth source of threat to cultural validity in assessment and clinical diagnosis. Leong (1986) pointed out that Asian Americans with limited English proficiency may be misunderstood by clinicians, resulting in challenges in conducting clinical interviews and reaching an accurate diagnosis (Sue, 1981; Sue & Morishima, 1982). Additionally, Shuy’s (1983) review of the literature revealed that the use of dialects or non-standard English by patients may interfere with the effective exchange of information, or even stimulate bias in the therapist performing the evaluation.
Leong and Kalibatseva (2016) also discussed the use of interpreters with racial and ethnic minority patients as another language problem in clinical diagnosis serving as a threat. Problems inherent in “interpreter-mediated interviews” are particularly relevant to immigrant/refugee clients since many of them may not speak or understand English (Goh et al., 2004; Lee, 1980). Studies have shown that the use of interpreters may result in distortions that may negatively influence the diagnostic evaluation process (Searight & Searight, 2009).
Inappropriate Use of Clinical and Personality Tests
The fifth source of threat to cultural validity concerns the inappropriate use of clinical and personality tests (Leong & Kalibatseva, 2016). Many personality and diagnostic tests were developed and normed on European American populations and their predictive validity with racial/ethnic minority groups has not been established. Studies of clinical and personality test results for Asian American participants have tended to show them as having more severe symptoms and profiles than White participants (Leong, 1986). Despite a convergence of data that Asian Americans may have more neurotic and disturbed personality profiles on objective self-report measures, these results need to be interpreted with caution for several reasons. First, there is an absence of culture specific test norms since most of these measures were developed and normed on White samples. Second, very few of the clinical diagnostic instruments have been translated into Asian languages and their validity remains undetermined for patients with English as a second language (Leong & Kalibatseva, 2016). Therefore, personality and clinical tests need to be administered and interpreted with caution when working with racially/ethnically diverse clients.
Overall, the threats to cultural validity that Leong and Kalibatseva (2016) discussed portray research methodology concerns with racially and ethnically diverse populations within a clinical setting. The consequences of lack of cultural validity include under- or over-pathologizing, misdiagnosis, and mistreatment for racial/ethnic minorities. These consequences fall under the larger category of racial health disparities.
Health Disparities
In 2002, the landmark report “Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care” was published (Institute of Medicine, 2003). The report presented evidence that racial/ethnic minorities received poorer quality of healthcare after accounting for socioeconomic status differences and other access-related variables. The report examined patient-level variables (i.e., preferences, treatment refusal, and clinical appropriateness of care), healthcare system-level factors, and care process-level variables (i.e., bias, stereotyping, and uncertainty). Communication barriers and mistrust (possibly due to past negative experiences) were related to poorer treatment among racial/ethnic minorities. Still, unconscious bias and stereotyping of service providers were most strongly associated with misdiagnosing and under-treating racial/ethnic minority patients.
Twenty years later, the proceedings of a workshop “Unequal treatment revisited” reports that racial/ethnic disparities in child/adolescent health, maternal health, mental health and substance use, aging, and disability persist (National Academies of Sciences, Engineering, and Medicine, 2024). Although the documenting of disparities and awareness of the negative effects of implicit bias and stereotyping have increased, there has been less work on interventions to address disparities. “Unequal treatment revisited” examines what could be done differently from various perspectives (e.g., clinician, community health, and primary care) and proposes new ways to examine data and measurement to advance health equity research. Next, we discuss behaviors and consequences related to conceptualizing, conducting, reviewing, and publishing psychological research.
Epistemic Oppression in Psychological Science
Buchanan et al. (2021) examined epistemic oppression in psychological science, which is defined as the systemic exclusion that prevents generating knowledge, advancing the field, and creating inherent flaws in research. One of the main challenges in psychological science has been to include and generalize psychological research to BIPOC (Black, Indigenous, and other People of Color) populations. Buchanan et al. (2021) stated that focusing on individual-level acts (e.g., the rejection of an article on BIPOC communities) ignores larger systemic disparities (e.g., diversity-related manuscripts get disparate reviews (King et al., 2018). The authors identified various systems that need reform “to eliminate white supremacy in psychological science” (p. 1098). Specifically, there are fewer studies focusing on BIPOC populations than White populations and those studies are often required to compare the BIPOC sample to a White sample in order to be published in top-tier journals. Current research approaches frequently rely on predominantly White convenience samples (e.g., psychology classes at predominantly White universities). However, these methodologies affect how research is conducted and analyzed and “reinforce whiteness” (p. 1099) as the norm and BIPOC people as unusual and less important to study (Buchanan et al., 2021).
Excluding BIPOC research and interpreting it in ways that may harm BIPOC communities are acts of epistemic violence and oppression with detrimental consequences for BIPOC communities and psychological science (Buchanan & Wiklund, 2020). Therefore, it is important for mainstream journals to invite research on BIPOC populations through special issues and to focus on the full experiences of BIPOC people (Buchanan et al., 2021). Additionally, federal funding agencies have traditionally awarded funding to Black scholars at half the rate of White scholars (Dzirasa, 2020). Funding inequalities have harmful consequences for Black scholars’ tenure and promotion in academic institutions (Buchanan et al., 2021). Therefore, having funding from different agencies (e.g., federal, state, and private) for research in BIPOC communities is of utmost importance to address these disparities. Overall, journals need to demonstrate their commitment to diversity and inclusion through actions (i.e., training editors and reviewers in the value of publishing BIPOC-focused research and promoting BIPOC scholarship).
Researchers need to evaluate the appropriateness of the methodology they use for recruitment and conceptualization in research with BIPOC populations (Buchanan et al., 2021). In order to encourage the recruitment of more diverse samples, journals can set minimum requirements for sample diversity in papers and require justification for using predominantly White samples (Buchanan et al., 2021). In addition, journals can promote a Diversity Science approach, which focuses on understanding the sociocultural reasons that cause and maintain group differences (Miller et al., 2019). Moving in this direction, the APA established the Equity, Diversity, and Inclusion (EDI) framework specifically for journal editors to enhance their EDI efforts. The framework is comprised of four areas of focus: equitable content, inclusive science, a diverse community, and an inclusive publishing industry. Particularly, a diverse community refers to the diversification of the community of authors, editors, and readers by addressing systemic and institutional barriers faced by underrepresented scholars and researchers (APA, 2021b). Thus far, several editors of APA journals have expressed a commitment to the utilization of the EDI framework as a mode of promoting EDI in their editorial practices. Some examples include Cultural Diversity & Ethnic Minority Psychology, the Journal of Diversity in Higher Education, and the American Journal of Orthopsychiatry (APA, 2024).
Furthermore, continuing to use the same research methods without questioning their origin or cultural influences can have negative consequences in perpetuating Whiteness as the norm. Community-based participatory research (CBPR), person-centered quantitative approaches, qualitative methodologies, and mixed-methods approaches may be more inclusive methodologies that amplify BIPOC voices (Buchanan et al., 2021). CBPR is a research approach that demands the involvement of all stakeholders throughout every stage of the research process; this paradigm offers a unique opportunity to engage in culturally appropriate research as the population under investigation is actively involved in the research process. Despite the appropriateness of this methodology for elevating the voices of marginalized communities, less than 1% of peer-reviewed psychology articles employed CBPR from 2004 to 2014 (Rodriguez Espinosa & Verney, 2021). Where quantitative research is concerned, person-centered methodologies are contrary to the variable-centered approaches employed in most quantitative studies and assert that a given sample may include multiple subpopulations marked by different sets of parameters and reject the assumption of population homogeneity (Morin et al., 2016). Person-centered quantitative research is a useful way to acknowledge and investigate the impact of intersectionality among populations with multiple marginal identities. Qualitative approaches offer opportunities to improve the relationship and dialogue between researchers and the communities at the center of their research and in turn reduce the likelihood of researchers imposing their assumptions onto diverse populations. Mixed-methods research offers opportunities to combine the best of both quantitative and qualitative methods, providing maximum flexibility to achieve the study’s goals (Fassinger & Morrow, 2013). This breadth of options indicates that a variety of methodological approaches can be used to advance an inclusive approach to research, and underscores the need for researchers to intentionally select inclusive approaches within the research paradigm dictated by the study’s research question.
A specific practice that journal editors may consider to promote the selection of inclusive research practices at each stage of the research process is requiring reflexivity statements of authors seeking to publish in their journals. “Reflexivity is a form of critical thinking which aims to articulate the contexts that shape the processes of doing research and subsequently the knowledge produced (Lazard & McAvoy, 2020, p. 160).” This may aid in addressing issues of bias, both implicit and explicit, as reflexivity requires researchers to question how their positionality impacts every stage of the research process from the identification of a research topic to designing, conducting, and writing the research report. This constant interrogation of the impact of self on the research process necessitates the unpacking of the partial, positioned, and affective perspectives one brings to the research (Lazard & McAvoy, 2020). Therefore, journals that require reflexivity statements not only encourage researchers to engage in a practice that promotes the reduction of bias at every stage of the research process, but also offers readers important context through which to interpret research results. Consequently, reflexivity statements have become commonplace, especially in qualitative work in psychology and in the broader social sciences. However, dissenters of this practice have highlighted problems with positionality (i.e., Savolainen et al., 2023) that should be considered by journal editors considering reflexivity statements as a required component of submitted articles.
The language used in psychological science can further contribute to intentional or unintentional messages about inferiority of groups based on race. Buchanan et al. (2021) recommended using system centered language, which acknowledges the intergenerational systems that have maintained oppression and racism. APA has encouraged the use of bias-free and person-first language and adding system centered language would ensure the use of language in psychological science that accurately portrays the role of historic oppression. Likewise, providing a definition of race as a political and social construct not influenced by biology is important in recognizing the sociopolitical nature of race and resulting inequalities (Helms, 2020). Moreover, reporting the race, ethnicity, and nationality of participants in the Methods section would assist in understanding better the generalizability of the findings.
Recommendations
This paper uses the Antecedents, Behaviors, and Consequences (ABC) model to outline the ways in which research methods and practices in psychological science continue to demonstrate racial inequalities and put people of color and scholars of color at harm. APA Resolution to Dismantle Systemic Racism (American Psychological Association, 2021c) was an initial attempt to propose changes in addressing racism. This paper specifically focuses on recommendations within the domain of Science, acknowledging that it is also interconnected with the remaining three domains (Education, Practice, and Public Interests).
The criticisms of Psychology for conducting WEIRD science (Henrich et al., 2010) have highlighted the importance of evaluating the validity of our scientific results based on our narrow and skewed sampling practices. Leong and Brown (1995) reviewed the vocational literature and recommended that we attend to both the cultural validity and cultural specificity of our research findings. According to Leong and Brown (1995), cultural validity and cultural specificity offer two dimensions along which we can improve our research by directly examining the impact of cultural differences on our research approaches. Cultural validity concerns examining and improving the construct, concurrent, and predictive validity of established theories and intervention practices in diverse contexts and with culturally diverse populations. Cultural specificity involves developing and examining the applicability of concepts, constructs, and models unique to distinct cultural groups for explaining and predicting human behavior and development.
The cultural validity approach to research reflects an etic perspective more in line with cross-cultural psychology that aims to identify universal laws of behavior across different cultural groups. Determining the extent to which our scientific constructs applied to people across cultural groups typifies the cultural validity approach (e.g., Hardin et al., 2001). The cultural specificity approach to research reflects an emic perspective, more in line with multicultural psychology that seeks to discern elements indigenous and unique to people within particular ethnic groups that affect their vocational behavior and development. Determining the extent to which acculturation influences the behavior and development of Asian Americans exemplifies the cultural specificity approach (e.g., Leong & Chou, 1994). It is recommended that we adopt this cultural validity and cultural specificity approach in our research methods and practices in order to develop a more inclusive and diverse science of Psychology rather than a WEIRD science with limited cultural validity for most of the world’s population.
Another recommendation is for us to evaluate the measurement equivalence of our psychological tests and measures. Leong and Kalibatseva (2016) cited the main thesis from Leong et al.’s (2010) review of cross-cultural research, which pointed to the fundamental problem being the failure to examine the measurement equivalence of the tests and measures when applied to other cultural groups. Similarly, the measurement equivalence of personality and diagnostic tests is problematic in assessment and diagnosis. In particular, these tests may not be culturally appropriate if their linguistic, functional, conceptual, and metric equivalence have not specifically been established. Therefore, in using personality and diagnostic tests with racial and ethnic minority clients, there is a need to recognize that there may be important group differences in definitions of mental illness and mental health. Clinicians who use the existing diagnostic and personality tests to diagnose ethnic minority patients without being aware of these issues and limitations may formulate culturally invalid diagnoses (Leong & Kalibatseva, 2016). Therefore, it is important for clinicians to be educated on measurement equivalence of tests for different groups and to review the most up-to-date research on test norms for different racial/ethnic groups when administering specific measures.
Another relevant development with regards to research methods and practices is the adoption of Evidence-Based Practice (EBP) in Psychology. EBP was initiated by the pioneering work of Archie Cochrane (1972), who called for the field of medicine to be guided by a periodic and critical summary of the best available scientific evidence to inform current practice. This contrasts with the traditional approach of practicing medicine according to the knowledge and skills developed in medical school. A 2005 APA Presidential Task recommended the adoption of EBP within Psychology (American Psychological Association, 2006). The final report from the Task Force (American Psychological Association, 2006) also addressed the issue of treatment for racial and ethnic minority groups. The report noted that client’s characteristics, such as age, culture, race, ethnicity, gender, gender identity, religious beliefs, family context, and sexual orientation need special attention (APA, 2006). All of these attributes influence the client’s “personality, values, worldviews, relationships, psychopathology, and attitudes toward treatment” (p. 279). Culture influences the nature and expression of psychopathology as well as the explanatory models of health and illness, help-seeking behaviors, and expectations about treatment and outcomes (APA, 2006; Lopez & Guarnaccia, 2000). The report noted that future research needs to address a myriad of issues, including the effect of patient characteristics on seeking treatment, treatment process, and outcomes, as well as empirical evidence for the effectiveness of psychological interventions with racial/ethnic minorities.
In addition, the report made an important observation: “Evidence-based practice in psychology (EBPP) is the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences.” (p. 273). Since our knowledge of the influence of patient characteristics, culture, and preferences among racial/ethnic minorities in psychological assessment, case formulation, therapeutic relationship, and interventions is limited, our future research should be designed to explore them within the framework of EBPP. Ultimately, the goal of this research should be to answer the question “What works for whom, when, and under what conditions?” Our continued commitment to promoting Evidence-Based Practice (EBP) in Psychology should be represented by inclusive research methods that seek to acquire evidence to treat all groups and all persons.
As evidence-based treatments became more prevalent, a debate emerged about their appropriateness and effectiveness with ethnocultural groups who do not share the same language and/or cultural values as the population for which they were originally developed (Bernal et al., 2009). Since the 2000s, there have been multiple studies evaluating the effectiveness of cultural adaptations of evidence-based treatments for different racial and ethnic groups. Hall and colleagues (2016) conducted a meta-analysis of culturally adapted psychological interventions in 78 studies on psychopathology outcomes. The authors found that culturally adapted treatments performed better than treatment as usual or no treatment with a medium effect size. A systematic review of evidence-based intervention adaptation studies over the past 20 years produced an overarching data-driven framework (the Cultural Treatment Adaptation Framework) that systematizes common components and language for adaptations (Chu & Leino, 2017). Using cultural adaptations helped with engagement and treatment delivery. The fidelity to the main components of treatment was preserved in most culturally adapted treatments but additional treatment components were needed to address sociocultural needs (Chu & Leino).
Scholars have asserted that graduate training in psychology is synonymous with diversity science training, indicating that psychological science is itself diversity science (Mendoza-Denton & España, 2010). This fundamental belief is supported by the idea that the study of psychology is inextricably intertwined with diversity. This is because psychological processes and principles are influenced by the inseparable link between persons and contexts, which is impacted by culture. Therefore, the goal of graduate psychology training should be “to produce the most competent (and culturally humble) diversity scientists” (Neblett, 2019, p. 1217).
Recommendations for advancing diversity science training in research methodology should begin with training students to critically examine which topics are valued and who is conducting the research (Syed et al., 2018). Also primary is raising students’ awareness of racial/ethnic groups that are systematically excluded from human studies (Hewer, 2015). Likewise, graduate students should grapple with the harms of comparative research, especially comparing people of color to White people (Serafica & Vargas, 2006). These strategies will help students to consider the importance of representation from conceptualization to interpretation as well as issues with external validity and applications of study findings to people of color (Hewer, 2015). This is directly related to early antecedents of psychological racism as the founding fathers of the field, White males, intentionally selected topics and samples that strategically advanced White superiority by normalizing assessments on White people and subsequently using them on racial/ethnic minorities. This should advance discussions about the development of psychometric constructs within racial/ethnic groups prior to comparing across groups (Neblett, 2019).
Furthermore, Guthrie’s (2004) seminal work Even the Rat Was White highlights the need for open acknowledgment of the history of scientific racism in psychology and training that implores students to avoid deficit conceptualizations of Black populations in research. Instead, research methodologies that reflect a deconstructionist school of Black Psychology should be applied to all stages of research (Neblett, 2019). Graduate training should also encourage the use of statistical models to evaluate cultural differences in psychological constructs (Neblett, 2019). These recommendations offer opportunities to apply diversity to psychological science in meaningful ways.
Though not a comprehensive compilation, these recommendations represent a considerable starting point for addressing tensions (i.e., resistance) and challenges (i.e., difficult classroom dialogue) that currently exists in efforts to provide graduate students in psychology with diversity science training. For these recommendations to be effectively implemented, policy recommendations (i.e., a top-down commitment to and investment in diversity science training) must first be executed to establish institutional support and garner buy-in from students and faculty alike. These large-scale recommendations are outlined in Neblett’s (2019) recent work.
Relying on diversity science, research on race and racism, and the APA Race and Ethnicity Guidelines, Miller and colleagues (2019) offered five recommendations that can strengthen research methodology in psychological science. First, scholars need to be mindful of historical patterns of oppression and inequality. Awareness and self-assessment of one’s life experiences is particularly important for researchers who have benefited from White privilege. Second, adopting a racially diverse team science approach would strengthen research methodology if team members come from different disciplines and have different orientations. Third, using diverse samples will significantly improve the quality of race and racism research. In particular, expanding beyond the recruitment of WEIRD participants and finding ways to invite participants from populations that have been underrepresented in psychological research would contribute to advancing the field. Fourth, considering intersectionality and experiences of power and oppression shows the complexity of the human experience. Finally, promoting knowledge translation from the laboratory to the field and applying research to develop interventions and influence policy changes will result in more meaningful impact (Miller et al., 2019).
Roberts et al. (2020) proposed recommendations for journals and authors to address racial inequality in the publication process. These recommendations for journals include (1) communicating explicit commitment to diversity, (2) including diverse individuals in all levels, (3) evaluating participant diversity, (4) publishing public diversity reports, and (5) establishing a diversity task force. Author recommendations emphasize the need to describe and justify the racial composition of the sample, consider the generalizability of their findings, and include positionality statements.
In summary, we have sought to provide a series of recommendations to assist in the dismantling of racism within Psychology in the United States, which is the focus of the current special issue. Consistent with the scope and focus of the special issue, we have discussed the Antecedents, Behaviors, and Consequences of scientific racism in relation to research methods and practices. Our review is not designed to be comprehensive but instead one of many steps in this process. We believe that the goals outlined in the APA Resolution to Dismantle Systemic Racism (American Psychological Association, 2021c) are formulated as aspirations and target at a very high level. Therefore, we wanted to extend the Resolution by providing a more detailed analysis and specific recommendations in relation to promoting research methods that can help advance a truly diverse and representative science of Psychology.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
