Despite a growing body of research on the outcomes of holistic admissions and eliminating standardized test score requirements throughout education, few have documented how organizations transition to holistic review. Implementation, however, may help explain variation in impacts of holistic admissions. This article draws upon theories of organizational routines to examine adoption of holistic review in 13 STEM PhD programs from five universities. We conducted 60- to 90-min interviews with admissions leaders, including a COVID-19 transcript review activity. Data reveal change is multilevel, involving new policy/structural, practice/cultural, and cognitive/interpretive routines, which carry promise for disrupting institutionalized inequities where the politics of changing these routines can be managed. We discuss implications for policy, organizational practice, and future research on academic evaluations.
Association of American Medical Colleges. (2010). Roadmap to diversity: Integrating holistic review practices into medical school admission processes.
3.
Association of American Medical Colleges. (2014). Roadmap to diversity and educational excellence.
4.
AttiyehG.AttiyehR. (1997). Testing for bias in graduate school admissions. Journal of Human Resources, 524–548.
5.
BakerR.KlasikD.ReardonS. F. (2018). Race and stratification in college enrollment over time. AERA Open, 4(1), 2332858417751896.
6.
BalA.AfacanK.CakirH. I. (2018). Culturally responsive school discipline: Implementing learning lab at a high school for systemic transformation. American Educational Research Journal, 55(5), 1007–1050.
7.
BallD. L. (2018, July16). (What) does practice-based teacher education have to do with disrupting racism and oppression?AERA Presidential Talk
BartonK. (2015). Elicitation techniques: Getting people to talk about ideas they don’t usually talk about. Theory and Research in Social Education, 43(2), 179–205.
10.
BastedoM.JaquetteO. (2011). Running in place: Low-income students and the dynamics of higher education stratification. Educational Evaluation and Policy Analysis, 33(3), 318–339.
11.
BastedoM. N.BowmanN. A.GlasenerK. M.KellyJ. L. (2018). What are we talking about when we talk about holistic review? Selective college admissions and its effects on low-SES students. Journal of Higher Education, 89(5), 782–805.
12.
BelascoA. S.RosingerK. O.HearnJ. C. (2015). The test-optional movement at America’s selective liberal arts colleges: A boon for equity or something else?Educational Evaluation and Policy Analysis, 37(2), 206–223.
13.
BennettC. T. (2021). Untested admissions: Examining changes in application behaviors and student demographics under test-optional policies. American Educational Research Journal, 59, 180–216.
14.
BensimonE. M.DowdA. C.WithamK. (2016). Five principles for enacting equity by design. Diversity and Democracy, The Equity Imperative, 19(1), 1–18.
15.
BielbyR.PosseltJ. R.JaquetteO.BastedoM. N. (2014). Why are women underrepresented in elite colleges and universities? A non-linear decomposition analysis. Research in Higher Education, 55, 735–760.
16.
Blair-LoyM.MayorovaO. V.CosmanP. C.FraleyS. I. (2022). Can rubrics combat gender bias in faculty hiring?Science, 377(6601), 35–37.
17.
BourdieuP. (1987). Distinction: A social critique of the judgment of taste. Harvard University Press.
18.
BourdieuP.PasseronJ. (1990). Reproduction in education, society and culture (Vol. 4). Sage.
19.
CanningE. A.MuenksK.GreenD. J.MurphyM. C. (2019). STEM faculty who believe ability is fixed have larger racial achievement gaps and inspire less student motivation in their classes. Science Advances, 5(2), eaau4734.
20.
CharmazK. (2014). Constructing grounded theory: A practical guide through qualitative analysis. Sage.
21.
CheryanS.MarkusH. R. (2020). Masculine defaults: Identifying and mitigating hidden cultural biases. Psychological Review, 127(6), 1022.
22.
CreswellJ. (2009). Research design: Qualitative, quantitative, mixed methods approaches. Sage.
23.
D’AdderioL. (2008). The performativity of routines: Theorising the influence of artefacts and distributed agencies on routines dynamics. Research Policy, 37(5), 769–789.
24.
DevineP. G.ForscherP. S.CoxW. T. L.Anna Kaatz SheridanJ.CarnesM. (2017). A gender habit-breaking intervention led to increased hiring of female faculty in STEMM departments. Journal of Experimental Social Psychology, 73, 211–215.
25.
DiamondJ.LewisA. (2019). Race and discipline at a racially mixed high school: Status, capital, and the practice of organizational routines. Urban Education, 54(6), 831–859.
26.
EatonA.SaundersJ.JacobsonR.WestK. (2019). How gender and race stereotypes impact the advancement of scholars in STEM: Professors’ biased evaluations of physics and biology post-doctoral candidates. Sex Roles, 82(3–4), 127–141.
27.
Educational Testing Service. (2015). Avoid these five common mistakes when using GRE scores.
28.
FeingoldA. (1994). Gender differences in personality. Psychological Bulletin, 116(3), 429.
29.
FeldmanM. S. (2000). Organizational routines as a source of continuous change. Organization Science, 11(6), 611–629.
30.
FeldmanM. S.PentlandB. T. (2003). Reconceptualizing organizational routines as a source of flexibility and change. Administrative Science Quarterly, 48(1), 94–118.
31.
FoldesH. J.DuehrE. E.OnesD. S. (2008). Group differences in personality: Meta-analyses comparing five US racial groups. Personnel Psychology, 61(3), 579–616.
32.
ForscherP. S.MitamuraC.DixE. L.CoxW. T. L.DevineP. G. (2017). Breaking the prejudice habit. Journal of Experimental Social Psychology, 72, 133–146.
33.
GarcesL. M. (2012). Racial diversity, legitimacy, and the citizenry: The impact of affirmative action bans on graduate school enrollment. Review of Higher Education, 36(1), 93–132.
34.
GarcesL. M. (2013). Understanding the impact of affirmative action bans in different graduate fields of study. American Educational Research Journal, 50(2), 251–284.
35.
GarcesL. M. (2014). Aligning diversity, quality, and equity: The implications of legal and public policy developments for promoting racial diversity in graduate studies. American Journal of Education, 120(4), 457–480.
36.
GarcesL. M.BilyalovD. (2019). Navigating the quicksand: How postsecondary administrators understand the influence of affirmative action developments on racial diversity work. Teachers College Record, 121(3), 1–40.
37.
GarcesL. M.JohnsonB. D.AmbrizE.BradleyD. (2021). Repressive legalism: How postsecondary administrators’ responses to on-campus hate speech undermine a focus on inclusion. American Educational Research Journal, 58(5), 1032–1069.
38.
Grbic MorrisonE.SondheimerH. M.ConradS. S.MilemJ. F. (2019). The association between a holistic review in admissions and the diversity of accepted applicants and students matriculating to medical school. Academic Medicine, 94(3), 396–403.
39.
Grutter v. Bollinger (2003). 539 U.S. 306.
40.
HallS.MorleyD. (2019). Essential essays: Vol. 1. Foundations of cultural studies. Duke University Press.
41.
HarperS. R. (2010). An anti-deficit achievement framework for research on students of color in STEM. New Directions for Institutional Research, 2010(148), 63–74.
42.
HirschmanD.BerreyE.Rose-GreenlandF. (2016). Dequantifying diversity: Affirmative action and admissions at the University of Michigan. Theory and Society, 45(3), 265–301.
43.
HirschmanD.BoskE. A. (2020). Standardizing biases: Selection devices and the quantification of race. Sociology of Race and Ethnicity, 6(3), 348–364.
44.
HseeC. (1995). Elastic justification in decision making: How task-irrelevant but tempting factors influence decisions. Organizational Behavior and Human Decision, 62, 330–337.
45.
JohnsonJ. C.WellerS. C. (2002). Elicitation techniques for interviewing. In GubriumJ. F.HolsteinJ. A. (Eds.), Handbook of interview research: Context and method (pp. 491–514). Sage.
46.
JonesA.SchuhmannP.SoquesD.WitmanA. (2020). So you want to go to graduate school? Factors that influence admissions to economics PhD programs. Journal of Economic Education, 51(2), 177–190.
47.
KahnemanD. (2011). Thinking, fast and slow. Macmillan.
48.
KarenD. (1990). Toward a political-organizational model of gatekeeping: The case of elite colleges. Sociology of Education, 63, 227–240.
49.
KentJ. D.McCarthyM. T. (2016). Holistic review in graduate admissions. Council of Graduate Schools.
KyllonenP. C.WaltersA. M.KaufmanJ. C. (2005). Noncognitive constructs and their assessment in graduate education: A review. Educational Assessment, 10(3), 153–184.
54.
KyllonenP. C.WaltersA. M.KaufmanJ. C. (2011). The role of noncognitive constructs and other background variables in graduate education. ETS Research Report Series, 2011(1), i–133.
55.
LamontM. (2009). How professors think. Harvard University Press.
56.
LamontM. (2012). Toward a comparative sociology of valuation and evaluation. Annual Review of Sociology, 38, 201–221.
57.
LanginK. (2019). Ph.D. programs drop standardized exam: Amid concerns about diversity and the test’s utility, many are moving away from the GRE. Science, 364(6443), 816.
58.
LemannN. (2000). The big test: The secret history of the American meritocracy. Macmillan.
59.
LewandowskyS.EckerU. K. H.SeifertC. M.SchwarzN.CookJ. (2012). Misinformation and its correction. Psychological Science in the Public Interest, 13(3), 106–131.
60.
LieraR.HernandezT. E. (2021). Color-evasive racism in the final stage of faculty searches: Examining search committee hiring practices that jeopardize racial equity policy. The Review of Higher Education, 45(2), 181–209.
61.
LipskyM. (1980). Street level bureaucracy: Dilemmas of the individual in public services. Sage.
62.
MaitlisS.SonensheinS. (2010). Sensemaking in crisis and change: Inspiration and insights from Weick (1988). Journal of Management Studies, 47(3), 551–580.
63.
MaxwellJ. A. (2013). Qualitative research design. Sage.
64.
McGeeE. O. (2020). Interrogating structural racism in STEM higher education. Educational Researcher, 49(9), 633–644.
65.
MilkmanK. L.AkinolaM.ChughD. (2015). What happens before? A field experiment exploring how pay and representation differentially shape bias on the pathway into organizations. Journal of Applied Psychology, 100(6), 1678.
66.
MilkmanK. L.RogersT.BazermanM. H. (2008). Harnessing our inner angels and demons. Perspectives on Psychological Science, 3(4), 324–338.
67.
MillerC.PosseltJ. (2020). Equitable admissions in the time of COVID-19. Physics, 13, 199.
68.
MillerC.StassunK. (2014). A test that fails. Nature, 510(7504), 303–304.
69.
MillerC. W.ZwicklB. M.PosseltJ. R.SilvestriniR. T.HodappT. (2019). Typical physics Ph. D. admissions criteria limit access to underrepresented groups but fail to predict doctoral completion. Science Advances, 5(1), eaat7550.
70.
MillerC. W.ZwicklB. M.PosseltJ. R.SilvestriniR. T.HodappT. (2020). Response to comment on “typical physics Ph. D. admissions criteria limit access to underrepresented groups but fail to predict doctoral completion.”Science Advances, 6(23), eaba4647.
71.
OrtizA. M. (2003). The ethnographic interview. In StageF. K.ManningK. (Eds.), Research in the college context: Approaches and Methods (pp. 35–48). Brunner-Routledge.
72.
OswaldF. L.HoughL. M. (2011). Personality and its assessment in organizations: Theoretical and empirical developments. In ZedeckS. (Ed.), APA handbook of industrial and organizational psychology (Vol. 2, pp. 153–184). American Psychological Association.
73.
PerezR.AcuñaA.ReasonR. D. (2021). Pedagogy of validation: Autobiographical reading and writing courses for first-year, low-income students. Innovative Higher Education, 46(6), 623–641. https://doi.org/10.1007/s10755-021-09555-9
74.
PloyhartR. E.HoltzB. C. (2008). The diversity–validity dilemma: Strategies for reducing racioethnic and sex subgroup differences and adverse impact in selection. Personnel Psychology, 61(1), 153–172.
75.
PoonO.LeeD.GalvezE.SerranoB. C.RazaA.ChunN.EnglerJ. S. (2020, April). Race-conscious and unconscious holistic admissions [Paper presentation]. Association for the Student of Higher Education Annual Conference, New Orleans, LA, United States.
76.
PosseltJ. R. (2014). Toward inclusive excellence in graduate education: Constructing merit and diversity in PhD admissions. American Journal of Education, 120(4), 481–514. https://doi.org/10.1086/676910
77.
PosseltJ. R. (2015). Disciplinary logics in doctoral admissions: Understanding patterns of faculty evaluation. The Journal of Higher Education, 86(6), 807–833.
78.
PosseltJ. R. (2016). Inside graduate admissions: Merit, diversity, and faculty gatekeeping. Harvard University Press.
79.
PosseltJ. R. (2020). Equity in science: Representation, culture, and the dynamics of change in graduate education. Stanford University Press.
80.
PosseltJ. R.HernandezT. E.CochranG. L.MillerC. W. (2019). Metrics first, diversity later? Making the short list and getting admitted to physics PhD programs. Journal of Women and Minorities in Science and Engineering, 25(4), 283–306. https://doi.org/10.1615/JWomenMinorScienEng.2019027863
81.
PosseltJ. R.HernandezT. E.VillarrealC. D.RodgersA. J.IrwinL. N. (2020). Evaluation and decision making in higher education: Toward equitable repertoires of faculty practice. Higher Education: Handbook of Theory and Research, 35, 1–63.
82.
PowellW.DiMaggioP. (1991). The new institutionalism in organizational analysis. University of Chicago Press.
83.
QuinnD. M. (2020). Experimental evidence on teachers’ racial bias in student evaluation: The role of grading scales. Educational Evaluation and Policy Analysis, 42(3), 375–392.
84.
RayR.BrownJ. (2015). Reassessing student potential for medical school success: Distance traveled, grit, and hardiness. Military Medicine, 180(Suppl. 4), 138–141.
85.
RayV. (2019). A theory of racialized organizations. American Sociological Review, 84(1), 26–53.
86.
RayV.PurifoyD. (2019). The colorblind organization: Race, organizations, and the organizing process. Research in the Sociology of Organizations, 60, 131–150.
87.
RodgersA. (2020, April). Toward a racialized cultural historical activity theory in higher education [Paper presentation]. American Educational Research Association, San Francisco, CA, United States.
88.
RosingerK.Sarita FordK.ChoiJ. (2021). The role of selective college admissions criteria in interrupting or reproducing racial and economic inequities. Journal of Higher Education, 92(1), 31–55.
89.
SackettP. R.SchmittN.EllingsonJ. E.KabinM. B. (2001). High-stakes testing in employment, credentialing, and higher education. American Psychologist, 56(4), 302–318.
90.
SaldañaJ. (2013). The coding manual for qualitative researchers. Sage.
91.
SchmidtF. L.HunterJ. E. (1998). The validity and utility of selection methods in personnel psychology. Psychological Bulletin, 124(2), 262.
92.
SedlacekW.BrooksG. (1976). Racism in American education. Nelson-Hall.
93.
SpillaneJ. P.HopkinsM.SweetT. M. (2018). School district educational infrastructure and change at scale. American Educational Research Journal, 55(3), 532–571.
94.
Stevens. (2007). Creating a class. Harvard University Press.
95.
StraussA.CorbinJ. (1994). Grounded theory methodology: An overview. In DenzinN. K.LincolnY. S. (Eds.), Handbook of qualitative research (pp. 273–285). Sage.
96.
Students for Fair Admissions v. President and Fellows of Harvard College. (2023). 600 U. S. 39.
97.
ThornhillT. (2019). We want Black students, just not you: How White admissions counselors screen Black prospective students. Sociology of Race and Ethnicity, 5(4), 456–470.
98.
VerostekM.MillerC. W.ZwicklB. (2021). Analyzing admissions metrics as predictors of graduate GPA and whether graduate GPA mediates Ph.D. completion. Physical Review Physics Education Research, 17(2), 020115.
99.
WangW.HsiehJ. J. (2006). Beyond routine: Symbolic adoption, extended use, and emergent use of complex information systems in the mandatory organizational context (ICIS 2006 Proceedings, Paper 48). http://aisel.aisnet.org/icis2006/48
100.
WarikooN. (2018). What meritocracy means to its winners: Admissions, race, and inequality at elite universities in the United States and Britain. Social Sciences, 7(8), 131.
101.
WeissR. S. (1995). Learning from strangers: The art and method of qualitative interview studies. Simon & Schuster.
102.
WilsonM.DePassA.BeanA. (2018). Institutional interventions remove barriers to recruit and retain diverse biomedical PhD students. CBE Life Sciences Education, 17(2), ar27.
103.
WitzburgR. A.SondheimerH. M. (2013). Holistic review: Shaping the medical profession one applicant at a time. The New England Journal of Medicine, 368, 1565–1567. https://doi.org/10.1056/NEJMp1300411.
YoungN. T.CaballeroM. D. (2021). Physics Graduate Record Exam does not help applicants “stand out.”Physical Review Physics Education Research, 17(1), w010144.
