Jason Chin (Australian national; Academic google), Alexander Holcombe (Sydney; Academic google), Kathryn Zeiler (Boston University; Academic google), Patrick Researcher (Busara Center; Academic google) and Ann Guo (Melbourne), Meta-research, psychology, and the law: a case study on implicit bias45 J. Legal Prof. 229 (2021):
When can the scientific findings of experimental psychology be applied with confidence to legal questions? And when applications have clear boundaries, are they easily recognized by legal commentators? To address these questions, we review recent findings from an emerging field of inquiry research (ie, meta-research). We find that many aspects of experimental psychology research practices and reporting threaten the validity and generalizability of legally relevant research findings, including those on which courts and regulatory bodies rely. As a case study, we assess the empirical claims behind commentators who claim that implicit bias profoundly affects legal procedures and practices, and that training can be used to reduce that bias. We find that these claims carry many indications of unreliability. Only limited evidence indicates that interventions designed to reduce disruptive behavior through implicit bias training are effective, and the research area shows many signs of publication bias.
To examine whether law journal articles acknowledge these limits, we collected a sample of 100 law journal articles mentioning “implicit bias training” published between 2017 and 2021. Of those 100 articles, 58 recommend implicit bias training and only 8 of those 58 express any skepticism about its effectiveness. Overall, only 19 articles express skepticism about implicit bias training. We end with recommendations for law journal authors, researchers, and practitioners for a more credible application of psychological findings in legal research and policy. Our focus is on how empirical research can best be used to solve our most important social problems, including racism.