Jonathan Schooler: How replicable can psychological science be?: A highly powered multi-site investigation of the robustness of newly discovered findings

There has been an increasing concern among scientists regarding irreproducibility of scientific findings in general, and psychological findings in particular. To date, the understanding of reproducibility has been impeded by two related challenges:

1. the lack of transparency of the scientific record, and
2. the retrospective nature of reproducibility studies.

In order to overcome these obstacles, four labs conducted a large scale multi-site prospective multi-replication study.

1. Each lab independently discovered new psychological findings that were then systematically replicated by the originating laboratory and by the others, following a complex pre-specified sequence of various replications and analyses. In so doing, this project
2. developed a gold standard for replication protocol, in which every effort was made to design experiments and implement replications in a manner that simultaneously maintained ecological validity while maximizing the likelihood of full replicability, and tested whether the replications of newly devised experimental protocols are associated with declining effect sizes, even when all reasonable efforts are made to minimize such declines.

Although the project is still underway, preliminary analyses indicate that when a gold standard approach is applied psychological findings are highly robust.
  

Jonathan Schooler

Professor, University of California at Santa Barbara

Jonathan earned his BA at Hamilton College in 1981 and his Ph.D. at the University of Washington in 1987. He joined the psychology faculty of the University of Pittsburgh as an assistant professor that same year and became a research scientist at Pittsburgh’s Learning Research and Development Center. Named a full professor in 2001, he moved on to the University of British Columbia (UBC) in 2004 as professor of psychology, Canada Research Chair in Social Cognitive Science, and senior investigator at UBC’s Brain Research Centre.In 2007, he joined the faculty at UCSB. Jonathan pursues research on consciousness, memory, the relationship between language and thought, creativity, problem-solving, and decision-making. He is particularly interested in exploring phenomena that intersect between the empirical and the philosophical such as how fluctuations in people’s awareness of their experience mediate mind-wandering and how exposing individuals to philosophical positions alters their behavior. He is an Osher Fellow at the Exploratorium Science Museum in San Francisco, as well as a fellow of the Association for Psychological Science and the Society for Experimental and Social Psychology. His work has been supported by the National Institute of Mental Health, the Unilever Corporation, the Center for Consciousness Studies, the Office of Educational Research, the Bower Foundation, the U.S. Department of Education, the John Templeton Foundation, the Imagination Institute, and the Fetzer Franklin Fund.He currently is on the editorial boards of Consciousness and Cognition, Social Cognitive and Affective Neuroscience, Journal of Imagination, Cognition and Personality, and Psychology of Consciousness: Theory Research and Practice. Dr. Schooler is the author or co-author of more than two hundred papers published in scientific journals or edited volumes and was the editor (with J.C. Cohen) of Scientific Approaches to Consciousness, which was published in 1997 by Lawrence Erlbaum.