Steven Goodman: Statistical methods as social technologies versus analytic tools: Implications for metascience and research reform

Statistical methods can be viewed through different lenses; as technical tools to detect signals and quantify uncertainty, or as foundations for making epistemic claims. The use of similar technical tools in different sciences can obscure the profoundly different epistemic cultures across the sciences. Elements of these cultures include what constitutes sufficient grounds for making a claim, e.g. that an intervention “works”, and who is qualified to make that judgment.

If these differences are not acknowledged, which they rarely are, debates about how to improve the use of statistics in science often miss the mark, never reaching closure. This will be shown with recent empirical work on whet- her specialized statistical review is needed for research papers, efforts of a major science funder (PCORI) to improve statistical practice in its funded research, and controversy about the use of statistical significance in published research. I will discuss what this means for metascience and for improving statistical practice.
  

Steven Goodman

Professor of Medicine and Health Research and Policy, Stanford University

Steve Goodman, MD, MHS, PhD is Associate Dean of Clinical and Translational Research and Professor of Medicine and Epidemiology in the Stanford School of Medicine. He is chief of the Division of Epidemiology and directs a newly established office in the School of Medicine to improve “researcher readiness” and the reproducibility of laboratory and clinical research. He is co-founder and co-director of the Meta-research Innovation Center at Stanford (METRICS), a group dedicated to examining and improving the reproducibility, integrity and efficiency of biomedical research. His research is in the methods and philosophical foundations of statistical inference, particularly the proper measurement, conceptualization and synthesis of research evidence, with an emphasis on Bayesian approaches. He also has worked on the connections between ethics and scientific methods, particularly in interventional research.

Finally, he has a strong interest in developing curricula and new models for teaching the foundations of good scientific practice.Among his current national positions and recognitions included chairing the Methodology Committee of PCORI (Patient Centered Outcomes Research Institute), being awarded the 2016 Spinoza Chair in Medicine from the University of Amsterdam for his work in scientific and statistical inference, serving as scientific advisor to the national Blue Cross-Blue Shield technology assessment program and being senior statistical editor at the Annals of Internal Medicine, since 1987. Before coming to Stanford in 2011, he was at the Johns Hopkins Schools of Medicine and Public Health, where he directed their cancer center’s Division of Biostatistics and Bioinformatics and the Dept. of Epidemiology’s doctoral program.