Andrew Gelman: Embracing Variation and Accepting Uncertainty: Implications for Science and Metascience
The world would be pretty horrible if your attitude on immigration could be affected by a subliminal smiley face, if elections were swung by shark attacks and college football games, if how you vote depended on the day within your monthly cycle, etc. Fortunately, there is no good evidence for these and other high-profile claims about the effects of apparently irrelevant stimuli on social and political attitudes and behaviors.
Indeed, for theoretical reasons, we argue that it is not possible for these large and persistent effects to co-exist in the real world. But if the sorts of effects being studied vary greatly by person and scenario, then simple experiments will not yield reliable estimates of effect sizes. It is necessary to instead embrace variation, which, in turn, requires accepting uncertainty. This has implications for the practice of science and for the proper understanding of replication and other aspects of metascience.
Professor, Columbia University
Andrew Gelman is a professor of statistics and political science at Columbia University. He has published research articles on statistical theory, methods, and computation, with applications in social science and public health. He and his colleagues have written several books, including Bayesian Data Analysis, Teaching Statistics: A Bag of Tricks, Regression and Other Stories, A Quantitative Tour of the Social Sciences, and Red State, Blue State, Rich State, Poor State: Why Americans Vote the Way They Do.His ideas on metascience include type M and type S errors, the folk theorem of statistical computing, the freshman fallacy, the time-reversal heuristic, the Armstrong principle, the Javert paradox, Eureka bias, Clarke’s law, the piranha problem, and the garden of forking paths.