May 10, 2023
On this episode of Unsupervised Learning, Razib talks to Adam Mastroianni, who runs the Experimental History Substack. Mastroianni was the inaugural guest on the Intrinsic Perspective podcast, hosted by Erik Hoel, where they discussed his post, The rise and fall of peer review - Why the greatest scientific experiment in history failed, and why that's a great thing (see also his follow-up, The dance of the naked emperors). Mastroianni opened a can of worms; the post has more than 800 likes and more than 330 comments. Razib asks Mastroianni about the fiercely positive and negative reactions to his contention that modern peer review has outlived its utility. They also unveil the historically contingent origins of the practice in the mid-20th century, and how it came to be seen as a holy enterprise necessary to science. Both agree that scientific publishing needs a paradigm shift; a topic that Razib tackled in 2014 with the Genome Biology comment Dragging a scientific publishing into the 21st century.
Razib and Mastroianni then discuss Experimental History, a Substack devoted to social psychology and meta-science. Why has Mastroianni decided to devote a substantial amount of energy to this project, as opposed to just publishing in journals? Experimental History touches on some of the experimental social psychology research Mastroianni has been involved in, but it also focuses on some of the generally understood findings in psychology and neuroscience, and why they’re true or false. In a world of academic science saturated with Ph.D. level researchers, Razib and Mastroianni explore the communication possibilities inherent in the Substack model.
Finally, Mastroianni unpacks his opinion that even many of the robust statistically significant findings in social psychology don’t matter. He believes that the lack of a single theory blocks proper understanding in psychology, and many of the results in his field are both uninteresting and fail to lead to a nontrivial increase in knowledge.