The reproducibility crisis has affected the scientific community for more than a decade now. This crisis has raised doubts about the validity of scientific studies and papers published in academic journals. When scientists and researchers attempted to reproduce the experiments in these papers, they were unable to yield the same results. Recently, researchers could reproduce only 8 out of 50 cancer research papers in the long duration of 5 years. After cancer biology, as a recent study has shown, the reproducibility crisis has spread to social sciences.
Reproducibility in Social Sciences
A recent study in Nature’s Human Behavior illustrates the reach of this crisis. A collaborative team of researchers from a group of universities decided to look into it. The team attempted to reproduce the findings in 21 studies originally published in Science and Nature, two leading scientific journals. These articles were based on several topics of social science such as link between analytical thinking and religious skepticism, anxiety and performance during exams etc.
The results obtained confirmed the reproducibility crisis in the social sciences field. More than one-third (8 of 21) of the published studies could not be reproduced. In addition, the evidence found in the other studies showed weak links between the findings in the original studies and the conclusions of the reproducibility tests.
What do the Researchers Say?
Since two of the most prestigious journals in the scientific community had published these articles, these results came into notice. Brian Nosek of the University of Virginia in Charlottesville says that these types of high-profile findings can receive significant media attention. However, the attention is not always positive; some of it may be negative as well.
Using the study of stress and performance during exams as an example, Nosek feels that these studies still yield important information. However, he did admit that some findings in the study could not be replicated. He says that this could be due to some minor change, like the differences in the sample sizes used in the experiments. In that case, Nosek says, “studies that obtain a significant result are likely to be exaggerations of the actual effect size.”
Reproducibility in Crisis: True or False?
Some claim the findings are exaggerated while some claim them to be simply false. However, this is still a serious issue for the larger scientific and academic communities. According to Magnus Johannesson of the Stockholm School of Economics, the results show that cautious interpretation of ‘statistically significant’ scientific findings is a must. Even if the most prestigious journals publish the research, the replication of the experiment is mandatory. To address this, the academic community conducted several reproducibility studies over the years. One example is the Reproducibility Project: Cancer Biology (RP:CB) that began in 2013. This project set out to test the reproducibility of 50 significant studies in cancer biology. To date, the results for 10 replication studies have been published on eLife.
Future of Scientific Reproducibility
The implications of the reproducibility crisis are wide-ranging in social sciences and academia as a whole. Two years ago, the field of psychology made headlines when two classic studies failed reproducibility tests. According to John Ioannidis of Stanford University, more such studies could improve the accuracy of the knowledge produced. Ioannidis and many others see the attention that reproducibility crisis received as beneficial for the scientific and academic communities. More such research projects, like the Reproducibility Project and the study in Nature’s Human Behavior, will bring out the reproducibility crisis in academia, thereby eliminating it.
In the end, the initiative taken to investigate the validity of scientific studies can help society in endless ways.
If this approach works in academia, what other areas of life can benefit from this investigation and scrutiny? Please share your thoughts with us in the comments section below.
