Evaluating the necessity of replicability in knowledge production: insights from natural sciences and history

Replicability is complicated and multifaceted, especially when looked at from a scientific point of view. When the results of two studies agree, it is most likely that the claim is based on reliable information as experimental findings can be due to chance. Therefore, replicability aims to reduce the likelihood that a finding is due to chance, it is an indicator of accuracy. As a child, I remember being told by my science teacher that the results from my experiment would only be valid if it had been done three times. In my essay, I will examine whether replicability is a necessary element in the production of knowledge through the Human and Natural sciences. I chose these areas of knowledge inspired by the following claim “Replicability is almost universally accepted as the most important criterion of genuine scientific knowledge.” (Hubbard). Across the Natural sciences replicability is considered an essential criterion for evaluating the quality of scientific knowledge and controlled experiments are often used to study cause and effect. Considering the Human sciences, replicability is necessary to produce knowledge in some cases, however, it may be more difficult to achieve due to the subjective nature of the subjects. Achieving replicability can ensure that we are not dealing with a single coincidence and are instead dealing with regular, repeatable events that others can test (Becker). This enables positive scientific outcomes that may be revolutionary for our future. So, I will assess the necessity of replicability together with its incapacity in certain circumstances to produce knowledge in the Natural and Human sciences.

Firstly, in the Natural Sciences, the scientific method along with the correct amount of data requires replicability to produce knowledge. Indeed, the self-correcting mechanisms of the scientific method depend on the ability of researchers to replicate the findings of published studies to strengthen evidence and build upon existing work (“Six factors”). Many factors can contribute to a result due to chance such as small sample size and low statistical power. In 1998 the physician Andrew Wakefield along with his colleagues published a case series in the Lancet, which suggested that the measles, mumps, and rubella vaccine was linked to autism in children (Rao). Despite his small sample size of 12 and the speculative nature of the conclusions, the paper was widely publicized, and there was a decrease in vaccinations because parents were concerned about the risk of autism after vaccination (Rao). Immediately after the release of the paper, studies were conducted refuting his claim between the vaccine and autism. By having a larger sample size and replicating the effects of the vaccine on children they proved his claim wrong. Wakefield’s data was insufficient due to his small sample size, it was unrepresentative which increased the chance of assuming as true a false premise. Across Wakefield’s study, the lack of replicability led to the production of a false theory that was mistaken for knowledge. Therefore, replicability was proven necessary in the production of credible knowledge. Fundamentally, this emphasises that replicability is expected to produce a reliable scientific claim, although in some cases it may be impossible due to certain circumstances. This links to scientific certainty as the claim isn’t influenced by perspectives or community bias as it is justified. However, it is difficult to meet these expectations. Scientists have the responsibility to ensure that their findings are reliable to avoid negative influences on our society. Moreover, it is a necessity for them to provide accurate scientific outcomes allowing the scientific world to evolve. Scientific findings in biomedical research are often unable to replicate to verify the reliability of the initial findings, wasting time and resources as well as jeopardizing the credibility of scientific findings. (“Six Factors”).

Secondly, regarding the natural sciences more generally, there have been multiple replication failures in Biology that have proven that once replicability is done correctly using the correct methodology, it is necessary to produce valid knowledge. Two different labs thought they used what they thought was the same method to test breast tissue (Bissel), but their results were different. When the experiment was done in both labs simultaneously, the researchers noticed that one lab moved the cells gently while the other used a more forceful method. Since these methods are commonly used, neither researcher thought to go into more detail about the mixing process (Bissel). Cells in this environment are extremely sensitive, and any shift in their environment can alter results (Bissel). This links to Stanford medical researcher John Ioannidis’s claim that the importance of reproducibility does not have to do with “ensuring the ‘correctness’ of results, but rather with ensuring the transparency of exactly what was done in a given line of research” (Ioannidis). Before this change was made to how the experiment was done, they were unaware that the mixing process could affect the results. This case reinforces the fact that although replicability was impossible due to a methodological issue, replicating it correctly, could have led the scientists to produce relevant knowledge on breast tissue.

On the other hand, in the natural sciences, there are exceptions where replicability is impossible to produce credible knowledge and alternate methods are used in Biology. Phylogenetic analysis is the study of how species evolve through genetic changes (Dutta). In this field of Biology, replicability is not necessary to produce knowledge as it is impossible to replicate historical events on certain species that have evolved in the past in a laboratory. In 2014, scientists discovered the dinosaur species Dreadnoughtus schrani which is the largest land animal for which body mass can be accurately calculated (Lacovara). This species of dinosaurs originally came from Argentina, and by nearly completing the skeleton of the dinosaur, they provided new insight into the morphology and evolutionary history of this species (Lacovara). Indeed, the scientists used different methods to produce knowledge on this species by using measurements of the dinosaur’s anatomy, statistical analysis, and comparison to already known dinosaur species.

Although, the Human Sciences differ greatly from the Natural Sciences, replicability is difficult to produce knowledge due to the high risk of failure. The field of Psychology is undergoing a replication crisis as several research findings do not replicate. The term “crisis” is indicative of an emergency, emphasizing that replicability may be necessary to produce knowledge, but it is failing to do so in this AoK. After a paper is published, other researchers might cite it in their work spreading errors (Piper). The fact that the replicability of published findings is <30% in social psychology and ~50% in cognitive psychology has therefore justifiably stimulated much concern and debate (Lewandosky). Brian Nosek is a psychologist from the University of Virginia in Charlottesville, and he recruited 270 scientists to reproduce the findings of 100 previously published studies (Raloff). The results of his study presented that only 35 of those experiments could be replicated (Raloff). This links back to the idea of a replication crisis; these studies may not have been possible to replicate due to unachieved statistical significance or unexpectedness of the initial studies.

Finally, in certain circumstances, replicability is deemed impossible to produce knowledge in psychology due to the unpredictable nature of humans however this inability allows us to produce knowledge alternatively. If the replication study collects new data but follows different methods, then it counts as a ‘conceptual replication’ (Holbrook). Psychology is the study of the mind and behavior, as human behavior is unpredictable and influenced by numerous complex factors, it can therefore be difficult to replicate experiments on humans and obtain the same results. The Stanford prison experiment was conducted by the social psychologist Dr Philip Zimbardo in 1971 (Konnikova). He stimulated a prison environment where paid participants were assigned different roles. Zimbardo concluded that when innocent people have power over others, they abuse this power whereas the powerless are driven to submission (Resnick). Around the world, the experiment was replicated. In Iceland the experiment was replicated considering human ethics and morals and while treating the patients with respect. As Iceland took a different approach to the experiment, they did not have the same results as the original one. That is to say, Icelandic prisons focus on a rehabilitative approach rather than a tyrannical one (“Stanford Prison”). Subsequent replications have been unsuccessful in obtaining the same results as the original study. In the natural sciences, if the outcome was to be different, we would believe that it is invalid or wrong which isn’t the case in the human sciences. This experiment allows the exploration of a new avenue on how different circumstances lead to different human behaviors. Consequently, the different results that have been published are producing knowledge that has been unknown until now. Evidently, there are no laws that govern our behavior and psychologists are aiming to look for factors that do influence our behavior.

In conclusion, replication is necessary for the natural and human sciences to produce knowledge given certain conditions. Replicability has been shown to contribute to data integrity in several fields by allowing scientists to confirm a discovery by doing the same experiments repeatedly. Scientific findings are supposed to be objective, that is independent of our wants or preconceptions, with the right training and apparatus this should lead to similar results producing reliable knowledge. Yet, as observed through the counterclaims, replicability may not always be necessary across these AoK’s. Through the examples studied, replication may not be possible due to failure, the unpredictable nature of human behavior and if a discovery can solely be traced from a unique event. Therefore, the enquiry whether replicability is necessary for the production of knowledge is ambiguous as it depends on the possibility of a study to be replicated or the many components that allow a study to be replicated.

Becker, Markus, and Nathalie Lazaric. "The influence of knowledge in the replication of routines." Core.ac.uk, 18 Feb. 2010, core.ac.uk/download/pdf/47835348.pdf. Accessed 14 Feb. 2023.

Bissell, Mina. "Reproducibility: The Risks of the Replication Drive." Nature, vol. 503, no. 7476, Nov. 2013, pp. 333-34, https://doi.org/10.1038/503333a. Accessed 14 Feb. 2023.

Dutta, Sanchari Sinha. "What is Phylogenetic Analysis?" News Medical, 9 Mar. 2021, www.news-medical.net/health/What-is-Phylogenetic-Analysis.aspx. Accessed 15 Feb. 2023.

Guttinger, Stephan. "The Limits of Replicability." European Journal for Philosophy of Science volume, 15 Jan. 2020. Springer Link, https://doi.org/10.1007/s13194-019- 0269-1. Accessed 11 Dec. 2022.

Holbrook, J. Britt, et al. "The humanities do not need a replication drive." CWTS, 21 Jan. 2019, www.cwts.nl/blog?article=n-r2v2a4&title=the-humanities-do-not-need-a- replication-drive. Accessed 12 Dec. 2022.

Hubbard, Raymond. "The Importance of Replication Research—Significant Sameness." Corrupt Research: The Case for Reconceptualizing Empirical Management and Social Science, 2016. SAGE, https://dx.doi.org/10.4135/9781506305332. Accessed 14 Feb. 2023.

Ioannidis, John P. a. "How to Make More Published Research True." PLoS Medicine, vol. 11, no. 10, 21 Oct. 2014, p. e1001747, https://doi.org/10.1371/journal.pmed.1001747. Accessed 14 Feb. 2023.

Konnikova, Maria. "The Real Lesson of the Stanford Prison Experiment." The New Yorker, 12 June 2015, www.newyorker.com/science/maria-konnikova/the-real-lesson-of-the- stanford-prison-experiment. Accessed 14 Feb. 2023.

Lacovara, Kenneth J., et al. "A Gigantic, Exceptionally Complete Titanosaurian Sauropod Dinosaur from Southern Patagonia, Argentina." Scientific Reports, vol. 4, no. 1, 4 Sept. 2014, https://doi.org/10.1038/srep06196. Accessed 15 Feb. 2023.

Lewandowsky, Stephan, and Klaus Oberauer. "Low Replicability Can Support Robust and Efficient Science." Nature Communications, vol. 11, no. 1, 17 Jan. 2020, https://doi.org/10.1038/s41467-019-14203-0. Accessed 14 Feb. 2023.

Paloš, Andrijana Perković. "Replicability in the Humanities." The Embassy of Good Science, 21 Apr. 2021, embassy.science/wiki/Theme:D4b207e5-cbcc-452b-aced- 4d7357c8dcb0#cite_note-:0-1. Accessed 12 Dec. 2022.

Peels, Rik. "Replicability and replication in the humanities." Research Integrity and Peer Review, 9 Jan. 2019, https://doi.org/10.1186/s41073-018-0060-4. Accessed 12 Dec. 2022.

Piper, Kelsey. "Science has been in a 'replication crisis' for a decade. Have we learned anything?" VOX, 14 Oct. 2020, www.vox.com/future-perfect/21504366/science- replication-crisis-peer-review-statistics. Accessed 11 Dec. 2022.

Raloff, Janet. "When a study can't be replicated." ScienceNewsExplores, 11 Sept. 2015, www.snexplores.org/article/when-study-cant-be-replicated. Accessed 14 Feb. 2023. Rao, T. S. Sathyanarayana, and Chittaranjan Andrade. "The MMR vaccine and autism -

Sensation, refutation, retraction, and fraud." Indian Journal of Psychiatry, Apr. 2011. National Library of Medicine, https://doi.org/10.4103/0019-5545.82529/. Accessed 11 Dec. 2022.