I will be the first to admit that, as a science student, the scientific method chapter of each textbook was no more than a list to be memorized for the first exam and forgotten until my next science course. In developing a research experiment, however, I have come to realize that the scientific method resembles more of a road map, applicable and necessary to our everyday lives. Whether questioning why my taillight is out and diagnosing the problem, or reproducing the methods of a recipe, elements of the scientific method allow us to investigate the issues we face.
We can start by asking the right question
When designing experiments, scientists must use the scientific method rigorously to construct creative and testable questions, analyze their results, and communicate their conclusions. As part of this process, they are also responsible for examining the work of their peers. Reproducibility is a fundamental tenet of the scientific method that helps to guide the development of questions, for instance: has this been done before and how consistent were the results? The ability to reproduce an experiment and draw similar conclusions supports the validity of those conclusions.
Despite the fundamental nature of reproducibility, a survey by Nature found that scientific research may be in the midst of a reproducibility crisis. Out of 1500 responses, over 70% of researchers could not reproduce results from another scientist’s experiment and 50% could not reproduce results from their own experiment. While these responses seem to undermine confidence in scientific results, it is a small sample size and there are many factors that can contribute to the reproducibility of results in a lab.
A case study in reproducibility
One of the greatest contributors to irreproducibility is experimental design. The implementation of certain methods can exist as a gradient across labs. To highlight disparities in experimental reproducibility, let's look at a case study involving one of “hottest,†yet broadest, topics in environmental research: combating the effects of agricultural drought in crop species.
Drought is an issue of increasing concern as it lowers crop yield. According to the Intergovernmental Panel on Climate Change, we will most likely experience increases in both the frequency and intensity of droughts on a global scale during the twenty-first century. While it is reasonable to fear drought and widespread famine, it is 2017 and not 2012. A worldwide network of dedicated individuals are developing sustainable solutions to mitigate the potential consequences of drought.
How do researchers study drought?
Though droughting a plant may seem straightforward, consistently recreating this agricultural phenomenon is difficult within a laboratory setting. Essentially, drought occurs when plants have limited access to water, but the term itself is vague as there are many legitimate ways to invoke ‘drought' or water limitation.
In a laboratory setting, drought is most frequently performed as a dry down or through chemical stresses. Dry downs represent methods that physically withhold water. Like neglecting to water a houseplant, withholding water causes the soil to subsequently dry down. Chemical stresses introduce a chemical to the soil that prevents water uptake in the roots. While these are all valid ways to implement drought, each method invoked may elicit a similar or distinct plant response. Plants respond to their environment, and though these methods limit water, the variation by which they do so can alter the plant response.
Comparing Apples and Oranges
Results gathered utilizing these different types of water limitation methods are often compared, but that raises the question, should they be? According to this review article, results should be reproducible across different types of drought implementation methods, but does comparing them equate to comparing apples and oranges?
I attempted to answer these questions with a greenhouse study on Helianthus annuus, or cultivated sunflower. Sunflower is an agriculturally important commodity and the third-largest produced and distributed oilseed crop in the world. As such, many researchers are interested in studying the effects of drought on sunflower and how to best improve sunflower production even in the face of drought. To determine the comparability of different dry down and osmotic treatments, I grew multiple plants under each condition and measured their physical traits.
Implementation is subject to a gradient
The results of my experiment indicated that these drought methods performed similarly in some regards, yet differed greatly in others. For example, overall plant growth measured comparatively. On the other hand, root measurements were significantly different from each other. Intuitively, this makes sense as roots are below ground and are directly exposed to the specific type of drought stress, yet it is still surprising to see tissue above ground responding similarly despite the variable response below ground.
Though these results did not confirm reproducibility across methods for the whole plant, they did not discount the comparison of those methods either, further illustrating that implementation is subject to a gradient.
This investigation is not to say that scientific results should be inherently doubted, but rather that reproducibility should be considered when analyzing academic research. Reproducibility of results serves as a quality checkpoint in the scientific method; take the inconsistent results and irreproducibility of Dr. Andrew Wakefield's experiment linking autism and vaccines for example.
In a publish or perish world, the allure of novelty can outweigh the incentive to reproduce previously published results. Further, confidence and trust in the peer review process can allow researchers to accept data others produce without reproducing it themselves. That being said, reproducibility is attainable and a majority of scientists greatly value it. This is best illustrated in the same Nature survey, where 66% of researchers had some established protocol for redoing experiments and comparing results.
Personally, this principle of the scientific method has allowed me to appreciate the intentionality and authenticity of the academic community. As scientists, especially budding ones like myself, it's our job to uphold the standards of scientific research in our crusade for truth, education, and improvement. Don't just memorize the scientific method, live it.
About the Author
 Liana Mosley is a student from Savannah, GA majoring in Nutrition Science and minoring in Music at the University of Georgia. Aside from working as an undergraduate research assistant in the Burke lab, she enjoys running around Athens, singing in the practice rooms of Hugh Hodgson, and sharing her love for UGA with high school students on campus tours. For more information, contact her at ljm87515@uga.edu. |
