Lower pvalue threshold to 0.005
I a comment was published in Nature Human Behavior that suggested lowering the pvalue threshold from 0.05 to 0.005. Subsequent discussion has occurred on Andrew Gelman’s blog and in an associated paper that suggests we need to remove null hypothesis significance testing from its gatekeeper role in science, i.e. papers can only get published if they have a pvalue of less than 0.05.
My opinion is much more in line with the discussion and paper on Gelman’s blog. I often tell students of my entry into statistics where I came from being a research engineer and a lab tech in a fruit fly genetics lab to wanting to obtain a master’s degree in statistics so that “I would know exactly how to analyze any type of data set.” I quickly realized the statistics world was much bigger than I realized and that statistics is not a black-and-white process. Similarly, I discourage colleagues and students from asking what is the “right” analysis? to asking what analysis can answer my scientific questions of interest.
I believe we teach statistics students a process that suggests there is one correct way to analyze a data set and one and only one result from that analysis that is scientifically meaningful. The implicit suggestion here is that every data set as a single method of analysis and that if the result isn’t scientifically meaningful then it should be thrown in the trash. In my opinion, this education process has led partially, if not wholly, to the current issue in science of replicability.
In my opinion, we need to start teaching students what the results of a statistical analysis really mean and get them away from thinking of a significant pvalue as the ultimate result of the scientific process. To pass the buck slightly down the road, in order for students to truly understand what the results mean they typically need to have a stronger mathematical background than they currently have. So let’s work toward boosting the mathematical capability of all students toward the goal of improving their ability to interpret science.
blog comments powered by Disqus