When we view research, we typically see those that are deemed “successful” research. What that means is that the test worked — that they succeeded in rejecting the null hypothesis, aka that there was no effect at all. However, when we’re only looking at what worked — we’re not looking at what didn’t work. And that means we’re missing out on half the puzzle.
Before we dive in any deeper on this, I want to give a little bit of an introduction to the economy of academia. Essentially the goal within research is to obtain tenure, or “indefinite professorship”. Once you get to tenure, you can do whatever research you want; the department or university effectively no longer has any say in the matter. However, before this, the goal is twofold: to produce a large amount of research, and to produce effective research. That is because this is really all the university is looking for when they’re looking for professors to tenure. And so, we end up having a bias towards small sample quick studies done in order to produce successful results. The conclusion? A system that is completely unsustainable for science.
The goal of science ought to be to find what works and what doesn’t work. When we scrap out what doesn’t work, our overall view of the world ends up getting hurt by it. Of course, a lot of this is driven by the aforementioned tenure track system, prioritizing the research “that works”. What universities ought to understand is that null effects can be just as powerful as significant effects. When you avoid one, you become blind in one eye. You lose depth. And, if we want to continue sustaining our technological trajectory, we cannot lose depth.
…
Anyway, that’s all for this one. If you want to keep in touch, check out my biweekly newsletter! Following this will give you the low-down of all the new stuff I’m working on, as well as some things I found interesting throughout the biweek. As an added bonus, you’ll also receive the Top 10 Tools I Use on a Daily Basis to help better manage your workload and do higher quality work in a shorter amount of time.