Many people seem to believe that science is an end-all be-all when it comes to discerning fact from fiction. But is our current theory of the experimental method really that powerful, or does it come with its own imperfections?
As you can imagine, science — more specifically the experimental or “scientific” method — comes with its own circle of competency. Science seems to work very well for stuff that is built into the physical world, stuff like physics and chemistry and biology where we can experience everything with our own five senses. But when we start getting into the abstract world, things begin to falter.
Take psychology, for instance. Psychology has no physical elements (beyond sensation and perception) and so it is based entirely on the abstract world of the individual, and how their minds might adapt and change over time. Just as a thought exercise, let’s say that Milgram’s Experiment is now known by the entire world’s population. Well, if the whole world knows it can be tricked by authority figures, a good chunk of the population will make the conscious decision to not get fooled. Therefore, if you were to then try the same experiment, you would likely find the results to no longer be statistically significant.
This is the key conundrum behind the replication crisis, the idea that old tricks which used to work on people now no longer do, and it’s causing us to have to rewrite most of our psychological knowledge. Some scientific diehards say the fault is at psychologist’s shoddy use of statistics, but the major culprit is instead likely that the scientific model as we know it just doesn’t work in this case. A ball won’t suddenly wise up and stop accelerating at 9.80665 m/s2, but humans can. So tight control of experimental values and logical statistics no longer work in this realm.
Same can be said in adjacent fields such as economics. There is no real “scientific” study of economics because any rule of economics can simply “go out of fashion”. The chemical properties to not go through fashion trends, but quantitative trading strategies certainly do. It’s not to say that all of our knowledge here is useless, or that studying these subjects are a waste of time. In fact, quite the opposite! When your entire model is broken, there’s a lot of interesting work to be done in fixing it. But first you have to realize that science isn’t perfect.
