Discussion about this post

User's avatar
James Smoliga, DVM, PhD's avatar

As somebody that works in (and debunks) research within the biological and medical sciences, I find it fascinating how much overlap I see between what you describe and what I find in very different disciplines.

With that first decoy example having a correlation of r = -0.29... Even if the study was well performed, a correlation <0.30 is weak at best. And, I would argue, it is often the result of statistical noise that is bound to be found when one does a sufficient number of analyses. Indeed, I have shown this in my own work debunking the concept of "digit ratio" (a field in most correlations are in the 0.2 to 0.3 magnitude).

So many other examples of weak science (or even theory) being supported as facts.

This is why, as somebody who values science and the scientific method, I am skeptical anytime I see research quoted or a headline proclaiming "new study finds" about any topic. I want to trust science and scientists, I really do. But there are so many incentives to confirm hypotheses, rather than rigorously test them, that a healthy sense of skepticism is necessary.

We need to actually dig into the methods and results ourselves to critically evaluate them, otherwise it's hard to know if what is being summarized accurately reflects the research itself. That includes evaluating the authors' own conclusions to determine if they are directly supported by the data or reaching too far.

2 more comments...

No posts

Ready for more?