I was merit hired from a summer work-study job in a basic medical science research lab halfway through my university BS degree - a dream job that fit perfectly with a very competent EE BS followed by a Molecular Biophysics and Physiology PhD lab-head and after a month and a half working there we had peer review publishable data and he created a position for me.
A dream job, man lab-tech, me, 'boss' and first 2 years we were on fire including a Science published co-authorship paper {in here}
.. and I was considering getting a Masters or maybe PhD and stay in the field. Lab-tech left to enter medical school and a woman labtech was hired and we got along okay but I started to wonder if she had a college degree because of how clueless she was, and although I was helpful until I realize that she resented it and would rather be wrong and waste entire days and rare medical samples.
She was pretty much the average of incompetence labtecs around the department - all women for white man genociding HR policies under the excuse of Affirmative Action .. the workplace started to become toxic to me even though I was always too busy to interact with them much, yet they so useless to lab that they rarely had anything to do and so drama, Witch-Whisper-Web lie spreading reputational-destroying productivity and personally destructive ..
.. one day our women labtech entered the lab and went straight to me and said my laughter was bothersome to the cnt across the hall .. and I was speechless and should have left that once dream-job become psychologically painful. I should of left because many women that worked near hated a man that had Joy in life and work, and had time and co-vomit-poison friends that also hated any man that was productive, and loved-life.
I had another Job in an EE manufacturing and R&D company, will entire professional engineers as decent hard-work men, and I liked that job (and when I left they hired 3 people to cover what I was doing) but about two years into that job the co-owner asked for a high priority task done soonest and I couldn't fit it in so we went through the other projects and found some things the women office manager could do, so .. she did not want to do it and kept wasting my time and eventually I told her I did have time for her foot dragging, go talk to Jack about or leave me alone and do some work.
Well, within a short time an older woman friend that worked on the line stop by my office to warn me that Ms Better-dead was passing ugly roomers about me, but would specify. The job went from very productive very good working environment to some kind of endless vile-horse-shit random unusually problems and hasseals.
I would rather walk Infront of a fire-squad and be killed then be near those kind of inhuman horrors .. if a man acted half of psychotic I would consider his arranged death to be a safety matter for me and others.
Oh, you say you once knew a co-worker that as far as you could tell would not be better for all in grave, and I say -FU, every time I hear about a school, workplace, or public shooting I always wonder if he killed the Witch and how many of her minions and if managers and owner may Just they killed also for forcing him to work in a vicious toxic hell-hole to eat and pay-rent. In a just world those people would hang over the corpse of their entire older adult blood-line and maybe their social friends blood-line.
Well, I missed my relevant to article point but I better stop here of I might say something that hurt VagFeelies.
My later wife worked in the same department doing molecular biology research as a Post-Doc and while getting to know her she happened to complain that her lab co-worker Masters or Post-doc also was untrustworthy, and she caught her making-up data, and future-wife said she had run into similar women researcher in her previous lab. The PI would say what he was expecting from her experiment and like magic she would get that result and super-fast, and PI liked her efficiency and would start to complain to future wife if she got different results or took a bit longer.
After marriage she ran a core microBiology lab that supported a number of researchers in a few departments and she never hired a woman I just realized that. She was a fucking sharp productive hard-working professional and so like me found most meritless-male-hating cnt quota-women hires a F-ing workplace and productivity destroying hazards, less useful than a pile of two-week old poison vomit.
Synagogue of Satan's Golems everywhere and destroying every profession and any man that sniffs of masculine virtue .. Fatherless Witch Raised baby-murdering sexually-pedo mutilating children's minds-body-souls as Satanic 5th column FemNazi women - destroying the West and our good names, our murderers love-joy-hopes of Life in every family and every place men used to be productive and safe in.
We desperately need more statistical and scientific literacy in this world….thank you so much for the detailed analysis of why EVERYONE should question every single “studies show”. Close to my heart:). I found this article fascinating as a former research psychology major and current K-12 educator. I love that you are addressing the issues with even peer-reviewed literature, to say nothing of works that have undergone no such process and are nonetheless touted as “fact”. I quite literally confront it and its effects daily in public education, not least around what practices are “quality teaching” (from what I’ve seen in what passes for the literature in my M.Ed. program and now, I’d say magnify the flaws in general psychological study by a factor of at least 50 for those focused on education, and then design an industry around them). Policy decisions, curriculum, and the endless and almost ubiquitously useless professional development and constant educational “reforms” that dictate the currents of my career all boil down to decidedly NOT robust evidence. Seriously, it’s bad. Thank you for this analysis!
I stopped trusting any studies less than 20 years old in a very wide range of topics and groups of people that are Woke-Protected. Average IQ in nations are worthless, for example.
What topics and groups you ask? Any topic or subjects-groups you would as soon put you dick in a blender that relay on Soros Foundation Sicked mind-raping Wikipedia entries. Last I looked their entry on 'Race' was a F-ing joke. I included some relevant information on the recent Democratic Abortion protection bill that did not go through, a loop-hole that allowed partial birth unborn baby torturing to death for Sacrifices to Satan or a particular Demon for unHoly favors and powers. Well, I corrected a statement and included the basic information and reference URL and the 'editor' removed it and threatened to lock-out my account for no sensible reason.
Google removed National Blood Bank studies that routinely show near 1/3 of children have wrong father listed, based on Blood-types and such. 1/3 of woman will lie like-damaging psychological crippling lies to entire extended families, to husband, to the child, and the real father, all those she pretends to love most. And if she caught - is she publicly tasered till pissing and screaming? No, she divorce-rapes her victim husband and destroys the family and the children's hopes for success in life.
But Satan's minions - I mean Women!, don't worry Google will support your soul-murdering lies.
Similar for a 3year comprehensive study of rape accusations filed with a police department that takes seriously investigating them well .. 40% were proven false, but who know how much higher the real number is? But it was on the FBI web-site and then it suddenly was not, and I found it in an archive.
I sometimes work with 3 different AI systems and I spent many hours finding the demonic presences in them so I can know what subjects I can trust working with it.
All the 'Professional' that actively lied and silently lies around the Demonic Covid social personal and economic attack and mass murders .. any Professional that is doing well in many of 'political' fields are assuredly Satanic Tools. No one not sacrificing trafficked children to Satan can do well these days in this deep level of Hell.
Perhaps not a coincidence that all of the claims above support conclusions whose polar opposite would not be allowed to be published at all. Of course you would expect contorsions, especially in a field with inherently low levels of precision.
I completely agree with your suggestion not to take a reported finding too seriously until it has been independently and directly (not conceptually) replicated in a study with high power. However, this suggestion applies not only to reported findings but also to reported "failures to replicate." Therefore, I think you need to inquire more deeply before saying things like this: "In this context, it was then not surprising that the long-term success rate for replications in psychology has hovered right around 50% (e.g., Boyce et al., 2023; Open Science Collaboration, 2015; Scheel et al., 2021)." The tables in OSC2015 indicate 92% power for their replications. However, few realize that those power calculations were based on inflated observed effect sizes reported in the original (also low power) experiments, not on a reasonable estimate of their true effect sizes. When you take into account the inflation of a reported effect size using a p < .05 filter (the filter used in the original studies), statistical power in OSC 2015 was not far from 50%, which means that what they reported is not far from what you'd expect if all of the original experiments reported true positives (not 50% false positives). The findings that fail to replicate when tested again with high power are not so much the everyday studies conducted by experimental psychologists (contrary to what OSC2015 implies) but the studies associated with sensational claims -- that is, studies associated with low prior odds of being true. This is why people are pretty good at predicting which findings will replicate and which will not. You say that the 50% successful replication rate baseline might be too high, but I think it is almost certainly too low. My 2 cents!
Great article. Just the perfect combination of snark and great reasoning.
A quibble.
I work in a complex business field across a wide variety of industries, and was CEO / CTO for a non-profit research organization which worked to create reliable worldwide standardized measures, practices and processes in addressing business issues.
One simple measure of complex failure was poor “perfect order reliability”. The calculation involved reducing the problem to four types of failure. In business it’s slightly different than science but conceptually identical.
The standard is that a purchased entity can be late, incomplete, damaged, or have inaccurate paperwork (wrong invoice).
To calculate reliability you first classify orders in those four attributes, sum the number which have none of those attributes, and divide by the entire base count.
A common arithmetic failure is to either multiply % in each category of failure in a sample set, or to add - both are completely wrong. [I burst out laughing when I first read the origin of “intersectionality” a completely false set of concepts. They fail at elementary arithmetic.]
Consider two sets of four orders. In the first set only a single order fails in all four syndromes. In the second set each order fails in its own way.
Multiplying the %’s (75% of the orders are perfect) in the first case gives you a reliability level of 31%. The actual reliability is 3 perfect of four or 75%.
In the second case you have a reliability level with the same logic of 31% but the reliability is actual 0% - there were no perfect orders.
Data samples must not be correlated, overlapping - ideally “orthogonal” vectors so to speak when you look at reliability metrics.
Companies often claim great % reliability, often in the 80%+ range, but when you apply the correct measure they are quite bad. It is not unusual to encounter a company who on a month to month basis, has 0% reliability. I also encounter companies who have terrible reliability and simply reject the metric. It’s quite crushing to realize customers dislike doing business, and easier to make excuses. Customers abandon the company.
I would apply the same logic to your observation.
Consider 6 papers: a single paper can be faulty due to faulty citations, faulty assertions, faulty data/arithmetic, lacking contrary evidence, censored, evidence, MSU (and I note as well citing outdated evidence). Faulty data/arithmetic is so widespread that it’s not hard to glance through papers and get a feeling instantly that there’s a problem.
The same issue is in play as orders.
Consider two sets of papers, one has a single paper defective in 6 categories; the other has each paper defective in one category. Both situations might seem to be 38% reliability but of course the first has 83% reliability and the second has 0% reliability.
In the business world i consider the orders for a company as a whole over a time period in benchmarking against other companies in the same or similar industries.
You may consider a few things to strengthen your case. First, I’d partition the data by field and then by institution - there are a huge number of research institutions (analogous to companies) and their order fulfillment is research (papers and other artifacts, like data, hypotheses, experiment designs, materials, patents, methods, trade secrets - other IP)
I would apply your imperfect paper set to the partitioning logic I outlined to all the output of by industry by institution, and you can easily see exact outcomes.
When we apply perfect order logic to a naive company the results are usually crushing, but smart companies simply begin partitioning failure by syndrome and attacking the core problems.
Institutions and industries (sciences) are not used to being measured by reliable research output, often only by citation indexing. These would all be naive companies in the control processes business use.
Your thesis was that 75% of psychology research is faulty. By analogy to naive companies, it is entirely possible that entire institutions in the field have zero credible research - none at all.
An example is in fields contemplating “gender identity”. The term was invented by two scientists in the 60’s to support positions they believed true with trans delusional patients, patients with transvestic fetishism, and a smattering of children with puberty dysphoria.
The principal experiments (not research: anecdotal rather than systematic) were both grotesquely unethical and masked the facts that the research subjects committed suicide as an outcome.
“Gender identity” is not a scientifically supported term in any dimension, similar to implicit bias or stereotype falsification.
Use of it in research (to me) invalidates the premises involved. That means that, for instance, all documents from the APA, WPATH and others in the field of “gender identity” are faulty as a starting point, citing an entity “gender identity” which itself a combination of MSU, censorship, faked/faulty/masked data.
I’m more cheerful than you, while being very pessimistic. I know empirically that generation of faulty work products can be corrected by systemic efforts at the institutional level.
Another great article. I can see why you use Gould's definition of a fact but it is still a bad definition and this is perhaps why you have often found yourself using scare quotes around the word ‘fact’. A fact is a way that the world IS, more technically, it is a state of affairs that obtains. States-of-affairs, and therefore facts, are not the kind of thing that can be confirmed or disconfirmed.
What can be confirmed are statements that state states-of-affairs. A statement stating a fact is a truth. Scientific statements are used to make claims about what the facts are and such claims can be the outcome of a body of scientific research. If that body of research is sufficiently comprehensive and methodologically sound many of its claims are scientifically justified statements and belief in them is justified. It is such scientifically justified statements that can be said to be ‘confirmed to such a degree that it would be perverse to withhold provisional assent’. Statements that state a fact can never turn out to be false but scientifically justified statements can turn out to be false.
Consequently, it is impossible for Gould’s definition to be good for two reasons. Gould's definition should instead be given as a definition of when a claim is a scientifically justified statement and when belief in that claim is scientifically justified. I would still reject it as a definition, though. Rather, his perverse-to-withhold-assent condition is a usefully short generally reliable indicator of a scientifically justified claim.
I get it but, as usual, this approach has a very strong subdisciplinary bias. In my area, cognitive psychology/cognitive neuroscience, I hardly know any (systematically) non-replicable findings. Stroop effect, Simon, flanker, memory effect, recency etc., learning, attentional blink, you name it. Some higher-order interactions may be context-dependent, but the basic effects (orignally obtained with dramatically small samples, sometimes just the two authors) are super solid. So most of this handwaving business may be restricted to effects and phenomena that are more penetrated by culture. That's not so surprising, but please don't over-generalize from your own (admittedly messy) field...
I think you might be confusing "social psych is really bad" with "cognitive is really good." There is good evidence for the former, not so much the latter.
How much grant money is available for replication studies? Grantors aren't interested in reinventions of the wheel: received wisdom/urban myth/widely held misconception:?
Well, I was thinking of the theoretically relevant outcomes of theoretically motivated, theoretically important experiments conducted by experienced researchers (that‘s what my attentional span is optimized processing), but not of cherry-picked (based on what graduate students find „interesting“ and, I guess, unlikely to replicate at face value) studies of unknown relevance, scored by two unexperienced graduate students according to „subjective replicability“ using unknown, if any, criteria…
Great article Lee! The default position in science is to remain skeptical until there is sufficient evidence to cross Gould’s threshold. Unfortunately I think too many people including psychologists want to claim that they have special knowledge. Perhaps too many PhDs do this to justify the costs of earning a doctorate. I used to be guilty of it
I was merit hired from a summer work-study job in a basic medical science research lab halfway through my university BS degree - a dream job that fit perfectly with a very competent EE BS followed by a Molecular Biophysics and Physiology PhD lab-head and after a month and a half working there we had peer review publishable data and he created a position for me.
A dream job, man lab-tech, me, 'boss' and first 2 years we were on fire including a Science published co-authorship paper {in here}
Full Science paper in this article;
AI generated audio overview
https://notebooklm.google.com/notebook/216ccfdf-a26d-486d-a3d7-4ee77cd7fe79/audio
"Multiverse Journal - Index Number 2215: 21st May 2025, Some Background, Example of Basic Medical Science Research Results"
https://stevenwork.substack.com/p/multiverse-journal-index-number-2215
.. and I was considering getting a Masters or maybe PhD and stay in the field. Lab-tech left to enter medical school and a woman labtech was hired and we got along okay but I started to wonder if she had a college degree because of how clueless she was, and although I was helpful until I realize that she resented it and would rather be wrong and waste entire days and rare medical samples.
She was pretty much the average of incompetence labtecs around the department - all women for white man genociding HR policies under the excuse of Affirmative Action .. the workplace started to become toxic to me even though I was always too busy to interact with them much, yet they so useless to lab that they rarely had anything to do and so drama, Witch-Whisper-Web lie spreading reputational-destroying productivity and personally destructive ..
.. one day our women labtech entered the lab and went straight to me and said my laughter was bothersome to the cnt across the hall .. and I was speechless and should have left that once dream-job become psychologically painful. I should of left because many women that worked near hated a man that had Joy in life and work, and had time and co-vomit-poison friends that also hated any man that was productive, and loved-life.
I had another Job in an EE manufacturing and R&D company, will entire professional engineers as decent hard-work men, and I liked that job (and when I left they hired 3 people to cover what I was doing) but about two years into that job the co-owner asked for a high priority task done soonest and I couldn't fit it in so we went through the other projects and found some things the women office manager could do, so .. she did not want to do it and kept wasting my time and eventually I told her I did have time for her foot dragging, go talk to Jack about or leave me alone and do some work.
Well, within a short time an older woman friend that worked on the line stop by my office to warn me that Ms Better-dead was passing ugly roomers about me, but would specify. The job went from very productive very good working environment to some kind of endless vile-horse-shit random unusually problems and hasseals.
I would rather walk Infront of a fire-squad and be killed then be near those kind of inhuman horrors .. if a man acted half of psychotic I would consider his arranged death to be a safety matter for me and others.
Oh, you say you once knew a co-worker that as far as you could tell would not be better for all in grave, and I say -FU, every time I hear about a school, workplace, or public shooting I always wonder if he killed the Witch and how many of her minions and if managers and owner may Just they killed also for forcing him to work in a vicious toxic hell-hole to eat and pay-rent. In a just world those people would hang over the corpse of their entire older adult blood-line and maybe their social friends blood-line.
Well, I missed my relevant to article point but I better stop here of I might say something that hurt VagFeelies.
God Bless., Steve
Oh, yea, I remember.
My later wife worked in the same department doing molecular biology research as a Post-Doc and while getting to know her she happened to complain that her lab co-worker Masters or Post-doc also was untrustworthy, and she caught her making-up data, and future-wife said she had run into similar women researcher in her previous lab. The PI would say what he was expecting from her experiment and like magic she would get that result and super-fast, and PI liked her efficiency and would start to complain to future wife if she got different results or took a bit longer.
After marriage she ran a core microBiology lab that supported a number of researchers in a few departments and she never hired a woman I just realized that. She was a fucking sharp productive hard-working professional and so like me found most meritless-male-hating cnt quota-women hires a F-ing workplace and productivity destroying hazards, less useful than a pile of two-week old poison vomit.
Synagogue of Satan's Golems everywhere and destroying every profession and any man that sniffs of masculine virtue .. Fatherless Witch Raised baby-murdering sexually-pedo mutilating children's minds-body-souls as Satanic 5th column FemNazi women - destroying the West and our good names, our murderers love-joy-hopes of Life in every family and every place men used to be productive and safe in.
We desperately need more statistical and scientific literacy in this world….thank you so much for the detailed analysis of why EVERYONE should question every single “studies show”. Close to my heart:). I found this article fascinating as a former research psychology major and current K-12 educator. I love that you are addressing the issues with even peer-reviewed literature, to say nothing of works that have undergone no such process and are nonetheless touted as “fact”. I quite literally confront it and its effects daily in public education, not least around what practices are “quality teaching” (from what I’ve seen in what passes for the literature in my M.Ed. program and now, I’d say magnify the flaws in general psychological study by a factor of at least 50 for those focused on education, and then design an industry around them). Policy decisions, curriculum, and the endless and almost ubiquitously useless professional development and constant educational “reforms” that dictate the currents of my career all boil down to decidedly NOT robust evidence. Seriously, it’s bad. Thank you for this analysis!
I stopped trusting any studies less than 20 years old in a very wide range of topics and groups of people that are Woke-Protected. Average IQ in nations are worthless, for example.
What topics and groups you ask? Any topic or subjects-groups you would as soon put you dick in a blender that relay on Soros Foundation Sicked mind-raping Wikipedia entries. Last I looked their entry on 'Race' was a F-ing joke. I included some relevant information on the recent Democratic Abortion protection bill that did not go through, a loop-hole that allowed partial birth unborn baby torturing to death for Sacrifices to Satan or a particular Demon for unHoly favors and powers. Well, I corrected a statement and included the basic information and reference URL and the 'editor' removed it and threatened to lock-out my account for no sensible reason.
Google removed National Blood Bank studies that routinely show near 1/3 of children have wrong father listed, based on Blood-types and such. 1/3 of woman will lie like-damaging psychological crippling lies to entire extended families, to husband, to the child, and the real father, all those she pretends to love most. And if she caught - is she publicly tasered till pissing and screaming? No, she divorce-rapes her victim husband and destroys the family and the children's hopes for success in life.
But Satan's minions - I mean Women!, don't worry Google will support your soul-murdering lies.
Similar for a 3year comprehensive study of rape accusations filed with a police department that takes seriously investigating them well .. 40% were proven false, but who know how much higher the real number is? But it was on the FBI web-site and then it suddenly was not, and I found it in an archive.
I sometimes work with 3 different AI systems and I spent many hours finding the demonic presences in them so I can know what subjects I can trust working with it.
All the 'Professional' that actively lied and silently lies around the Demonic Covid social personal and economic attack and mass murders .. any Professional that is doing well in many of 'political' fields are assuredly Satanic Tools. No one not sacrificing trafficked children to Satan can do well these days in this deep level of Hell.
lack of verification and falsehood are two different things.
Perhaps not a coincidence that all of the claims above support conclusions whose polar opposite would not be allowed to be published at all. Of course you would expect contorsions, especially in a field with inherently low levels of precision.
Freudian Slip "Footnoes."
I completely agree with your suggestion not to take a reported finding too seriously until it has been independently and directly (not conceptually) replicated in a study with high power. However, this suggestion applies not only to reported findings but also to reported "failures to replicate." Therefore, I think you need to inquire more deeply before saying things like this: "In this context, it was then not surprising that the long-term success rate for replications in psychology has hovered right around 50% (e.g., Boyce et al., 2023; Open Science Collaboration, 2015; Scheel et al., 2021)." The tables in OSC2015 indicate 92% power for their replications. However, few realize that those power calculations were based on inflated observed effect sizes reported in the original (also low power) experiments, not on a reasonable estimate of their true effect sizes. When you take into account the inflation of a reported effect size using a p < .05 filter (the filter used in the original studies), statistical power in OSC 2015 was not far from 50%, which means that what they reported is not far from what you'd expect if all of the original experiments reported true positives (not 50% false positives). The findings that fail to replicate when tested again with high power are not so much the everyday studies conducted by experimental psychologists (contrary to what OSC2015 implies) but the studies associated with sensational claims -- that is, studies associated with low prior odds of being true. This is why people are pretty good at predicting which findings will replicate and which will not. You say that the 50% successful replication rate baseline might be too high, but I think it is almost certainly too low. My 2 cents!
Great article. Just the perfect combination of snark and great reasoning.
A quibble.
I work in a complex business field across a wide variety of industries, and was CEO / CTO for a non-profit research organization which worked to create reliable worldwide standardized measures, practices and processes in addressing business issues.
One simple measure of complex failure was poor “perfect order reliability”. The calculation involved reducing the problem to four types of failure. In business it’s slightly different than science but conceptually identical.
The standard is that a purchased entity can be late, incomplete, damaged, or have inaccurate paperwork (wrong invoice).
To calculate reliability you first classify orders in those four attributes, sum the number which have none of those attributes, and divide by the entire base count.
A common arithmetic failure is to either multiply % in each category of failure in a sample set, or to add - both are completely wrong. [I burst out laughing when I first read the origin of “intersectionality” a completely false set of concepts. They fail at elementary arithmetic.]
Consider two sets of four orders. In the first set only a single order fails in all four syndromes. In the second set each order fails in its own way.
Multiplying the %’s (75% of the orders are perfect) in the first case gives you a reliability level of 31%. The actual reliability is 3 perfect of four or 75%.
In the second case you have a reliability level with the same logic of 31% but the reliability is actual 0% - there were no perfect orders.
Data samples must not be correlated, overlapping - ideally “orthogonal” vectors so to speak when you look at reliability metrics.
Companies often claim great % reliability, often in the 80%+ range, but when you apply the correct measure they are quite bad. It is not unusual to encounter a company who on a month to month basis, has 0% reliability. I also encounter companies who have terrible reliability and simply reject the metric. It’s quite crushing to realize customers dislike doing business, and easier to make excuses. Customers abandon the company.
I would apply the same logic to your observation.
Consider 6 papers: a single paper can be faulty due to faulty citations, faulty assertions, faulty data/arithmetic, lacking contrary evidence, censored, evidence, MSU (and I note as well citing outdated evidence). Faulty data/arithmetic is so widespread that it’s not hard to glance through papers and get a feeling instantly that there’s a problem.
The same issue is in play as orders.
Consider two sets of papers, one has a single paper defective in 6 categories; the other has each paper defective in one category. Both situations might seem to be 38% reliability but of course the first has 83% reliability and the second has 0% reliability.
In the business world i consider the orders for a company as a whole over a time period in benchmarking against other companies in the same or similar industries.
You may consider a few things to strengthen your case. First, I’d partition the data by field and then by institution - there are a huge number of research institutions (analogous to companies) and their order fulfillment is research (papers and other artifacts, like data, hypotheses, experiment designs, materials, patents, methods, trade secrets - other IP)
I would apply your imperfect paper set to the partitioning logic I outlined to all the output of by industry by institution, and you can easily see exact outcomes.
When we apply perfect order logic to a naive company the results are usually crushing, but smart companies simply begin partitioning failure by syndrome and attacking the core problems.
Institutions and industries (sciences) are not used to being measured by reliable research output, often only by citation indexing. These would all be naive companies in the control processes business use.
Your thesis was that 75% of psychology research is faulty. By analogy to naive companies, it is entirely possible that entire institutions in the field have zero credible research - none at all.
An example is in fields contemplating “gender identity”. The term was invented by two scientists in the 60’s to support positions they believed true with trans delusional patients, patients with transvestic fetishism, and a smattering of children with puberty dysphoria.
The principal experiments (not research: anecdotal rather than systematic) were both grotesquely unethical and masked the facts that the research subjects committed suicide as an outcome.
“Gender identity” is not a scientifically supported term in any dimension, similar to implicit bias or stereotype falsification.
Use of it in research (to me) invalidates the premises involved. That means that, for instance, all documents from the APA, WPATH and others in the field of “gender identity” are faulty as a starting point, citing an entity “gender identity” which itself a combination of MSU, censorship, faked/faulty/masked data.
I’m more cheerful than you, while being very pessimistic. I know empirically that generation of faulty work products can be corrected by systemic efforts at the institutional level.
Grand. Academic circle jerking.
Another great article. I can see why you use Gould's definition of a fact but it is still a bad definition and this is perhaps why you have often found yourself using scare quotes around the word ‘fact’. A fact is a way that the world IS, more technically, it is a state of affairs that obtains. States-of-affairs, and therefore facts, are not the kind of thing that can be confirmed or disconfirmed.
What can be confirmed are statements that state states-of-affairs. A statement stating a fact is a truth. Scientific statements are used to make claims about what the facts are and such claims can be the outcome of a body of scientific research. If that body of research is sufficiently comprehensive and methodologically sound many of its claims are scientifically justified statements and belief in them is justified. It is such scientifically justified statements that can be said to be ‘confirmed to such a degree that it would be perverse to withhold provisional assent’. Statements that state a fact can never turn out to be false but scientifically justified statements can turn out to be false.
Consequently, it is impossible for Gould’s definition to be good for two reasons. Gould's definition should instead be given as a definition of when a claim is a scientifically justified statement and when belief in that claim is scientifically justified. I would still reject it as a definition, though. Rather, his perverse-to-withhold-assent condition is a usefully short generally reliable indicator of a scientifically justified claim.
I get it but, as usual, this approach has a very strong subdisciplinary bias. In my area, cognitive psychology/cognitive neuroscience, I hardly know any (systematically) non-replicable findings. Stroop effect, Simon, flanker, memory effect, recency etc., learning, attentional blink, you name it. Some higher-order interactions may be context-dependent, but the basic effects (orignally obtained with dramatically small samples, sometimes just the two authors) are super solid. So most of this handwaving business may be restricted to effects and phenomena that are more penetrated by culture. That's not so surprising, but please don't over-generalize from your own (admittedly messy) field...
I think you might be confusing "social psych is really bad" with "cognitive is really good." There is good evidence for the former, not so much the latter.
https://royalsocietypublishing.org/doi/pdf/10.1098/rsos.231240
Replication Rates
Cognitive: 58%
Social: 36%
How much grant money is available for replication studies? Grantors aren't interested in reinventions of the wheel: received wisdom/urban myth/widely held misconception:?
Well, I was thinking of the theoretically relevant outcomes of theoretically motivated, theoretically important experiments conducted by experienced researchers (that‘s what my attentional span is optimized processing), but not of cherry-picked (based on what graduate students find „interesting“ and, I guess, unlikely to replicate at face value) studies of unknown relevance, scored by two unexperienced graduate students according to „subjective replicability“ using unknown, if any, criteria…
Great article Lee! The default position in science is to remain skeptical until there is sufficient evidence to cross Gould’s threshold. Unfortunately I think too many people including psychologists want to claim that they have special knowledge. Perhaps too many PhDs do this to justify the costs of earning a doctorate. I used to be guilty of it