The Bizarre Bad Criticisms of Our Merit Paper

In this essay, I review some of the more common and prominent criticisms that have appeared regarding our recent paper on merit in science (open access and truly accessible to lay readers). In general, they are really bad. (In a subsequent essay, I will cover how one might actually criticize our paper coherently). Our paper briefly lit up the editorial pages (WSJ here, paywalled last I checked; NYTimes here), blogosphere (more on this later) and even the Editor of apex journal Science weighed in (more on this later too). Synopsis of our paper Unsafe Science here. In short, we argued that the very concept of merit in science and academia is under sustained (mostly) ideological assault from those advancing postmodern and “critical theory”¹ activist agendas

Card received at the Krewe de Joan of Arc Mardi Gras Parade in New Orleans, 2023.

Really Stupid Criticisms

One blogger went on a bizarre rant on the grounds that it was absurd for us to claim that a journal’s failure to publish our article violated our free speech rights.

It would have been absurd, had we claimed it. This delusional critique was by Scott Lemiux, who is described as a professor of political science. Presumably, that means he has a Ph.D. For further insight into why someone with so much education can write something so stupid, I highly recommend Taleb’s Intellectual Yet Idiot. This is not just name-calling. Read it yourself; its an idiotic adolescent rant.

Another vein of stupidity is the “straw man” critique. Supposedly, when we argued that many prominent postmodern and critical theory perspectives reject merit and objectivity, that’s a straw man argument, an absurd caricature because, duh, no one is so stupid as to reject objectivity and merit right? This was in some of the PNAS reviews of our article that led to its rejection there, and it was all over academic twitter.

Richard Delgado was one of the most prominent critical race theorists of the last 30 years. Shall we see what he wrote? Fom Delgado & Stefanic’s 2001 book on Critical Race Theory:

“For the critical race theorist, objective truth, like merit, does not exist, at least in social science and politics. In these realms, truth is a social construct created to suit the purposes of the dominant group.”

Nothing quite says “some prominent critical race theorists reject merit and objectivity” as a prominent critical race theorist literally rejecting merit and objectivity.²

Another stupid attack, by a Distinguished Professor of Politics, Peter Dreier, declared our paper a “fraud,” literally, a fraud.

Did Dreier expose us as making up data? No. This would be impossible, in that ours was a scholarship-informed opinion piece, not an empirical study. Did he expose plagiarism? Did he show that we cited papers as making claims they did not make? Did he show that we systematically ignored relevant evidence opposing our perspective. No, no, and, you guessed it, no. So what grounds did he think he had for characterizing the paper, or its “mistreatment” as a fraud? Read it. He does not even attempt to do so. Instead, he rants about climate denialism and corporate corruption of science.

He does say this:

To bolster their point that science has been kidnapped by leftists, including the editors of scientific journals, they claim, falsely, that “the paper was rejected by several prominent mainstream journals.”

In fairness, it was only formally rejected by one after a full review process. However, we sent inquiries about whether they would even consider the paper to a slew of journals, and they all told us to go fly a kite. So the only “fraud” here is Dreier calling this false.

He knows this. From his blog:

What she and other co-authors had actually done was, in Krylov’s words, make “informal inquiries” to journal editors, about whether they might consider the article.

So he knows exactly what we did and knows that it was rejected at a slew of journals and still calls the claim that it was rejected “fraud.”

Stupidity or malice? You decide. This might help inform that decision:

This practice violates scientific norms of submitting articles to journals anonymously to avoid potential bias.

No it doesn’t. He is clueless. This is a common and conventional practice (although the practice of submitting without an inquiry is also normal when one knows the type of article is the type of thing that journal publishes). The idea of submitting an informal inquiry (“Would you consider this paper?”) is to not waste anyone’s time submitting an article to a journal that would never even consider it.

Dreier’s moral condemnation is typical of political sectarianism. Academic extremists routinely sociopathologically denounce, on supposed ethics grounds, academic behavior that is normative, if they do not like the argument. My favorite example of this was this denunciation of normal peer review processes by an outrage mob of a paper that was critical of diversity, equity, and inclusion initiatives (among other sins). From the online petition to retract the paper (which succeeded):;

“This essay was submitted, sent out for peer-review, accepted, and published. This does not happen by accident. We, as a group, denounce the essay itself and the process by which it was published” (emphasis added). .

They denounced normal academic publishing processes because the mob did not like the paper. Much as did Dreier.

---

Then there is this douzy by PZ Myers (see the “screechy monkey” comment on the Wikipedia talk page screenshotted below).

I don’t know much about him, but people tell me he has a widely-read blog and that he was once a prominent New Atheist. Here is what the Talk portion of his Wikipedia entry says:

I also looked him up on Google Scholar, he has some pubs on zebra fish in the 1980s and 1990s, and a smattering of other things.

Ole PZ makes a zillion different points, nearly all wrong regarding our paper, but here I’ll just focus on one as illustration. He claims:

The first big problem, and one that plagues the whole paper, is that merit isn’t actually defined.

This is Figure 2 from the paper:

Of course, the “importance” of any given discovery, talk or paper may be pretty subjective until the fullness of time has weighed in. As our paper repeatedly acknowledged, scientists’ biases may creep in to influence judgments of merit. Nonetheless, we now know that discoveries that cigarettes cause cancer, that bacteria, not stress, cause ulcers and that genes influence many physical and psychological characteristics are pretty important, each of which was doubted, controversial or even dismissed at the time. It also took some time to discover that certain ideas lack merit (e.g., thalidomide is not safe to administer to women who are or might be pregnant; the implicit association test does not measure “unconscious racism”).

Moving right along, here are some quotes from PZ’s blog:

It’s an expansion of the Grievance Studies nonsense, and Boghossian is one of the authors, while Helen Pluckrose and James Lindsay are cited, and if authors are not on the staff of the University of Austin, at least some of them are publishing opinion pieces in Quillette. Basically, it’s a collection of right-wing ideologues complaining about ideology, unaware that it’s ideology all the way down.³

Then screw you, McWhorter. (John McWhorter is one of the co-authors of the Merit paper).

And of course, in the inevitable, all-too-familiar move among the progressive left when dealing with those whose views they oppose:

That was seriously one of the worst papers I’ve ever read, and I read creationist drivel — it’s just that the Krylov paper actually has some respectable authors attached to it, although their respectability is greatly diminished by their association with the racist apologetics in this badly written paper.

Faux Sophisticated Criticisms

If you go to the NYTimes reader replies to Pamela Paul’s essay on our paper, you will discover that most people were overwhelmingly supportive. The first two (when I checked, on 6/6/23, both were dated May 5). Marvin wrote: “"Citation justice" is one of the most absurd and petty proposals I've ever heard of.” And “L” wrote, “I find this [that merit needs to be defended] baffling and shocking.” But when the NYTimes curated “readers’ responses” they published a handful that were mostly critical-to-hostile. According to one, we “exacerbate the panic over the alleged subservience of academia to ‘so-called’ political correctness.” Another is a bizarre mini-rant about genetics and scientific racism. But my “favorite” is this one:

This is a brilliant work of bullshit, by someone who has written a book about identifying bullshit. (Nietzche quotes about abysses and monsters come to mind). Bullshit refers not to lies, exactly, but to a flagrant disregard for the truth. Each of Bergman’s statements are, by themselves, at least mostly true (not so sure about the “implication” part but whatever) but the totality is bullshit. They are red herrings, irrelevant to both Pamela Paul’s NYTimes essay, and, more important, to our paper (did Bergstrom even read the actual paper?). Here is what Pamela Paul actually wrote:

The authors argue that science should instead be independent, evidence-based and focused on advancing knowledge.

This sounds entirely reasonable.

Now, it is certainly true that papers are usually rejected for the reasons Bergstrom listed. Of course, it is also true that most academic papers are not on politicized topics, so the possibility of ideology infecting science does not even come up for, as Bergstrom put it, “the vast majority of submissions.” Regardless, we do not need generalities to understand why our paper was rejected. We have the editorial letter “explaining” the rejection! As described in the WSJ editorial, we were told by an editor that the concept of merit has been “legitimately attacked as hollow” and that some of its points were “hurtful.”⁴ Indeed, the nature of the criticisms, not the mere fact of being rejected, is what reveals the editor’s and reviewers’ political biases. The ideas that merit has been legitimately attacked as hollow, and that a paper on merit could be “hurtful” to anyone, are superficial progressive platitudes (see this excellent essay by someone who wants Democrats to win elections, The Democrats’ Merit Problem).

The issue is not that the paper was rejected at PNAS (and elsewhere); it is the ridiculous reasons for the rejection. And Bergman’s whole “misleading lay readers about how science works” point? He is damn right, that is bad. He should know better than to do it. The idea that scientists’ biases may distort their responses to and reviews' of others’ work is not controversial, and has been known for decades. Bergstrom knows this.

Ignorant, malicious or tribal dogmatism? You decide.

---

Editor-in-Chief (Holden Thorp) at the apex science journal, Science, weighed in with an opinion piece that was … IDK what it was. Its tone was as if it was some sort of refutation of our paper, but it did not bother to actually address any of our arguments.

The article was titled “It matter who does science” as if we argued that it did not. Here is what we actually stated:

Self-correction is facilitated by pluralism to maintain intellectual diversity and maximize the chances of uncovering provisional truths. Intellectual diversity ensures vigorous skeptical vetting of scientific claims by a critical mass of doubters who ultimately accept being bound by objective truths once they have been rigorously determined by extensive evidence.

So what does Thorp object to? He makes some patently absurd statements, such as:

the public has been taught that scientific insight occurs when old white guys with facial hair get hit on the head with an apple or go running out of bathtubs shouting “Eureka!”

and

It has somehow become a controversial idea to acknowledge that scientists are actual people

This is the Editor in Chief at Science. Publishing this opinion piece in … Science. It is idiotic nonsense.

He then goes on to pull a subtle bait and switch. See if you can catch it:

One view⁵ is that objective truth is absolute and therefore not subject to human influences. “The science speaks for itself” is usually the mantra in this camp.⁶ But the history and philosophy of science argue strongly to the contrary. For example, Charles Darwin made major contributions to the most important idea in biology, but his book The Descent of Man contained many incorrect assertions about race and gender that reflected his adherence to prevalent social ideas of his time. Thankfully, evolution didn’t become knowledge the day Darwin proposed it, and it was refined over the decades by many points of view. More recently, pulse oximeters that measure blood oxygen levels were found to be ineffective for dark skin because they were initially developed for white patients.

Did you catch it? The fact that science has gotten some things wrong or that scientists’ biases have, sometimes, misled them to advance false conclusions, is presented as if it invalidates the reality of objective truths. It does nothing of the kind. Indeed, the way Darwin’s incorrect assertions about race and gender, and the fallibility of pulse oximeters were discovered, was by subsequent scientists debunking false claims and replacing them with true ones. The failures of pulse oximeters was discovered because it was objectively true that they were ineffective for people with darker skin.

The entire notion of scientific validity rests on the existence of objective truth, and without it, science is meaningless. Thorp baited you with the implication that there is no objective truth and switched in scientists’ biases and errors as if it refutes the existence of objective truth. Which it cannot possibly do because to know that an error was made or a conclusion is biased implies that one has access to objective truths that debunk those errors and biases.

An excellent essay on this controversy by a bio-ethicist at Merck (which includes some thoughtful criticisms of our paper) puts it this way:

Thorp adopts a questionable strategy known as the motte-and-bailey tactic⁷, employing it fallaciously and deceptively. He presents the easily justifiable opinion encapsulated in "It matters who does science" (the [uncontroversial and easily defensible] "motte"), while conveniently avoiding any arguments that challenge Abbot et al.'s initial claim. Thorp's unspoken and potentially harder-to-defend propositionthat "merit should (to some degree) be replaced by social engineering or identity-based policies" (the "bailey") remains unsupported and unaddressed in his discourse.

Thorp is a highly accomplished chemistry professor. Were he to speak about, say, compounds used in genetic therapy, you should probably listen. But he is writing about social engineering. What expertise does a chemist have about social engineering? Ballantyne (2018) described Holden Thorp, whoops, I mean, epistemic trespassing thus:

Epistemic trespassers are thinkers who have competence or expertise to make good judgments in one field, but move to another field where they lack competence—and pass judgment nevertheless. We should doubt that trespassers are reliable judges in fields where they are outsiders.

Another chemistry professor, Henry Bauer, sees this clearly, when referring to another chemist, John Herbert, who made arguments similar to those of Thorp’s⁸:

[Chemists'] write in authoritative tone about citizenship and civic responsibility, ethics, morality, social roles and responsibilities, and even how individuals should be evaluated by the scientific community and by the whole society. It is far from obvious that chemists have, individually or collectively, the background of cosmopolitan knowledge of civilization’s intellectual infrastructure and understanding to speak in this way; they are ultracrepidarian [a person who criticizes, judges, or gives advice outside their area of expertise; i.e., they do not know what they are talking about, or at least not any more than anyone else.]

Most Criticisms of Our Merit Paper are Inherently Logically Incoherent

To criticize our paper is to argue that it is bad or unjustified in some way. However, to make these sorts of arguments, the critics must have some standard for truth. If they do not, then they cannot possibly know our paper is wrong, biased, misguided, hurful, or anything else.

Street art, Lisbon, Portugal, in the Alfama district.

Implicitly, then, they believe that getting at the truth is possible because they are making a truth claim when arguing our paper is wrong, hurtful, etc. If we are wrong and they are right, then they themselves are promoting claims that are actually true! That is, their claims have merit, whereas our’s don’t. Anyone who believes the critics claims have merit (including the critics themselves) implicitly accepts our central argument that science has to be judge on its merits, even if they pose as critiques of our paper.

There is a structure to this sort of logical incoherence. It is impossible for there to be no scientific merit and for the claim that there is no scientific merit to … have merit. The claim (or implicit argument) that “there is no scientific merit” is logically incoherent in the same way that “All statements are false” is logically incoherent. Both are self-negating. If “all statements are false” is true, then “all statements are false” is also false, so that some statements are true. If “there is no scientific merit” is true then “there is no scientific merit” has no scientific merit, and there is scientific merit. Self-falsifying claims may feel good (to some people) and signal tribal allegiances but exclude themselves from being taken as serious scientific truth claims.

There is one exception to my argument that the critics are incoherent. If they literally believe there is no truth, only politics and power, then, sure they can criticize our paper. They could do it on the grounds that our politics are bad, theirs are good, and they have the power to force their political values on to the rest of us. But if that is their argument, it vindicates the claims in our paper about the politicization of science.

Epilogue

Anna Krylov, the main force of nature behind the merit paper, has also created this website, indefenseofmerit.org, that curates a lot of the essays, blogs, and podcasts discussing our paper. Jerry Coyne, over at WhyEvolutionisTrue has a slew of entries on some of the critical responses to our paper (such as here, here, and here).

The critics reviewed herein are, by many measures, really smart, accomplished people. They are all academics with PhDs, and, often, long lists of scientific publications. Make of that what you will.

---

Footnotes

1. Even the name, critical “theory” is a misnomer. Critical theories have no resemblance to scientific theories (which typically require testing hypotheses and predictions against empirical data, usually with some potential for falsification). Critical theories are generally little more than sustained editorializing masquerading as scholarship. See this excellent guest post by Michael Mills for more on this, although this meme does a pretty good job:

Courtesy of @thewoketemple, who you can find on Twitter.

2. In fairness, Delgado & Stefanic state “objective truth, like merit, does not exist, at least in social science and politics.” What “at least” means here is deeply unclear. But let’s take the most charitable interpretation, which is that it means “only” in social science and politics. 1. It is still a rejection of objective truth and merit, albeit, in this interpretation, it is not a rejection of, say, a heliocentric solar system or quantum mechanics. 2. It is still a rejection of merit and objectivity (in politics and social science); and 3. It is still absurd. Consider politics. It means that the statement “The First Amendment states that "‘Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances” is not objectively true. Nor would statements be objectively true such as “FDR won four consecutive Presidential elections” or “Political polarization in the U.S. increased substantially from 1990 to 2020.” Now let’s consider social science. It would mean that it is not objectively true to state that “IQ test scores correlate with academic achievement, income, and even mortality” and that “Black people are underrepresented, compared to their population proportion in the U.S., in U.S. academic STEM fields.” To refer to those statements as “not objectively true” would be silly. This sort of thing is emblematic of the misguided thinking seen throughout the critics’ analysis: The existence of bias, which surely objectively (smirk) exists sometimes, does not invalidate the existence of objective realities. And on the natural sciences?

3. PZ Myers and “rightwing ideologues.” I ran an Indivisible group in 2017. Most of the co-authors did not announce their politics or party affiliation, but those who mentioned it to me were mostly Democrats or liberals.

4. The specific quote in the paper that was deemed “hurtful” is: “one would be hard-pressed to give examples of institutional features today that foster discrimination and are responsible for the dearth of minority scientists in STEM.” It was also deemed “patently false” by the editor who did not provide a shred of evidence, a single citation, for such discrimination. I mean, if its “patently false” it would be easy to provide the evidence, right? Go here to find a video, 52-54 minutes in) of DEI advocates admitting that they do not know of a single Black person denied an academic job because of hiring discrimination. See also the suit brought by Asian groups against Harvard & UNC to be heard at SCOTUS — which require lower academic accomplishments to admit Black students than for other groups (whether this is justified by affirmative action concerns is irrelevant — this is the opposite of anti-Black racism). Go here for some of the absurd stuff cited as evidence for systemic racism at apex journal, Nature, such as a grandstanding Tweet and the (literal, night time) dream of a Black woman.

5. The “one view” Thorp refers to, by linking, is our paper on merit.

6. "“The science speaks for itself” is usually the mantra in this camp.” This sentence comes right after citing our paper, so, presumably, this means us. We neither wrote nor implied anything so simplistic because, duh, science does not always speak for itself. So he presents something I, and I suspect most of my co-authors, agree with as if it debunks our analysis. Shoot me now.

7. Motte and bailey is a well-known rhetorical trick. The term was inspired by medieval defense systems. A well-fortified castle (the motte, from “mountain” because castles were often on mountains, which were easier to defend) overlooks and defends a much larger territory (the bailey). Invaders attack and take the hard-to-defend bailey, but the defense holes up in the castle and fights them off till they give up and retreat. They defenders then retake the bailey.

In rhetoric, the trickster makes an extreme claim, like “science is filled with racism!” When questioned, asked for evidence, or challenged, they retreat to the defensible motte (“scientists are imperfect humans!”). The challengers, often befuddled by the trick, then say some version of “oh, good point” and go away. But once you know the trick, it is easy to see through it and call bullshit. Such as here:

8. John M. Herbert, Martin Head-Gordon, Hrant P. Hratchian, Teresa Head-Gordon, Rommie E. Amaro, Alán Aspuru-Guzik, Roald Hoffmann, Carol A. Parish, Christina M. Payne, Troy Van Voorhis, “Words matter: on the debate over free speech, inclusivity, and academic excellence”, Journal of Physical Chemistry Letters, 13 (2022) 7100-7104; https://pubs.acs.org/doi/10.1021/acs.jpclett.2c02242?ref=pdf