The Journal of Open Inquiry in the Behavioral Sciences
A New Open Access, Open Peer Review, Open Inquiry Journal
The Journal of Open Inquiry in the Behavioral Sciences (JOIBS) is an initiative of The Society for Open Inquiry in the Behavioral Sciences (SOIBS). This essay was written by Cory Clark and me collaboratively; we are among the 10 SOIBS founders; I am editor-in-chief of the journal and Cory is an associate editor.
Conventional Peer Review is Broken
It never delivered what it promised — assurances that only the best, most credible, true findings would make it into the “peer reviewed literature” — despite involving an immense cost in person power and actual dollars. Journal prestige is often taken as a marker of quality. Prestige is usually highly correlated with “impact” (usually some function of citation counts). Amusingly, however, there is little or no connection between prestige, impact, or conventional measures of scientific quality (see the extensive list of references at the end).
Conventional Journals Instantaneously Retract Papers Because of Mobs and Take Years to Retract Papers Because of Fraud
Existing journals contribute to publication bias by incentivizing scholars to report only their statistically significant tests and perfect packages of studies. Existing journals delay scientific progress by sending papers through multiple rounds revision—each lasting several weeks or even several months. Existing journals routinely kowtow to outrage mobs and retract “offensive” papers that show no evidence of fraud, plagiarism or mistreating participants and yet take months, sometimes years, to retract papers after being shown they were published with bona fide fraudulent data.
Conventional Journals Unjustifiably Prioritize Reviewers Over Authors
Existing journals prioritize reviewer and editor perspectives over author perspectives. There is no good intellectual or scientific reason for this prioritizing. It is true that, sometimes, reviewers make good points and the authors improve the paper by addressing them. But it is also true that reviewers sometimes make bad points and when editors prioritize reviewers it can cause authors to make changes that they believe worsen the paper. Furthermore, it is true that reviewers sometimes just make annoying trivial points and, even though addressing them may not worsen a paper, doing so can be a ridiculous timesuck for an author who might have been able to accomplish something worthwhile with that now-forever-lost time.
Worse, most existing journals hold themselves and their authors to radically different standards of accountability. There is no accountability for reviewers. If they are hostile, petty, and counterconstructive,1 the worst that happens is that the editor ignores them. More likely, the editor will treat their comments as serious criticism and now authors have to deal with an arrogant, overconfident, demanding, incompetent or ax-grinding militant as if they are scientifically serious.
Conventional Journals are Riddled with Biases
Existing journals prioritize the work of certain types of scholars (e.g., prestigious ones) creating a Matthew Effect in which the rich get richer. Existing journals prioritize certain types of findings (e.g., surprising and novel ones [i.e., the kind most likely to be incorrect]) at the expense of higher quality but less flashy findings. Some existing journals obstruct or even explicitly ban findings that violate the current editors’ personal moral or political (usually progressive) sensibilities.
Conventional Journals are Exploitative
Existing journals charge thousands for open access or paywall research so that sometimes not even authors themselves can access their own work. Existing journals profit off the free labor of authors and reviewers, and some threaten their authors who post their own work on their personal websites. Existing journals cave to social media mobs and retract papers because they have intense financial stakes in need of protection.
How is JOIBS Different? We Count the Ways
Behavioral science needs a new journal, one that eschews these many dysfunctions.
1. We won’t contribute to publication bias
Give me your tired, your poor,
Your huddled masses yearning to breathe free,
The wretched refuse of your teeming shore.
Send these, the homeless, tempest-tossed to me,
I lift my lamp beside the golden door!”
Oh whoops. That’s the poem on The Statue of Liberty. But, the JOIBS spirit is quintessentially identical, just applied to behavioral science.
Send JOIBS your rejected papers that cannot be published elsewhere.
Send us your null results,
even null results across many studies.
Send us your replication studies,
even your failed replications.
Send us your incremental or niche findings.
Send us your findings that disconfirm
what “every” social scientist “knows”
real or imagined behavioral science consensuses
famous claims advocated by famous professors at prestigious Universities
Send us your politically incorrect (or correct) findings.
Send us your findings that “do not fit” with other journals.
Send us the work you do not wish to publish elsewhere.
Send us the work you fear to publish elsewhere.
Send us the work you wish to get out quickly.
Send these, your “homeless” papers, to JOIBS.
The lamp of open inquiry beckons and welcomes them all.
We’re here to share science, not control narratives. At JOIBS, your paper will be evaluated, and you will receive constructive criticism and feedback, but publication will not be censored or blocked by reviewers’ or editors’ subjective judgments, biases, personal animosities, political axes or even Great Gnashing of Teeth.
2. Reviews are published
We’ve all experienced bad faith reviewers who make weak or inappropriate criticisms to kibosh papers they dislike for reasons unrelated to quality. At JOIBS, our reviewers are accountable too, with all reviews published alongside our articles.
Reviewing is a time consuming and thankless job. Not at JOIBS. Our reviews all get published as peer commentaries and contribute to public scientific discourse (and will be listable as publications on CVs).
3. JOIBS is working hard to expedite the peer review process
It should never take years for a paper to go from submission to publication. We aim to improve our papers in just one round of reviews (where feasible) and give authors primary control of over the revisions they choose to make and pace of the acceptance and publication.
3. No exorbitant open access fees
Authors and reviewers work hard and shouldn’t have to pay exorbitant fees for open access to the articles they contributed to. At JOIBS, we have a low, one time submission fee ($75), that goes toward supporting our journal management and nothing else.
4. JOIBS is Committed to Never Retracting a Paper in Response to Social Media Mobs or Online Petitions
JOIBS Policy on Retractions:
We will base retraction on C.O.P.E. guidelines, which can be found here: https://publicationethics.org/files/retraction%20guidelines.pdf. We reserve the right to retract a paper if errors go uncorrected for more than a month (as described in Copy-Editing, Proofing, and Errors). We will never retract a paper in response to social media mobs, open or private letters calling for retraction, denunciation petitions, or the like if such petitions fail to show that the published paper meets C.O.P.E. guidelines for retraction. Similarly, authors may only retract their own published papers if they meet C.O.P.E. guidelines. Critics are welcome, however, to publish their criticisms as part of JOIBS’ policy of open peer review.
How to Submit to JOIBS
Instructions on how to submit can be found here and here. You can proceed directly to submission by going here. You can find our published articles here.
A summary of one was posted at Unsafe Science, here, on political tribalism and its associated biases in judgments. In social psychology, there is a raging debate about whether conservatives are more biased than liberals, or whether bias is about equal. The paper published in JOIBS linked above found “A pox on both your houses” — that is, it found that liberals were more biased than conservatives (it is just one paper, no one is saying “THIS ENDS THE CONTROVERSY,” but such work must be publishable in order to evaluate the controversy and whether the controversy is actually even the right controversy).
JOIBS is an initiative of The Society for Open Inquiry in the Behavioral Sciences (SOIBS)
You can find out more about SOIBS here and here. We are scholars and practitioners in the behavioral sciences committed to free inquiry and truth seeking. In healthy scientific fields, ideas are debunked rather than censored, and their proponents are debated rather than punished. We are dedicated to maintaining open inquiry, civil debate, and rigorous standards in the behavioral sciences. Become a member here.
Some Sources On the Failures of Peer Review
Academic articles in peer reviewed journals
The irony of us referencing peer reviewed articles to support our argument that peer review is deeply dysfunctional and in need of reform is not lost on us. But our point is not that everything peer reviewed sucks. It doesn’t. Whatever is good and worthwhile, however, is because the researchers are good and doing worthwhile work; it is not because of any added value provided by existing peer review. Peer review does sometimes do some good, but it does at least enough bad to neutralize whatever good it does.
https://www.pnas.org/doi/abs/10.1073/pnas.1707323114
https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1559-1816.1975.tb00675.x
https://link.springer.com/article/10.1007/bf01173636
https://www.mdpi.com/2075-4698/10/4/82
https://link.springer.com/article/10.1007/s12144-023-04739-2
https://journals.sagepub.com/doi/full/10.1177/25152459211007467
https://www.frontiersin.org/articles/10.3389/fnhum.2013.00291/full
https://www.frontiersin.org/articles/10.3389/fnhum.2018.00037/full
https://link.springer.com/article/10.1007/s11192-016-2227-4
https://www.science.org/doi/10.1126/sciadv.abd1705
https://www.jstor.org/stable/25662607
https://journals.sagepub.com/doi/full/10.1177/1745691612464056
https://www.journals.uchicago.edu/doi/full/10.1093/bjps/axz029
Essays, Op Eds, Blogs
These are often better than the peer reviewed articles because they are very broad and comprehensible, yet also cite many many more peer reviewed articles than we have here.
https://www.nytimes.com/2023/05/04/opinion/science-evidence-merits.html
https://www.experimental-history.com/p/the-rise-and-fall-of-peer-review
https://getsyeducated.substack.com/p/pnas-is-not-a-good-journal
https://unsafescience.substack.com/p/apex-peer-reviewed-journal-nature
Counterconstructive. Cory made this word up. Its a good one, though.
 | A guest post by
|
I was really hoping for a revival of the Journal of Polymorphous Perversity, but I'll settle for this.
The paper my colleagues and I wrote was probably rejected from, or told it was a bad fit for, around 10 journals. Thankful that we found a home with JOIBS! In addition to all the issues raised in this article, another problem is that it takes a huge amount of time to reformat articles for each new submission. This is yet one additional disincentive to do work that is controversial, admittedly incremental (IMO much of published social science is incremental yet described otherwise), or likely to produce null results.