> My personal impression is that the volume of academic publishing has increased so much that it's impossible for readers to really keep up, and making it more difficult for scientific consensus to form completely.
I've noticed this as well, and it just makes me think that quality reviews become more important over time.
Unfortunately, in my experience most reviews basically mirror what a couple recent reviews said, adding a few new papers. This is assumed to be up-to-date when in fact if the older reviews missed some important older papers, it's not up-to-date. And that's what I see: important papers missed by reviews in the past continue to be missed. I don't know if this experience is valid outside of fields other than my own, however. In my PhD I've tried to comprehensively review the literature and I've found quite a few important missed papers.
I think few people actively pick "which truth they want to reinforce, and what truth they want to suppress." My impression is that literature reviews are done more out of convenience than an intentional desire to distort the literature.
> My friends and I discussed just putting the commentary on an online archive, but it wasn't clear anyone would make the connection with the paper.
I knew there were things like what I was mentioning, but couldn't remember the names of them. Thanks.
I don't think any of these biases are necessarily consciously enacted, but I think they exist in some of the ways you mention. There's a sort of echo chamber effect or positive feedback loop with citations.
Somewhere I remember reading a bibliometric analysis of citation patterns in the nutritional sciences regarding the effects of salt. There were two huge clusters of papers, one basically "salt is basically fine" and the other is "salt should basically be avoided". The papers in a cluster cited each other a lot, and not so much papers in the other clusters.
I agree that quality reviews become more important over time, but there's the issue of "according to who?" I suppose this becomes an expert judgment call and is the nature of these things, as it always has been, but I feel like as things become more parochial and balkanized the meaning of a "good review" changes somewhat, or becomes harder to agree on.
I've noticed this as well, and it just makes me think that quality reviews become more important over time.
Unfortunately, in my experience most reviews basically mirror what a couple recent reviews said, adding a few new papers. This is assumed to be up-to-date when in fact if the older reviews missed some important older papers, it's not up-to-date. And that's what I see: important papers missed by reviews in the past continue to be missed. I don't know if this experience is valid outside of fields other than my own, however. In my PhD I've tried to comprehensively review the literature and I've found quite a few important missed papers.
I think few people actively pick "which truth they want to reinforce, and what truth they want to suppress." My impression is that literature reviews are done more out of convenience than an intentional desire to distort the literature.
> My friends and I discussed just putting the commentary on an online archive, but it wasn't clear anyone would make the connection with the paper.
I think PubPeer is designed for this situation: https://pubpeer.com/