Pet peeves of peer review

I've been doing peer reviews for about six years and for some reason have been invited to do a bunch more recently. During the Munin conference keynote by Dan Quintana and James Heathers on peer review, some of my pet peeves of peer review came to mind.

These are by no means all my pet peeves of peer review, considering that peer review is ever evolving as a practice.

The invitation doesn't indicate the length of the manuscript

If I am going to spend time on something, I would like to be able to estimate how much time it is going to cost. If I am invited to review a 5 page opinion piece, I could do that in a shorter time than a 30 page experiment. I end up having to infer a lot of information about what is actually being asked of me.

Too short deadlines

In 2015, I researched how publication delays have evolved over the years (spoiler: with time it takes longer it seems). Empirical investigations by others confirmed that. I'm all for reducing unnecessary delays!

Nonetheless, it's not reasonable for an invitation, which adds work to my schedule, to come with a one or two week deadline. Most often, I cannot commit the time that quickly to provide a thorough peer review, without it coming at the expense of my own work or personal time.

Authors uploading wrong files

It doesn't happen often, but when it does, it's rather disruptive. I start the review, and then I get a notice that some file should've been something else. It's especially difficult when I was rounding up my review, as I then have to go back and check everything again with these new files.

Figures and tables at the end of the manuscript

The flipping back and forth drives me nuts — it literally decontextualizes the information. I tend to review on my tablet nowadays, and I don't really know how to open the document twice at the moment.

Open data that isn't open

I am one of those reviewers that will actually click the links to the shared data, materials, preregistration code. It happens to me sometimes too, but it is a pet peeve of mine when I hit a 404 error or a permission error. I can't do my review if I can't check the files that are part of the manuscript.

Citing in preparation work

I've done it myself, and sometimes you have to. I cannot rely on these references though, as I don't know whether I'll ever be able to find the information that's mentioned. A situation I cited an in preparation piece, ended up never being published and that whole citation is virtually worthless, despite being part of an argument.

Not hearing the result of the process

In many cases I end up not hearing what happens with the manuscript. Did other peer reviewers see similar points? What did the editor decide? Was there another round? Did the paper get published at the journal I did the review for? I often end up in the dark.

The 10,000 unconsented account creations

Often journals create accounts for you when they want to invite you to review. I know because I often first get an email that an account was created, and then the invite. Note that this is unconsented information processing, which is technically a violation of the GDPR 🤷‍

What are your pet peeves of peer review? I'd be interested to hear what your pet peeves are. Send me an email if you'd like to share ☺️