Slip to main content

Peer Review Bias: What Is It, and What Causes It? | Lex Academic Blog

Peer review is vital for ensuring the academic and editorial quality of published articles and books. But it is far from perfect. Bias in the peer review process is well known, much discussed and, unfortunately, difficult to identify due in part to the anonymity of the reviewers (and sometimes the authors). The term ‘peer review bias’ refers to any violation of neutrality in the assessment of a piece of work offered for publication. If we accept that peer review contributes to scholarly rigour, we must also acknowledge that bias undermines it. As researchers, we therefore have a stake in eliminating bias and improving the peer review process. In this post, we explore the forms peer review bias can take and the ways different peer review processes enable and prevent it.

Forms of Bias

What constitutes a violation of neutrality in the assessment of a submission? The most clear-cut example occurs when there is a conflict of interest, which could lead the reviewer to accept or reject a submission because that is what most benefits them. A failure or inability to assess the quality of a submission, owing to a lack of subject knowledge or time, is another obvious case of dubious peer review.

Bias may also occur when the reviewer disagrees with the argument put forward by the author, which may cause them to impose corrections that require the author to modify conclusions or add or remove nuance. Differing views are inevitable and should be welcomed by the author as ways of furthering scholarly debate and adding to the overall quality and completeness of their argument. However, reviewers need to present their view as an alternative to, rather than a substitute for, the author’s argument. To do otherwise is to stifle debate and give the false impression of a consensus.

Of course, an overwhelming majority of reviewers can be relied on to declare a conflict of interest and evaluate a different point of view with a measure of objectivity. A reviewer’s assessment of a submission can nonetheless be influenced by subtle yet powerful biases regarding, for example, an author’s gender, language, nationality, race or scholarly reputation. It is easy to imagine, for instance, that a reviewer in a male-dominated field might, even subconsciously, look less favourably on a female-authored submission. In addition, we should remember that reviewers are influenced by their experiences of peer review as authors, as well as by culturally variable norms regarding feedback and research quality. A particularly stringent reviewer might reject a high-quality piece of work, while another, more lenient reader might take a chance on a slightly less compelling submission.

Forms of Peer Review

There are several systems of peer review, each with its own advantages and disadvantages. In single-blind peer review, the reviewer knows the identity of the author, but the reviewer is anonymous. While this ensures that the reviewer is free of the author’s influence, it can too easily lead to bias. It allows for discrimination, benefits well-known authors and thus disadvantages early career researchers. It also exacerbates the very obvious power imbalance in the peer review process. Readers involved in undergraduate and postgraduate assessment will appreciate that the way anonymity works in single-blind peer review is at odds with what is widely considered to be good marking practices. In higher education, it is the student, not the marker, who is anonymous because it is recognised that this encourages fair assessment. Even a PhD thesis examiner is not anonymous. The anonymity of the holder of power in single-blind peer review is, then, somewhat exceptional in the wider context of academia.

Because of these issues, double-blind peer review, in which both the author and the reviewer are anonymous, is now common practice in the humanities and social sciences. Anonymity protects the author from discrimination and the reviewer from negative comeback. This system is nonetheless imperfect. In fields with only a handful of experts, it is possible to deduce the identity of the author or reviewer. Authors also forfeit their anonymity if they self-cite.

In response to the shortcomings of blind reviewing, there has been a partial shift towards open peer review, in which the identities of all parties are known. While it can lead to conflict, this system offers greater transparency, encourages constructive criticism and provides an opportunity for dialogue between author and reviewer following completion of the peer review process. Reviewers should be receptive to such conversations, just as authors should welcome constructive criticism, for both foster scholarly debate and collegiality between researchers.