Abstract
Organizational Psychology Review’s mission is to advance research and practice in organizational psychology by stimulating the development of high-quality theory. In pursuit of this mission, OPR publishes conceptual contributions, meta-analyses, and integrative reviews of the literature capturing the state of the science. In an earlier editorial, Stephen Humphrey reflected on the question of what makes for a good meta-analysis (Humphrey, 2011). OPR has also launched a Special Forum on Good Theory to stimulate discussion of and reflection on the question of what is good theory in organizational psychology (van Knippenberg, 2011, 2012). Ferris, Buckley, and Hochwarter (2012) and Wiesenfeld and Brockner (2012) were the first contributions to the Forum, and the current issue adds Shalley (2012) as a third contribution, with more to come in the following issues of OPR.
As we announced earlier, OPR issued a continuing Call for Review Proposals. The rationale for this call is to lower the threshold for taking on the task of writing a review of the literature by limiting the initial investment to the submission of a proposal rather than a full-blown review article. In doing so, we hope to alleviate author worries that their review might be seen as not moving enough beyond a summary of the state of the science (i.e., and therefore not providing enough of a value-added contribution). The bottom line is that we want to see more integrative reviews of the literature, because these can make important contributions to the field.
Urging authors to submit review proposals (or review papers – the proposal stage is by no means a prerequisite; it aims to be a “service” to authors only) while at the same time recognizing the risk that reviews are seen to provide a limited value-added contribution, we beg the question of what would make for a good review of the literature. What do we want to see in review papers to conclude that they provide sufficient value-added? I know better than to claim that this editorial can comprehensively answer the question of what would make for a good review of the literature, but its intention is to at least make a humble start in answering that question.
Given that OPR also publishes conceptual contributions and meta-analyses, an obvious starting point in addressing the question of what makes for a good review is to ask how a review is different from a conceptual paper, and when and why a “narrative” review would be preferable over a meta-analysis.
The first question is a somewhat awkward one, because there is no hard cut distinguishing a review from a conceptual analysis. Conceptual analysis should be sufficiently grounded in an established (i.e., published) conceptual and empirical basis. Put differently, conceptual contributions are rooted in a review of the literature. Conversely, reviews of the literature should do more than summarize what is out there; they should advance original thinking that builds on an integration of the literature reviewed. In other words, they should contribute conceptual insights. It is therefore hard to say where review ends and conceptual contribution begins. Rather, whether the claim to provide an integrative review of the literature or the claim to advance new theory is more appropriate is a judgment call. That being said, reviews of the literature would in substantial part (even when not exclusively), advance conclusions that are based on empirical evidence rather than (only) on new theory. In contrast, conceptual contributions would only advance propositions that still await testing. Put in a different way, conceptual contributions more strongly bear the burden of novelty, whereas reviews more strongly bear the burden of evidence.
What this also means is that undertaking a review only makes sense to the extent that there is a sufficient body of evidence available. This is not to say that only “big” areas of research are worthy of review, but the research field should be substantive enough to allow for a review that advances a series of nontrivial new insights from an integration of theory and empirical evidence. This should not be taken to imply that reviews need to include all available evidence. Unlike meta-analyses that should aim to include each and every study relevant to a hypothesis test, narrative reviews need to be representative in their coverage of the literature, not exhaustive. For smaller fields of research there can be value-added in an exhaustive review because this in and of itself can provide a valuable reference source, but given the inevitable space constraints of journals, representativeness becomes the defining criterion for inclusion of studies especially where the body of evidence is more substantive. A key skill in writing a review then, is the ability to communicate an understanding of the field that “by implication” answers questions readers might have about the reasons not to include certain studies. More to the point, a good review is written in such a way that it does not invite the criticism that studies that would have changed the review’s conclusions are not included.
This is not to say, however, that reviews need to limit their conclusions to those that can be based on empirical evidence. Indeed, an important function of a review can be to identify questions that should be studied – a good review can lead to the conclusion that what is needed to further develop the field is research addressing different issues than have been addressed in the past. Balance is required here, however; the core of the review is conclusions based on an integration of earlier work, and a review that mainly focuses on what yet needs to be done is unbalanced. In fact, such an approach would suggest developing a conceptual contribution instead, because this can be a more focused discussion of new theory and avenues for research.
Another consideration is that where evidence-based conclusions are concerned, meta-analyses have a couple of important advantages over narrative reviews. Indeed, in their ability to quantify conclusions and test hypotheses, they are clearly superior to narrative reviews. From that perspective then, conclusions that by and large are summaries of empirical evidence warrant meta-analysis probably more than narrative review – or at least lead to stronger, more impactful conclusions when advanced based on meta-analysis. The weakness of meta-analyses in comparison to narrative reviews, in contrast, is that they are limited to where there is empirical evidence in sufficient numbers. Reviews of the literature can put a lot of emphasis on a limited number of studies, or even on a single study, and may also advance conclusions on the basis of an integration of empirical and conceptual work. Viewed from that perspective, a strong review would advance as its main conclusions such conclusions as could not be more powerfully derived from meta-analysis.
A good review would also clearly identify problems to address in reviewing and integrating the literature. To state the obvious, a review of a field with extremely consistent and uncontested findings is likely to mainly lead to conclusions that researchers in the field subscribed to already. A good review thus would identify a clear need for the review to address unresolved issues. It needs to advance new conceptual insights that move beyond the published body of work on which it is based.
In answering the question of what makes for a good review, this editorial aims to make a modest contribution by advancing some issues for prospective authors to consider. In developing a review of the literature, it may be good to explicitly ask (oneself first, to somehow filter into the writing), which unresolved issues the review is addressing and why a literature review is more appropriate than a meta-analysis or a conceptual contribution. It may also be good to ask whether studies that are not included in the review would challenge its conclusions. First and foremost, one would ask the question, however, which conceptual contribution the review makes – what are the new insights advanced?
This editorial also hopes to be yet another invitation to authors to develop integrative reviews of the literature that make a value-added conceptual contribution. Review proposals as well as fully developed review papers can be submitted through the OPR website, and the editorial team is ready, willing, and able to answer any queries regarding your potential plans to develop a review for OPR.