Abstract
Since the last BJOT editorial on reviews (Unsworth, 2017), we have been delighted with the number and increased quality of submissions we have received in the review category. To continue to support authors to submit the highest quality review articles, this editorial presents further guidance on the kinds of reviews BJOT prioritises for consideration for publication. It also provides information on the critical appraisal and quality review tools that authors can use to review both literature and health applications (software ‘apps’). When submitting a review, please also see the journal’s web page and submission guidance for information on our current policy regarding review submission, as this information changes as we update and revise our priorities for the journal. For example, at present we particularly encourage systematic reviews, and we plan to include at least one systematic review in any special issue.
While BJOT will consider many different types of review articles as submissions, those most likely to be published are systematic reviews, scoping reviews and formal reviews of software apps that, in each case, include either critical appraisal or a quality assessment of the studies or apps included.
Systematic reviews provide a rigorous synthesis of information to answer a specific question or summarise a well-defined topic. When undertaking a systematic review for BJOT, there is an expectation that the review has been registered on a site such as PROSPERO (National Institute for Health Research, York University), and that a team of authors has usually undertaken the meticulous work required to locate and critically appraise the articles found. Findings are synthesised using meta-analysis when possible. Templates and software are available to assist authors in these processes from the Cochrane Collaboration (www.cochrane.org), Joanna Briggs Institute (www.joannabriggs.org) and Covidence (www.covidence.org). Authors should report their review using the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines (Moher et al., 2009). Excellent guidance on writing a systematic review can be found in articles by Harris et al. (2013) and Pati and Lorusso (2018), as well as in the text by Gough et al. (2013). As an example, a systematic review by Novak et al. (2013) examined the best available intervention evidence for children with cerebral palsy. Using bubble diagrams in red (to indicate ‘do not do it’), amber (‘probably do it, proceed with caution’) and green (‘do it’), the authors succinctly conveyed the types and quality of evidence across 131 outcomes for 64 interventions, identifying 15 green light interventions.
Scoping reviews also systematically search for literature, but aim to map an area of literature and identify what work has been completed, what research remains to be done and where systematic reviews may be required. Hence, it would be considered inappropriate to conduct a scoping review on a narrowly defined topic that is well researched, possibly with some randomised controlled trials, in which authors only locate a dozen or so articles to review. In fact, we urge authors to run a test search to identify potential articles to determine if their question is better answered using a systematic review or scoping review before committing to the rigorous search process required. When scoping reviews are conducted, we strongly prefer that if 30 or fewer articles are included, the authors provide justification of why a systematic review with specific research question was not warranted.
When a scoping review is undertaken, we normally expect that some form of quality review of these articles is also included. Although inclusion of a critical appraisal is not documented in scoping review methodologies (Arksey and O’Malley, 2005; Levac et al., 2010), it is increasingly common for authors to do so, and for journals to expect this of authors submitting work for consideration. The rationale for this compels adoption: if authors have mapped a relatively small number of articles in a topic area, then the reader immediately questions the quality of the articles presented to get a clearer picture of the priority of future research needed. For example, a scoping study mapping a topic into three areas may identify five high-quality randomised controlled trials in one area, but only two or three medium-quality descriptive cohort studies in the remaining two areas. This quality assessment provides more valuable information than simply suggesting that all studies contribute equally to knowledge in the field. To give a pertinent real-world example: Schultz Petersen and colleagues (2019) presented a scoping review on research investigating the effects of occupation and activity-based interventions used in mental health. A total of 21 studies and four systematic reviews were mapped (so a relatively small number), the level of evidence was rated against the Oxford Centre for Evidence-Based Medicine Levels of Evidence (see Appendix 1), and their methodological quality was reviewed using the CASP tool (2) qualitative (see Table 1).
Appraisal tools for use in systematic reviews and other forms of review research (alphabetical order by title).
Health apps have become big business internationally; the United States Food and Drug Administration (FDA) estimated that in 2017, 325,000 health apps were available on mobile smart phones, and that there were over 3.7 billion mobile health app downloads that year, internationally. With such a plethora of choice, it makes sense that occupational therapists have already undertaken reviews of apps in a certain category and presented their work to BJOT for publication. Two forms of systematic review can be undertaken in relation to apps: (a) a review of the literature reporting on the apps (see for example Munter-Mas et al., 2019); or (b) a review of the apps themselves. While a systematic review of 45 frameworks for reviewing apps has recently been completed (Moshi et al., 2018), the authors concluded that there were gaps with each of the frameworks identified. A simple approach to evaluate mental health apps by the American Psychiatric Association has been included in Table 1 and Appendix 1 as an example. This approach was then expanded and harmonised by Hensen et al. (2019) with aspects of other review tools noted by Moshi et al. (2018). Authors are encouraged to examine and select a review framework that is suitable for the type of apps under consideration when preparing an app review. An example of a review of health apps was presented by Dubey and colleagues (2014) who undertook a review of smartphone applications as a source of information on stroke. These authors presented a relatively simple review of 93 apps rated across cost, usefulness and scientific validity.
When undertaking any form of review, authors should also consider contextualising the work by citing other reviews completed in the topic area and justify why the new review is required. Rather than detracting from the new review, placing it within the broader context is very helpful for readers and ensures the contribution of the new review to advance knowledge in the field is clear. Finally, there are many critical appraisal or quality assessment tools to help authors review articles and then summarise findings for readers using either a narrative or score. To assist authors in selecting the most appropriate tool to review the studies located in their systematic, scoping or app review, Table 1 provides a summary of tools available, and Appendix 1 presents several websites in which either these tools, or comparative information about these tools, can be located.
Writing a scholarly review requires considerable art and science. Information location, synthesis and critique are essential to provide timely and accurate summaries of available best evidence to support clinical practice. However, as previously noted (Unsworth, 2017), the production of excellent reviews that guide and promote best practice are meaningless if the findings are not lifted from the page and enacted in practice. Development of guidelines and support for clinicians to translate review findings into routine occupational therapy practice is also required. We look forward to continuing to work with authors to deliver readers with informative, thought-provoking reviews that promote best practice in occupational therapy and contribute to the advancement of knowledge.
Footnotes
Research ethics
Ethics approval and consent were not required for this editorial.
Declaration of conflicting interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this editorial. Carolyn Unsworth is an Associate Editor – Reviews, and an Editorial Board member for BJOT.
Funding
The author received no financial support for the research, authorship, and/or publication of this editorial.
Appendix 1: Websites for appraisal tools and resources
App Evaluation model: American Psychiatric Association
Critical Appraisal Skills Program (CASP): economic evaluation studies
Centre for Evidence Based Medicine, Oxford University
Checklist to a evaluate a report of non-pharmacological trial (Clear NPT)
Cochrane Collaboration
Consensus-based Standards for the selection of health Measurement Instruments (COSMIN)
Covidence
Jadad Scale (modified version)
Joanna Briggs Institute
Mixed Methods Appraisal Tool (MMAT) (2018 edition)
Newcastle–Ottawa Quality Assessment Scale
Newcastle–Ottawa Quality Assessment Scale (adapted for cross-sectional studies)
http://doi.org/10.1643/theoncologist.2016-0429 (in supplementary materials)
Oxford Centre for Evidence Based Medicine
Risk of Bias in Non-randomised studies of interventions (ROBINS-I)
Physiotherapy Evidence Database (PeDRO)
Scottish Intercollegiate Guidelines Network (SIGN)
Temple University Critical Appraisal Tools