While several thousand primary research articles are published every year, secondary literature such as reviews are used to collect large amounts of research data into coherent, manageable pieces. They provide excellent sources of information for novices or for the seasoned veteran. Review writing is not just an art but a rigorous scientific task that must be systematically undertaken. In an article titled “On the benefits of systematic reviews for wildlife parasitology”, published in the journal International Journal of Parasitology Parasites and Wildlife in 2016, authors Haddway and Watson actively encourage authors of review articles to avoid non-systematic reviews.
They start by pointing out that systematic reviews require significant investments of time and resources to ensure that reliable methods are used to aggregate literature and understand the subject under study to avoid biases. In contrast, non-systematic reviews may exhibit biases including biases in publication selections, lowering their intended reliability. The authors provide an ideal example of how to conduct a systematic review in their paper.
The best part of the paper is perhaps table 1 that lists the different types of reviews that they identified in wildlife parasitology, an aggregative field that combines ecology and parasitology.
1. “Configurative narrative integrative reviews” typically analyze qualitative metrics with little attention paid to quality and ill-defined criteria for inclusion or exclusion in the review. These are useful introductory notes to an area of work.
2. “Aggregative scoping reviews” typically analyze literature by vote-counting or categorizing but pay little attention to quality and possess ill-defined criteria for inclusion or exclusion. These provide a precursory evaluation of the scope of research in the field.
3. “Aggregative full literature reviews” typically are exhaustive searches that analyze quality data, placing them in categories based on some metric. These add to research in a field by synthesizing evidence provided in empirical data into coherent arguments.
4. “Aggregative meta-analysis reviews” typically use well-defined quantitative methods following exhaustive literature searches to synthesize statistically important arguments from multiple sets of empirical data, overcoming biases that arise from homogenous populations.
5.“Aggregative systematic reviews” are high in detail with quantitative and qualitative appraisals of all data available in a given area including grey literature, with assessments of quality and liability.
The authors expand on “domains” that affect the quality of reviews and give some advice for avoiding landmines that I think are best implemented during the writing process. I have summarized them here, but I suggest that the original paper be read for context (Link).
1. Transparency: To ensure that science remains repeatable and verifiable, the authors suggest that information on search strings and databases be provided.
2. Comprehensiveness: To avoid publication selection biases given the power of reviews in driving decision-making, the authors suggest that multiple databases be used with attention paid to non-significant results in grey literature.
3. Vote-counting: The authors posit that categorizing data into positive, negative or non-significant bins, aka vote-counting, should be avoided so that magnitude and variability of data that arises from heterogeneity and effect size are captured.
4. Critical-appraisal: Studies included in an review should be assessed for their reliability to ensure that only good quality data is included. The authors provide examples for why accuracy, validity and biological relevance may vary for each study based on its experimental design (sample size, control matching and inherent genetic variability).
5. Evidence for effects: The authors caution fellow writers on the over-arching tendency to declare the lack of evidence of effect as the evidence of no effect. They recommend extreme care in the interpretative phase so that a lack of studies is not interpreted as a lack of effect.
I found that the advice prescribed in the article to be broadly applicable to review article writing in the biological sciences. It is especially relevant to graduate students writing their literature review chapters for their dissertations, and having recently written my own recently, I have found ways to improve it before final submission. Let us continue to make efforts to contribute to reliable science!
Reference:
Haddaway, Neal R., and Maggie J. Watson. "On the benefits of systematic reviews for wildlife parasitology." International Journal for Parasitology: Parasites and Wildlife 5.2 (2016): 184-191.
