Printer Friendly

Enhancing the quality and transparency of systematic reviews/ Mejorando la calidad y transparencia de las revisiones sistematicas.

Systematic reviews (SR) have been important tools for determining the magnitude of an effect, with appropriate methodology, rigor and scientific quality (1-3). This epidemiologic design was developed to conduct an exhaustive, systematic and explicit assessment of the literature, based on a clearly research question, an explicit methodology, a critical appraisal using a variety of tools and a qualitative summary of the evidence (3). On the other hand, the meta-analysis (MA), is the statistical analysis used in the synthesis of the evidence at the end of a very well performed systematic review3. It compares head to head interventions, however nowadays, we have another tool to perform indirect or mixed comparisons (Network meta-analysis) (4,5). This new statistical tool evaluates the effectiveness when comparing different treatments with similar characteristics, which have not been directly compared in a study. Unlike the traditional meta-analysis, this new tool compares the results of different studies that have a point or a common intervention without a direct comparison (5-7).

Nowadays, when attending patients, professionals need high-quality evidence to base their practice, however, the number of studies is increasing exponentially and It might be very difficult keeping pace with and assessing the evidence presented. At the same time, SRs/MA are growing in number but not always in quality, comprising a number of ways in which bias can be introduced in SRs such as: 1) Inappropriate methodological quality of primary studies; 2) Publication bias (Statistically significant results and those in English language are more likely to be published); 3) Inclusion criteria influenced by the most favorable outcome and the results of the primary studies; 4) inappropriate statistical approach; among others bias (8,9) that need to carefully assess in order to make good decisions in clinical and population settings.

The intention of this editorial is to show some key notes, relevant to evidence synthesis that can be applied by any health-related professionals and researchers.

Steps to follow in Systematic reviews

The synthesis of the evidence gathers a methodologically correct design that requires a working group and a protocol to be developed. All protocols for systematic reviews must be written according to PRISMA-P (10) and registered in recognized database, for example in the international prospective register of systematic reviews (PROSPERO) ( from the University of York and the National Institute for Health Research. This allows to be transparent regarding the methods and purpose of this important kind of research.

People who want to perform systematic reviews or to use information in clinical or field settings must adhere to the following requirements outlined below, in order to achieve consistency and comparability (3,11):

* Establish a clear and concise research question.

* Configure a reproducible search strategy (no limited neither to one language nor to one database).

* Locate and select studies (Published and unpublished).

* Extract the data.

* Assess the quality of the evidence according to the kind of study (Cochrane risk of bias tool, Newcastle--Ottawa Scale (NOS), MINORS, ROBINS-I, QUADAS2 and Grading of Recommendations Assessment, Development and Evaluation (GRADE), among others).

* Analyze and describe the results.

* Perform MA if it is appropriate (fixed or random effects, meta-regression, network meta-analysis, heterogeneity assessment and sensibility).

* Write the manuscript according to PRISMA (12).

Assessing the quality of systematic reviews

Additional to these important issues, health professionals must assess the quality of these manuscripts in order to apply to their patients or analyze in a journal club. This is a fundamental issue to deal with when interpreting the evidence, since it is needed that readers balance the numerical results against the quality of the study to accept the recommendations stated.

Important and standardized tools have been developed to both critically appraise and adequately reporting systematic reviews. These approaches lead to be transparent in science, nonetheless, good reporting (PRISMA statement) is not synonymous to high methodological quality. Two of the most worldwide used tools are: 1)Assessing the methodological quality of systematic reviews (AMSTAR2) (13) and 2)Critically Appraisal Skills Programme (CASP) ( Either one must be used in order to critically appraise systematic reviews in clinical or academic settings as previously stated.

Furthermore, according to Taylor et al. (14), ten questions described in Table 1, are one easy way to critically appraise this kind of study, which contains information regarding: The question, the appropriateness of the design, the methods, the statistical analysis and the conflicts of interests.

Lastly but not the less important, readers need to consider the effect that the author's conflict of interest might add to the effect size. It is important to elucidate whether the research has industry influence regarding the arguments assessed for policy decision-making, since conflict of interest are frequently inappropriately described in editorials, comments, letters, perspectives and obviously also in systematic reviews.



(1.) Ferreira GI, Urrutia G, Alonso-Coello P. Systematic reviews and meta-analysis: scientific rationale and interpretation. Rev Esp Cardiol. 2011; 64(8):688-96. doi: 10.1016/j.recesp.2011.03.029.

(2.) Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta analyses: the PRISMA statement. PLoS Med. 2009; 6(7): e1000097. doi: 10.1371/journal.pmed.1000097.

(3.) Garcia-Perdomo HA. Evidence synthesis and meta-analysis: a practical approach. Int J Urol Nurs. 2015; 10(1): 30-36. doi: 10.1111/ijun.12087.

(4.) Catala-Lopez F, Tobias A. Sintesis de la evidencia clinica y metaanalisis en red con comparaciones indirectas Clinical evidence synthesis and network meta-analysis with indirect-treatment comparisons. Med Clin. 2013;140(4):145-192. Doi: 10.1016/j.medcli.2012.09.013.

(5.) Garcia-Perdomo HA. Network meta-analysis, a new statistical technique at urologists' disposal to improve decision making. Int Braz J Urol. 2018; 44(3): 422-8. doi: 10.1590/S1677-5538. IBJU.2018.03.02

(6.) Mills EJ, Thorlund K, Ioannidis JP. Demystifying trial networks and network meta-analysis. BMJ. 2013; 346: f2914. doi: 10.1136/bmj.f2914

(7.) Catala-Lopez F, Tobias A, Cameron C, Moher D, Hutton B. Network meta-analysis for comparing treatment effects of multiple interventions: an introduction. Rheumatol Int. 2014; 34(11):148996. doi: 10.1007/s00296-014-2994-2.

(8.) Egger M, Dickersin K, Smith G. Problems and limitations in conducting systematic reviews. In: Smith G, Egger M, Altman D (Ed). Systematic reviews in health care: Meta-analysis in context. Second edition. London: BMJ Publishing Group Ltd.; 2001. p. 475.

9. Page MJ, Altman DG, McKenzie JE, Shamseer L, Ahmadzai N, Wolfe D, et al. Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis. J Clin Epidemiol. 2018; 95:7-18. doi: 10.1016/j.jclinepi.2017.11.022.

(10.) Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4: 1. doi: 10.1186/2046-4053-4-1.

(11.) Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. The Cochrane Collaboration; 2011. Available from: https://handbook-5-1.

(12.) Moher D, Liberati A, Tetzlaff J, Altman D. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ Br Med J. 2009; 339: b2535. doi: 10.1136/bmj.b2535.

(13.) Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358: :j4008. doi: 10.1136/bmj.j4008

(14.) Taylor P, Hussain JA, Gadoud A. How to appraise a systematic review. Br J Hosp Med. 2013;74(6):331-4. doi: 10.12968/ hmed.2013.74.6.331Taylor P, Hussain JA, Gadoud A. How to appraise a systematic review. Br J Hosp Med. 2013;74(6):331-334. doi: 10.12968/hmed.2013.74.6.331. Colomb Med. (Cali) 49(4): 251-53

Herney Andres Garcia-Perdomo

Universidad del Valle, Escuela de Medicina, Grupo de Investigacion UROGIV, Cali, Colombia

Corresponding author:

Herney Andres Garcia Perdomo, Associate Professor. Escuela de Medicina . Grupo de Investigacion UROGIV . Universidad del Valle. Cali, Colombia. E-mail: herney.
Table 1. Ten questions to easily assess systematic reviews.

1. Is the study question relevant?

2. Does the study add anything new?

3. What type of research question is being asked?

4. Was the study design appropriate for the research question?

5. Did the study methods address the most important potential
sources of bias?

6. Was the study performed according to the original protocol?

7. Does the study test a stated hypothesis?

8. Were the statistical analyses performed correctly?

9. Do the data justify the conclusions?

10. Are there any conflicts of interest?
COPYRIGHT 2018 Facultad de Salud-Universidad de Valle
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2018 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Editorial
Author:Garcia-Perdomo, Herney Andres
Publication:Colombia Medica
Date:Oct 1, 2018
Previous Article:El Observatorio de Cancer de Cali, una prioridad.
Next Article:Mobile device for thrombolysis decisions for telestroke/ Dispositivo movil para decisiones de trombolisis mediante Telestroke.

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters