Printer Friendly

The research interview: Q&A with the authors of "Avoidable Waste of Research Related to Inadequate Methods in Clinical Trials".

In a recent issue of the BMJ, Yordanov et al. report on their study of hundreds of randomized clinical trials. (1) They were interested in the concept of "wasted research related to inadequate methods" and looked at trials included in reviews recently published in the Cochrane Database of Systematic Reviews. For trials at "high risk of bias" in at least 1 domain, they assessed whether simple and inexpensive methodological adjustments at the planning stage could have decreased the risk of bias. The domains included sequence generation for randomization of participants, allocation concealment (so that researchers cannot influence how participants are assigned), blinding of participants and personnel, blinding of outcome assessment, and incomplete outcome data.

Victoria White, MA, ELS, editor of the AMWA Journal, conducted an email interview with the study's first 2 authors, Youri Yordanov, a physician and doctoral student at INSERM in Paris, and Agnes Dechartres, MD, PhD, a researcher at INSERM.

Q: It would appear that your research is of interest to medical writers in various ways. It highlights the idea that too much research yields findings that are not very interpretable. It also suggests various stages in which research waste could have been prevented, partly through better reporting of methods. Before we get into the design of your study and its important findings, can you define some key terms and concepts for us so that we will understand how they are being used in this study? In particular, what are research waste, risk of bias, and inexpensive methodological adjustments?

A: Research waste related to inadequate methods refers to trials with methodological problems resulting in high risk of bias (eg, inadequate randomization method resulting in high risk of selection bias) that could have been avoided by better planning of the trial.

Risk of bias refers to the tool developed by the Cochrane Collaboration for assessing risk of bias. (2) This tool includes elements associated with treatment effect estimates such as randomization method, allocation concealment, blinding, exclusion of patients from analysis.

By inexpensive methodological adjustments, we are referring to methodological adjustments that could have been easily performed from the planning stage of the trial and that represented less than 5% of the total cost of the trial as assessed by experts in the field.

Q: With our medical writer audience in mind, what do you see as your key findings?

A: In our study, we aimed to evaluate if part of the waste of research related to inadequate methods in clinical trials included in recent Cochrane reviews could have been avoided by simple and inexpensive methodological adjustments at the planning stage of the trial. We found that such adjustments could decrease the number of elements at high risk of bias in nearly half of trials and could transform all elements at high risk to low risk in 12% of the 142 trials in our sample (95% CI, 7% to 18%) (Figure 1). In a simulation study correcting for incomplete

reporting, this avoidable waste represented 42% (95% CI 36 to 49%).

Q: How did you go about evaluating the trials for risk of bias?

A: We evaluated the risk of bias using the Cochrane risk of bias tool, a tool including elements associated with treatment effect estimates. (2) In this study, we initially relied on the risk of bias assessment conducted by the review authors and then reassessed risk of bias for a sample of trials.

Q: Can you briefly describe the simulation study you did to re-estimate the avoidable waste if all trials were adequately reported?

A: Many trials in our sample did not adequately report all elements so that the risk of bias could not be assessed. Therefore, we could not fully assess the avoidable waste due to inadequate methods. For example, in our sample, allocation concealment was not reported in 54% of trials, so risk of bias for this element and for those trials was unclear. So, the second part of our study was a simulation of what would be the avoidable waste due to inadequate methods if all methodological elements were adequately reported. To do so, we considered all elements that were not adequately reported as missing data and used multiple imputation [a statistical technique] to attribute a high or low risk assessment to these domains. Then, we re-estimated the avoidable waste by assessing whether simple and inexpensive adjustments could transform elements at high risk into low risk. After this imputation, we estimated that the avoidable waste related to inadequate methods could represent 42% (95% CI 36% to 49%). This means that if all trials were adequately reported, simple and inexpensive adjustments could transform all elements at high risk to low risk in 42% of trials that initially had one element at high risk of bias.


Q: You mentioned that in 54% of the trials in your sample, allocation concealment was not reported. Just so that we are clear, you are not saying that 54% of trials don't report this. Your sample didn't consist of trials in general, but rather it consisted of 200 trials for which Cochrane review authors had already determined that 1 domain was at high risk of bias. Is that correct?

A: Yes, from the random sample of trials confirmed to have at least one trial at high risk of bias, information on allocation concealment was not reported in 54% of the trials.

Q: In the article, you and your coauthors mention that it would be beneficial to have "more active implementation" of the CONSORT guidelines, which describe how to report various aspects of randomized controlled trials. Do you have suggestions for how to more actively implement the guidelines?

A: There is evidence that the quality of reporting in trials remains insufficient. In our study, we found that 74% of the trials in the sample had at least 1 methodological element at unclear risk of bias because of incomplete reporting. However, there is a massive diffusion of reporting guidelines such as the CONSORT Statement and endorsement by journals. Turner et al. estimated that over 600 journals have endorsed the CONSORT Statement. (3) Many editors endorse the CONSORT Statement, but they only mention the existence of reporting guidelines in the instructions to authors. Some editors have implemented more active policies, such as asking authors to submit a checklist at the submission or acceptance stage or asking peer reviewers to evaluate the quality of reporting using the guidelines. A study showed that active implementation of CONSORT abstract guidelines was associated with an improvement in the reporting of abstracts. (4)

Q: What are the limitations of your research? And where might you go next with this topic?

A: Our study has several limitations. Our sample of trials is from Cochrane reviews, which may have excluded trials not meeting certain methodological criteria. Also, we relied on review authors' risk of bias assessment and reassessed risk of bias for a random sample of 200 trials at high risk of bias. We may thus have underestimated the number of trials at high risk of bias. Even if we attempted to provide a classification of all methodological problems in as much detail as possible, feasibility and costs of adjustments may vary across trials for the same methodological problem.

In our study, we focused on the waste related to inadequate methods. As outlined by Chalmers, waste of research occurs at all stages of clinical research, from the choice of questions that are not relevant to under-reporting of trial results. (5) So, there is a lot to do to decrease the waste at each stage.


1. Many clinical trials reported in the medical literature are not adequately designed, potentially biasing the outcomes.

2. Simple, inexpensive changes in trial design may be able to influence the value of trial results.

3. Even trials that are properly designed may not be adequately reported; clear reporting can increase the probability that the results of properly designed trials can be interpreted correctly.

4. Adhering to the reporting guidelines in the CONSORT Statement could reduce research waste.


(1.) Yordanov Y, Dechartres A, Porcher R, Boutron I, Altman DG, Ravaud P Avoidable waste of research related to inadequate methods in clinical trials. BMJ. 2015;350:h809.Epub

(2.) Cochrane Handbook for Systematic Reviews of Interventions: The Cochrane Collaboration; 2011.

(3.) Turner L, Shamseer L, Altman DG, Schulz KF, Moher D. Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. Syst Rev. 2012;1:60.

(4.) Hopewell S, Ravaud P, Baron G, Boutron I. Effect of editors' implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ. 2012;344:e4178.

(5.) Chalmers I, Glasziou P Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86-89.
COPYRIGHT 2015 American Medical Writers Association
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2015 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:White, Victoria J.
Publication:American Medical Writers Association Journal
Article Type:Interview
Geographic Code:1USA
Date:Jun 22, 2015
Previous Article:Patient decision support in renal care: a clinical perspective.
Next Article:Elixir: The American Tragedy of a Deadly Drug.

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters