Correcting the medical literature: ethics and policy.

Editors and publishers take our responsibilities seriously. There are international congresses on peer review in biomedical publications, the most important contributions from which are published in classic, special issues of JAMA. (1-3) A wide range of topics is discussed, including the nature of peer review, whether reviews should be anonymous, whether reviewers should be blinded to the identity of the authors, techniques to minimize publication bias, ways of dealing with plagiarism, the impact of fraudulent research on scientific literature, the effects of institutional prestige and author nationality on reviewers' recommendations, and many other topics.

While the importance of correcting medical literature after fraudulent publication has been addressed, surprisingly little attention has been paid to the important issue of correcting the medical literature after publication of bad papers. As noted in a previous editorial, (4) the peer-review process is intended to detect poorly designed or misleading articles before they are published. The process is flawed, however; substandard articles can appear in any journal.

Editors try to avoid this unfortunate occurrence in several ways. Usually, we assign submitted manuscripts to more than one reviewer, and we select reviewers who should be knowledgeable about the subject matter being reviewed. Moreover, we encourage our editorial boards to return manuscripts that are outside their areas of expertise and to suggest alternate or supplemental reviewers. For example, it is common for reviewers to request that a manuscript receive additional review by a statistician, to be certain that no sophisticated errors go unrecognized, especially if such errors might affect the validity of the conclusions.

Despite our best efforts, once in a while we publish something that should not have been published. Hence, it is surprising that so little attention has been paid to the obligation of a journal to acknowledge this situation and correct the literature when correction is warranted. This shortcoming is true not only in international symposia, but also in our field.

Otolaryngology journal editors meet twice a year (at COSM and at the AAO-HNS meeting) to discuss a variety of subjects ranging from standardization of terminology to the problem of duplicate publication, but this issue has not been on our agenda. I suspect that most of us have assumed that such corrections are handled adequately through letters to the editor, editorials, or retractions. However, addressing such issues occurs at the discretion of the editor in most cases. So, response to such events varies.

The existence of this problem was brought to my attention recently through my correspondence with the Journal of the American College of Surgeons (JACS), certainly a deservedly well-respected publication. In October, JACS published an article on voice changes after thyroidectomy. (5) While I was delighted to see an article on voice in the College's journal, I was dismayed at its quality. The clinical examinations were not sufficiently sophisticated to meet the current standards of clinical care, let alone those of clinical research. No blinded, subjective analysis of the clinical data was included. Preoperative and postoperative laryngeal electromyographic data were not provided, so the status of laryngeal nerve function remains unknown. The voice-analysis system used is not particularly sophisticated, and details of recording protocols were not provided. Also, the parameters selected for reporting were inappropriate. The analysis program used generates data on a variety of other measures, but this paper excluded those data and provided no information as to why they were excluded.

More importantly, the errors in the data reported in this article are profound in their naivete. For example, shimmer was described as a measure of intensity. This is completely inaccurate. Shimmer is a perturbation measure that describes cycle-to-cycle amplitude variation and has no predictable correlation with vocal intensity. Intensity should be measured in decibels SPL under carefully controlled conditions.

This article was fatally flawed and would not have been accepted in any otolaryngology journal. Immediately after the article was published, I wrote a letter to Timothy J. Eberlein, MD, editor of JACS, and I received an admirably prompt, standard rejection letter indicating that the rejection "usually reflects issues of timeliness, a backlog, or beliefs that the material, when complete, did not fit our readership." I would have accepted this outcome happily if my letter had been rejected because another letter had been accepted and was to be published to point out the shortcomings of Sinagra et al's article. However, follow-up contacts with the journal have indicated that this is not the case. Hence, readers of JACS, most of whom are not sophisticated in their analysis of voice literature, are likely to believe the unsupportable conclusions stated in this article. For otolaryngologists who read other voice literature, this may not be a problem; but for thyroid surgeons who primarily read general surgical literature and who may rely on this article, it is a problem.

Readers of all journals should be aware not only that the peer-review process is imperfect, but also that the ethics and policies involved in a journal's response to the publication of bad literature have not been discussed widely and that the response of any journal when such issues arise may lie in the hands of one person. Editors are well-meaning, but we are not always expert in the subject matter of every paper published in our journals. Also, we are laced with numerous pressures regarding journal space and other matters, and we are not blessed with (or bound by) standard policies or guidelines that help determine our response when the validity of a published article is challenged.

These facts highlight not only the need for all of us to read critically, but also the need for more formal review of this subject by publishers, editors, and ethicists interested in optimizing the peer-review process and the quality of medical literature.

References

(1.) JAMA 1990;263:1309-1456.

(2.) JAMA 1994;272:79-174.

(3.) JAMA 1998;280:203-306.

(4.) Sataloff RT. Statistics in clinical practice (editorial). Ear Nose Throat J 2005;84:7.

(5.) Sinagra DL, Montesinos MR, Tacchi VA, et al. Voice changes after thyroidectomy without recurrent laryngeal nerve injury. J Am Coll Surg 2004;199:556-60.