In societies with a free and open judiciary system, individuals are permitted to challenge a judge's verdict, ability to remain impartial, and conduct. In the first situation, a higher court of appeal typically handles the matter.1 In the second, the judge is disqualified from overseeing a case if an objective observer raises reasonable questions about the judge's impartiality.2 In the third situation, a governing body, such as a judicial council, hears the complaint. To minimize the likelihood of such events, judges are appointed based on their legal knowledge, prior experience, and a historical demonstration of impartiality and fairness.

Although most medical journals and their editors conduct themselves fairly, there are some differences between their practice and that of a judiciary body. In this article, I pose four questions to medical journal editors concerning their impartiality and training as researchers and editors, and the options available to those who disagree with a decision rendered by an editor or editorial board.

Are you free? Editorial freedom versus economic and political pressure. Editors must be free to make decisions about journal policy and the publication of manuscripts without coercion.3 This is the basis of editorial objectivity, and the foundation on which the advancement of both human well‐being and science rest.4 Those who choose (or are chosen) to become medical journal editors tend to do so because they believe in and wish to uphold this fundamental principle.3 However, it has become apparent that editorial decisions are not always independent or unfettered. In fact, external and internal pressures partially influence an editor's selection and acceptance of manuscripts.

Editors have led themselves to believe that they publish works that address the needs of a principal ‘target’ audience. However, in a study comparing the relative importance of topics selected for publication by JAMA, readers agreed with only three of the top ten subjects selected by the journal's editorial board members.5 A similar discordance was found among peer reviewers, experts and the editors of the Annals of Internal Medicine.6 Thus, if editors do not consistently meet the needs of their readers, on what basis do they make decisions about what they publish?

The influence of pharmaceutical and diagnostic test firms on medical research7 and physician practice patterns8 has received much attention. However, there has been little focus on how the private sector bears on medical journal editors and publishers. In a 1995 survey of 221 North American senior medical journal editors, only 57% believed that journals were responsible for ensuring the honesty of pharmaceutical advertisements, while 40% were in favour of subjecting advertisements to rigorous peer review.9 In response to a commentary questioning the ability of editors to judge the appropriateness of drug advertisements, one editor wrote: ‘Given our heavy reliance on openness in the editorial process and an already considerable but appropriate restriction on the claims that are made in drug ads, the fundamental problem with [this] suggestion is that they must surely stem from a belief that we physicians are far too gullible to understand that an ad is an ad, and a belief that editors don't know that they are judged by what they publish.’10

What is now known is that physicians, under the influence of pharmaceutical advertising and promotions, are much more impressionable than was originally believed.11 A systematic review, recently published in JAMA, highlighted the fact that pharmaceutical advertising and promotions influence physician prescribing practices,12 even among those still in training.13 Frequent and intense drug promotion is correlated with increased prescribing volumes, and more expensive and less appropriate prescribing. Drug advertisements often omit negative points about a drug and offer ‘red herring’ surrogate endpoints while using images (including sexual imagery) to appeal to personal desires.11 Moreover, referenced articles in pharmaceutical advertisements are often of poor methodological quality.14 A study of five major Russian medical journals, comprising 207 distinct advertisements, found that only 40% mentioned the generic name of the drug, 45% mentioned the drug's indication, 11% mentioned safety warnings and contraindications, and 2% provided references. These phenomena arose despite the presence of Russian law prohibiting the omission of essential drug information in its advertisements.15 If journal editors once believed that pharmaceutical advertisements were ‘benign’ and their readers impervious to the negative effects of such ads, one must question whether editors themselves have been influenced by pharmaceutical advertising and sponsorship.

Some medical journals benefit from the publication of subsidized symposia under the name of the parent journal. In a study published in 1992, 42% of 625 symposia published within medical journals had a single pharmaceutical sponsor.16 Published pharmaceutical‐sponsored symposia were more likely to use misleading titles and refer to drugs by their brand name, and less likely to be peer‐reviewed with the same degree of rigor as other journal articles.16

Arguments may be put forward that the above examples are rare, or arose in another era, or in countries with less rigorous editorial policies. Nonetheless, they reveal weaknesses in the policy of objectivity claimed by medical journal editors. As recently as 1997, most editors had not yet required their authors to disclose any potential financial conflict of interest. Only 16% of 1396 highly ranked scientific and biomedical journals had a conflict‐of‐interest policy in effect, and less than 1% of articles disclosed any personal financial conflict of interest on the part of their authors.17 In response, several prominent medical editors have taken a more active stance on this issue. Even so, the private interests of the corporate sponsor are satisfied by editorial policy: ‘[A] sponsor should have the right to review a manuscript for a defined period (e.g. 30 to 60 days) before publication to allow for the filing of additional patent protection, if required … but the sponsor must impose no impediment, direct or indirect, on the publication of the study's full results, including data perceived to be detrimental to the product.’18

Why might journal policy permit a private, arms‐length pharmaceutical sponsor to be privy to unpublished data before it can be shared with other clinicians and researchers?19 Why should editors appease the economic interests of corporate sponsors and effectively muzzle scientists, even in a time‐limited fashion?20 Certainly, there is no editor‐industry conspiracy, and researchers themselves delay publishing data to allow for patent application21 but it is clear that medical journal editors are under greater pressure than they are willing to admit.

Another challenge facing medical journal editors is the need to publish studies that improve their journal's profile, vis‐à‐vis its impact factor22,23 and media exposure.24 This, in turn, augments the sale of journal subscriptions and article reprints.25 Because medical publishing is a competitive business, an editor must consider not only the clarity and scientific validity of a manuscript, but its novelty and timeliness as well.24,26 One former editor of JAMA labelled this influential factor as the ‘sexiness' of the manuscript.25 For a popular journal, the sale of reprints and syndicated research can turn big revenues,25 particularly in the context of product promotion, as evidenced by an advertisement on the British Medical Journal's website: ‘Reprints are invaluable for direct marketing, exhibitions/seminars and sales support campaigns and for mailing new product information to doctors [and for distribution] to conference delegates and visitors at exhibitions.’27

Beyond their susceptibility to advertisers' and their own economic interests, editors may also face tremendous pressures from the journal's governing body. We have recently witnessed the firing of the chief editors of both JAMA28 and the New England Journal of Medicine.29 Both dismissals arose out of strong philosophical differences between the American Medical Association29 and the Massachusetts Medical Society28 and their respective editors. Even if editors respond to such divergence by claiming freedom and independence,28,29 there is no guarantee of an effective change in this relationship: the editor of JAMA made this point twice prior to his own dismissal.3,30

Who made you the boss, and why? Training and appointment of editors. The position of editor‐in‐chief is held in high regard. However, senior editorial appointments often occur independently of a formal, regulated process. For example, in a survey of the editors‐in‐chief of 191 peer‐reviewed clinical journals, 32.5% stated that they were elected or selected by a scientific society, editorial board or research committee, while 21% were nominated by the previous editor.31 Moreover, formal editorial training has been quite limited. In the same survey, 45% of senior editors had no formal training in editing, and 35% had not previously served on the journal's editorial team. When questioned, 69% of editors believed that some form of editorial training would be helpful; surprisingly, 26% did not.31

It has been suggested that medical journal editors should have clinical or research experience in, and a substantial knowledge of, the topics within the scope of their journal.32 However, after a detailed search of the internet, using both PubMed Medline and Google search engines, suggested requirements for training of journal editors as researchers or in the critical appraisal of medical information could not be found. Few individuals in society (perhaps with the exception of politicians) are placed in a position of power and authority without some evidence that they have ‘worked from the ground up’. Those trusted with the selection and dissemination of new and potentially valuable research ideas must be treated no differently.

Does the boss make mistakes? Avoiding errors and editorial malpractice. Applied statistics remains the principal method for analysing medical research data. Because editors often do not understand complex biostatistical methods, most rely on the expertise of statisticians.33 However, in a survey of 114 biomedical journal editors, only a third maintained a policy guaranteeing statistical review for all accepted manuscripts, while 31% of small and 58% middle‐sized journals maintained a statistical consultant on staff.34 Even among journals that do consult with a statistician, the application35 and reporting36 of commonly used statistical methods, even at the most basic levels, remains poor.36,37 For example, in a random sample of 27 randomized clinical trials published within five major medical journals, 81% explicitly reported a single primary outcome, and only 41% provided confidence intervals for the point estimates of the efficacy of the interventions. The clinical importance of the findings was discussed in 20 studies (74%), of which five (25%) provided justification for their clinical interpretation of the results.36 Simply stated, editors cannot guarantee that basic statistical concepts have been properly evaluated and presented in the manuscripts that they publish.38

To assess the appropriateness and scientific quality of a manuscript, editors rely on peer reviewers9,39 defined by one editor as ‘professionals whose standing is equal to or above that of the authors’.40 However, the use of expert peer reviewers introduces several problems. First, there is often poor agreement between reviewers about the acceptability of research manuscripts, which was demonstrated by kappa scores ranging from 0.08 to 0.28 among peer reviewers of two neuroscience journals.41 In the same study, comparably low agreement scores were observed for assigning priority to manuscript publication.41 A similar lack of agreement between reviewers was observed in a study of research manuscripts submitted to the Annals of Internal Medicine.6 Country of origin and language of composition also appear to influence a peer reviewer's judgment of a manuscript. For example, in a study of original manuscript submissions to Gastroenterology, US reviewers were significantly more likely than non‐US reviewers to recommend acceptance of papers submitted by US authors.42 Non‐US reviewers also ranked US papers slightly higher than non‐US papers, albeit to a lesser degree than their US counterparts.42 In another study, a non‐authentic but realistic short manuscript was randomly sent to more than 150 Scandinavian referees; one version was written in English, the other in the referee's national language.43 The reviewers gave a significantly higher acceptance rating to the English‐language version.43 Finally, the majority of editors do not believe in querying their reviewers about any potential financial conflict of interest.9

If medical editors depend on a system of peer review that is both imprecise and susceptible to considerable variation and bias,44,45 editorial decisions may be weakened. If a novel study is submitted to a journal, representing serendipitous but important findings,46 or the ‘controversial or non‐conformist views of sceptics and dissident minorities, ‘will the peer review system be fair, or will ’highly innovative science be stifled by the conflict of interest and reviewer shortcomings underlying the review system?’.47

Can there be another boss too? Judging the judges. Abuse of power and misconduct by medical editors is probably quite uncommon.48 But beyond being accountable to themselves, and, in the case of larger medical journals, editorial board members, editors' decisions cannot be challenged or reversed. Although the reader might be deemed the ultimate judge, they cannot judge that which has not been published.

To resolve disputes between a medical journal editor and a researcher or reader, an appeal process, such as that proposed by Altman in 1994,48 is required. This should comprise an external appeal mechanism (when a dispute cannot be resolved from within), and an international council that would develop a code of conduct for editors with a corresponding taxonomy of inappropriate editorial conduct.48 Surveyed nearly 5 years after this proposal was made, most journal editors agreed with the formation of such a system,49 but little has materialized since then.

Creating a separate ombudsperson for every journal, as the Lancet50 does, is a novel idea. The Lancet's model, initiated in 1996, allows direct contact with the ombudsperson and bypasses the editors entirely. This process, which handled 20 complaints during its first 18 months, has been successful from the perspective of the Lancet's editor.50 For smaller journals, which form the majority of medical journals, individual ombudspersons might be impractical. To address this, the publisher could appoint a non‐partisan individual who covers all the journals it represents. Alternatively, the ombudsperson might be appointed by a group of journals within a certain geographical region or area of speciality.

To conclude, this article was not written to denounce medical journal editors or suggest that they be replaced by another body. Journal editors should remain at the helm of medical journal science, especially since the editors of prominent medical journals have raised a number of the shortcomings described herein. Indeed, the World Association of Medical Editors (WAME) was founded in 1995 for this purpose.32 Nonetheless, because such self‐reflection remains in its infancy and the position of medical journal editor is not without bias, arbiters who are independent of the journal editorial board and the publisher are needed. To generate better feedback to editors and researchers alike, we also require candour about the qualitative aspects of selecting and editing a medical manuscript.51

Peer reviewers, the bedrock of medical journal objectivity, require more training and experience. One simple solution might be for editors to provide them with a short review of best practices along with the checklist of core elements to consider.52,53 Although peer reviewers are in short supply, it is important that editors, using validated rating instruments,3,54 only retain those who can do a proper job. Similarly, medical journal editors need more formal training, including a solid grounding in the principles of critical appraisal, practical biostatistics, and, perhaps, the science of arbitration. Formal editorial training55 and a demonstration of previous competence in research should be prerequisites for the appointment of a senior journal editor. Such demands may be high, but it is in their hands that society places its most highly regarded vehicle for advancing health science.56

In setting the stage for future obstacles yet to face medical editors and researchers alike, we may draw wisdom from those intimately tied to both: ‘we need less research, better research, and research done for the right reasons’.57 If we refuse to engage in such a debate, we may forfeit our freedom and independence. ‘Years ago society accorded science the privilege of governing itself. If it finds that its trust continues to be betrayed, the politicians will step in by default.’58

Ms Marian Vermeulen, MHSc, both provided helpful comments and edited this manuscript. Dr Ray was previously an editorial fellow at the Annals of Internal Medicine. He is currently an editorial board member of the journals Current Controlled Trials in Cardiovascular Medicine, and Evidence‐Based Obstetrics and Gynecology. He is also an honorary member of the World Association of Medical Editors. As an active medical researcher, he has had many manuscripts rejected by various medical journals.

References

1
Canadian Judicial Council. The Conduct of judges and the role of the Canadian Judicial Council. 2000. Ottawa, Canada: Canadian Judicial Council, pp. 1–7 [http://www.cjc‐ccm.gc.ca/english/CofJ‐E.pdf] Accessed: 1 October 2001.
2
Citizens for Legal Responsibility. Disqualification of judges. Morton Grove, Illinois: Citizens for Legal Responsibility [http://www.clr.org/Judicial‐disqualification.html] Accessed: 2 October 2001.
3
International Committee of Medical Journal Editors. Editorial freedom.
Br Med J
 
1988
;
297
:
1182
.
4
Lundberg GD. The role and function of professional journals in the transfer of information.
Int J Technol Assess Health Care
 
1988
;
4
:
51
–8.
5
Lundberg GD, Paul MC, Fritz H. A comparison of the opinions of experts and readers as to what topics a general medical journal (JAMA) should address.
JAMA
 
1998
;
280
:
288
–90.
6
Justice AC, Berlin JA, Fletcher SW, Fletcher RH, Goodman SN. Do readers and peer reviewers agree on manuscript quality?
JAMA
 
1994
;
272
:
117
–19.
7
Walsh ME, Graber GC, Wolfe AK. University‐industry relationships in genetic research: potential opportunities and pitfalls.
Account Res
 
1997
;
5
:
265
–82.
8
Dixon T. Pharmaceutical advertising: information or influence?
Can Fam Physician
 
1993
;
39
:
1298
–304.
9
Wilkes MS, Kravitz RL. Policies, practices, and attitudes of North American medical journal editors.
J Gen Intern Med
 
1995
;
10
:
443
–50.
10
Rennie D. Editors and advertisements: what responsibility do editors have for the advertisements in their journals? (editorial)
JAMA
 
1991
;
265
:
2394
–6.
11
Mansfield PR. How does pharmaceutical company promotion affect prescribing? First International Conference on Improving Use of Medicines: ICIUM. Chiang Mai, Thailand, April 1997. [http://www.who.int/dap‐icium/posters/4P7_fintext.html] Accessed: 14 November 2001.
12
Wazana A. Physicians and the pharmaceutical industry: is a gift ever just a gift?
JAMA
 
2000
;
283
:
373
–80.
13
Anonymous. Drug‐company influence on medical education in USA (editorial).
Lancet
 
2000
;
356
:
781
.
14
Lexchin J, Holbrook A. Methodologic quality and relevance of references in pharmaceutical advertisements in a Canadian medical journal.
CMAJ
 
1994
;
151
:
47
–54.
15
Vlassov V, Mansfield P, Lexchin J, Vlassova A. Do drug advertisements in Russian medical journals provide essential information for safe prescribing?
West J Med
 
2001
;
174
:
391
–4.
16
Bero LA, Galbraith A, Rennie D. The publication of sponsored symposiums in medical journals.
N Engl J Med
 
1992
;
327
:
1135
–40.
17
Krimsky S, Rothenberg LS. Conflict of interest policies in science and medical journals: editorial practices and author disclosures.
Sci Eng Ethics
 
2001
;
7
:
205
–18.
18
Davidoff F, DeAngelis CD, Drazen JM, Hoey J, Hojgaard L, Horton R, Kotzin S, Nicholls MG, Nylenna M, Overbeke AJ, Sox HC, Van Der Weyden MB, Wilkes MS. Sponsorship, authorship, and accountability.
Lancet
 
2001
;
358
:
854
–6.
19
Stacey J. The press embargo: friend or foe? (letter)
JAMA
 
1985
;
254
:
1965
.
20
“Muzzling scientists” (news report).
Br Med J
 
1991
;
302
:
199
.
21
Blumenthal D, Campbell EG, Anderson MS, Causino N, Louis KS. Withholding research results in academic life science. Evidence from a national survey of faculty.
JAMA
 
1997
;
277
:
1224
–8.
22
Sperti G, Ierardi C. The ranking of Italy's publications in the biomedical field: an objective evaluation.
Cardiologia
 
1999
;
44
:
51
–4.
23
Marban E. Circulation research impact factor sets new record (editorial).
Circ Res
 
2001
;
89
:
101
.
24
Nelkin D. Scientific journals and public disputes.
Lancet
 
1998
;
352(Suppl.)
:
SII25
–8.
25
Lundberg G. Lundberg: serious journals can be sexy. Inside Medill News. 18 February 1999. [http://www.medill.northwestern.edu/inside/1999/lundberg.html#top] Accessed: 16 November 2001.
26
Radford DR, Smillie L, Wilson RF, Grace AM. The criteria used by editors of scientific dental journals in the assessment of manuscripts submitted for publication.
Br Dent J
 
1999
;
187
:
376
–9.
27
BMJ Publishing Group. Reprints. [http://www.bmjpg.com/template.cfm?name=reprints] Accessed: 15 November 2001.
28
Fletcher SW, Fletcher RH. Medical editors, journal owners, and the sacking of George Lundberg.
J Gen Intern Med
 
1999
;
14
:
200
–2.
29
Hoey J. When journals are branded, editors get burnt: the ousting of Jerome Kassirer from the New England Journal of Medicine.
CMAJ
 
1999
;
161
:
529
–30.
30
Lundberg GD. House of Delegates reaffirms editorial independence for AMA's scientific journals (editorial).
JAMA
 
1993
;
270
:
1248
–9.
31
Garrow J, Butterfield M, Marshall J, Williamson A. The reported training and experience of editors in chief of specialist clinical medical journals.
JAMA
 
1998
;
280
:
286
–7.
32
Squires BP. A global network for medical journal editors.
CMAJ
 
1995
;
152
:
1757
–9.
33
Marks RG, Dawson‐Saunders EK, Bailar JC, Dan BB, Verran JA. Interactions between statisticians and biomedical journal editors.
Stat Med
 
1988
;
7
:
1003
–11.
34
Goodman SN, Altman DG, George SL. Statistical reviewing policies of medical journals: caveat lector?
J Gen Intern Med
 
1998
;
13
:
753
–6.
35
Bagley SC, White H, Golomb BA. Logistic regression in the medical literature: standards for use and reporting, with particular attention to one medical domain.
J Clin Epidemiol
 
2001
;
54
:
979
–85.
36
Chan KB, Man‐Son‐Hing M, Molnar FJ, Laupacis A. How well is the clinical importance of study results reported? An assessment of randomized controlled trials.
CMAJ
 
2001
;
165
:
1197
–202.
37
Welch GE 2nd, Gabbe SG. Review of statistics usage in the American Journal of Obstetrics and Gynecology.
Am J Obstet Gynecol
 
1996
;
175
:
1138
–41.
38
Lukic IK, Marusic M. Appointment of statistical editor and quality of statistics in a small medical journal.
Croat Med J
 
2001
;
42
:
500
–3.
39
Callaham ML, Baxt WG, Waeckerle JF, Wears RL. Reliability of editors' subjective quality ratings of peer reviews of manuscripts.
JAMA
 
1998
;
280
:
229
–31.
40
Anonymous. On looking at ourselves (editorial).
Med J Aust
 
1993
;
159
:
6
.
41
Rothwell PM, Martyn CN. Reproducibility of peer review in clinical neuroscience: is agreement between reviewers any greater than would be expected by chance alone?
Brain
 
2000
;
123
:
1964
–9.
42
Link AM. US and non‐US submissions: an analysis of reviewer bias.
JAMA
 
1998
;
280
:
246
–7.
43
Nylenna M, Riis P, Karlsson Y. Multiple blinded reviews of the same two manuscripts: effects of referee characteristics and publication language.
JAMA
 
1994
;
272
:
149
–51.
44
Eldredge J. Characteristics of peer reviewed clinical medicine journals.
Med Ref Serv Q
 
1999
;
18
:
13
–26.
45
Anonymous. Dangers of over‐dependence on peer‐reviewed publication (editorial).
Nature
 
1999
;
401
:
727
.
46
Sommer TJ. Suppression of scientific research: bahramdipity and nulltiple scientific discoveries.
Sci Eng Ethics
 
2001
;
7
:
77
–104.
47
Stehbens WE. Basic philosophy and concepts underlying scientific peer review.
Med Hypotheses
 
1999
;
52
:
31
–6.
48
Altman DG, Chalmers I, Herxheimer A. Is there a case for an international medical scientific press council?
JAMA
 
1994
;
272
:
166
–7.
49
Good CD, Parente ST, Rennie D, Fletcher SW. A worldwide assessment of medical journal editors' practices and needs: results of a survey by the World Association of Medical Editors.
S Afr Med J
 
1999
;
89
:
397
–401.
50
Horton R. The journal ombudsperson: a step toward scientific press oversight.
JAMA
 
1998
;
280
:
298
–9.
51
Purcell GP, Donovan SL, Davidoff F. Changes to manuscripts during the editorial process: characterizing the evolution of a clinical paper.
JAMA
 
1998
;
280
:
227
–8.
52
Strayhorn J Jr, McDermott JF Jr, Tanguay P. An intervention to improve the reliability of manuscript reviews for the Journal of the American Academy of Child and Adolescent Psychiatry.
Am J Psychiatry
 
1993
;
150
:
947
–52.
53
Paice E. How to write a peer review.
Hosp Med
 
2001
;
62
:
172
–5.
54
Feurer ID, Becker GJ, Picus D, Ramirez E, Darcy MD, Hicks ME. Evaluating peer reviews: pilot testing of a grading instrument.
JAMA
 
1994
;
272
:
98
–100.
55
Kassirer JP. msJAMA: Why be a medical editor? (editorial).
JAMA
 
2001
;
285
:
2253
.
56
Vandenbroucke JP. 175th anniversary lecture. Medical journals and the shaping of medical knowledge.
Lancet
 
1998
;
352
:
2001
–6.
57
Altman DG. The scandal of poor medical research (editorial).
Br Med J
 
1994
;
308
:
283
–4.
58
Lock S. Ethics and journal publishing: taking the debate forward. Wanted? Standard operating procedures for journals.
Addiction
 
1995
;
90
:
1313
–15.