News Media Reporting of Medical Research Is Criticized

WHITE RIVER JUNCTION, VT, June 9 ? TV and newspaper reports emerging from major medical meetings are so overstated or so lacking in context that viewers and readers would be better off paying no attention to them whatsoever.

So declared a pair of Dartmouth researchers with the VA Outcomes Group here. Steven Woloshin, M.D., M.S. and Lisa M. Schwartz M.D., M.S., advised consumers to skip the medical news reported by the popular press because "ignoring a preliminary report about a weak study is preferable to being misled."

They reviewed 174 newspaper articles, including 32 front-page stories, and 13 radio and TV news stories, and reported their findings on the quality of the reportage in the June issue of the Medical Journal of Australia.

All articles were based on research reported at five scientific meetings in 2002 and 2003-the American Heart Association, 14^th Annual International AIDS Conference, American Society of Clinical Oncology, Society for Neuroscience, and the Radiological Society of North America.

Three physicians with clinical epidemiology training assessed the accuracy of reporting. using a number of criteria including reporting of study details such as size, design and quantification of results, reporting of study limitations, reporting of adverse events and warnings about the preliminary nature of research reported at meetings.

Among the findings:

• 34% of studies did not mention study size;
• 18% did not mention study design (more than a third of these were so ambiguous that the reviewers had to guess the design);
• 40% did not quantify the main result;
• 94% of reports about animal studies failed to mention that the results might not be applicable in humans;
• 90% of stories about uncontrolled studies failed to point out that the it was impossible to conclude that the result was associated to treatment;
• 12 news stories mentioned that the study was also reported in a medical journal;
• Only two of the news articles about the 175 unpublished studies noted that the findings were not published, might not have been peer reviewed, and could change prior to publication.

The authors wrote that research "presented at scientific meetings is generally not ready for public consumption: results change, fatal problems emerge, and hypotheses fail to pan out. Nonetheless, the presentations are often big news."

The authors conceded that their review is limited by its scope-they only analyzed articles from five meetings-that may have missed "better" coverage from other scientific meetings. But they think this is unlikely because the five meetings are considered major news events that draw top-level media.

Bias is also a potential problem, but the authors sought to "minimize subjectivity by creating and pretesting an explicit coding scheme, using two independent coders (one blinded to our hypotheses) and reporting only elements with very high inter-rater reliability," they wrote.

Finally, the authors concluded that it is unlikely that there will be less reporting at medical meetings in the future. So, better reporting will require better sources. "Press releases issued by meeting organizers, granting agencies and academic institutions should routinely included balanced data presentations (we favor tables with the absolute risks of outcomes for each study group) and study cautions," they advise.

The report by Drs. Woloshin and Schwartz followed a study published last March in Mayo Clinic Proceedings that claimed one in five newspaper articles about neurological disorders was wrong.

• Related articles: