While attending the annual Advancing Ethical Research Conference of Public Responsibility in Medicine and Research (PRIM&R) last month in San Diego, I had the opportunity to hear a talk by Dr. John Ioannidis, in which he debunked commonly accepted scientific “truths.” Calling upon his own work, which is focused on looking critically at published studies to examine the strength of their claims (see his heavily downloaded 2005 paper “Why Most Published Research Findings Are False”), Ioannidis raised important questions for those of us who think about research ethics, and who oversee and manage the research conducted at universities and scientific institutes across the country.
Ioannidis persuasively argued that our system for publishing only studies with statistically significant positive findings has resulted in a bizarre kind of reality where virtually no studies are ever reported that found “negative” results. Negative results are suppressed because nobody is interested in publishing them. Editors and reviewers have a major role in this problem; they choose not to publish studies that are not “sexy.” This artificially inflates the proportion of observed “positive” results and influences the likelihood a scientist will even write up a journal article because she knows what it takes to get published.
But isn’t there an ethical obligation to publish so-called negative results? In human research, people give their time and undergo risks for the conduct of a study. Their sacrifices are not meaningful if the results are never shared. Furthermore, negative results tell us something important. And if they are not published, some other research team somewhere else may unknowingly repeat a study, putting a new batch of subjects at risk, to investigate a question for which the answer is already known. Finally, to the extent a study is conducted using taxpayer dollars, the data derived should be considered community property, and there are opportunity costs associated with unnecessarily repetitive work.
Currently, Institutional Review Boards (IRBs) are charged with performing an analysis to determine whether the risks and discomforts to be experienced by study participants can be justified in light of the anticipated benefits to the subjects or to others/society (see 45 CFR §46.111: Criteria for IRB approval of research). Through the lens of Ioannidis’ talk, a meaningful consideration of risks and benefits should require the research team to demonstrate it has conducted a thorough review of what is known already about the phenomenon proposed for study. (Incidentally, IACUCs—the committees that oversee animal research — already require performance of a lit search at the time of study proposal to assure alternatives have been considered.) Such a review could only be considered exhaustive if previous results from similar experiments, both “positive” and “negative,” were available for consideration, both by by the researchers proposing a new study and by the IRB.
In most other contexts, we are encouraged to accentuate the positive: in our work, our personal relationships, even for purposes of improving our own health. But in matters of research, the knowledge that an experiment does not work, or that a new drug/device/procedure does not work, is important and beneficial. Failing to share such information, whether that failure is on the part of a researcher or an editor, is not just a missed opportunity to contribute to a shared body of knowledge. It also can be an ethical violation. Wasting precious resources on (inadvertently) repetitive studies destined to fail is a waste of time, effort, and money that could have been used for better purposes. And putting future subjects at unjustified risk of harm in such studies is unacceptable.