Publication bias is a major problem in clinical research. Because negative or neutral results are often not published, the published literature shows a bias towards significant (positive) results. Drugs appear to be more effective than they really are, and side effects seem less severe. Consequently, patients suffer and die unnecessarily.
This TED Talk by Dr Ben Goldacre explains why unreported instances of negative data are misleading and dangerous: Ben Goldacre: What doctors don’t know about the drugs they prescribe
A growing number of researchers and the public want to be able to see the full picture and have all clinical trial data made available for analysis. The AllTrials campaign, launched a year ago, “calls for all past and present clinical trials to be registered and their results reported. The campaign has published a detailed plan on how all clinical trials can be registered and all results reported.”
Sign the AllTrials petition.
It’s welcome news that the company Johnson & Johnson has recently decided to give its data from pharmaceutical clinical trials to researchers at Yale University. Even when research is published, the raw data are generally not made public. The Yale University Open Data Access Project (YODA) will receive all data from Johnson & Johnson and make it available to other researchers upon request.
Publication bias applies also to laboratory animal research.
A Dutch team surveyed all animal laboratories in the Netherlands to assess the extent of publication bias in animal research. They found that only up to half of laboratory animal research is published:
Publication bias is an important problem in laboratory animal research (LAR) according to laboratory animal researchers. We estimate that only fifty percent of LAR is published, but it may be far less in for-profit organizations given that their employees estimated that only ten percent of LAR gets published overall, including their own. Lack of statistical significance, technical problems, the opinions of supervisors and peer reviewers were considered important drivers of non-publication. Respondents thought that mandatory publication of study protocols, research results or the reasons why results could not be obtained may accelerate scientific progress.
But what about the published research. Is it well conducted?
The scientists who have their research published often omit key details. A 2009 review of published research using animals found many flaws:
Only 59% of the studies stated the hypothesis or objective of the study and the number and characteristics of the animals used. Appropriate and efficient experimental design is a critical component of high-quality science. Most of the papers surveyed did not use randomisation (87%) or blinding (86%), to reduce bias in animal selection and outcome assessment. Only 70% of the publications that used statistical methods described their methods and presented the results with a measure of error or variability.
A more recent analysis of published papers observed that voluntary reporting guidelines for animal research that were introduced in 2010 have been widely ignored. Although more than 300 research journals have endorsed the ARRIVE guidelines, funding bodies, authors, referees and editors disregard them. The authors noted:
Our survey of the literature uncovers worrying inadequacies in the reporting of experimental design, selecting appropriate statistical analyses, and applying key points in the ARRIVE guidelines.
So, even if you believe that animal research is useful and necessary to help us find cures for human diseases, how confident can you be that the research is of good quality and the publicly available evidence is complete and not distorted?