When the Best Evidence is not Available

Best practice relies on the best available evidence. There are comments throughout this site on what ‘best’ and ‘evidence’ means, but what about ‘available’? What if the best, most reliable and valid evidence has been conducted, but the results are not available to the researcher or practitioner? This could be the case if research has recently been carried out, but the results have not yet been reviewed and published. Another, more disconcerting reason for results being unavailable is that some results are simply never published. Various forms of bias exist in research evidence, and one of these forms is known as ‘publication bias’, the relatively low likelihood of negative results being published compared with positive results. This tendency to publish positive results has in fact led to a few journals being dedicated to negative results in an attempt to redress the balance, including the Journal of Negative Results, the Journal of Negative Results in Biomedicine  and – here's an interesting one – the Journal of Interesting Negative Results. Negative results are generally considered rather boring. They agree with the ‘null hypothesis’ that nothing has happened; the intervention had no effect. Positive results, on the other hand are much more exciting – lo and behold, the intervention worked and the world must be told! However, it really shouldn’t be like this. We should be hearing as much about findings that show the intervention does not work as those that show it does work; if the picture is skewed, we have an incorrect view of the effectiveness of interventions, and this is not good for health. It is also not good for health funding systems, because public and/or private funds may be over-spent on supplying interventions that are far less effective than we think.

A failure to publish results may also occur in soms cases due to conflicts of interest. A clear, very interesting and shocking example of this situation is hitting the news right now. You will no doubt be familiar with the drug Tamiflu. The story behind Tamiflu is a long one, but I will attempt to cram it into a nutshell (read more here). This drug was bought in large quantities worldwide (at a cost of almost US$7 billion) when the World Health Organisation declared an Influenza pandemic in 2009. However, evidence for the drug’s effectiveness had been provided by its manufacturer only in the form of unpublished data provided confidentially to only one research group, and had not been open to any form of external review. Years later, the data have still not been made available, but the good news is that this dreadful situation – a fortune spent by governments on a drug with no good evidence base – seems to have created shock waves that may result in real change. Ben Goldacre, a doctor, author of a web site and related books on misleading science, statistics and interpretations that impact negatively on health care, has joined forces with the British Medical Journal (BMJ) and others to found the AllTrials campaign, with the aim of ensuring that clinical trial data (past, present and future) are available, subjected to external scrutiny, and not locked away by those who stand to profit. If successful, this would mean that data from all clinical trials must be reported, and researchers would have access to the full range of results of those trials (not only on systemic drugs of course; on any health care interventions, including ocular therapeutics and vision therapies). Ultimately this would pave the way for systematic reviews and meta-analyses that include a full picture of the effectiveness of interventions, and for clinical decisions that are based on not only the evidence we are allowed to see, but all of the evidence.

Go to top