By Ingrid Torjesen, BMJ News, 22 August 2018.
Over a hundred academics, patient groups, lawyers, and politicians have now signed an open letter to the Lancet calling on the journal to commission an independent reanalysis of the data from the PACE trial—a study into treatment for myalgic encephalomyelitis (ME)/chronic fatigue syndrome (CFS), which it published the results of in 2011.(1)
The PACE trial showed that adding cognitive behavioural therapy and graded exercise therapy to usual specialist medical care moderately improved outcomes for people with ME/CFS but that adding adaptive pacing therapy was of no benefit.
The open letter—the signatories of which include academics from UCL, the London School of Hygiene and Tropical Medicine, Harvard, Berkeley, and Stanford—points out that the trial had “major flaws” and “unacceptable methodological lapses.”
For example, 13% of the participants qualified at baseline as “recovered” or “within the normal range” for one of the study’s two primary measures (self-reported physical function) but were still considered to meet the CFS criteria to enter the study.(2)
A five year battle by Australian patient, Alem Matthees, succeeded in getting Queen Mary University of London to release the original trial data under the UK Freedom of Information Act.(3, 4)
- See MEA blog: The Times ‘Call for review of ‘flawed’ ME research in Lancet letter' which includes the open letter sent by Prof. Racaniello and David Tuller, signed by academics, researchers, clinicians, patient representatives, and politicians.
A preliminary reanalysis of that data (5) concluded that the previously reported recovery rates were inflated fourfold and that the recovery rates in the cognitive behavioural therapy and graded exercise therapy groups were not significantly higher than in the group that received specialist medical care alone.(6)
The £5m (€5.6m; $6.4m) publicly funded PACE trial has had widespread influence on research, treatments prescribed, and medical and public attitudes towards the illness. On its publication patients expressed anger because they thought that it suggested that ME/CFS was all in the mind and could be cured by cognitive behavioural therapy and exercise, when in fact some patients reported that such treatments caused them harm.
In 2016, Simon McGrath, who has a biochemistry degree from the University of Oxford and is unable to work because of ME/CFS, wrote a blog for The BMJ arguing that the PACE trial shows why patients need to scrutinise studies about their health.(7)
Academics are now voicing concerns about the trial, he wrote, but “for many years, researchers and the medical establishment would not engage with patients who made the same criticisms—simply because, it seems, they were patients.”
Ten members of UK parliament are among the politicians who signed the open letter, including Carol Monaghan (Glasgow North West), who predicted during a debate in February that “when the full details of the trial become known, it will be considered one of the biggest medical scandals of the 21st century.”(8)
The Lancet was approached for comment.
Several ‘Rapid Responses' to the above article have since appeared in the BMJ online. Here are a selection:
1. Please do not misunderstand patient views
By Andrew J Kewley, Student, South Australia.
This editorial stated “On its publication patients expressed anger because they thought that it suggested that ME/CFS was all in the mind”.
In anticipation of any future comments about this, I'd like to point out that this is a mischaraterisation of patient views. I have communicated with a quite a number of patients who are critical of the trial in question and none of them were angry because “it suggested thats ME/CFS was all in the mind”. I don't know where this view comes from, but it seems to be a misunderstanding of patient views.
Patients were angry that the results of the trial were excessively spun, and that much of the outcome measures specified in the protocol were not reported in the manuscript published in The Lancet. In particular, the thresholds for “positive outcome” and “recovery” were heavily watered down such that they were no longer meaningful.
One of the major criticisms by patients was that the “recovery” (remission) criteria as described in the protocol was not used and in its place, a highly questionable and misleading criteria for “normal” function was used instead. This criteria was used to imply recovery in one of the associated editorials published at the time of the original manuscript.
The criteria for “normal” functioning in the trial was defined as a SF-36 physical function score above 60. A SF-36 physical function score of 60 is less than the 15th percentile of the overall population of men/women that were of the median age enrolled in this study. The median and mode for mostly healthy western women of this age is somewhere around 95/100. Since patients were screened for other conditions that would affect their physical functioning, one would expect far higher scores to be considered recovered.
The most strange aspect however was the entry criteria required a score lower than 70/100. Which is to say according to the thresholds used in the PACE trial, you could simultaneously be considered ill enough to participate and well enough to be considered recovered with the same score of 60/100.
The reanalysis by Wilshire et al. analysed the data as close as possible to the original protocol, given the data they had access to. they discovered that on the contrary to claims made by the authors, there was no difference in recovery rates between the randomised groups.
This lack of recovery from such treatments is much more in line with patient community experiences with these treatments. Namely that they might be useful for a minority of patients to manage living with a chronic illness more effectively, but that they should not be considered primary treatments that lead to remission.
This example is potentially a textbook example of how pre-publishing clinical protocols is important and how deviation from that protocol is used in an attempt to exaggerate the efficacy of an intervention in a clinical trial.
2. Re: Pressure grows on Lancet to review “flawed” PACE trial
By Jonathan CW Edwards, Professor in Medicine, Emeritus, UCL, Rayne Institute, 5, University Street, London WC1
Kewley (previous comment) makes the key point. The reason why I have signed the letter to The Lancet is that the PACE trial is methodologically so poor as to be uninterpretable.
Patients may have expressed concern that the trial inappropriately reinforces psychological theories but this seems to have been used as a smoke screen by the PACE authors and associates to obscure the fact that patient scientists have identified the flawed nature of the trial.
I have no personal or professional interest in ME but have been greatly impressed by the ability of ME/CFS patients to initiate the sort of critical review that should have been performed by peers.
The reason why the poor quality of this trial is now so important is that this sort of poor methodology, based on unblinded trials with subjective outcome measures, is widespread in the field of therapist-delivered treatments.
A house of cards has been built that will ultimately collapse as it emerges that the same results can be obtained with any form of treatment that deliberately aims to influence patients' judgement of their health status.
By Michael Sharpe, Professor, PACE trial team, University of Oxford, University of Oxford Department of Psychiatry.
This ‘news item' refers to a three times recycled letter that has been repeatedly sent to the editor of the Lancet journal by campaigners, about a study that was published in 2011. [1]
The study was a clinical trial which found that cognitive behaviour and graded activity therapies in addition to medical care were better treatment for patients with chronic fatigue syndromes (CFS) than either a form of pacing therapy or medical care alone. However these findings were not well-received by some, including the main authors of the letter referred to. They have long campaigned to stop research into such treatments for people with ME/CFS and to get published studies, such as this one, retracted.
The trial referred to in the letter was subject to rigorous monitoring whilst being done and extensive peer review before being published. Furthermore, and unusually, because of the campaign against it, has been subject to a number of further reviews after being published, all of which have found its conclusions to be sound. Although it is the largest trial to test these therapies, it is not the only one; other studies have also found cognitive behaviour and graded activity therapies to help patients.
Science does not progress by campaigners trying to stop research being done or by suppressing its findings, simply because they are unwelcome. Science works best by testing ideas by doing experiments and then seeking either replication or refutation of the findings; the PACE trial findings have so far been replicated a number of times. We look forward to them being subject to further testing by other researchers in future.
Click for more rapid responses…