ME Association Press Release, 21st March 2018
Benefits reported in a controversial medical trial part-funded by the Department of Work of Pensions were “not reliable,” a major study has found.
A large-scale, government-funded trial, known as PACE, claimed psychotherapy and exercise helped the estimated 250,000 sufferers of the devastating illness, M.E. (myalgic encephalomyelitis).
Manifesting as unrelenting fatigue and profound pain, the condition, also known as chronic fatigue syndrome, has no known cure and is made worse by exertion.
Sufferers are often confined to their beds, unable to walk, and need help even to shower – an action that could then lay them low for hours, days, weeks or longer.
When the results of the five-year PACE trial were published in 2011, researchers claimed that graded exercise therapy (GET) and cognitive behaviour therapy (CBT) were “moderately effective” forms of treatment.
The trial concluded that both treatments led to recovery in over a fifth of patients.
But PACE has since faced intense criticism from other researchers, clinicians, patients and charities, such as the ME Association, over how the results were obtained, analysed and presented.
Parliament has previously heard claims that the data was deliberately flawed to “remove people from long-term benefits and reduce the welfare bill”.
After a long legal battle, unpublished data from the trial was released and has now been independently reanalysed. The paper, published in the journal BMC Psychology, has found that the benefits reported for psychotherapy and exercise therapy are modest and not statistically reliable.
Lead author Carolyn Wilshire, said:
“Our reanalysis was designed to explore how the PACE trial outcomes would have looked if the investigators had adhered to the primary outcome they described in their original published protocol.
“We also looked into the published data on long-term outcomes to examine whether they had been influenced by the treatments patients had received after the trial had ended.
“We found that the groups receiving CBT or GET did not significantly outperform the control group after correcting for the number of comparisons specified in the trial protocol. Rates of recovery were consistently low and not significantly different across treatment groups.”
In surveys carried out by the ME Association, more than half of patients who had followed the recommended graded exercise programme saw a worsening in their symptoms.
Dr Charles Shepherd, Honorary Medical Adviser to the ME Association, today said:
“The ME Association has always been very critical of the way in which the PACE trial was designed, especially the lack of any objective outcome measures.
“And we have not been impressed by the way in which the results have been reported in medical journals, especially claims relating to recovery following CBT and GET.
“So, it comes as no surprise to find that a very careful re-analysis of some of the PACE trial data by Carolyn Wilshire and colleagues has concluded that impressive claims for recovery following CBT and GET are not statistically reliable.
“It is also very concerning to note that this data was only released through use of the Freedom of Information Act and a very costly Tribunal – which ordered the release of data.
“This sends a powerful message to the research community that they must be willing to share data where there are serious concerns about protocols or the reliability of results from a clinical trial.
“The ME Association believes that it is very important to encourage research data sharing and, where appropriate, independent reanalysis – which is why we made a significant financial contribution towards the processing fee for publication of this paper.
“The message is clear – CBT and GET are not effective ways of treating a serious neuroimmune disease. The sooner this message gets across to health professionals the better.”
The PACE trial data was used justify NHS recommendations of exercise and cognitive behaviour therapy and no changes were made as a result.
But a patient revolt has forced the government and NICE (the National Institute of Clinical Excellence) to review the guidelines used by UK doctors. That review may not be completed before 2020.
Rethinking the treatment of chronic fatigue syndrome—a reanalysis and evaluation of findings from a recent major trial of graded exercise and CBT
Authors:
Carolyn E. Wilshire, Tom Kindlon, Robert Courtney, Alem Matthees, David Tuller, Keith Geraghty and Bruce Levin.
Abstract:
Background
The PACE trial was a well-powered randomised trial designed to examine the efficacy of graded exercise therapy (GET) and cognitive behavioural therapy (CBT) for chronic fatigue syndrome. Reports concluded that both treatments were moderately effective, each leading to recovery in over a fifth of patients. However, the reported analyses did not consistently follow the procedures set out in the published protocol, and it is unclear whether the conclusions are fully justified by the evidence.
Methods
Here, we present results based on the original protocol-specified procedures. Data from a recent Freedom of Information request enabled us to closely approximate these procedures. We also evaluate the conclusions from the trial as a whole.
Results
On the original protocol-specified primary outcome measure – overall improvement rates – there was a significant effect of treatment group. However, the groups receiving CBT or GET did not significantly outperform the Control group after correcting for the number of comparisons specified in the trial protocol. Also, rates of recovery were consistently low and not significantly different across treatment groups. Finally, on secondary measures, significant effects were almost entirely confined to self-report measures. These effects did not endure beyond two years.
Conclusions
These findings raise serious concerns about the robustness of the claims made about the efficacy of CBT and GET. The modest treatment effects obtained on self-report measures in the PACE trial do not exceed what could be reasonably accounted for by participant reporting biases.
Media Coverage
The Times: Findings of £5m ME chronic fatigue study ‘worthless’
By Tom Whipple, Science Editor, 22nd March, 2018.
A landmark £5 million study that formed the basis for NHS treatment of ME sufferers has been heavily criticised for its “troubling” use of data, in new research that claims its recommendations were largely worthless.
Since 2011 the estimated 250,000 Britons with ME, also known as chronic fatigue syndrome, have been offered treatment based on exercise and cognitive behavioural therapy, after a study known as the Pace trial was published in The Lancet and seemed to show this approach improved symptoms.
Now, after a freedom of information request forced the scientists behind the 2011 study to release their raw data, other researchers have performed their own analysis. They said their study, published in the journal BMC Psychology, raised “serious concerns about the robustness of the claims” made about exercise and therapy.
Keith Geraghty, from the University of Manchester, said at the very least the £5 million trial represented wasted resources. “I would argue it could have been spent more wisely,” he added.
The authors of the original Pace trial stood by their findings.
BBC News: Chronic fatigue trial results ‘not robust', new study says
BBC Health, 22nd March, 2018
Fresh analysis of a controversial study, which recommended exercise and psychological therapy for people with chronic fatigue syndrome, suggests their impact is more modest than first thought.
The PACE trial found the treatments to be “moderately effective”, leading to recovery in a fifth of patients. But this new analysis finds “no long-term benefits at all”.
The authors of the original trial in 2007 said they stood by their findings.
That randomised trial was designed to examine the effectiveness of graded exercise therapy (GET) and cognitive behavioural therapy (CBT) for chronic fatigue syndrome (CFS), also known as myalgic encephalomyelitis, or ME.
Its findings were positive, but patient groups like the ME Association have always been critical of the way the trial was designed and the way the results were reported.
Goalposts ‘moved'
There has also been controversy over the release of data from the trial, with some arguing it should be made available to all researchers for further analysis.
The PACE trial reported that 59% of patients who received CBT and 61% who had exercise therapy had improved overall, compared with 45% in a control group.
When the results were re-examined, after data was obtained under a Freedom of Information request, researchers found that just 20% of CBT patients and 21% of GET patients improved, along with 10% of control patients.
Figures for those who recovered were originally reported as 22% for patients in each of the CBT and GET groups, but this reduced to 8% in the latest re-analysis.
Writing in the journal BMC Psychology, lead author Dr Carolyn Wilshire, from the University of Wellington in New Zealand, said the PACE trial moved the goalposts by changing the way treatment success was measured after the trial had begun.
She added: “Until there is positive evidence to suggest otherwise, the conclusion we must draw is that PACE's treatment effects are not sustained over the long term, not even on self-report measures.
“CBT and GET have no long-term benefits at all. Patients do just as well with good basic medical care.”
Modestly effective treatment
The ME Association, which part-funded the new study, said it was no surprise that “impressive claims for recovery following CBT and GET are not statistically reliable”.
Dr Jon Stone, consultant neurologist at the Western General Hospital in Edinburgh, said better treatments for chronic fatigue syndrome were needed, or more effective forms of rehabilitation.
“Until we have these, the question is whether it is better to offer a modestly effective treatment supported by data from many other trials, with a realistic discussion of its pros and cons, than none at all.”
The three authors of the original PACE trial – Prof Michael Sharpe, from the University of Oxford, and Prof Trudie Chalder and Dr Kimberley Goldsmith, from King's College London, said the new analysis had used only part of the data from the trial.
They also said many other trials and meta-analyses had replicated the findings of the PACE trial.
“In conclusion, we find little of substance in this critique and stand by our original reports.”
NICE is currently updating its guidance on the diagnosis and management of chronic fatigue syndrome.
Other press reports
https://www.huffingtonpost.co.uk/20…ic-fatigue-study…
https://www.belfasttelegraph.co.uk/…onic-fatigue-study…
http://www.malverngazette.co.uk/new…_of_chronic…/
https://www.shropshirestar.com/news…ndings-of-chronic…/
http://www.worcesternews.co.uk/uk_n…_of_chronic…/
http://www.thenorthernecho.co.uk/uk…_of_chronic…/
http://www.oxfordmail.co.uk/…/na…_of_chronic_fatigue…/
http://www.dailyecho.co.uk/…/nat…_of_chronic_fatigue…/
http://www.bournemouthecho.co.uk/ne…_of_chronic…/
http://www.swindonadvertiser.co.uk/…_of_chronic…/
http://www.thetelegraphandargus.co…._of_chronic_fatigue_study____not_reliable___/
http://www.southwalesargus.co.uk/ne…_of_chronic…/
http://www.theboltonnews.co.uk/news…_of_chronic…/
Findings of chronic fatigue study ‘not reliable' | County Times
Findings of chronic fatigue study ‘not reliable' | The Argus
Findings of chronic fatigue study ‘not reliable' | The Wiltshire Gazette …
Findings of chronic fatigue study ‘not reliable' | Lancashire Telegraph
http://www.leaderlive.co.uk/…/na…_of_chronic_fatigue…/
Boots: CBT for ME/CFS study questioned
https://www.webmd.boots.com/…/cbt-for-me-cfs-questioned
Science Media Centre
- CFS/ME The Illness and the controversy – 21st March 2018.
- Before the headlines analysis – Reanalysis of the PACE trial – 22nd March 2018
- Expert reaction to reanalysis of the PACE trial for chronic fatigue syndrome (CFS) treatments – 22nd March 2018
The BBC article: “The three authors of the original PACE trial – Prof Michael Sharpe, from the University of Oxford, and Prof Trudie Chalder and Dr Kimberley Goldsmith, from King’s College London, said the new analysis had used only part of the data from the trial.”
What a pathetic and disingenuous objection, after the PACE authors spent years fighting FOI requests for the data! Now they winge about the critique on the basis that the critiquers don’t have all the data!
As for John Stone, quoted in the BBC article as saying that CBT are GET are “modestly effective”, why does he claim that on the basis of a trial that didn’t control for placebo and therapist effects and was riddled with many other outstandingly bad flaws?
Is Sharpe, Chalder, White and Goldsmith really do have data that can rescue the PACE trial then let’s see it! Publish it and prove your case, Professor Sharpe. It is of course a daft thing to claim because if the PACE authors really did have such data they would have published it years ago.
What is becoming clear is that Sharpe et al are making absolute fools of themselves. That is on their heads; the problem is the damage they have done to extremely ill people. Nothing can undo that now because the damage has been done. The best thing Sharpe et al can do is to repudiate the PACE trial, see sense and admit reality.
Thanks for this. Is it too late to edit to include fellow researchers, ME expert clinicians, statisticians and journalists in this part “But PACE has since faced intense criticism from patients and charities, such as the ME Association, over how the results were obtained, analysed and presented.” It is not just patients who criticize PACE and similar papers.
Hi,
It is a good point and one that slipped past us. I have amended the sentence above to properly reflect that criticism of PACE comes from other researchers, clinicians, patients and charities.
Many thanks.