The SMILE trial was an attempt to determine the efficacy of the Lightning Process® when delivered in addition to specialist medical care in the treatment of ME/CFS for children and adolescents.
Rather than attempt to place all of our review on this blog, we have made it available to view online or download as a pdf. file.
The full review explains why we have reached the conclusions we have and we would recommend you try and read it if you are able.
Some extracts from the full review follow, beginning with a summary statement from Dr Charles Shepherd.
Dr Charles Shepherd, Hon. Medical Adviser, ME Association:
“The SMILE trial is one of the worst examples of a clinical trial supposedly designed to assess the acceptability, effectiveness and safety of a treatment for ME/CFS that I have come across. In fact, in several ways it is a lesson in how not to conduct a clinical trial in people who have ME/CFS.
“There was no adequate control group, no attempt to properly measure the effectiveness of the Lightning Process® as a stand-alone intervention, and no mention of the likely placebo effect in an unblinded trial that involved comparing treatment A (i.e. specialist medical care) with treatment A + treatment B (i.e. specialist medical care plus the Lightning Process®).
“There was a serious lack of objective outcome measures (especially in relation to measuring physical activity levels), no explanation of the likely impact on self-report outcome questionnaires (where those receiving the Lightning Process® may well have been aware of the alleged benefits), and the specialist medical care ‘control' seemed to involve a ‘pick and mix' approach that was not at all standardised.
“It is very hard to understand why the Science Media Centre went all-out in their attempt to promote this trial. We were not therefore surprised to read of the negative scientific reaction to their coverage of it – or that one expert from their panel compared neurolinguistic programming (a key component of the Lightning Process) to ‘pseudoscience’. However, we were surprised by the way in which the British and overseas media accepted the findings without question.
“The ME Association stands by its initial statement on the SMILE trial and does not recommend the Lightning Process® for people with ME/CFS. We have referred several Lightning Process® practitioners to the Advertising Standards Authority where therapeutic claims have been made that cannot be supported. We will continue to do so, if necessary.
“We welcome sound research into treatments that could produce better outcomes for children and adolescents with ME/CFS and any applications in this area made to the MEA Ramsay Research Fund.”
Science media centre: Inconvenient Truths
Science media centre: Prof. Dorothy Bishop
Media reaction to SMILE trial: The SMILE trial is published
ME Association statement: Lightning Process and SMILE trial in young people with ME/CFS
Overview of the study
You can read the full SMILE trial – which is open access – here.
And the previously published feasibility study, here.
The SMILE Trial randomly assigned 100 children aged 12-18 with mild/moderate chronic fatigue syndrome (CFS) to either a specialist medical care group (SMC-only), which served as the ‘control’ group, or a specialist medical care and Lightning Process® group (SMC+LP).
‘49 children were allocated to SMC-only, and 51 to SMC+LP. Participants’ mean age was 14 years, 76 were female and all described themselves as British. Participants were disabled by their fatigue: only seven were attending full-time school and 47 described themselves as attending 2 days or less school a week.’
The primary outcome measure of the trial was the SF-36 physical function questionnaire, which was completed before treatment and then at 6 and 12 months after treatment.
Secondary outcome measures included the Chalder fatigue score, the Hospital Anxiety and Depression scale (HADS), the Spence Children’s Anxiety Scale (SCAS), the visual analogue pain scale (VAS), and self-reported school attendance, measured as the number of days attended in the previous week.
Results revealed that both treatment arms improved physical function and decreased fatigue and anxiety, but that the SMC+LP group showed a greater improvement overall than the SMC-only group.
The trial also looked at the cost-effectiveness of SMC+LP by using quality-adjusted life years (QALYs), derived from the EQ-5D-Y questionnaires, but it did not appear to be more cost-effective than SMC-only, due in large part to the initial high cost of the Lightning Process® treatment, despite the concluding remarks.
The authors concluded:
‘The LP is effective and is probably cost-effective when provided in addition to SMC for mild/moderately affected adolescents with CFS/ME.’
The full review of the SMILE Trial can be downloaded or read online, as a pdf. file version. You may need to download Adobe Acrobat reader if you don’t already have it installed.
Study design
One of the biggest flaws in the study design was that it was not blinded, meaning both the participants and the providers knew which treatment they were receiving, and this could have led to an over-estimate of outcome effectiveness.
It left room for bias as the control group (SMC-only) may have been disappointed with their eventual treatment allocation and the SMC+LP group may have been exposed to positive expectations.
Can such treatment trials be blinded? On the one hand, the SMILE trial randomised assessment of trial data so the assessors could not determine who received what treatment.
But on the other, SMILE did not adopt a ‘sham’ treatment arm either of CBT or LP to try and determine if participant outcomes are the result of any ‘active’ therapy application.
Such an approach could have been extremely useful and quite innovative as psychotherapy trials in ME/CFS, and more generally, appear reluctant to test the impact of placebo.
Other concerns:
- In the feasibility study parental feedback suggested the benefits of the LP were emphasized, and most of the information given was about the LP, and that there was some disappointment when finally allocated to the SMC group.
This could have influenced subsequent outcomes although we don’t really know who completed the self-report outcome measures or the degree to which parents might be said to have influenced outcome measures.
- Three of the participants allocated to the SMC-only arm went on to seek private LP.
This may confirm that they only entered the study in the hopes of getting LP for free or could reflect how well LP was portrayed in the initial information given pre-randomisation.
It is not clear whether the data from these 3 participants was then used in the SMC or SMC+LP data at the end, but it could have affected the results.
- According to the diagram of the two study arms (Figure 1):
- 9 of the 51 participants allocated to the SMC+LP arm appear to have received SMC-only
- 39 of the 51 received the full LP course, yet there appears to be 44 participants included in the primary analysis
The numbers here do not seem to add up and it is not clear whether SMC-only participants were included in the SMC+LP data analysis; which would be contamination of the treatment group, affecting the validity of the results.
- The authors said that, ‘three participants in the SMC+LP arm received the LP course after completing the 6-month follow-up’.
This would have affected the results if they were included in the data analysis as the 12-month results would in fact be less than 6 months after LP treatment.
These areas of the study, largely confused by Figure 1 (below), are not very transparent so it is difficult to determine which participants were included in the final results.
- The authors said that, ‘Five participants withdrew from the study: two from the SMC-only and three from the SMC+LP arm’.
There are no reasons given that might explain why this was the case and yet it would have been useful qualitative data to collect and might have given us a better idea of the acceptability of the treatment amongst participants.
The full review of the SMILE Trial can be downloaded or read online, as a pdf. file version. You may need to download Adobe Acrobat reader if you don’t already have it installed.
Please help us continue our work
If you have found this review useful, then please donate – whatever you can afford – to help us continue with our work to make the UK a better place for people with M.E. Just click the button below to visit our JustGiving page:
Or why not join the ME Association as a member and become a part of our growing community? For a monthly (or annual) payment you will not only be helping to keep us doing what we do best, but will receive our exclusive ME Essential magazine.