Well, that took a while. Five years after the Ars Chris Lee indicated That the authors of the Homeopathy paper did nothing but present “magic” as an interpretation of their findings, the editors of the journal in which it was published I retreated. Backtrack comes on Extensive objections Of the paper’s authors, who continued to believe that their work was solid. But in reality, the exchange between editors and authors stumbled into details missing the real problem in the original paper.
The work described in the now pulled paper included a small clinical trial of treating depression with three groups of participants. One group received standard treatment, while another group received a placebo. The third group received a homeopathic treatment – meaning they got hydrated. According to the analysis in the paper, the water was more effective than the placebo or the standard treatment. But as Chris pointed out in his original critique, the authors jumped to the conclusion that treating people with water must therefore be effective.
The problem with this is that it ignores some equally applicable explanations, such as statistical luck in a very small study (only about 45 people per group) or that it was time spent with a homeopathic practitioner that made the difference, not water. These are problems with interpreting the results, not with the data. (This probably explains why the paper was published by PLOS ONE, where reviewers are asked to simply consider the quality of the data rather than the relevance of the results.)
This does not mean that there were no potential data issues. According to the rollback notice, other researchers criticized certain aspects of the research, prompting the journal to convene a committee that included three editors, an outside academic, and a statistician. They considered one of the issues Chris noted – the inability to rule out a placebo effect from the homeopathy process. But they also looked at how the authors chose different “treatments” that included differences in the preparation of the water. There were also questions about how the participants were diagnosed in the first place.
The authors of the original paper were given the opportunity to respond, and they did. But the PLOS ONE panel found their response inadequate, which led to a reversal.
In response to their refusal, the authors say they provided more than enough information for anyone skilled in homeopathy to replicate the study – they all seem to know precisely how to prepare water based on a patient’s symptoms.
But their response also spoils the game. They wrote: “The PLOS ONE editors did not explain the ways in which they found our study design inappropriate.” Instead, they mentioned it simply because homeopathic remedies are included [different] The effectiveness of homeopathic medicines [aka water], Any positive effect seen should be a placebo. “For the authors, the design of their study was aimed at ruling out the placebo effect so as to show any difference in the effect of homeopathy. They are upset because the editors did not see things that way.
And that is the actual problem here, rather than specific complaints about the methodology. The control groups don’t tell you anything about the specific mechanism driving any changes in the experimental group. It only allows you to determine when experimental conditions lead to a different result. The reason This difference is a matter of interpretation, informed by what we know from other scientific studies. If you see a difference, then you need to think of all the scientifically plausible mechanisms to explain this.
Based on what we know from other work, the editors of PLOS ONE are correct in considering that “homeopathy generates a placebo effect stronger than the pill” is a plausible mechanism. And they are right that “water is behaving magically” is not plausible remotely. The authors of the second paper prefer it, so their work does not belong to a scientific journal.
The somewhat disappointing thing is that our editors don’t even bother to announce this. Instead, the rollback notice is largely focused on experimental details, as if the paper could have been fixed by detailing the materials and methods section. It presents a misleading picture of the issues here. While there is some solace in getting the right result – the paper has been officially pulled – it would be more helpful if the result came for the right reasons.