Friday, 26 February 2016

Acupuncture for renal colic? How could such an awful paper get published?

The essential ingredients for conducting a randomized clinical trial are rather simple.

First you start with a research question that is hopefully feasible, interesting, novel, ethical and relevant. Then you come up with a hypothesis, null hypothesis and a primary outcome measure.

Next try to determine the smallest clinically meaningful difference that you think would be important to detect. Power your trial accordingly, get ethics approval and off you go!

Did this “RCT” do any of this?


Well, I guess they did do the “off you go” part.

This paper is so brilliantly awful that I will not discuss any of the results or conclusions as they are really irrelevant. However, I will focus on a few details.

This trial was conducted in three “clinics” in Turkey. Patients were evidently randomized (although there was zero mention of how they were randomized… not to be pedantic but this is important) to one of three arms; IM diclofenac, acupuncture, or intravenous acetaminophen (paracetamol). There was no primary outcome measure but they reported all kinds of measures over time on a VAS and VRS.

As mentioned previously, there was no power calculation performed but they managed to enroll about 40 patients in each arm. It doesn’t take a rocket scientist to conclude that they could only ever find very large differences with these small numbers (probably about 60mm on a VAS). In statistical speak, the possibility for type II error is huge.

Another question is ethics.

I do not believe that a Human Research Ethics Committee (HREC) in Australia would have approved the experimental design of this study. Before asking participants to take on some form of risk, pain or inconvenience, one needs to know if the study is designed to answer the research question. Otherwise it is a waste of time and exposes volunteers to unnecessary harm. This trial had no power calculation, no idea if it could answer any questions and as such could not be considered ethical.

Well done studies looking at acupuncture show that it is no better than placebo. (These are often conducted using sham acupuncture.) When we have an established treatment that is known to be effective, is it ethical to randomize a patient into a study arm that is placebo? For the methodology nerds, there was no clinical equipoise. This is mandatory before conducting an ethical randomized clinical trial.

There is so much more we could criticize about the methods, statistical analysis, bias’ and completely irrelevant discussion, but what’s the point.

Unfortunately a popular and generally well regarded educational program, EMRAP reviewed this paper and completely missed. It goes to show that even the best educational programs will get it wrong sometimes.

How did this paper get published? Not to be harsh, but this publication is not exactly a high impact journal. The process for peer review can be rather hit or miss in some of these fringe journals. In this case, it was a big miss.


Keynar M, Koyuncu F, Buldu I, et al. Comparison of the efficacy of diclofenac, acupuncture, and acetaminophen in the treatment of renal colic. Am J Emerg Med 2015;30:749-53.

No comments:

Post a Comment