ATTUD Journal Club Postings

How Clinical Trial Statistics Can be Misleading

A recent review paper (Rosen et al, Addiction, in press) quantified how some cessation statistics can be misleading. Lets see if I can summarize that without too much jargon.

The most commonly reported effectiveness statistics are the Odds Ratio (OR) and Relative Risk (RR). They are usually very similar and the RR is easier to understand. If the quit rate in the active group is 40% and in the control group is 20%, the RR is 2. It is important to remember the OR and RR are measures of the relative, not the absolute, increase in quitting. This is especially important to remember when examining long-term outcomes. For example assume that at the end of treatment, the quit rates are 40% and 20%. Now, assume 3 months later then are 20% and 10% and then even later they are 10% and 5%. The RR for all of these is 2. But the absolute difference in quit rates declines from 20% to 10% to 5%. This difference is call the Risk Difference (RD).

Many clinicians find the most useful statistic is the derivative of the RD, called the Number Needed to Treat (NNT) – sorry for all the jargon just bear with me. The NNT is the inverse of the RD and represents the number of smokers one would have to treat to get one extra long term abstinence. So the NNT in the above scenario increases from 5 smokers (when RD is 20%) to 10 smokers (when RD is 10%) to 20 smokers (when RD is 5%).

The Rosen article took data from the Cochrane reviews of medications for smoking cessation and from their data one can find the actual NNT at the end of treatment is 6, at 6 months was 8 and at 12 months was 12. So even though the commonly reported RR or OR does not decline, the actual benefit of treatment does decline.

So how good is a NNT of 6-12. There is a NNT Database. My reading is that these are very good NNTs. Very few treatments are less than 10 – mostly for antibiotic treatments For example, for most cardiovascular interventions like stents or aspirin the NNTs are over 100. And some, like aspirin to prevent heart attacks, are in the thousands.

Half of continuing smokers will die from smoking, thus, the NNT,not for cessation, but for mortality, would at the worst be 16 not 8. So, would you be willing to give 16 kids a vaccine if you knew it would save one kid’s life? Okay, would you be willing to treat 16 smokers to save one early death?