Monday, November 5, 2007

Unusual job for a quackbuster

snake oil
Looking for snake oil tricks in a big food and cancer report with a respectable multi authorship.

Last week I have come across this new report on food and cancer and its big media echo. The media spread messages like you should be as slim as possible in order to prevent cancer. Some days ago I had commented on a study finding only a very doubtful link between obesity and breast cancer in women. Obesity, let it be understood, is a lot more than just overweight, and this in turn is far away from "as slim as possible".

All of a sudden, I had a sense of déjà vu, and a scene from a western movie came to my mind: A snake oil peddler in front of a crowd, boasting with exaggerated claims about the benefits of his product. Scientifically proven, of course.

No one would expect to find the big five tricks, as may be described in the snake oil chapter of "Peddler's Handbook", in a serious modern cancer report with a multi-professor authorship. But I couldn't resist having a look for these tricks, and here is what I have found.

1. The China Trick
Make sure to label your product "Genuine Chinese Snake Oil", based upon a medicine that is thousands of years old, and therefore it must help against pain.
The names of organizations and persons behind the food and cancer report are really impressive, and it seems that for most media this alone has been sufficient to take the claims of the report for granted.

2. The Mrs Jones Trick
Present Mrs Jones to the crowd and have her confirm that she applied snake oil and later her pain disappeared. Make sure to have a Mrs Jones in every town.
This is one of the most effective tricks because people have a strong urge to see causes behind things. Thus, most people will believe that the pain of Mrs Jones disappeared because she had applied snake oil. But rheumatic pain comes and goes in phases anyway, and it is just a matter of chance that an application of snake oil is followed by a pain remission.
In the case of the food and cancer report, most data are just correlations of the Mrs Jones type, that is, consumption of certain foods or certain body mass indexes as well as certain cancers occur in the same population groups. Yet the authors of the report claim they have found a cause - more about the details in an upcoming post.

3. The Big List Trick
If people tell you that Mrs Jones is a single case which does not convince them, show them a long list of patients also successfully treated with your snake oil.
The food and cancer report takes together results of many studies of very different quality and size, claiming that the sheer number of studies makes the results more reliable.

4. The Molehill Mountain Trick
Look closely at any small effect and it will grow big. Sell mountains, not molehills.
Most of the data presented in the food and cancer report are presented in the form of relative risk differences, that is, as (bigger) percentage of a (smaller) percentage. By this method, small effects may be magnified by a factor of twenty, fifty, or more.

5. The Crowd Trick
Selling to a crowd is easier than selling at the door. Buying is contagious.
The big media coverage is just one side of this trick but not the only one. It may also have worked in the making of the report because most experts, for instance, share the view that body weight above a certain level is a cause of illness, if not an illness by itself. When such hypotheses are taken for granted, interpretation of data will be biased in some way.

Conclusion: As hard as it may be to believe, but even in this report, based on a huge number of scientific studies, authored by highly respected scientists, some old tricks of snake oil peddlers can be found. Thus, the claims of this report must be viewed with a skeptic eye. Tomorrow we are going to look at some details. Stay tuned.

Photo credit:

No comments: