Have a look at this graph and wonder how a group of experts can see there a convincing evidence of body fatness as a cause of cancer. (Taken from page 216 of the report on food and cancer mentioned in a previous post.)
Green dots and lines
The graph shows results of 41 studies on a link between body mass index (BMI) and pancreatic cancers. The square dots represent the relative risk average value of all cases analyzed in the study. The bigger the dot, the more cases. The horizontal lines represent the interval of 95 percent confidence. The shorter the line, the more reliable the values. As expected, the bigger the dots, the shorter the confidence lines in general.
The upper 21 are cohort studies, where body mass and cancer cases have been observed in a great number of persons over a long time. The lower twenty are case-control studies, where cancer cases and matched controls with the same characteristics but without cancer, have been compared. Such studies may be somewhat less reliable than cohort studies.
The upper diamond sums up the 21 cohort studies, its center is at a relative risk of 1.14 which means a relative risk increase of 14 percent. The width of the diamond represents the overall 95 percent confidence interval when all studies are taken together.
The lower diamond, summing up the twenty case-control studies, is somewhat wider than the upper one, telling us that these studies in general are somewhat less reliable. But this difference is not really big. The lower diamond is exactly at 1.00 which means that the risk did not change at all.
The results are not looking very consistent. Only one type of study is suggesting a link between BMI and pancreatic cancer, the other type does not show any such link. In both types of studies there are some suggesting even a risk reduction linked to a higher body mass - the dots to the left of the vertical line.
Yet the authors of the report conclude:
"There is ample epidemiological evidence, which is generally consistent, and there is a dose-response relationship. There is evidence for plausible mechanisms that operate in humans. The evidence that greater body fatness is a cause of pancreatic cancer is convincing."You may wonder if they have looked at the same data. Of course, a pink diamond that does not hit the vertical line means a statistically significant result. This is the case for the upper 21 studies, but the gap between the line and the diamond is only half as wide as the diamond itself. That is, the result is only weak.
In addition, it has already been magnified by a factor of more than hundred. In terms of snake oil tricks, this is selling molehills as mountains: The incidence of pancreatic cancer in the United States is 0.01 percent of the population every year, over a lifetime of 80 years we get a risk of 0.8 percent, and a relative increase of this risk by 14 percent translates to an absolute risk increase of 0.11 percent. The pink diamond, in reality, would be so small to just disappear behind the vertical line.
Of course, not all results of this report are so doubtful. But I have chosen this example to show how the authors deal with facts, and it leaves me with a lot of skepticism about the conclusions. One of my upcoming posts will deal with the question of cause.