Skip Navigation LinksHome > Blogs > FRESH SCIENCE for Clinicians > A Cautionary Tale: Avoiding the Hype of an Observational Stu...
FRESH SCIENCE for Clinicians
News about basic science of interest and relevance for cancer clinicians
Tuesday, August 16, 2011
A Cautionary Tale: Avoiding the Hype of an Observational Study

 

African-American women who have two or more full-term pregnancies have a greater risk of developing hormone-receptor negative breast cancer, researchers reported in today’s issue of Cancer Epidemiology, Biomarkers & Prevention. Breast-feeding, however, may mitigate that risk, according to the authors.

 

A quick look at the abstract supports those conclusions. But upon closer reading that scary headline, which has already been picked up by numerous media outlets, doesn’t seem quite so justified.

 

For example, although the researchers, led by Julie Palmer, ScD, Senior Epidemiologist at the Boston University School of Public Health, report a statistically significant p-value for the trend of increased births and increased risk of hormone receptor negative breast cancer, the point estimates for comparisons between groups do not reach statistical significance. The hazard ratio for risk of hormone-receptor negative breast cancer is 1.48 for women who have had three or more children, compared with women who had none, but the 95% confidence interval spans 1.

 

In other words, that comparison does not reach statistical significance, even though it is a comparison of the most extreme groups, 0 versus. 3+ children.

 

In fact, as I read the paper, it seemed that very few of their point-value comparisons reached statistical significance, though the trends did.

 

Much Ado about Little

So were the conclusions supported by the data or should the community consider them more hypothesis-generating than conclusive? Do they warrant media coverage with headlines like “High cancer risk in black moms who don't breastfeed” or “Causes of high incidence of breast cancer in African-American women identified”?

 

Biostatisticians and cancer experts seem to be less convinced.

 

“The evidence is very weak to nil for all of the authors' conclusions,” said Donald Berry, PhD, Professor of Biostatistics at MD Anderson Cancer Center.

 

“I have a particularly strong reaction to this: ‘Impact: Increased breastfeeding may lead to a reduction in the incidence of this breast cancer subtype.’ The support for this statement is essentially non-existent,” Dr. Barry said. “The interactions across the hazard ratios don't come close to being statistically significant.

 

"And inferences about subsets of subsets are notoriously difficult. Finally, this ‘Impact’ statement is one of cause and effect, something their study cannot and does not address.”

 

Lisa Schwartz, MD, Professor of Medicine at Dartmouth Medical School, points out that readers need to be cautious about p-values for trends. “It is always much easier to get a low value for a p of trend, than for the other measures,” she said.

 

Moreover, when she used the authors data to calculate the absolute risks of developing hormone-receptor negative cancer -- which Dr. Palmer’s team did not do -- they are very low.

 

“The 12-year risk of having an ER-negative tumor is 0.32%, so three out of 1,000 women in this population developed ER- PR- tumors,” Dr. Schwartz said. “And the difference, if you apply their adjusted hazards, is the difference between 0.32% and 0.47%.”

 

In other words, the absolute difference is between 3 and 5 cases per 1,000 women over 12 years.

 

Dr. Schwartz notes that with such a rare event, it would be hard for the authors to get precise estimates of risk, even with their cohort of 59,000 women.

 

Additionally, she says, it’s also hard to know how much confounding is going on. The women who had more children also tended to have other risk factors, such as lower education, lower socioeconomic status, and higher weight.

 

“I know the authors adjusted for all of these things…but once you see confounding, you worry that there is more confounding that you didn’t measure,” Dr. Schwartz said.

 

“The other thing I am nervous about – breast feeding may be good for a lot of reasons, it’s not like they are proposing a dangerous drug – but that is not statistically significant either, when they test the interaction between the number of children and the tumors,” Dr. Schwartz continued. “They have a lot of very borderline results, that are not statistically significant, so I think you have to be more cautious. We are not sure this is not due to chance."

 

Her level of confidence in the results? “I think it is more in the hypothesis-generating area, than the let’s worry people range.”

For his part, Otis Brawley, MD, Chief Medical Officer of the American Cancer Society, said he thought the overall question about parity and hormone-receptor negative tumors was interesting, because full-term pregnancies are usually associated with a decreased risk of breast cancer. (In fact, Dr. Palmer and colleagues saw the expected decrease in risk of hormone-receptor positive tumors and full-term pregnancies.)

 

But despite the interesting question of relationship between hormone-receptor negative disease and pregnancy, Dr. Brawley is cautious. “I don’t think you can draw a firm conclusion from one study,” he said. “If I were a female I would not change my behavior because of this paper one bit.”

 

Finally, he said, that differences reported between racial groups almost always turn out to be the result of cultural or environmental differences, not inherent biological traits.

 

Authors More Cautious in Interview

So given all of these comments and anticipating widespread media coverage, I went back to Dr. Palmer and asked her how she thinks the community should view her data.

 

“I think the most conclusive part is the finding of increased risk of ER- cancers with two or more births,” Dr. Palmer said. “I would never say it is absolutely conclusive, but the reason I don’t mind people like you reporting on this is because we are not the first ones to find this. Because our findings are so close to those of the Carolina Breast Cancer Study, it makes you believe more in the causal relationship.”

 

She continued on to say she wanted to highlight these findings because “it really may mean something about mechanisms” of tumor formation and she wanted other researchers to know about the data. In fact, she’s been surprised that the previous studies went largely unnoticed, even by epidemiologists, she said.

 

As for a more general message, she expressed a mix of caution and reiterating the point her group made in the paper. “The only public health message we have here,” she said, “has to do with the breast feeding -- and our data are so weak that I don’t think we can put it out there -- which is that the increased risk we see with more births was really only there for women who hadn’t breast fed. If that’s true it would have great public health importance.”

 

Caution Needed

So why am I spending my time – and yours – on this study if there is no public health message and numerous experts are skeptical of its conclusions?

 

I guess it’s because I worry that we, as a community of scientists and observers of science, make too much of these non-causal relationships. We are too ready to say that one thing statistically associated with another is a biological driver.

 

I know caution doesn’t sell papers or draw in journal readers, but I, for one, would prefer much more subdued headlines and more cautious conclusions -- even in scientific papers.

3/5/2012
Ben Creelan said:
Without having read the article, this sounds like a classic example of 'data dredging'. Fishing for statistically significant comparisons which were not prospectively defined at the beginning of the study. Aside from the statistical problem of multiple comparisons, lack of biological plausibility, and possibility of confounders, there may be another explanation for these findings. It is reasonable to expect that multiparous females will have more interaction with the healthcare provider. Therefore, they will have the opportunity to be screened and diagnosed with more incidental breast lesions than the nulliparous population.
About the Author

Rabiya S. Tuma, PhD
RABIYA S. TUMA, PHD, a Contributing Writer for Oncology Times, is an award winning journalist and a regular contributor to The Economist, and the Journal of the National Cancer Institute. Her work has appeared in a variety of publications including CR Magazine, Yoga + Joyful Living, O The Oprah Magazine, HHMI Bulletin, and the New York Times. Prior to launching her writing career, Rabiya earned her doctorate at the University of Washington and Fred Hutchinson Cancer Research Center and worked at a biotechnology firm in Eugene, Oregon. And though she traded a lab bench for a computer, she remains fascinated with the work that takes basic science into the clinic.

Her OT blog was recognized this year by the American Society of Healthcare Publication Editors (ASHPE) with a bronze award in the category of Best Blog.