24h-payday

Straight to the source

September 15th, 2009 by avatar

Reports of new research studies can be found almost everywhere on a daily basis: On the nightly news, in mainstream magazines, in forwarded e-mails, on Facebook and Twitter feeds. But can these reports be taken at face value?

Let’s look at a few examples:

First a small blurb published in Reader’s Digest in February 2007:

Fresh flowers can cheer up any drab room — not to mention score points with your sweetie. But make us kinder as well? So says Harvard psychologist Nancy Etcoff, PhD.

She sent 54 people either a mixed bouquet or a candle in a hurricane glass. Flower recipients said they felt more compassion toward others than those who got the candles, and reported more enthusiasm at work too. Place blooms in the kitchen or bedroom so you’ll see them first thing in the morning. That’s when moods tend to be lower and blossoms can provide their biggest boost.

This sounds like a very official, authoritative study. It comes from a “Harvard psychologist” and even details a little about the methodology. However, I searched pretty extensively and was not able to find that it had been published in any academic journal. The researcher, who is an instructor teaching one class at Harvard, does not even list it as part of her published research on her Harvard web page. The study is, however, extensively touted on the web page of the Flower Promotion Organization and the Society of American Florists, which includes as part of the study “documentation” an article by an interior designer on where to best place flowers in your home. To me, this brings the study into some question. Who paid for the study? Was it commissioned by one of the organizations that exist to convince people to buy flowers? If so, there is a possibility of a bias in the methodology or conclusions. It’s difficult to evaluate a study that has not been published. Why was it not published? I am left with more questions than answers, and I’d hesitate to place too much faith in this study.

As I suspect in this case, all too often media reports of studies are based on a press release alone, and the reporters never see the real study. Press releases are written in a way to make headlines, not necessarily for complete accuracy, and reporters under a deadline don’t always do much more than use the facts in the press release.

Another example, taken from my local newspaper:

SLTribune4-19-09

Wow, pretty interesting findings, aren’t they? Well, tracking down the actual study turned out to be difficult (you can read about it here) but when I did, it turned out to be a study on rats. Yes, rats. The newspaper article, with its references to “you” and “mom”, implied that the research was done with humans, but in fact it was animal research. Results from animal research may or may not be applicable to humans.

In another article from Reader’s Digest, the author lists a bunch of research findings about the potential harms of vitamins, among them this claim:

Researchers at the National Cancer Institute (NCI) found that men who took more than one multivitamin daily had a higher risk of prostate cancer.

However, the author oversimplified the NCI report in a way that can be easily misunderstood. To quote from the NCI report:

We found that multivitamin use was unrelated to overall risk of total and organ-confined prostate cancer. (p. 761)

The possibility that men taking high levels of multivitamins along with other supplements have increased risk of advanced and fatal prostate cancers is of concern and merits further evaluation. (p. 754)

A subtle difference, true, but also an important one. Multivitamins did not affect the overall rate of prostate cancer, or of the subgroup of cancers that have not spread. High levels of vitamin use (more than 7 multivitamins in a week, combined with other supplements) may have had an effect on the more rare, advanced forms of cancer.

That word “possibility” is very important. It signals that there is a gray area here where something has not been proven or disproven, but could go either way. It may be the vitamins and supplements contribute to the aggressiveness of cancers, but it also may very well be that they don’t. That’s why the authors recommended further research. When you read the word “possibility” think of it as a hypothesis – a guess – something that should be researched further.

In this study, researchers had a fairly small number of people with advanced cancer – only about 1/10 the numbers they had for localized cancer. This makes it hard to draw strong conclusions about the advanced cancers. Yet those conclusions are the very thing that the popular media article focused on!

These kinds of issues are fairly common in the popular media. Remember the writers and reporters at the various magazines, newspapers or web sites aimed at the general public are likely NOT trained in reading and understanding research. With this lack of knowledge, they are likely to make easy mistakes, like broadly generalizing and not picking up on subtleties like the prostate cancer article.

With a little education and a careful reading of research, you can skip the mainstream media, learn to read the actual studies yourself, and avoid pitfalls like these. Next time, I’ll discuss how you can find the studies to read them for yourself.

Uncategorized , ,

  1. avatar
    Tim
    September 15th, 2009 at 19:52 | #1

    Reminds me of when the U of U announced that they had achieved cold fusion.

  2. avatar
    NursingBirth
    September 15th, 2009 at 20:23 | #2

    GREAT post!! Even if I hear about a study in a newspaper or magazine I always look up the original study on the journal’s website to AT LEAST read the abstract myself! If it’s worth reading I then try to get the full article off interlibrary loan. IT is SO important to get your information from the source and not from someone else’s interpretation! Thank you for writing about this!

    ~Melissa
    http://www.nursingbirth.com

  3. September 19th, 2009 at 07:19 | #3

    Media is especially guilty of reporting “food news” this way. I recently attended a conference where Herald McGee, who writes for the NY Times, shared with the audience how he first got published in Science (egg white stabilization w/ copper), and then later discovering that his original hypothesis/conclusions were flawed–and having his requests for a change ignored. Nutrition news is perhaps the worst–for example, a recent presentation at the ACS conference about antioxidant levels in cereals got translated into “popcorn has high levels of antioxidants” on sites such as MSNBC, yet the paper has not yet been peer reviewed etc. As NursingBirth says, go to the source. Thanks for bringing this issue up, it’s a good reminder to us all.

  1. December 5th, 2009 at 19:11 | #1