Cherrypicking stats: bad form and not helpful

December 1st, 2009 by avatar

Science & Sensibility contributor, Andrea Lythgoe, has a great post up at her own blog. In The Doula Numbers Game, Andrea shows that many of us may be overestimating – and overstating – the beneficial effects of continuous support from doulas. She argues and I agree that using outdated statistics that yield “better” results could compromise our integrity. Moreover, doing so is not necessary to advocate for greater access to doulas.

Data from the Cochrane Systematic Review show more modest effects of doula support, but they still add up to “clinically significant” benefits, greater satisfacation, and no evidence of harm. Maternal-fetal medicine researchers who evaluated the evidence for a variety of obstetric interventions in the November 2008 issue of the American Journal of Obstetrics and Gynecology called doula support “one of the most effective interventions” (p. 446) for improving outcomes. And they did so without being wowed by the inflated early statistics. (They stuck to the Cochrane.)

It can be extremely difficult to look at research objectively. It is human nature to want to cherrypick the research that furthers our cause the most. We may try to find fault with statistics we don’t like and subconsciously ignore problems or limitations of statistics we do. But improving the safety and effectiveness of maternity care requires that we critically analyze the research, which means recognizing limitations and flaws in the studies we agree with and standing behind solid research even when we don’t like the conclusions. We need not worry. Even with a critical lens, research points to a need to radically reform our system to make it more mother-friendly.

Andrea finishes each post in her Understanding Research series with a familiar plea to practice, practice, practice finding and reading research literature. One of the skills we all should practice is to read the studies that seem to contradict our beliefs or biases. Often, these studies are flawed, and spending time reading them helps us hone our ability to spot methodological problems and logical inconsistencies in other research. Other times the research is valid, and we see circumstances where technology and medicine do in fact improve outcomes. Reading these studies can also shed light on important unanswered research questions.

I highly recommend that readers take a look at Andrea’s post for an example of thoughtful critical analysis of statistics on doula support in labor. It is hard to update our long-held beliefs or alter the ways we teach and practice. But this is just what we’re asking of our “medical model” counterparts. We should lead by example.

By the way, Andrea wrote her post for the upcoming Healthy Birth Blog Carnival.  There are so many other great contributions, too. I will have the Carnival up in the next 48 hours – promise!

Uncategorized , , ,

  1. avatar
    December 1st, 2009 at 14:46 | #1

    Thanks for this. It’s interesting, and it makes me feel like I can really trust this blog!

  2. December 1st, 2009 at 18:25 | #2

    Yet another fab post, Amy! I am happy to have found out about Andrea’s blog, too, and I’m going to forward her post to every doula I know. Thanks for keeping up the amazing work!

  3. avatar
    Judy Mercer
    December 2nd, 2009 at 07:34 | #3

    Good reminder Amy. Sometimes it is hard to be objective!

  4. December 2nd, 2009 at 16:07 | #4

    “We may try to find fault with statistics we don’t like and subconsciously ignore problems or limitations of statistics we do.”

    Excellent post.

    May I suggest two examples that need immediate correction?

    1. Infant mortality is not a measure of obstetric care; it is a measure of pediatric care. Therefore, it is misleading and disingenuous to quote infant mortality rates as an indictment of American obstetrics.

    The correct statistic is perinatal mortality. According to the World Health Organization, the US has one of the lowest perinatal mortality rates in the world, lower than Denmark, the UK and The Netherlands.

    2. The Johnson and Daviss BMJ 2005 study of homebirth does NOT show that homebirth is with a CPM is as safe as hospital birth. Indeed, it shows that homebirth with a CPM has nearly triple the neonatal mortality rate of moderate to low risk hospital birth.

    Johnson and Daviss withheld the scientific evidence about hospital neonatal death rates in 2000 because it made homebirth look so bad by comparison. Instead, they attempted to fool people by comparing homebirth in 2000 with a bunch of out of date papers extending back to 1969.

    A commitment to honestly appraising the scientific research about childbirth must include these facts.

  5. avatar
    December 2nd, 2009 at 16:36 | #5

    This is a great post, Amy. I’ll refer to it and refer it to others often. It should be required reading for everyone who has any interest in childbirth and maternity care. Now – to find a way for everyone to apply this (i.e. check their biases at the door before reading, commenting, judging, etc!!).

  6. avatar
    Jennifer V.
    December 2nd, 2009 at 21:40 | #6

    What is with the off-topic comment above that recommends correcting information not even in this post?

    Amy Romano, you are apparently now the reigning boss of All Childbirth Advocates, responsible for correcting every article and post on the internet so that it conforms with random bloggers’ standards. This post was about doulas and outdated research, but now you’ve been tasked with making corrections to things you didn’t even write about in this post.

    Chop chop.

  7. avatar
    Teri Shilling
    December 14th, 2009 at 20:23 | #7

    I was a little disappointed by this post…I am not sure of whom you are referring to as:
    “many of us may be overestimating – and overstating – the beneficial effects of continuous support from doulas.”

    As a DONA doula trainer I think most of us refer to the benefits as outlined in Hodnett’s et al meta-analysis of 15 trials from N. America, Europe, and Africa (10). (Described well in Andrea’s blog post)

    We talk about the HISTORICAL roots of the early studies in training but the accusation/assumption that doulas/childbirth educators were not up to date or quoting current research triggered a defensive cord for me. If readers walked away feeling that doulas aren’t as effective as they say they are, that saddens me. I hope they clicked and read Andrea’s entire post.

    I really think that most CBEs/doulas talk in general terms about the benefits – not quoting/promising certain %/outcomes if a woman in labor has continuous support.

    Just feeling a little defensive with the tone that doulas/cbes need to lead by example and “update our long held beliefs or alter the ways we teach.” The CBEs and doulas in my life certainly are committed to being evidence based and quickly incorporate new research!

    Thanks for your hard work in producing this blog. It certainly makes it easier for CBEs and doulas to keep up on the current research!

  1. No trackbacks yet.