Archive

Posts Tagged ‘meta-analysis’

Planned home birth and neonatal death: Who do we believe?

August 17th, 2010 by Amy Romano Amy Romano

The (in)famous Wax home birth meta-analysis hit the scene over a month ago. But the buzz doesn’t seem to be dying down. In the weeks since the original pre-publication and press release, editors at The Lancet and BMJ have both weighed in, and there’s a steady stream of media attention. While all of the media have dutifully quoted midwives in leadership positions saying the meta-analysis is flawed (an assessment with which I agree), I still keep coming back to the question I asked in my earlier postdid we need a meta-analysis to establish the neonatal outcomes of planned home birth? We had, after all, a very large, methodologically rigorous study on home birth safety involving over a half million women that was published less than 2 years ago. Won’t that suffice?

I had a chance to interview two of the researchers who conducted that study when I was in Vancouver for the Normal Labour & Birth International Research Conference. Simone Buitendijk, MD, is Professor of Maternal and Child Health and Midwifery Studies at the Academic Medical Center of Amsterstam and heads up the Child Health Programme at the Netherlands Organisation for Applied Scientific Research. Ank de Jonge, the study’s lead author, is a practicing midwife with a PhD in public health who works at the Midwifery Science section within the EMGO Institute for Health and Care Research at VU University Medical Center in Amsterdam. I gained some new insights from them about their research and the Wax meta-analysis. Based on those interviews, and despite having written about the meta-analysis twice already, I thought it was time to ask anew: which is the “better” evidence for determining neonatal outcomes of planned home birth: the de Jonge cohort study or the Wax meta-analysis? Let’s have a look at some objective criteria and see how each study measures up.

Study size (home birth group):

  • Wax: 9,811
  • de Jonge: 321,307

That’s right, the Dutch neonatal mortality analysis is 33 times the size of the neonatal mortality meta-analysis. And believe it or not, this was BRAND NEW news to me that I didn’t realize until I spoke to de Jonge and Buitendijk. Although I had access to the full-text of the Wax meta-analysis and in fact looked critically at it (heck, I blogged about it!), I completely missed the fact that while the de Jonge study was “included” in the meta-analysis, it was excluded from the analysis of neonatal mortality, which was the major finding given so much attention by the media.  On the one hand, I’m pretty embarrassed to have made such a major error. On the other hand, it just underscores how misleading it can be for professionals or lay people to read headlines about a meta-analysis of “hundreds of thousands” of births finding triple the neonatal death rate.  Wax’s neonatal death data don’t come from hundreds of thousands of births at all. Not by a long shot.

Mechanism to ensure data were from planned home births:

  • Wax: mechanism varies across the included studies. In Pang et al., which contributed 63% of the home birth data and accounted for 12 of the 18 neonatal deaths in normally formed newborns, researchers relied on birth certificate data that did not differentiate between planned and unplanned home births, and assumed that any birth certificate for a baby born at home at or beyond 34 weeks, signed by a midwife, nurse, or doctor was a planned home birth, a method that has not been scientifically validated and has been widely criticized. Unplanned home births are riskier than planned home births with qualified attendants.
  • de Jonge: midwives routinely record the planned place of birth in a national perinatal database that covers 99% of births and is linked to another database of neonatal deaths by a validated method. Planned place of birth was unknown for 8.5% of the population, and the outcomes of this group were analyzed separately and reported.

Definition of neonatal death:

  • Wax: death of a live-born infant between 0 and 28 days
  • de Jonge: death of a live-born infant between 0 and 7 days (the World Health Organization definition of early neonatal death)

The appropriate definition of neonatal death has been a major bone of contention in the comments on this and other blogs that criticized the Wax meta-analysis.  Both 0-28 days (neonatal death) and 0-7 days (early neonatal death) are accepted definitions. Proponents of using early neonatal death argue that it is more sensitive to events occurring around the time of birth, such as hypoxic injury resulting from inadequate fetal monitoring or a sudden emergency like a cord prolapse or placental abruption. Indeed, some of the late (8-28 days) neonatal deaths reported in Wax resulted from sudden infant death syndrome, a condition that has nothing to do with planned place of birth. On the other hand, proponents of using 0-28 day mortality point out that some babies experiencing severe hypoxic injury in labor or birth may be kept alive for many days in a modern neonatal intensive care unit.  Their deaths should be counted as birth-related even if they don’t die as soon after birth.

Regardless of which is the more appropriate measure, I was shocked by something de Jonge and Buitendijk revealed in their interview. Wax never contacted them to ask for their 8-28 day mortality data. It is standard practice among researchers who conduct meta-analyses to contact the authors of the original papers to obtain unpublished data, clarify methodologies, or ask for data in a compatible format. One would think that if Wax was truly interested in whether planned home birth caused neonatal death up to 28 days, he would be very motivated to get his hands on the Dutch data set. And while de Jonge and Buitendijk told me that those data are not as complete as the early neonatal death data (because some pediatricians don’t reliably enter their patients’ data), they do in fact have the data up to 28 days and would have supplied it to Wax had he asked. Instead, they have done the analysis themselves and submitted it for peer review.  (Therefore, we’ll have to wait for the results.)

What were the characteristics of the population?

  • Wax – no requirements for home birth eligibility were defined for inclusion in the meta-analysis. Individual studies included in the meta-analysis varied in their mechanisms for determining eligibility. As noted above, the largest study that contributed the majority of neonatal deaths relied on birth certificates. Women with any of 18 medical conditions documented on the baby’s birth certificate were excluded. Neither the study authors nor Wax and colleagues comment on whether this is a reliable method for defining “low-risk”. (As someone who routinely completed birth certificates when I was practicing, my guess is that it isn’t.)
  • de Jonge – National guidelines (”Obstetric Indication List“) define who is eligible for primary midwifery care and home birth. These conservative guidelines ensure that the population of women having planned home births are healthy and at very low risk of complications.

The Dutch study has been criticized because it is, well, Dutch – midwifery and home birth in the Netherlands are highly regulated and integrated into the system, and there are clear eligibility guidelines. The same isn’t true of the United States, so we can’t generalize the results here or elsewhere where home birth is marginalized (e.g., Australia). What the Dutch study gives us, though, is a clear model to emulate in order to make sure home birth is as safe as it can be – regulate midwifery, provide continuity of care for women who need to be referred, and make sure only low-risk women are having home births. Instead of acknowledging this and moving forward to optimize safety, Wax and colleagues chose to mash together data from five different countries and four different decades with no attention paid to which women were and were not eligible and spit out an authoritative answer to the question, “Is home birth safe?” “Is home birth safe?” is a bogus question to which there is no answer. Context, training, system integration, and perhaps above all else the characteristics of the population matter. Any study worth its salt will describe these factors in as robust detail as is feasible. Combining and meta-analyzing data from dissimilar contexts may make sense in other areas of health care, but when context is everything, what’s there to gain?

A note about comments: please keep it civil and on point. I’m OK with debate, discussion, and disagreement. Name-calling, personal attacks, and other degrading commentary will be deleted or edited.

Amy Romano Uncategorized , , , ,

Meta-analysis: the wrong tool (wielded improperly)

July 10th, 2010 by Amy Romano Amy Romano

A lot has been said about the new meta-analysis of home birth. (Here is an excellent summary from Jennifer Block.) Canadian physician Michael Klein has been widely quoted as saying that the meta-analysis, a potentially valuable statistical tool, was performed poorly because the researchers included studies using discredited methodology, as well as studies that are decades old. “Garbage in, garbage out.” I totally agree with this assessment. I also take issue with the fact that the researchers did not display the standard “forest plot” that customarily accompanies a meta-analysis to illustrate how the relative magnitude of observed differences in the individual studies and the pooled analysis. And I’m perplexed by the use of a fixed-effects model for the analysis of neonatal death.

But I want to take a step back and ask a larger question - is meta-analysis even appropriate for the study of home birth?

Meta-analysis is a statistical process that pools data from multiple studies. It is intended to achieve two related goals:

  • have adequate statistical power to detect differences in rare but clinically important outcomes (such as perinatal mortality among babies of healthy women)
  • establish a definitive answer to an important clinical question, so that policies and practices can adapt to conform to the new “truth” and other researchers don’t have to study the issue anymore.

Let’s look at these two issues separately in the context of the Wax meta-analysis.

Statistical Power

Lack of statistical power could not possibly be the rationale for conducting a meta-analysis on the safety of home birth. That’s because there already is a study large enough to detect differences in intrapartum and neonatal death. In fact, it contributed 94% of the data on planned home birth in the meta-analysis (321,307 of 342,056 planned home births). That study found virtually identical rates of neonatal death in both the planned home and planned hospital births*, with relatively narrow confidence intervals. Neonatal deaths on day 0-7 occurred in 3.4 per 10,000 of each group and when combined with intrapartum mortality and adjusted for confounding factors, the relative risk was 1.00 (95% CI 0.78 to 1.27). That means that there was a 95% likelihood that planned home birth results in somewhere between a 22% reduction and a 27% increase in intrapartum or neonatal mortality.)

By adding a bunch of smaller, older, and flawed studies, excluding the intrapartum deaths (which may be affected by intrapartum events and therefore are potentially modifiable by the birth setting) and adding deaths that occurred between 8-28 days (which are less likely to be related to intrapartum events and therefore are less modifiable by birth setting), we suddenly have nearly three times the neonatal mortality rate with planned home birth and a confidence interval you could drive a truck through?  (a 95% chance that home birth increases the risk of neonatal death by somewhere between 32% and 625%)  Hmmm…

Definitive “truth”

The other reason to undertake meta-analysis is to definitively settle a clinical question. Meta-analysis, after all, holds a privileged place atop the evidence pyramid, where it is considered the “best evidence.”  But is a deeply flawed meta-analysis really better than an adequately powered, methodologically sound study? The answer, of course, is no. All the meta-analysis does in such cases is separate the reader from the primary source of the data so that they can’t assess it for themselves, while putting the evidence-based stamp of approval on whatever statistics the meta-analysis software spits out. But people with a political motivation to authoritatively declare a certain definitive truth may realize that most people don’t bother to check to see if a meta-analysis is done appropriately or critically assess the quality of the included studies. They just go, “Oh look, there’s a meta-analysis of home birth and it said it’s 3 times riskier than hospital birth. That settles that! It’s a meta-analysis, after all!”

So if not a meta-analysis, then what?

OK, so if meta-analysis was not the right tool, what is?  And can we stop studying the safety of home birth now that we have that large study that contributed 94% of the home birth data to the meta-analysis?

The way I see it, the large study that showed equivalent perinatal outcomes between home and hospital birth tells us definitively that home birth can be safe. But it doesn’t tell us that home birth is intrinsically safe. We need to continue to study home birth using all of the tools in the research toolbox, qualitative and quantitative, to determine under what circumstances home birth is safe and how to optimize care and outcomes in all birth settings. And we need to stop pushing home birth underground in the United States where it remains a fringe alternative, poorly integrated with the maternity care system, with no standard safety net in place for women who begin labor with the intention to birth at home but turn out to need hospitalization in order to birth safely. Shame on the American Journal of Obstetrics and Gynecology for making this task even more difficult than it already was, by publishing and publicizing a junk meta-analysis.

*edited 7/12/2010 to correct a (serious) error. Sentence previously read “virtually identical rates of neonatal death in both the planned and unplanned home births.”

Amy Romano Uncategorized , , , ,