24h-payday

A Case of Statistical Malpractice? Predicting the Risk of Uterine Rupture

December 12th, 2009 by avatar

‘Tis the season for the Society for Maternal-Fetal Medicine to publish the abstracts for their forthcoming annual meeting. Every year around this time I receive the gift of an electronic Table of Contents alert for the Supplement to the American Journal of Obstetrics and Gynecology that lists conference sessions.  MFM doctors do interesting research, and their conference, which I have never attended, always has several sessions that look fantastic along with others that make me cringe (like a recent year’s session plugging this “exciting innovation“).

Nestled among the 800+ abstracts was one that I would put in the cringeworthy column, not for the focus of the research but for the complete mismatch between the reported findings and the researchers’ conclusions. [Emphasis mine]

Frequent epidural dosing is a marker for impeding uterine rupture in patients attempting vaginal birth after cesarean (VBAC)

Alison Cahill, Anthony Odibo, Jenifer Allsworth and George Macones
Washington University in St. Louis, St. Louis, Missouri

Objective
To estimate the association between epidural dosing and risk of uterine rupture in women attempting VBAC.

Study Design
A nested case-control study within a multicenter retrospective cohort of >25, 000 women with a prior cesarean was performed, comparing cases of uterine rupture to women without rupture (controls) while attempting VBAC. Extensive data extraction included all medications in 15-minute increments. In women who attempted VBAC with an epidural anesthetic, dose timing, frequency, and quantity were compared between cases and controls. Time-to-event analyses were performed to estimate the association between epidural dosing and risk for uterine rupture while accounting for duration of labor and confounding effects.

Results
Of 804 women in the nested case-control study; 504 (62.7%) had an epidural, with no statistical difference in epidural usage rates between cases and controls (70.4% v. 62.4%, p=0.09). Women who experienced uterine rupture were > 4 times more likely to require epidural dosing in the 60 minutes prior to delivery (aOR 4.1, 2.4 – 6.7, p <0.01). Cox-regression analysis revealed a dose-response relationship between number of doses in the final 90 minutes of labor and risk of rupture, after adjusting for prior vaginal delivery, and oxytocin exposure.

Conclusion
Clinical suspicion for uterine rupture should be high in women requiring frequent epidural dosing during a VBAC trial.

What’s the problem here?  This is a classic example of reporting the “hazard ratio” (e.g., “4 times more likely”) in lieu of the more appropriate statistics, which in this case would be the “positive predictive value”. It is indeed noteworthy that women destined to experience uterine ruptures self-administer more anesthesia in the minutes prior to the event, but should “clinical suspicion be high” every time a woman in a VBAC labor pushes the epidural button frequently? At least from the data reported in the abstract, the answer is: we have no idea.

To get an answer we need much more data. Specifically, we need to know:

  • how many women pushed the epidural button frequently
  • how many of them had a uterine scar rupture
  • how many women did not push the button frequently
  • how many of them had a uterine scar rupture

These data would help us calculate the sensitivity and specificity of epidural dosing in predicting uterine scar rupture, which in turn tell us the likelihood of a “false positive” (a woman requests frequent doses of epidural but does not have a scar rupture) and a “false negative” (a woman doesn’t request frequent epidural dosing but does have a scar rupture).

Sensitivity and specificity are especially important in predicting something that occurs rarely, such as uterine scar rupture in a VBAC labor. Reporting that something is “4 times more likely” could still be a small risk in absolute terms, if the baseline risk is low. In the case of VBAC, this kind of reporting could in fact be hazardous, because it is likely that many women and even many obstetricians overestimate the baseline risk of uterine scar rupture and of rupture-related morbidity and mortality.  So quadrupling it would falsely elevate risks even further. Let’s take for example statistics put forth by a spokesperson for the American College of Obstetricians and Gynecologists. In a letter to a mother who appealed to the College to make VBAC more accessible, he notoriously overestimated the risks.

In two percent of [VBAC labors] the result can be a rupture of the old scar. If this happens, then death of the baby is almost certain and death of the mother is probable. Even if the mother does not die, virtually 100% will lose their child bearing ability.

In this scenario, anything associated with a 4-fold increase in uterine rupture would result in 6 additional babies dying plus 6 additional mothers dying or needing hysterectomies for every 100 VBAC labors. Looking at these data, it’s easy to justify doing a cesarean when the woman begins asking for epidural top-ups even if top-up requests have a low predictive value.

But the uterine scar rupture rate is in fact 0.5-1%, and in only about 5% of ruptures is the baby likely to die. Maternal mortality is rarer still, and the likelihood of either maternal mortality or hysterectomy is actually higher with repeat cesarean surgery than it is with planned VBAC. Quadrupling these risks might result in 15 excess fetal/newborn deaths per 10,000 VBAC labors. This may still seem to be an unacceptable risk, but it’s nothing close to 6 per 100. In this scenario, it’s a little more difficult to justify going straight to a cesarean for every woman requesting more anesthesia.

I’ll give the researchers the benefit of the doubt. It is clear that they understand the distinction between relative risk and predictive value, since they’ve published papers on the topic before that appropriately concluded that obstetric variables poorly predict the likelihood of scar rupture. They may also have been severely limited by journal space constraints in preparing their abstract for publication. But I’ll call “statistical malpractice” on them for publishing a conclusion that suggests that the predictive value is high without providing any data to support it.

FYI, this topic should be familiar to anyone listening to the news lately, as false positive are at the crux of the debate about the new mammography guidelines. The New York Times ran a piece explaining concepts of risk and predictive value just last week, with the decidedly unsexy title, Mammogram Math. It’s a  great read for anyone who wants to know more about interpreting statistics about risk.

Uncategorized , , ,

  1. December 13th, 2009 at 00:17 | #1

    >>> What’s the problem here? This is a classic example of reporting “relative risk” (e.g., “4 times more likely”) in lieu of the more appropriate statistics, which in this case would be the “positive predictive value”. It is indeed noteworthy that women destined to experience uterine ruptures self-administer more anesthesia in the minutes prior to the event, but should “clinical suspicion be high” every time a woman in a VBAC labor pushes the epidural button frequently? At least from the data reported in the abstract, the answer is: we have no idea.

    Amy – I have to massively disagree with you here.

    The authors have performed a case control study, which is an appropriate study design when looking at rare events.

    For those that do not know, a case control study is where you select a number of cases (people who have the outcome you care about – uterine rupture in this case) and then select a number of controls who did not have that event, who are matched to the cases as closely as possible. You then look at particular exposures you are interested in (in this case frequency of epidural rebolusing) and see if that exposure is more common in the cases than in the controls. From this you can calculate an odds ratio, which is the odds that a case had a certain exposure relative to the odds that the controls had an exposure. For very rare events, such as uterine rupture, the odds ratio will be numerically similar to, but not the same as, a relative risk.

    Amy has commented that they have reported relative risk, which they have not, and could not in this study design. She goes on to say that they should somehow have come up with a positive or negative predictive value, which also is impossible in this study design.

    To create a positive or negative predictive value, one must be working with a true population with a true prevalence. This is because PPV and NPV are very linked to prevalence, while sensitivity and specificity are prevalence independent.

    For this reason is impossible in this study design to create a positive or negative predictive value. Why? Because the prevalence in study isn’t real! The study designers chose all the cases, and then chose a number of controls, probably 1:1 or 1:2 matched. Say it was 1:2. That would create a prevalence of 33% for uterine rupture – obviously not the real prevalence of uterine rupture. That’s why you can’t calculate the stats that Amy desires from this type of study.

    Note that this study was a nested case control inside a big cohort study. The only way they could have gotten the numbers Amy wants would have been to run the analysis on the entire cohort. The abstract states that they did detailed data abstraction from case and control charts, which amounted to a few thousand charts. The overall data set was > 25,000 women. Could they have done that – yes, but it would have amounted to abstracted all the data for this little analysis from 25,000 charts, something that would be unlikely to be done.

    Ultimately, these authors did an appropriate study design to look at a certain risk factor for an uncommon event. They showed that there is a 4x odds ratio for requiring many epidural boluses in women who are experiencing uterine rupture. Their study design is correct, and assuming that they controlled for bias correctly, the conclusion should also be correct.

    That doesn’t make the study too meaningful to me, however. If you were to tell me that a patient experiencing a uterine rupture might need more pain control in their epidural, my response would be “DUH!”. Sort of an obvious result. When it comes to VBACs, I’m always on the lookout for signs of uterine rupture, and having a patient need many reboluses would already peak up my ears, with or without this study.

    But the study design is correct, and there cannot be relative risks from it.

    Nicholas Fogelson

  2. December 13th, 2009 at 00:51 | #2

    Your comments do point out a inherent weakness in Case Control studies, though. As you say, we want relative risk, and its brother in arms likelihood ratio. As case controls cannot provide it, they are less useful to use than well designed cohort studies, and the ultimate study, the prospective randomized trial.

    The problem is that it hugely expensive and near impossible to do a prospective trial to look at very rare events. As uterine ruptures are so rare, you would have to collect detailed data prospectively for tens if not hundreds of thousands of women to be able to make a statistically significant statement about the relative risks of various precursors to uterine rupture.

    Sometimes these kinds of studies are done, but they are usually NIH or industry sponsored, and are multicenter (Like the Preterm Prediction Study, the MAGPIE study, or the FASTER trial.)

    I suspect that everything we know about uterine rupture comes from case control studies, and similarly does not provide the data you were looking for. Its just too rare an event to look at it that way.

  3. December 13th, 2009 at 08:26 | #3

    Thanks for weighing in, and for explaining case control study design to readers. You’re right that the researchers could not calculate the predictive value of the event given their study design. But this research team has shown an interest in establishing the obstetric factors that are predictive of uterine rupture, so why use a study design that can’t shed any light on that? And more importantly, why publish a conclusion that implies that the factor is in fact statistically predictive?

    I recognized (though I didn’t say as much) that a study that could yield predictive value would be expensive, complicated, and probably not feasible at all. As you say, it is not “news” that extreme pain is a sign of impending uterine rupture. This research team tried to quantify that, presumably to come up with some sort of clinical guidance as to how many boluses are “too many”. I stand by my argument that you can’t answer that question with the data they report, which as you point out is probably because their study didn’t generate that data.

    The media and the public often confuse concepts of increased risk (whether OR, RR, or Hazard Ratio, which is actually what they in fact reported in this study – i have edited my original post to reflect this) with predictive value. As my example, the mammography example, and countless others show, the two are quite different especially when events are rare, and we as a culture have a tendency to overestimate risks of rare events, something that certainly seems true of VBAC as the Hale quote illustrates. I saw this abstract as a means to show that “4 times more likely” may or may not be clinically meaningful. I have no bones with how they conducted the study, which may have been impeccable in its design. But without more information, I don’t think this data alone can tell us anything about how our “clinical suspicion” should correlate with epidural doses.

  4. December 13th, 2009 at 12:07 | #4

    This is loosely related, but I’ve heard stories about women seeking a hospital VBAC who are denied a trial of labor if they refuse to consent to an epidural. I Facebooked a call for stories out of curiosity.

  5. avatar
    Susan Hodges
    December 13th, 2009 at 13:27 | #5

    I would be curious to know if the role of labor augmentation (with pitocin or other drug) was addressed in this study. The original study that started the trend against allowing VBACs (sorry, I don’t have the citation in front of me) actually found that using certain drugs to intensify labor caused the increase in uterine rupture found in the study, not VBAC labor itself. My understanding is that pitocin-stimulated labor is much more painful than unmedicated labor. I would be interested if women wanting to up their epidurals were also having the pitocin (or other labor stimulating drug) levels increased. Might the demand for more pain relief be associated with increased pitocin (or other drug), which might also be associated with increased risk of rupture? Seems like this would be an important factor to know about.

  6. December 13th, 2009 at 13:42 | #6

    Related to Jill’s point – There have been many discussions recently on the various cesarean/VBAC listserves regarding hospitals both refusing epidural in VBAC, and others requiring it upon admission for anyone requesting VBAC. I saw a mother whose OB told her they would “require” her to have the epidural catheter placed as soon as she was admitted, even though she did NOT want an epidural at all. This mother, by the way, instead walked into a different “VBAC-banning” ER just a few days later and had her baby completely unmedicated.

    But I like NF’s quote – and shouldn’t this really be what we’re explaining to women about UR? “I suspect that everything we know about uterine rupture comes from case control studies, and similarly does not provide the data you were looking for. Its just too rare an event to look at it that way.”

    Too rare an event. Unlike the doubtful, impending catastrophe that many OBs/Hospitals make it out to be.

  7. December 13th, 2009 at 14:29 | #7

    @Amy Romano

    >>> I saw this abstract as a means to show that “4 times more likely” may or may not be clinically meaningful. I have no bones with how they conducted the study, which may have been impeccable in its design. But without more information, I don’t think this data alone can tell us anything about how our “clinical suspicion” should correlate with epidural doses.

    I don’t really agree with this statement. Basically you are saying that the case control design is unable to provide any risk information, and is therefore not useful. If we take this as true we would never be able to study any rare condition. In this study they found a 4x hazard ratio for uterine rupture in patients who needed rebolusing of their epidural. Hazard ratios and odd ratios are not relative risks, but for rare events they will be mathematically similar to relative risk – so as best as we will ever be able to tell, women who are requiring many epidural boluses are about 4 times more likely to have an impending uterine rupture than women who do not have this need.

    The underlying rate of uterine rupture of 1% includes all uterine ruptures from pre labor all the way through the third stage. The women in this case are all late in labor, so at that point the population does not have a 1% risk of rupture, as they have already passed through a substantial amount of the rupture period without experiencing a rupture. Ie if a raft has a 1% risk of toppling after going through ten rapids, after it has successfully navigated 8 of the rapids it has a substantially less than 1% rate of toppling thereafter.

    This study, though limited in study design, suggests that there is somewhere around a 4x likelihood of rupture in women who are experiencing enough pain to require epidural rebolus, but given that the underlying rate of rupture is so low, this would not necessarily necessitate any action.

    Ultimately these folks are looking at a surrogate for pain, which is already a well established sign of uterine rupture. If my patient is doing a VBAC and is having severe continuous pain, I’m going to be concerned about rupture, with or without this study.

  8. December 13th, 2009 at 18:06 | #8

    Yeah, I thought about the role of Pit or other agents that are known to increase the rupture rate. The abstract was probably too short to deal with it, but I’d like to know if there were any data that you can find (now or in the future) that see if they looked at women in various risk profiles (e.g. completely natural vs. Pit), or if they just lumped all attempted VBACs into one group, without regard to any other known factor in uterine rupture.

  9. December 13th, 2009 at 19:15 | #9

    “But the uterine scar rupture rate is in fact 0.5-1%, and in only about 5% of ruptures is the baby likely to die. Maternal mortality is rarer still, and the likelihood of either maternal mortality or hysterectomy is actually higher with repeat cesarean surgery than it is with planned VBAC.”

    Thanx so much for this- excellent info
    I am one of those women who is currently fighting for the chance to have a TOL for VBAC. I am very interested in where you got these numbers (above) from and if you can direct me to the study or paper you’re taking it from. In particular, I’m researching the statistical differences between a VBA2/3/4C, and a woman having a 3/4/5th C/S with regards to uterine rupture, infant/maternal morbidity, hysterectomy etc… If you can throw me any info I’d be hugely grateful.

  10. December 14th, 2009 at 00:44 | #10

    @Kathy
    Pitocin increases the rupture rate from 0.5% to 1% in most studies.

  11. December 14th, 2009 at 21:08 | #11

    @Jill–Unnecesarean
    I know that the “Hey, hey it’s a VBAC” board (that’s really what it’s called) on alt.life has anecdotal reports of epidurals being required for VBAC attempts. However, I think this study shows an association (if that), not causality. However, I think there is something to be said for not requiring one. Why mask a major symptom of the most pertinent complication? At least, why force someone to?

    Dr. Fogelson, what about reanalyzing the data as a retrospective cohort study on the original cohort? Would that be problematic? You mention using the existing cohort, then say a cohort study would be impractical.

  12. December 15th, 2009 at 01:24 | #12

    @MomTFH

    In order to do the case control, they had to do detailed chart abstraction from the few thousand charts that were involved. In order to do a retrospective cohort analysis, they would have had to abstract that data from over 25,000 charts, which would likely have been impractical. Big prospective cohort studies can be done, but are very costly and need to be custom designed for the desired outcome from the beginning. This study was an add-on analysis on another cohort not originally designed for this outcome – as such they probably did not have all the data they needed in the original dataset and needed to look at the charts. I hope they had some medical students!

    >> However, I think this study shows an association (if that), not causality.

    This is entirely correct. There is no reason to believe that bolusing an epidural causes uterine rupture. It is likely a marker for increasing pain, which is a sign for uterine rupture.

  13. December 17th, 2009 at 15:59 | #13

    “Ultimately these folks are looking at a surrogate for pain, which is already a well established sign of uterine rupture. If my patient is doing a VBAC and is having severe continuous pain, I’m going to be concerned about rupture, with or without this study.”

    I agree with this, with or without this study, pain is an indication of uterine rupture. I would hope that any good clinician, though, wouldn’t just jump the gun, but use other factors to decide if a section is needed. Assessing the type of pain (with contractions or not, where it is located, how severe) should all be a part of this also. What kind of pain the women had in the study…and if this was a indicator of whether or not they used the epidural more.

  14. December 17th, 2009 at 18:35 | #14

    @Rachel
    >> I agree with this, with or without this study, pain is an indication of uterine rupture. I would hope that any good clinician, though, wouldn’t just jump the gun, but use other factors to decide if a section is needed.

    No question

  15. avatar
    pinky
    December 21st, 2009 at 08:49 | #15

    It makes sense that someone who is rupturing her uterus would require more pain medication. I am told it is extremely painful to rupture your uterus. However, I don’t see what this study tells us or how it helps us. If a vbac has consistent pain, she will get a c-section. We could wait until the FHR drops and does not come back, but I think that may be too late. I think we need to focus on avoiding the first c-section.

  1. December 13th, 2009 at 20:37 | #1