24h-payday

Archive

Posts Tagged ‘becoming a critical reader’

Jumping to conclusions: Popular media spins an abstract into headlines.

February 23rd, 2012 by avatar

A new study has been making the rounds of the popular news sites.  The abstract – 65: Neonatal outcomes associated with intended place of birth: birth centers and home birth compared to hospitals  The abstract of the study is published in AJOG It was presented at the Society for Maternal-Fetal Medicine 32nd Annual Meeting.

It is strange that this abstract is getting so much attention. With only an abstract available it is impossible to judge the study’s merits. We look forward to the publication of the study. At this point we have to reserve judgment for later.  We simply don’t have the data available to determine the strength or validity of the study.  That said, it is amazing that the findings presented in the abstract are getting so much attention.

Here are some of the many articles, with varying perspectives, discussing it: 

I found this article to be neutral verging on steering families toward hospital birth:

 “Babies born at home were more than twice as likely to have an Apgar score of under 7 as children born in a hospital or at a birthing center, and also had double the chances of having a seizure….

The overall number of kids who had seizures was low — less than 1 percent at any location.

Prior research has shown that babies with lower Apgar scores are more likely to have complications after birth, such as needing breathing assistance, going to the ICU, having seizures or having developmental issues, Cheng said.”

Study Weighs Pros, Cons of Home or Hospital Birth: More seizures, lower Apgar scores found in home or hospital birth  

 

This article has a positive spin for homebirth:

 “But when a certified midwife was present, it seems babies born at home may fare as well as those born in hospitals, said study researcher Dr. Yvonne Cheng, an obstetrician and gynecologist at the University of California, San Francisco.

“It’s not just about where you deliver, but perhaps who you deliver with,” Cheng said.

Home births are known to be associated with fewer obstetric interventions — that is, women in labor at home receive fewer epidurals and less pain medication.

Women must weigh the benefits of home births against the risks to make an informed decision about where to give birth, Cheng said.”

Midwives make homebirth safer for babies  

 

 This article seems to treat the study in a neutral manner:

 “Women who have home births or plan to deliver at home have lower rates of cesarean delivery; however, their babies are more likely to have neonatal seizures and lower Apgar scores if a certified midwife is not in attendance, according to research presented here at the Society for Maternal-Fetal Medicine 32nd Annual Meeting.”

Home Births Associated With More Seizures, Lower Apgar Scores

 

This one uses bad data to back a claim:

“…recent evidence suggests that while the absolute risk of planned home births is low, such births carry a neonatal death rate at least twice as high as that of planned hospital births. Neonatal death occurred less than once in 1,000 hospital births, compared with two in 1,000 home births, said an American Journal of Obstetrics & Gynecology meta-analysis published in September 2010.”

Home births rise despite higher neonatal mortality rate: Although the vast majority of deliveries occur in hospitals, more women who want a less institutional experience are opting to give birth at home.

This AMA article is citing the Wax et al study.  Science and Sensibility has discussed the vast array of errors and misinformation in the Wax study on four separate occasions:

Others have cited Wax et al, although not explicitly such as this one: Homebirths up Dramatically, but are they safe?

There were numerous letters written to AJOG with regards to the flaws in the study, as well.  So, to have the Wax et al study brought up again is inappropriate and poor science.  It feels to me like a scare tactic or propaganda.

Given that we don’t have all the information, I question the journalistic integrity with which the articles above are written.  It’s always a good headline – about the dangers of home birth.  It’ll get links clicked, newspapers sold and running commentary on social media sites.  However, without proper analysis of the data things are potentially misrepresented.  Once we gain access to the full study, Science and Sensibility will be able to respond appropriately.

Some questions we hope to answer:

  • What data were used? How strong is the data set?
  • Many home births are not reported as such, so data will be lacking.  How is this accounted for?
  • Does the legal status of a homebirth midwife impact outcomes?  Especially because the author states that CNMs have better outcomes than do CPMs or DEMs. We are not aware of research that supports this.
  • Is it considered homebirth if the mother was transferred from home to hospital mid-labor if her intention was to have a home birth?

For more on recent perspectives on homebirth please visit the Homebirth Consensus Summit.

Let’s get the discussion going here.  What are your thoughts on homebirth?

Evidence Based Medicine, Guest Posts, Home Birth, Metaanalyses, Midwifery, News about Pregnancy, Research , , , , , , , , , , , , , , ,

Becoming a Critical Reader: Journal papers that aren’t studies

November 23rd, 2010 by avatar

In this, my last article for the “Becoming a Critical Reader” series, I want to discuss a few types of articles that are frequently found in journals, but are not studies. These can provide important information, and are not by any means worthless, but they are not what we traditionally think of as research.

Opinion pieces – Most issues of a journal will have one or more editorials. In these, the authors get to give their opinion. And like in the rest of life, opinions are not the same as fact. These articles give the reader a chance to see how one individual reads, interprets and applies the research in their practice. Editorial pieces can be unsigned, representing the editorial board of the journal, or can be written by and represent the opinion of an individual.

Practice guidelines – These are statements that organizations put together to help their members practice. Journals are a way that some organizations will spread the word about a new, updated practice guideline, express opinions about current guidelines, etc.

Media reviews – Some journals, like the Journal of Perinatal Education, will publish reviews of various types of media: books, videos, patient education programs, anything that might be relevant to their readers.

Case reports – Sometimes unusual cases come up, and they don’t happen often enough to have a formal study. While you certainly can’t make a definitive statement about a treatment based on a case study, they make for interesting reading. Occasionally a case study might be published, which leads to a reader thinking “I’ve seen that, too!” and before long, care providers can be communicating and sharing ideas for these kinds of oddball situations.

Some journals, like the New England Journal of Medicine present a regular clinical case study in each issue, as sort of a “how would you handle this?” with the resolution as well.

Patient Education materials – Some journals, like the Journal of Midwifery and Women’s Health, publish handouts that their subscribers can print out and use in their practice.

Reviews or critiques of other studies – You may see responses to and critiques of studies also published in journals. Sometimes these are short and take the form of a letter to the editor or news brief. Sometimes they are full-fledged articles. It is always a good idea to go back and read the original study yourself and form your own opinion!

Continuing Education modules– Usually connected with a systematic review, practice guideline, or other article, these short quizzes can be returned to the journal or organization for CEUs.

Reports on a conference or meeting – If the journal is published by an organization that holds an annual meeting, they may include information presented at those meetings in the journal. This could be abstracts, papers that were presented, information on how many attended, or a travelogue.

Personal narratives– Occasionally you will find articles describing one person’s experiences in an area related to the journal. You may have a report on one midwife’s efforts to improve neonatal resuscitation training in Africa, or a doctor’s ponderings on the difference between health care now and health care when he began his practice 25 years ago.

While I won’t be going into questions to ask about each of these, I hope that you’ll remember to check your own biases and read them with a critical eye.

Uncategorized ,

Becoming a Critical Reader: Questions to ask about systemic reviews and meta-analyses

November 14th, 2010 by avatar

Systemic reviews are generally considered to be at the top of the evidence pyramid, providing one of the best sources of information. But just like any other type of research, a systemic review is only as good as the work and data that goes into it. A systemic review carefully looks at all of the evidence using a rigorous, predefined system of methods, and draws conclusions based on the information gathered. That rigorous, predefined system of methods is critical to a good systemic review. The researchers go through several steps:

  • Select a specific, well-defined question
  • Lay out their criteria for searching and selecting the evidence
  • Conduct a very thorough search of all the available literature
  • Evaluate the studies, rejecting the studies that are of poor quality
  • Review the studies that make the cut
  • Make a recommendation for practice

A systemic review differs from a general review of the literature in the methodology used. A systemic review starts by formulating specific criteria that will be used to judge which studies will be included and which will be excluded. The criteria are set before any of the studies are reviewed; ideally this will prevent bias and make for a stronger, more valid result.

Some systemic reviews include a meta-analysis, where statistical techniques are used to combine the results of the included studies and use the larger sample size to draw a stronger conclusion. But don’t assume that all meta-analyses use the systemic review process! It’s entirely possible to conduct a meta-analysis of a group of studies chosen in an incomplete or biased manner. The questions below can help you identify which meta-analyses use the systemic review process.

When reading a systemic review or meta-analysis, here are some questions to consider:

1. How well is the question defined? There should be a clear statement of what the review would like to show. Then double check the results to make sure that question actually got answered.

2. Is it the right question? If two things are being compared, is the comparison appropriate? Do all the studies included use the same comparison/control?

3. Do the authors describe their search? Was it thorough? Authors should discuss how they went about searching for the articles they evaluated. A thorough researcher will look at multiple databases, use variations of the key words, and include studies published in other languages.  Limiting to studies published in English is convenient, but you may miss valuable research. Unpublished studies are also sometimes included, as it can be difficult to get a study published if your results showed no dramatic differences. This can help avoid publication bias, but unpublished studies should still be thoroughly checked for quality. The search and selection methods should be so clearly outlined that someone could duplicate them.

The Cochrane Collaboration is best known for conducting systemic reviews. In the Cochrane organization, reviewers publish their protocols before conducting the review. (If you are a Lamaze member, you can access these protocols on the Cochrane site by logging in through the Lamaze Member Center.) This is not generally done elsewhere, but in the published paper, the reviewers should explain their protocol. Any potential conflicts of interest should also be addressed.

4. Did the authors evaluate the quality of the studies reviewed? Not all studies are of equal quality, though sometimes studies with quality issues can still provide useful information. One example of this is the Cochrane Review on skin-to-skin, which included some studies that did not have completely random groupings if it appeared that the groups were otherwise equal.  For this reason, many systemic reviews will rate the quality, size and applicability of the studies as they evaluate them, and assign them a weight so that the most appropriate studies are more heavily represented in the results.

5. Are there any biases in the inclusion/exclusion criteria? Read through them very carefully and evaluate this aspect. Easier when you’re already familiar with the studies out there, or if the excluded studies are listed for your viewing. Overly restrictive criteria lead to smaller sample sizes and less reliable results.

6. Were the outcome measures clearly defined? What are the benefits or risks the researchers were looking at? Are outcomes lumped into groupings of debatable usefulness? Do they matter? As Amy Romano recently pointed out sometimes the outcomes have little or no real life importance.

7. Are there biases in how subgroups are analyzed? In one interesting study researchers created random data with the roll of the dice. The studies used dice that were identical other than color, and each time a 6 was rolled, it was counted as a patient death. The researchers then manipulated subgroup analysis to show that red dice had significantly higher death rates than other colors!

Don’t assume that the general inclusion/exclusion criteria were applied to each subgroup. As we learned with the Wax meta-analysis on home birth, selective subgroup analysis does happen.

Bias in subgroup analysis is less likely if the subgroups were defined before the reviewers selected the included studies. Cochrane systematic reviews generally are conducted this way.

8. How old is it? As new research is done, even systematic reviews become outdated. It’s hard to give a specific age that is “too old”, though, since some topics will have many new studies each year, and might be outdated at 18 months old while others rarely get anything new, and might still be relevant three years after publication.

9. How can I apply this? Again, this is always your last, but very important, consideration. Is this information that is applicable to your practice? To the population you serve? How can you best use this information?

Systemic reviews are among the best types of evidence out there, but only if they are done well. Read carefully, and consider the quality of the analysis as well as the quality of the included studies.

Next up: A roundup of other types of articles you may encounter in journals.

Uncategorized , , ,

Becoming a Critical Reader: Questions to Ask About Literature Reviews

July 7th, 2010 by avatar

A literature review is one person’s attempt to summarize what the literature says about any given topic. Many pieces of original research will have a mini-literature review as a part of the study to help place that study in context, but many times you will come across a literature review published on its own. These literature reviews are not the same as a systemic review or meta-analysis (more on those coming in a future article) and are less rigorous, but can still yield valuable information. Some good questions to ask:

1. Was the topic well identified? Each literature review should contain a clearly defined statement of what the author is trying to learn more about.

2. Was search thorough? The article should detail exactly how they conducted a search for relevant articles, and how articles were included or excluded. For example, it is very common for authors to limit their review to studies published in English or in the last 5 years, etc.

3. Were the included studies reviewed well? The author of the literature review should point out any important strengths or weaknesses found in the studies reviewed.

4. Was any important study (that should have been included) left out? At first, this will be a hard question to answer, unless you do your own searching. But as you read, read, read and become more familiar with the research on a topic, you may know of studies or authors who commonly write on your topic of interest and you’ll be able to spot things left out.

5. Were conclusions consistent with the information presented? You want to watch out for authors making big leaps or ignoring problems. Try to be fair and balanced as you determine if the author was fair and balanced.

6. And finally, what does this mean for me? That will vary widely based on your personal situation. As a reader, you may be a nurse, midwife, childbirth educator, doula, doctor, or parent. You may have more than one role. Carefully think about how this may – or may not – apply to you in your various roles. (Notice this is always the last question on my lists? There’s a reason!)

Literature reviews are close cousins to the next type of summarizing papers we will look at: meta-analyses and systemic reviews. Watch for those articles coming soon!

Uncategorized ,

Becoming a Critical Reader: Questions to Ask About Qualitative Research

May 17th, 2010 by avatar

Qualitative research generally describes and sometimes explains. It doesn’t try to prove anything. Because qualitative research is more interested in depth, the sample sizes tend to be much smaller. Once you’ve determined that what you have here is an original piece of quantitative research and you’ve already considered the basic questions here and here, you’re ready for the specific questions:

1. Did the researcher have a clear question? Remember in qualitative research, the authors are not trying to prove any hypothesis, but they still should have an idea of what they are setting out to do. It can be as broad as “To explore college students’ beliefs about childbirth and midwifery” or as specific as “to assess certified nurse-midwives’ (CNMs’) knowledge of female Genital Cutting (FGC) and to explore their experiences caring for African immigrant women with a history of genital cutting.”

2. How was the data collected? Was data collection well described?

Among the possible methods are:

Survey – this could be numerical data from a questionnaire or open ended questions with respondents filling out their thoughts and feelings. Sometimes researchers will use an existing tool that has been validated. This means that the survey, test, etc. has been studied and found to consistently give good results. Sometimes there isn’t a validated tool available, so researchers make their own or adapt an existing one. The researcher should describe the tool used.

Interview – Can be done either free-form or with a prepared set of questions/prompts. If a set of prepared questions is used, you’ll often see it (or excerpts) in the published study.

Observation – Can be either overt, where the subjects know they are being observed, or covert, where subjects do not know they are observed. Because being observed sometimes changes behavior, covert observation does have a place. Look for a description of how the observations were recorded. Did the observer make notes? Tally the number of times something happened? Video and audio recordings are sometimes used, but how are they analyzed?

Focus groups – small groups gathered together to discuss issues.

3. What was the setting? The setting is important to the generalizeability of the study. Generalizeability refers to whether or not the study can be applied beyond the scope of the study. Looking at how laboring women communicate with hospital staff probably doesn’t apply to home births, and vice versa. Consider cultural issues as well. While a study on how Hispanic women feel about breastfeeding is valuable, it won’t apply to a Muslim or Eastern orthodox woman.

4. How was the data evaluated/coded? In much of qualitative research, the authors do something called a thematic analysis. This kind of analysis looks for recurring themes. Sometimes the researchers will use coding where they have a list of themes they are looking for and will assign these codes whenever they see them. For example, one study (Ayers, S. (2007) Thoughts and Emotions During Traumatic Birth: A Qualitative Study. BIRTH 34:3 September 2007) listed one possible theme of “Negative Emotions” and assigned that code whenever an interview transcript showed words like “scared” “upset” or “shock”. Look into how the process was done. Was there one person doing the analysis or was it done by different researchers? Did the researcher analyze her own data or have a different person do it? Having multiple people look into the data makes for a stronger study. If the authors describe their analysis using terms you don’t understand, look them up.

5. What is the researcher’s perspective? Is the researcher coming from a medical point of view? A public health point of view? A consumer point of view? Consider how this point of view may have impacted the implementation of the study and the analysis.

6. What was the underlying framework?

Phenomenological – this research framework tries to gain insight into other people’s experience. You’ll often see this framed using the phrase “the lived experience.” Here is an example of a phenomenological study.

Ethnographic Research tries to understand various cultures and social systems. An ethnographic researcher might observe a group of labor and delivery nurses to figure out their social system, or interview student midwives about their interactions with preceptors and each other. Here is an example of an ethnographic study.

Grounded Theory goes beyond simply describing and understanding and actually tries to draw a conclusion that is “grounded” in the data collected. Here is an example of a qualitative study using a grounded theory approach.

There are many other possible approaches, including “integrated” designs which may include aspects of various frameworks.

7. How were the subjects / setting chosen? Often with qualitative research the researchers will use a convenience sample, where they look at a population that they have easy access to, or a quota sample, where they look at the first number of subjects that they can find, up to a set amount. Consider how similar the subject and setting are to your practice. The more similar, the more applicable.

8. And finally, what does this mean for me? That will vary widely based on your personal situation. As a reader, you may be a nurse, midwife, childbirth educator, doula, doctor, or parent. You may have more than one role. Carefully think about how this may – or may not – apply to you in your various roles.

Careful consideration of all these aspects can help you come to conclusions about this study and its usefulness. Next up: Questions specific to quantitative research.

Uncategorized ,