Science & Sensibility

A literature review is one person’s attempt to summarize what the literature says about any given topic. Many pieces of original research will have a mini-literature review as a part of the study to help place that study in context, but many times you will come across a literature review published on its own. These literature reviews are not the same as a systemic review or meta-analysis (more on those coming in a future article) and are less rigorous, but can still yield valuable information. Some good questions to ask:

1. Was the topic well identified? Each literature review should contain a clearly defined statement of what the author is trying to learn more about.

2. Was search thorough? The article should detail exactly how they conducted a search for relevant articles, and how articles were included or excluded. For example, it is very common for authors to limit their review to studies published in English or in the last 5 years, etc.

3. Were the included studies reviewed well? The author of the literature review should point out any important strengths or weaknesses found in the studies reviewed.

4. Was any important study (that should have been included) left out? At first, this will be a hard question to answer, unless you do your own searching. But as you read, read, read and become more familiar with the research on a topic, you may know of studies or authors who commonly write on your topic of interest and you’ll be able to spot things left out.

5. Were conclusions consistent with the information presented? You want to watch out for authors making big leaps or ignoring problems. Try to be fair and balanced as you determine if the author was fair and balanced.

6. And finally, what does this mean for me? That will vary widely based on your personal situation. As a reader, you may be a nurse, midwife, childbirth educator, doula, doctor, or parent. You may have more than one role. Carefully think about how this may – or may not – apply to you in your various roles. (Notice this is always the last question on my lists? There’s a reason!)

Literature reviews are close cousins to the next type of summarizing papers we will look at: meta-analyses and systemic reviews. Watch for those articles coming soon!

Andrea Lythgoe Uncategorized becoming a critical reader, Understanding Research

Many readers of this blog follow our Understanding Research series developed by our very own Andrea Lythgoe, LCCE. Here’s another learning resource for those interested in deepening their understanding of the fundamentals of evidence-based healthcare.

Consumers United for Evidence-Based Healthcare, a collaborative project of the U.S. Cochrane Center has developed a free online course in six parts that look in-depth with real-world examples about how evidence can be the foundation for action. As the project director, Kay Dickerson, says, “We’re only going to get evidence-based healthcare in this country through consumer activism.”

Lamaze International along with other maternity care advocacy organizations including Childbirth Connection and Our Bodies Ourselves are members of CUE. Here’s an inspiring video about CUE and the importance of consumer involvement in pushing for evidence-based care in the U.S.

Amy Romano Uncategorized systematic review, Understanding Research

Qualitative research generally describes and sometimes explains. It doesn’t try to prove anything. Because qualitative research is more interested in depth, the sample sizes tend to be much smaller. Once you’ve determined that what you have here is an original piece of quantitative research and you’ve already considered the basic questions here and here, you’re ready for the specific questions:

1. Did the researcher have a clear question? Remember in qualitative research, the authors are not trying to prove any hypothesis, but they still should have an idea of what they are setting out to do. It can be as broad as “To explore college students’ beliefs about childbirth and midwifery” or as specific as “to assess certified nurse-midwives’ (CNMs’) knowledge of female Genital Cutting (FGC) and to explore their experiences caring for African immigrant women with a history of genital cutting.”

2. How was the data collected? Was data collection well described?

Among the possible methods are:

Survey – this could be numerical data from a questionnaire or open ended questions with respondents filling out their thoughts and feelings. Sometimes researchers will use an existing tool that has been validated. This means that the survey, test, etc. has been studied and found to consistently give good results. Sometimes there isn’t a validated tool available, so researchers make their own or adapt an existing one. The researcher should describe the tool used.

Interview – Can be done either free-form or with a prepared set of questions/prompts. If a set of prepared questions is used, you’ll often see it (or excerpts) in the published study.

Observation – Can be either overt, where the subjects know they are being observed, or covert, where subjects do not know they are observed. Because being observed sometimes changes behavior, covert observation does have a place. Look for a description of how the observations were recorded. Did the observer make notes? Tally the number of times something happened? Video and audio recordings are sometimes used, but how are they analyzed?

Focus groups – small groups gathered together to discuss issues.

3. What was the setting? The setting is important to the generalizeability of the study. Generalizeability refers to whether or not the study can be applied beyond the scope of the study. Looking at how laboring women communicate with hospital staff probably doesn’t apply to home births, and vice versa. Consider cultural issues as well. While a study on how Hispanic women feel about breastfeeding is valuable, it won’t apply to a Muslim or Eastern orthodox woman.

4. How was the data evaluated/coded? In much of qualitative research, the authors do something called a thematic analysis. This kind of analysis looks for recurring themes. Sometimes the researchers will use coding where they have a list of themes they are looking for and will assign these codes whenever they see them. For example, one study (Ayers, S. (2007) Thoughts and Emotions During Traumatic Birth: A Qualitative Study. BIRTH 34:3 September 2007) listed one possible theme of “Negative Emotions” and assigned that code whenever an interview transcript showed words like “scared” “upset” or “shock”. Look into how the process was done. Was there one person doing the analysis or was it done by different researchers? Did the researcher analyze her own data or have a different person do it? Having multiple people look into the data makes for a stronger study. If the authors describe their analysis using terms you don’t understand, look them up.

5. What is the researcher’s perspective? Is the researcher coming from a medical point of view? A public health point of view? A consumer point of view? Consider how this point of view may have impacted the implementation of the study and the analysis.

6. What was the underlying framework?

Phenomenological – this research framework tries to gain insight into other people’s experience. You’ll often see this framed using the phrase “the lived experience.” Here is an example of a phenomenological study.

Ethnographic Research tries to understand various cultures and social systems. An ethnographic researcher might observe a group of labor and delivery nurses to figure out their social system, or interview student midwives about their interactions with preceptors and each other. Here is an example of an ethnographic study.

Grounded Theory goes beyond simply describing and understanding and actually tries to draw a conclusion that is “grounded” in the data collected. Here is an example of a qualitative study using a grounded theory approach.

There are many other possible approaches, including “integrated” designs which may include aspects of various frameworks.

7. How were the subjects / setting chosen? Often with qualitative research the researchers will use a convenience sample, where they look at a population that they have easy access to, or a quota sample, where they look at the first number of subjects that they can find, up to a set amount. Consider how similar the subject and setting are to your practice. The more similar, the more applicable.

8. And finally, what does this mean for me? That will vary widely based on your personal situation. As a reader, you may be a nurse, midwife, childbirth educator, doula, doctor, or parent. You may have more than one role. Carefully think about how this may – or may not – apply to you in your various roles.

Careful consideration of all these aspects can help you come to conclusions about this study and its usefulness. Next up: Questions specific to quantitative research.

Andrea Lythgoe Uncategorized becoming a critical reader, Understanding Research

Quantitative research seeks to prove something through experimentation and statstics. Once you’ve determined that what you have here is an original piece of quantitative research and you’ve already considered the basic questions here and here (insert link to article 4: Original Research), you’re ready for the specific questions:

1. How many groups are compared? Did the authors show that the groups are statistically similar? Look for a table showing things like basic demographic information comparing the groups. Some studies will only have one group. In this case, the authors may be comparing the group at two different times (like before and after a treatment) or they may be comparing the study group to the general population using existing data. If the authors are using existing population data as a control, they should demonstrate that the study group is similar to the general population.

2. Did the authors prove correlation or causation? This is a very important distinction to understand.

Correlation just demonstrates that two things tend to happen together. They could be completely unrelated. To use a fictional example, you might find a correlation between mothers with blue eyes and the number of towels used in the labor room, but it does not mean that the blue eyes are the reason for the increased towel use. A correlation can be positive (when the rate of variable A increases, so does the rate of variable B) or negative (when the rate of A increases, the rate of B decreases). You’ll sometimes hear things described as “associated with” – this is generally referring to correlation.

Causation requires statistics and probability to determine if the connection is likely to be because of the variable tested. Researchers must create two groups of participants who are similar in every way except the intervention that they are testing. This can be done by randomizing participants into two groups or using statistical procedures to control for differences.

(This is very much an oversimplification. I’ll be doing a series on statistics later that will explore these concepts further.)

3. Are the tables, charts, and graphs understandable? Do they relate to the conclusions? Could they mislead someone who does not read the text?

4. Is this study population applicable to my practice or situation? Look at the criteria for including (or excluding) the study population. Read it over and see who the study was done with. A study done only with low risk first time moms may not be applicable to a diabetic woman pregnant with her fourth baby. On the flip side, sometimes studies that look at a specific population can provide very useful and helpful information for that specific population. Just make sure you know what the study population is, and recognize that you cannot accurately apply that information to a wider or different group.

5. Are the findings really significant? There is a difference between statistical significance and clinical significance. A student once showed me a study of castor oil induction where the authors reported a significant difference in APGAR scores between the two groups. While the calculated p value was less than .05, the two groups average Apgar scores were 9.78 and 9.71. The babies in BOTH groups had good outcomes – the difference simply didn’t mean much clinically.

6. Is the study size sufficient? In quantitative research, a bigger sample size usually helps. The more you have in the study, you’re better able to find statistical differences. It isn’t just the overall study size, either. Many studies will run analysis of smaller subgroups. So to use another hypothetical example, a study looking at a new drug might have 5,000 women in the study, but if the authors report that “among women who have 6 or more previous pregnancies, the risk is lowered” – you should find out how many women are in that subgroup. If there were only 15 in that subgroup, it might be hard to make a valid conclusion. It is very common to look at subgroups by the number of previous pregnancies, by race, or by other categories.

7. What is being tested, and what is it being compared to? Some studies will have one or more experimental groups and a “control” group as a comparison. This control group will either have no treatment, a placebo treatment, or the current “standard” treatment. Ethically, you cannot test a new cancer drug by giving cancer patients in the control group no treatment, but you can compare a new drug against the current treatment. Make sure that (within the bounds of ethics) the researchers have chosen an appropriate comparison group.

8. What were the outcomes measured? How were they measured? Every study has at least one independent variable – the thing(s) the researchers are trying to learn about. They choose certain things, called outcomes (or dependant variables) to watch for. An example of this would be an epidural study that compares those who have early epidurals with late epidurals. The timing of the epidural would be the independent variable. The outcomes are chosen by the researcher, and could include things like cesarean rate, epidural complications, APGAR scores, etc. The study should clearly outline which outcomes they were interested in and how they were measured.

9. If applicable, did the researchers do a good job of “blinding”? Blinding is the term for keeping from the study participants and staff which group they are in. This is common in drug trials. Sometimes people are helped by simply believing that something will help – the well known “placebo effect”. If the participant does not know which drug they are taking (experimental drug, standard treatment, or “fake” drug with no effects) the researcher can better determine which effects truly are from the drug being tested.  The staff is also blinded whenever possible to avoid accidentally or subconsciously biasing the result. Sometimes blinding is simply not possible, but whenever possible, it is a helpful technique.

10. And finally, what does this mean for me? That will vary widely based on your personal situation. As a reader, you may be a nurse, midwife, childbirth educator, doula, doctor, or parent. You may have more than one role. Carefully think about how this may – or may not – apply to you in your various roles.

Remember that not every study is perfect. Finding a minor flaw in a study does not necessarily invalidate the whole study. You as the reader need to remember to be objective and ask yourself if the study does a good enough job of showing what it set out to do. Because of our differing perspectives and biases, it is possible to come to a different conclusion than another reader. Also, each study should be considered in the context of all the other research done on the topic. Right now that seems overwhelming, doesn’t it? Our next type of article, literature reviews, will give you insight on how you can view studies in the context of other research.

Andrea Lythgoe Uncategorized becoming a critical reader, Understanding Research

When most people think of reading research, they think of reading an original research study. And there are plenty of those published in journals all the time. Original research generally falls into two main categories: quantitative research and qualititative research. It’s really too bad the terms are so similar, because it can be easy to confuse the two words, when the kinds of research are actually very different.

Quantitative research looks at hard numbers. There will be lots of statistical analysis of data. These are your drug trials, your complication rates, your outcome data. The information the researcher gathers will be counted, measured, or categorized. Quantity is so important in this kind of research, it gives quantitative research its name.

Qualitative research is more about gaining insight, understanding experiences, or learning how societies and systems function. These are done through interviews, observation, etc. It gives voice to the study participants, and very often you’ll find lots of quotes from study participants in the articles. Since qualitative research looks to go deeper into the meaning of events, study sizes are often much smaller. Amy has written a great article http://www.scienceandsensibility.org/?p=1058 on why qualitative research is valuable, and if you haven’t read it, you’ll find it is well worth it.

And then there are studies that are a combination of the two. A good example of this is the recent PLACE study on what happens when you take the bed out of the birth room. In this piece of original research, the authors looked at both quantitative data – the medical outcomes (like cesarean rates, APGAR scores, rates of pitocin use, etc.) and the qualitative data (how the mothers, nurses, and midwives/doctors felt about the change).

This study is also a good example of a pilot study. A pilot study is a small study, not big enough to reach any statistical significance (if it is a quantitative study). The whole point of a pilot study is to find out if the study design will work for a large scale study. Because this was a pilot study, the qualitative portion (which focused on how well this change would be received) was vitally important. In the conclusion, you’ll read:

Our next trial will use a different mattress, which does not promote easier access to the woman (and thus to the application of constraining technologies), but which does make it physically more comfortable for practitioners to give care and for women to use.

This decision was based on the qualitative portion of the study. Without the qualitiative portion, the researchers would not have known that the staff thought the mattress was too heavy. The quantitative portions of the study, (where the researchers looked at birth outcomes, length of labor, etc.) showed promise and no increase in problems. Since the researchers encountered a lot or resistance from staff, they wrote that “the pilot trial results may increase the comfort level of other staff members.”

Another way to look at the study is to decide if it is a prospective study or a retrospective study. A prospective study is one that starts before the variable being studied. Let’s use the topic of pitocin induction as a variable.

A prospective study of pitocin induction would start while the moms are still pregnant. Researchers would recruit the women as study participants and divide them into groups. Some would be induced, some would not, and the researchers would collect data before, during and after the induction. A prospective quantitative study would look at the data on things like timing of the induction, length of labor, cesarean rates, etc. A prospective qualitative study might interview the mother before and after, or record the birth and analyze what she says about her experience during the labor, etc.

A retrospective study is one that begins after the fact. It looks back on what has already happened. A retrospective quantitative study on pitocin induction might go back and look at all the births in a certain hospital over a 3 month period of time and look at those induced by pitocin to spontaneous labors. Researchers could compare the timing of the induction, length of labor, cesarean rates, etc. A retrospective qualitative study might find 15 women who experienced a pitocin induction in the last year and ask them to describe their birth experience to the researcher.

Both prospective and retrospective studies have their strengths and weaknesses, and we’ll get more into that in our methodologies section, but for now, just think about the two as you read original research.

In summary, with original research, you’ll want to determine if the study is:

• Qualitative, quantitative, or a combination
• Large enough to reach a statistical conclusion or a pilot study
• Prospective or retrospective

In our next two articles we’ll take a closer look at analyzing qualitative and quantitative research.

Andrea Lythgoe Uncategorized becoming a critical reader, Understanding Research