Science & Sensibility

Pretty much everyone would agree that there is bias in research. Most people would say that bias is inherently bad. While it absolutely can be a bad thing, it can’t be completely eliminated. So what can be done about bias in research?

There are many kinds of bias:

• Researcher bias: researcher sets out wanting to the study to prove something, and intentionally or unintentionally manipulates the study to make sure that happens
• Sponsor bias: The organization that sponsors the study either encourages researcher bias or manipulates the publication of the data. Some studies might be completely suppressed, some might have overly inflated press releases touting minimal results.
• Publication bias: Journals must be selective in what they publish due to space limitations, but I think it is fair to say that some journals may choose not to publish a study that might anger its audience.

But today I want to focus on READER bias:

Your first job in the critical reading of an article is to check your bias. We are all human, and so we all have bias. Sometimes it is hard to see your own biases. Take a look at the pictures below. In the first picture, we can tell that there is something there, but it is difficult to see. In this case, the letters are lined up with our angle of vision.

In this second picture, the letters are running the opposite way as our line of vision, and as you can see, suddenly that bias is crystal clear!

The same is true with our reading of the research. The biases that we have act as a filter that alters our reactions to the research. If we already have our minds made up that induction of labor = bad, then any research on labor induction is going to be seen through that filter. Any research that seems to place induction in a favorable light will be seen has highly suspicious. Any minor flaws will be exaggerated. Any research showing bad outcomes from inductions will likely get a “free pass” and flaws may be overlooked.

Murray Enkin, author of “A Guide to Effective Care in Pregnancy and Childbirth”, said this:

Perhaps the most important bias of all resides in the (potential) reader, who determines how (or if) the results will be read and interpreted.

I would agree with him. I have, over the years, seen the best and worst of research used to back up various points, ignoring the quality (of lack of it!) as long as it agrees with them. This is a normal human tendency, and one that is at the heart of many discussions about the available research.

But the good news is that reader bias isn’t impossible to overcome.

The solutions? Awareness of bias and a change of perspective! As you read, consider how this research might be read and understood by someone with a completely different perspective. When you read a study that really resonates as a great study with you, play “devil’s advocate” and pick it apart. Be merciless in looking for flaws, weaknesses and the other types of bias listed above. The same is true of seeing an article you disagree with. Look for strengths and solid evidence. Have an open mind to other possibilities. Sometimes when doing this, you’ll be able to see some aspects you would never have noticed otherwise.

So, here’s an exercise for you. Take a few minutes, and write down what your biases are when it comes to research. Which kinds of research, which methods, which topics do you particularly feel drawn to? Which ones seem silly or useless? For inspiration, you may want to read a personal commentary article written by Murray Enkin (2008) where he goes through his own personal biases. The things he feels a bias for or against may not be the same for you. I know I have a disagreement with one of his stated preferences. But taking the time to carefully think through your own personal biases, to clearly acknowledge the filters through which you view the research, can only help you as you try to step back and make a critical analysis of the research.

Reference: Enkin, M. W. (2008) Biases in evaluating research: Are they all bad? Birth: Issues in Perinatal Care. 35(1). 31-32.

Andrea Lythgoe Uncategorized becoming a critical reader, bias, Understanding Research

We’ve finished our series on finding research articles, the next series will focus on how to critically read articles in order to evaluate them. I think we all know there is good research and bad research, so now we’ll explore how to tell the difference.

You’ve found an article that looks promising. Now what?

First, you’ll want to give it a quick read-through. You’ll be going back later to do the real critical analysis, so don’t get bogged down in the details at this point.

Start with the abstract, a short summary of what the article contains. The abstract will give you a brief overview of the article and its contents. Keep in mind that abstracts are not comprehensive, and are written by the study authors. They are in a sense similar to theatrical trailers. You can often get the basic gist of what the article is about from the abstract, but just like a well-done movie trailer can make a bad movie look good, you can’t gauge the quality of an article by the abstract alone. Read the whole thing!

You’ll often hear references to “studies”, “the research” or “the literature” – but not all articles published in professional journals are the same. In this initial read-through, you’ll want to determine what kind of article this is and if it fits your needs.

Among the kinds of articles you may find in professional journals are:

Literature Reviews – These articles attempt to do a search of all available research on a particular topic and summarize the research for the reader.

Case Reports – Generally these articles focus on more rare events or novel experiments. They may help the author find others who have experienced similar rare events, or “show off” the author’s creative solutions to such rare events.

Opinion piece – In these articles, the author shares a personal opinion on any topic or thoughts on a piece of research. It is not uncommon for the editor of a journal to publish a comment piece on a new study being published in that same issue.

Original research – This is generally what people are referring to when they talk about “studies”. In original research, the authors have done something from scratch. This may be an experiment that was conducted, a new analysis of information previously gathered (like data extracted from chart reviews), interviews, surveys, etc. We will go into much more detail on original research in future articles.

Systemic review/Meta-analysis – We will get into the details of these in a later article, but for now, just know that these kind of articles take the data from previously published studies and analyze that data to come up with a conclusion. The most well-known of these types of articles are the Cochrane Reviews.

Next you’ll want to make sure you’re clear on one thing: What did the authors set out to do? When you do the critical reading, you’ll be keeping that purpose in mind as a measuring stick of sorts to see how well the authors or researchers accomplished that goal.

And that’s it for the initial read through! Now that you have an overall sense of what the article you’ve found is all about, we’ll get ready to do the real critical analysis.

Tomorrow: how to detect bias in research!

Andrea Lythgoe Uncategorized becoming a critical reader, Understanding Research

While I’ve spent a lot of time teaching about databases and indexes, they certainly are not the only way you can track down studies. In this final installment of the tracking down studies part of the Understanding Research series, I’ll show you how you can track down studies in other ways.

Last May I read an interesting blog post on Teri Shilling’s blog.

She posted about some research she’d read about in a magazine. The study had found that attending a childbirth class was the only significant predictor of whether or not a woman was satisfied with her childbirth experience.

Naturally, I wanted to know more, and being the “go straight to the source” type, I wanted to see the actual study. Teri had listed a few key pieces of information:

Study author = Angelina Arcamore

Study author’s place of work = Villanova University

Teri’s source: CBEreporter article written my Marsha Rehms

I started by asking Teri what she knew.

Next I tried doing a search in EBSCO and PubMed for the author. Nothing. Tried searching for  the key words “Childbirth Education” and “satisfaction”. A few results, but nothing like what I was looking for.

Did a general Google searches for “Angelina Arcamore”, “CBEreporter” and for “Marsha Rehms”. Nothing helpful there.

So my only remaining clue was Villanova University. I went to their web site:

I searched their site for “Arcamore” and got nothing. Then I tried “childbirth education” and got a single hit:

And voila! I found the faculty bio page of Angelina Arcamone!

Notice her name is actually spelled “Arcamone” instead of “Arcamore”! Somewhere along the way, a letter got misread as an r instead of an n. Very easy mistake.

So then I go back and do EBSCO, PubMed and Google Scholar searches for the correctly spelled name.

Still no results.

But remember how the faculty bio page had an e-mail address on it? Well, I decided to write to her and ask about the study. Within 24 hours she responded with a very nice e-mail:

While I found the information she sent me to be helpful, I am looking forward to seeing her study published!

Did you like this step-by-step of how I tracked down this study? You can find more articles like this on my web site.  If you have a study you’re having trouble finding and would like me to give it a try, send me an e-mail andrea [at] lythgoes [dot] net and I just might do it here on Science & Sensibility!

Andrea Lythgoe Uncategorized childbirth education, tracking down studies, Understanding Research

This article is part of the Understanding Research series.

So you found a few good articles, but you wish you could have found more. Sometimes all it takes is finding one good study, and you can use that one to find more!

There are several ways to do this:

1. Most studies will have a section discussing the literature previously published on the topic. Dig up some of those studies using the information listed at the end. And then check the bibliographies of THOSE studies, too. In the research world, this is called “backwards chaining,” and works best with a more recent study. Sometimes you’ll get lucky and in the database you’ll see a list of the studies cited by the one you’re looking at, like this list of references connected with a study I found on EBSCO:

2. Unless your study is brand spanking new, there’s a good chance someone else has cited it in a newer study. So as you’re searching, watch for places where you can see a list of studies that cite the one you’re reading. Google scholar nearly always includes a “Cited by” link under each search result. Meta-analyses and systemic reviews are GREAT finds if you can find one, as they tend to look at many studies all at once. This particular example of studies that cite the one I was interested in comes from EBSCO:

3. In the database where you found the study, look at the “related articles” that share key words with your article. Google Scholar and PubMed generally have excellent links to related articles. They may be articles that share keywords, authors, etc. with the study you are looking at. PubMed flags any articles that are a critique or review of another article as well.

4. In most databases, the author’s name is a link. Clicking on it will search for any other articles by that author. Since researchers sometimes tend to focus on particular topics, you can often find similar studies. Sometimes this works very well – as in the case of someone like Susan Ludington-Hoe, who has published quite a bit on the topic of kangaroo care. She also has a very distinctive name. If you find a study by someone with a more common name, like say MJ Anderson, there may very well be more than one of those. You could revise those results or combine an author search with a keyword search in this case.

I’ve said it before in this series and I’ll say it again. The BEST way to get good at finding articles is practice, practice, practice! Try different indexes, try different approaches, and talk to a librarian for the tips and tricks!

Andrea Lythgoe Uncategorized tracking down studies, Understanding Research

This article is part of our Understanding Research series.

MeSH – maybe you’ve seen it, that weird word that is always in caps, except for the little e. What is it and why should I care? Well, MeSH is an acronym for Medical Subject Headings, and in essence it is a way of having a standardized set of keywords that everyone uses to index research. It is run by the US National Laboratory of Medicine and is updated once a year.

It can be used a few different ways.

First, you can use it as a thesaurus to find out other terms you might use for your searching. So if you are searching for postpartum depression, you can enter that term here and find something that looks like this:

You can see the official MeSH heading of “Depression, Postpartum” and below it several other Entry Terms that could be used as search terms. You can also see that before 1995, the indexing for postpartum depression was different than it is now. If you’re looking for studies older than 1995, using those as keywords may be helpful as well. (Though in this example, they are probably too broad!)

MeSH keywords are organized in a tree with similar keywords near them. By clicking on the “Concept View,” you can see how postpartum depression fits into two different areas, pregnancy and mental health:

Searching the MeSH keywords can be an excellent way to find the generic and alternate brand names for drugs. Look at all the possible keywords that come up in a search for Cytotec:

But where MeSH keywords really shine is within PubMed!

For example, if I wanted to find studies on the risks of induction of labor, I could do a simple PubMed search like this:

2440 results is a lot to slog through, and a quick scan of the first page of results shows things like a case report of a second trimester uterine rupture and an analysis of the risk factors for bladder injury during a cesarean section. So you’ve got a lot of stuff that really doesn’t apply to what you want to know.

So let’s try using MeSH! Click on the MeSH Database in the left hand sidebar and do a regular keyword search there.

In the MeSH database, I did a search for “Induction of Labor” and this is the MeSH result:

You can see that the MeSH keyword is a little different than my search, as the result is for Labor, Induced instead if Induction of Labor. The MeSH term is defined so you can ensure you’re at the right result. In this case there was only one MeSH keyword result, but it is possible to get multiple results and the definition can help you ensure you are looking at the correct one for your search.

Once you’re at the keyword entry that is the right one for your search, you can click on the “Links” in the upper right hand corner to complete a PubMed search for all articles that have this keyword. When I did this, I got 7,064 results – that’s 354 pages of citations to wade through, far more than I want to do. So I go back and try it again, this time using the “major topic” version of the PubMed search, so that I only get articles where induced labor is a major topic. This narrows it down to just under 4,800 results. Still too many!

But notice those subheadings just below the definition? Add a checkmark to the most applicable subheadings and then use the dropdown menu below to send these subheadings to the search box.

Your results will be limited by the subheadings you chose. Maybe you see more than one key word that fits what you want to know. Go ahead, click all the ones that fit.

For our example, I chose “Adverse effects” “Complications” and “mortality” as the three that I wanted to search. I can choose to add them to the search box with any of the major operators: AND, OR and NOT. I’m going to choose OR to get results that include any one of “Adverse effects” “Complications” and “Mortality.” I also checked the box that limited the results to major topics only.

Now I have 279 articles with the main topic of complications, adverse effects, or mortality in induction of labor! Much better than the 2,440 results we got before – far less to wade through, and they are much more on topic.

Next time: Using one great study as a tool to find more like it!

Andrea Lythgoe Uncategorized tracking down studies, Understanding Research