Why Experience Is Not Evidence

I’ve noticed that when I’m conversing with someone who believes something non-scientific (vaccinations are dangerous, for instance, or herbal remedies are safer than pharmaceutical drugs), the rationale they give to support their belief system generally starts along the lines of, “I knew someone who…” While it’s tempting to build our beliefs on the sum of our experience (and the experiences of others), our experiences don’t count as scientific evidence.

Here’s an example to illustrate my point. My experience is that the Earth is flat, and that the sun moves around the Earth. When I look outside, I see a flat horizon. I can watch the sun come up in the morning, move in a half-circle, and then go down at night. Clearly, the sun is circling around a flat planet. These observations of mine aren’t unique; the geocentric universe was so accepted at one time that to claim otherwise was considered heresy. It wasn’t until the 16th century, and Ferdinand Magellan’s circumnavigation of the planet, that the last of the semi-plausible arguments for a flat Earth subsided (though fringe fanatics persist to this day).

Similarly, what my experiences — or yours, or your brother’s, or a friend of a friend’s — teach me about other topics in science may or may not lead me to valid conclusions. Anti-vaccination websites are rife with stories of severe adverse reactions to vaccines; these self-reported events help to fuel the scientifically-unfounded anti-vaccination movement. However, there are logical fallacies associated with accepting anecdotal evidence in scientific decision-making. For instance, the fallacy of post hoc ergo proctor hoc is Latin for “after the fact, therefore because of it.” We fall victim to this logical fallacy when we believe that event B, which followed event A, was caused by event A. Another way to state this is correlation is not causation. If you hear of a case in which a child had his or her 12-month vaccinations (the first ones to include the oft-maligned measles, mumps, and rubella or MMR shot) and then developed autism symptoms, you’d be tempted to ascribe the symptoms to the vaccination. You’d be particularly inclined to do so if the child in question were yours, a relative’s, or the child of a close friend, because you’d be looking for a cause outside genetic predisposition. However, there are very few reliable early indicators of autism, and the disease generally goes undiagnosed until at least a year of age, per the National Institute of Neurological Disorders and Stroke. This means that in almost all cases of autism, the diagnosis will follow administration of the MMR.

Another problem with accepting self-reported experience as evidence is that self-report is a notoriously faulty way to gather information, due to differences in the ways that we each interpret and communicate what we experience and observe. For simplicity’s sake, I’ll stick with anti-vaccination sentiment. Parents who choose not to vaccinate may state that they’ve heard about too many bad vaccine reactions to justify vaccinating. Many of these “bad vaccine reactions,” however, are perfectly normal immune system responses, and pose no harm whatsoever to the vaccinated child. Fevers are normal after vaccination, and occur occasionally after a shot — for instance, they affect about 15% of children after an MMR. To refer to a fever as a “bad” reaction is to misunderstand the way the immune system (and vaccination) works. A vaccination stimulates the immune system to learn to recognize a particular pathogen, so that later exposure triggers immune “memory” cells that can quickly respond to the threat and prevent illness. When your immune system is activated, you’ll often develop a fever; this is your body’s way of helping the immune cells out, as many viruses and bacteria proliferate less effectively at elevated temperature. Similarly, muscle soreness, crankiness, headache, and many of the other mild side effects of vaccination aren’t signs that something’s gone wrong — they’re signs that everything is working as it should. The U.S. Centers for Disease Control and Prevention (CDC) — a government agency that in no way profits from vaccination development or administration — maintains data on adverse vaccine reactions. Truly bad vaccine reactions are exceedingly rare. The most common of these are allergic reactions, which vary in their prevalence with the type of vaccination. In the case of the MMR, they occur in about one child for every 1,000,000 immunized (a very small risk, particularly in light of the fact that three of every 1000 children who get the measles die of complications due to the disease). Other truly awful adverse reactions to vaccinations, such as life-threatening seizures or brain damage, occur so rarely that there’s no way to be sure such reactions are attributable to the vaccine at all. Think of it this way: you could get hit by a meteor after being vaccinated, but that wouldn’t mean that the vaccine caused the meteor strike. Similarly, just because a child has seizures in the days after a vaccination doesn’t mean the vaccination caused them.

Some anti-vaccination websites cite reactions reported on the Vaccine Adverse Event Reporting System (VAERS), another government service. It’s important to note, however, that anyone can enter anything — verified or not — into the VAERS database. Dr. James R. Laidler, for instance, rather infamously entered into the VAERS that an influenza vaccination turned him into The Incredible Hulk. He was contacted (eventually) by a representative from the database, who suggested that his reaction was quite unusual and asked his permission to delete it. Had he refused, the report would have remained. Bottom line is that the VAERS may be full of fallacious reports (intended or not), as it is an open database.

The point here is that experience — even when that experience appears completely legitimate — isn’t the same as scientific evidence. I experience that the Earth is flat, but I have to trust in the evidence that it is not. Similarly, while friends, acquaintances, and people on the Internet report adverse reactions to vaccines that scare me (and force me to remind myself of all the safety and efficacy data that supports vaccination every time I take my baby to the pediatrician), I have to remember that the post hoc ergo proctor hoc fallacy is common, meaning a bad bout of teething replete with days of crying and sleeplessness could be mistakenly attributed to 6-month vaccinations — and reported as such — by worried parents. Similarly, a parent’s natural concern for their child’s well-being may skew their perspective and could cause overstating of a mild fever or post-vaccination crankiness. Experience is interesting, and if enough people have the same experience, it warrants further investigation. Even collective experience, however, doesn’t replace true scientific evidence — with the exception of a very few astronauts and circumnavigators, we’re all just taking it on faith in science that the Earth is round.

 

What do you think it takes for experience to count as evidence?

 

References

Laidler, J. Chelation and Autism. Posted 27 July 2005, Accessed 16 Sept 2011.

National Institute of Neurological Disorders and Stroke Autism Fact Sheet. Accessed 16 Sept 2011.

U.S. Centers for Disease Control and Prevention Possible Side-Effects From Vaccines. Accessed 16 Sept 2011.

Vaccine Adverse Effect Reporting System (VAERS)

Advertisements

14 Comments (+add yours?)

  1. D Hidenindeflor
    Sep 19, 2011 @ 05:13:21

    if experience is not evidence, then there is no such thing is evidence, because an observation, the core of the scientific method, is an experience by the observer…hence, giving birth to paradoxes like Shroedinger's cat or the gold foil experiments w/sub-atomic particles being both wave and particle depending on the observer. If you count no experiences as evidence, then there is no way to follow the scientific method and therefore, no way to know anything in the world.

    Reply

    • SquintMom
      Sep 21, 2011 @ 23:35:57

      I appreciate your comment, and see where you're coming from. In the interest of a short title, I went with "Why Experience Is Not Evidence," but perhaps a more accurate (albeit non-user-friendly) title would have been "Why Personal Experience Isn't Necessarily Representative of Greater Scientific Truth, i.e., Evidence." The core of the scientific method depends upon converting experience into evidence through inductive and deductive reasoning based upon reproducibility and the potential of the experience to be generalized to other situations and/or other observers.It's not that no experience is evidence. Rather, it's that not all experience is evidence, and the two terms can't and shouldn't be used interchangeably.

      Reply

  2. Philippos
    Sep 19, 2011 @ 21:55:33

    I found out about your blog from Ben E. Thanks for doing this.

    Reply

  3. Anna
    Sep 22, 2011 @ 04:33:34

    Check out Paul Offit's new book, deadly choices. I guarantee you will love it!

    Reply

  4. Ladygoogoogaga
    Oct 23, 2011 @ 00:28:06

    Wow, you are super smart…..I don’t know what else to say except that if there is any evidence AT ALL that a vaccine could cause a problem with my child, then I cannot risk it…..

    Reply

    • SquintMom
      Oct 23, 2011 @ 02:28:20

      Thanks for your comment! I agree that the evidence suggests that a vaccine poses a small risk, but there’s also no doubt that the diseases themselves pose a larger risk. For instance, as pointed out in the article linked to below, the risk of death from measles is NINE TIMES HIGHER than the risk of a moderate, non-fatal side effect from the MMR vaccine. This holds true for other vaccines as well; in all cases the risk of death from the disease itself is many, many times higher than the risk of even a moderate vaccine side effect, and the risk of death from the disease itself is higher by many orders of magnitude (thousands, tens of thousands, hundreds of thousands of times higher) than the risk of a severe vaccine side effect. In a perfect world, we’d be able to keep our children from ever being exposed to any risk, but unfortunately, the world doesn’t work that way, and risk is inevitable in any course of action or inaction. As such, it makes sense to select carefully the risks we take, and choose the least risky course.

      http://www.highlighthealth.com/resources/anti-vaccination-movement-endangers-public-health/

      Reply

  5. Trackback: Taking The Bull By The…(In Support Of Moms Who Circumcise) |
  6. Megyn @Minimalist Mommi
    Nov 23, 2011 @ 22:07:59

    I think I get frustrated with this too when “normals” present scientific data as truth. I often see other moms presenting some anecdotal “evidence” along with some scientific paper summarized to prove their point. When I’ve actually read the journal article (where most are correlational), I wonder how these people can jump to such conclusions. Thank goodness for people like you and the scientific method!

    Reply

  7. Trackback: Non-Vax Nonsense | SquintMom.com
  8. Trackback: Should I Vaccinate My Child for Chickenpox? | SquintMom.com
  9. Trackback: What The Science Says About Circumcision: Part 2 — The Risks | SquintMom.com
  10. Trackback: Should I Vaccinate My Child for Chickenpox? | Celebrating Entropy Should I Vaccinate My Child for Chickenpox? | disorder. can't avoid it…might as well enjoy it!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: