Vaccines, Climate Change, and the Misuse of "Research"

Because of the increase in measles cases across the United States, a lot has been written recently about vaccines and the anti-vaccine movement. I've avoided blogging on the subject, partly because the discussion has been a lot of heat and not very much light and partly because it's not so much a debate or a conversation as a minority of folks screaming at the top of their lungs while holding fingers in their ears lest they listen to anyone else. Frankly a lot of it is pretty ugly, so while I have taken some pains to make sure I read some "authentic" voices on the anti-vaccine side, on the whole it all just makes me sad.

I do think that the continual parade of claims and counter-claims, of facts and factoids and made-up nonsense, is counterproductive because it isn't an argument about science or what's true. The primary drivers, as they so often are, amount to tribalism - people protecting their self-identities. I read this piece the other day and was glad to find a medical professional who gets this. Click the link - it's well worth a read.

So what do I have to add to this conversation? I want to focus on the use - and more particularly the misuse - of one word, "research". I have seen this word misused in recent weeks by anti-vaccine parents, I have seen it misused in exactly the same way by climate change "skeptics" in recent years, and in similar fashion by partisans on any number of issues over time.

As an illustration, let me borrow words from an anti-vaccine parent posted to the internet (underlining added):

You want to vaccinate? Go for it. I choose not to. But shouldnt we all have the freedom to choose what's right for our family? My unvaccinated child is not putting your vaccinated (and therefore protected-right?) child at risk. In fact it's the opposite. Your vaccinated child sheds his/her vaccine for days putting my child, and others who cannot vaccinate due to medical reasons, at risk. But I'll assume that risk and hope his immune system is strong enough to protect him. You do your thing. I'll do mine. Just stop assuming I don't care about others, because I do. I care enough to watch Vaccine Nation to understand herd immunity and research vaccines for myself. I've read the package inserts from the manufacturer websites. Have you?

Now, to be clear - I do not doubt this person's sincerity or motives. I believe this author genuinely believe these words. The problem here is a fundamental misunderstanding of the term "research".

I see this claim a lot, not just from anti-vaccine folks but from climate change skeptics, gun-rights advocates, and many other partisans for various issues. They will claim to have "researched" something to come to their own conclusions - a subtle cultural argument that appeals to Americans' individualism. After all, who wouldn't want to come to their own conclusions instead of being one of the "sheeple" who follow the herd, right?

The problem is that what these people call "research" consists only of reading stuff that other people have written. The most generous thing that could be said is that they are engaged in literature review - reading the research of others in order to get a sense of what's already out there in the field. Lit reviews, as anybody trained in any academic or scientific field knows, are a necessary precursor to doing research. But they are not research itself.

Actual research involves:

• Formulating a falsifiable hypothesis
• Designing a legitimate and non-biased test for that hypothesis including valid and reliable operational measures for the key concepts
• Gathering original data to perform that test
• Analyzing the results
• Carefully drawing conclusions by comparing the data with the hypothesis
and then (the gold standard)
• Allowing the entire process to be author-blind reviewed by other established experts in the field who are qualified to judge the work.

The parent cited above, and all of this ilk, haven't done any of this. All they have done is read some other stuff that other people did, usually based on non-falsifiable hypotheses to begin with (for example, "the CDC is the center of a global conspiracy to cover up the truth.")

Moreover partisans on a given issue don't even do lit review right, because they only read or watch the stuff that already agrees with their point of view. They will occasionally cite a peer-reviewed article, often misreading its conclusions in the process (since they are rarely, themselves, actual experts in the field). They dismiss studies that draw different conclusions and evaluate studies with the grossest and most obvious of double standards. Watching a documentary film made by this man hardly qualifies as a thorough review of the relevant literature on any subject.

This, of course, is why actual expertise is worth something. People who are qualified to conduct actual research in a given field have to spend years learning how to do it right. Because of my education, I am qualified to do research on political phenomena. I am not qualified to do research on vaccines. And neither are nearly all of the anti-vaccine voices out there.

So the next time you run into someone spouting what appears to be a questionable point of view on a given subject and claiming to have "done their own research", call them on it. Ask what that "research" consists of. Ask what data they have collected, what testing and analysis methods they used, and what hypothesis they were testing. I expect the answers you get will be much more heat than light, because this isn't really about science. But at least people should admit that.