Antivax: Sometimes what you omit defines the most effective lies

As ever, the twitter backlash against the Australian Vaccination Network continues. One particular item has grabbed my attention. @nocompulsoryvac, a twitter account run by Meryl Dorey on behalf of the AVN, tweeted this little gem today, on a claimed 155% risk increase for Autism.

155% greater risk of autism in vaccinated boys vs the unvaccinated. When is the medical community going to pay attention?

Well, this piqued my interest. That's a fairly spectacular figure, representing a significant 2.5x increase in risk. Wow. And it's quite a specific number too. Must be conducted in quite a rigorous manner, eh?

Well, no. Not really.

You see, even though Meryl left that little gem unattributed*, I decided I was going to hunt down this data point and see if it was really all it was cracked up to be.

So off I went to my favourite search engine, and typed in "155% autism risk vaccine". And what do you know? I found the "study", first result out of the gate.

It turns out that the rigorous, statistically controlled, epidemiological study in question was... a phone survey. By a marketing company. Funded, to the tune of US$200,000 by... Generation Rescue, a partisan antivax organisation founded by a ditzy celebrity and dedicated to proving the MMR/Autism link by any means possible.

Oh I'm sorry, did I say all that? I meant concerned parents' group honestly investigating the MMR/Autism link.

DING-A-LING-A-FUCKING-LING

So aside from the immediate issue with how a poorly designed phone survey somehow gets conflated with real medical research, there are other issues. It seems there was no medical follow-up to confirm diagnoses, no control for known risk factors such as past familial neurological trouble, no reporting on exactly what the poll questions were, and finally no review of the data by any other group - all of which raise serious flags about the reliability of said "study"

But do you know what the best part was?

No, really? Do you want to know?

Well, it seems Meryl can't even report her own partisan studies correctly. The actual results are summarised in the article thusly:

-- Among more than 9,000 boys age 4-17, vaccinated boys were 2.5 times (155 percent) more likely to have neurological disorders, 224 percent more likely to have attention deficit hyperactivity disorder, and 61 percent more likely to have autism.

-- For older vaccinated boys in the 11-17 age bracket, the results were even more pronounced. Vaccinated boys were 158 percent more likely to have a neurological disorder, 317 percent more likely to have ADHD, and 112 percent more likely to have autism.

I've highlighted where Meryl fucked up, not because I think that you, dear reader, are a bit thick, because you're not. You're smart and good looking, and probably get far more casual sex than the average person. I highlighted in in case Meryl comes by, because, you see, she needs help. Meryl's target figure (autism) was 61%. 155% referred to all 'neurological disorders', including the rather fluffily defined ADHD.

Now, it's been a while since I did Statistics 101, but the numbers also seem a little skewed, and oddly reported. The two age groups quoted, for example, are actually just one group, with a subset pulled out to highlight the intended result. What we have is 4-17, and the subset 11-17. What are the numbers for the 4-11 group? We're not told, but we can extrapolate, and someone clever with Maths could probably tease out the numbers. Yes, folks, the 4-11 numbers WOULD be lower, significantly lower, than either of the two groups we quoted. There's no way around that. If your overall sample set has a 61% rate, but one half has a 112% rate, what does the unreported group have? That's right, an even lower rate**. So MMR (and other vaccines) are administered around school starting age, and then very little happens until the age of 11. That's stretching post-hoc ergo propter hoc a bit far, even for Meryl.

Secondly, what we're quoting here is a percentage increase in a rate that itself is not reported. Dr Ben Goldacre devotes an antire chapter of "Bad Science" to "Bad Stats", and this is precisely where he starts out. The number best suited to this kind of reporting is the 'Natural Frequency', that is a number such as "1 in 5000 people". However it's far more sensational, if your study gives you small results, to report the 'relative risk increase'. Let's just break this into another paragraph to make this a little easier, and just make up a hypothetical study of 10,000 subjects

  • Background rate of event: 1/10,000, or 100 per million
  • Reported rate for study: 3/10,000, or 300 per million
  • Relative Risk Increase: 300%

Fuck! 300%! Call the cops! Call the fucking coastguard! Do something! That's a 300% fucking increase!!! Aaaaaaagh!

No, wait. That's an increase of only two reported cases per 10,000 results. We just managed to find two more people than we expected by chance. Out of 10,000 people. That's almost certain to happen if you run more than one study.

This is entirely statistically insignificant. But it's a 300% increase. That's even more then Generation Rescue found in their study. Holy crap, I'm onto something!

Not only that, given that the incredible publicity being generated around the purported vaccine/autism link, I would positively expect misreported results in this kind of survey. A kid that misbehaves a little, in the mind of a mother exposed to too much Jenny on Oprah, is suddenly an ADHD case. A socially awkward kid is suddenly autistic. You can see where I'm going here.

In fact, given the methodology and the background surrounding this specific manufactured 'controversy', I'm really, really surprised the numbers weren't even more skewed.

So, in summary, Meryl is lying again. SURPRISE!

 

* If there's one thing antivaxers are good at, it's making assertions without attribution. For a period on the AVN's FaceBook page, I made it my business to correctly attribute quotes lifted wholesale by AVN supporters.
** Math geeks, I'm not sure I got the calculations correct, but is it possible there would be a negative result in the lower group?

 

posted @ Monday, August 17, 2009 5:37 PM

 
 
 

Comments on this entry:

# re: Antivax: Sometimes what you omit defines the most effective lies

Left by Jason at 8/17/2009 5:49 PM
Gravatar
OK, I'm just doing a little math on Gen Rescue's figures here. Bear with me if this looks a little abstruse, but as I said earlier it's been a while since stats 101

Let's assume the GR stats are correct.
Let's assume the cohorts

overall cohort: 10,000 subjects
4-11 cohort: 5,000 subjects
11-17 cohort: 5,000 subjects

Let's further assume that the previously reported rate of autism is 10 incidences per 10,000 subjects, for ease of doing the numbers.

GR reports a 61% overall autism increase for the entire Cohort, and a 112% increase solely for the 11-17 sub-cohort

so our overall hypothetical cohort shows a rate of 16 incidences of autism.

but our upper cohort of 5000 showed an increase of 112%, or 5*112%, or 11 when rounded

so the overall group had 16 cases, which appears above average. But the lower cohort? EXACTLY THE EXPECTED FIVE.

No wonder they left it out. Now I know this is proving my point with hokey maths, but since I don't have the real figures, this is all I've got

comments?

# re: Antivax: Sometimes what you omit defines the most effective lies

Left by DrRachie at 8/17/2009 6:41 PM
Gravatar
Gee, you say she didn't reference her figures? That's funny cause earlier today she said;

"Without a reference, there is no way to say where this is sourced. While the AVN always refers to references, unfortunately, the media and the medical community don't in general seem to adhere to the same practices".

Lying again?

# re: Antivax: Sometimes what you omit defines the most effective lies

Left by Jason at 8/17/2009 6:59 PM
Gravatar
Here's the link to Generation Rescue's reporting of the survey.

http://www.generationrescue.org/survey.html

One part I find interesting is a quotation from a CDC spokesperson, which seems to add some validity to the methodology, however if you parse the sentence correctly, it does nothing of the sort.

"the consistency of prevalence estimates across the two surveys supports high reliability or reproducibility of parental report of autism and reliability is one important component of validity."

Note the bit I bolded. All this says is that one phone survey gives much the same results as another phone survey. It says nothing about how the responses compare to actual diagnoses. It just means the survey firm was honest about conducting the survey, NOT that the parental responses lined up with an actual medical diagnosis.

GR even use it in their criticisms/responses section, dishonestly

"Criticism: Parent responses is not a reliable way to gauge either a child's diagnosis or whether or not a child has been vaccinated.

Response: We would point to our "Methodology" section above and cite the CDC, who also uses a parent phone survey to gauge prevalence of NDs in children. We generally mimicked their approach. "


So what if you mimicked their approach? The answer still does not address the criticism that parental report is no substitute for proper diagnosis, especially in a media saturated world with Jenny on Oprah convincing nervous parents that their kids may be sick because of vaccines. Just because the CDC says one phone survey gives similar results to another, doesn't mean that the results in any way line up to medical fact.

All it means is one set of parental reports is much the same, statistically, as another set. Frankly, the CDC statement is being misrepresented.

# re: Antivax: Sometimes what you omit defines the most effective lies

Left by DrRachie at 8/17/2009 7:00 PM
Gravatar
Yeah Jason you can't be right here;

*If there's one thing antivaxers are good at, it's making assertions without attribution.

Because of what I said earlier (see above).

Meryl told me that vaccines cause they disease(s) they are supposed to prevent, that they contain toxic levels of poison, that they are the greatest hoax of the 20th century, that they don't work, that they cause eczema, asthma, autism, SIDS, shaken baby syndrome and she got it all from medical journals.

She says so here:

"All of our information is sourced from medical journals, referenced. We certainly do not give out misinformation." (1)

and "I don’t agree with the Skeptics that the AVN are not giving factual information, because that’s all that we provide. All out information comes from mainstream medical sources.." (1)

Radio interview, North Coast Radio, August 6 th 2009.

See? She gets it from mainstream medical journals. She says so, on radio. Must be true.

# re: Antivax: Sometimes what you omit defines the most effective lies

Left by DrRachie at 8/17/2009 7:56 PM
Gravatar
Meryl knows all about conducting scientific studies, see the following;

"...we surveyed our readers, ...we asked a whole series of questions....whether their children were vaccinated, how many of them were vaccinated, how many of them were unvaccinated, and what the incidence of asthma and other conditions were.. the final number was that the children who were vaccinated were six times more likely to be asthmatic than the children who were unvaccinated.

And we tried to get that published in the Medical Journal of Australia, but they wouldn’t publish it, they wouldn’t even look at it. And it was done very scientifically, we actually had a scientist doing the entire study, writing the questions, tabulating it, used a special computer program for this so it would be fit for publication, but they wouldn’t publish it.

And this is what happens, unfortunately. When something comes out that questions medical procedures, it doesn’t get published in medical journals".

(Transcript, webinar, "Meryl Dorey - Voodoo Children", July 31st, 2009).

Imagine that! A special computer programme! To make it fit for publication! And a scientist was somewhere near the computer! WOW, sciencey!

# re: Antivax: Sometimes what you omit defines the most effective lies

Left by Jason at 8/17/2009 7:59 PM
Gravatar
I now have the PDF of the poll questions, and frankly despite GR's satement's to the contrary, they do lend themselves to self-selection. First question out of the trap:

"This is SurveyUSA calling Sonoma County parents with a private, confidential survey about vaccinations and children's health"

And the hot button issue with vaccinations and children's health right now is... guess what? Oh yeah, Antivax paranoia. How much more likely are you to enter this survey if you're an antivax mommy than if you're just an average mum looking after average kids? Self selection FTW, folks.

Look, I know it's difficult to design a survey like this, but hey, come on.

# re: Antivax: Sometimes what you omit defines the most effective lies

Left by Edgar Allen at 8/17/2009 9:56 PM
Gravatar
You people are evil and should be kept away from children. My child had vaccinations and almost immediately became an annoying little shit.

# Reverend

Left by Dave The Happy Singer at 8/17/2009 10:21 PM
Gravatar
'Edgar Allen'. I lol'd.

# re: Antivax: Sometimes what you omit defines the most effective lies

Left by Dave The Happy Singer at 8/17/2009 10:22 PM
Gravatar
Wait. The 'title' in the comment box didn't mean what I thought it meant.

# re: Antivax: Sometimes what you omit defines the most effective lies

Left by Jason at 8/17/2009 10:43 PM
Gravatar
Would you like me to change it to "subject"? Or hide it entirely? All things are possible in the land of the web.

# re: Antivax: Sometimes what you omit defines the most effective lies

Left by reasonablehank at 10/1/2009 2:07 PM
Gravatar
@Edgar Allen. Thank you for your evidence confirming my own unsupported belief. Now I know that it not my poor parenting skills which are the cause of the annoying "little-shit"ness as displayed by my offspring. AVN, here I come.
Comments have been closed on this topic.
«October»
SunMonTueWedThuFriSat
293012345
6789101112
13141516171819
20212223242526
272829303112
3456789
 
Vaccination Saves Lives: Stop The Australian Vaccination Network
 
 
Say NO to the National School Chaplaincy Program