How not to use maths to support your assertions

I've written before on the topic of percentages, and more specifically the inability of many people to correctly employ, disseminate and understand them. A striking example of this failure has cropped up recently on the Australian Vaccination Network's Facebook page, a well-known venue for such failures.

This post links to an article on the website of one Greg Beattie, an antivax parent who tried, and failed, to take on the Australian Human Rights and Opportunities Commission, on the topic of the right to refuse non-vaccinated children admission to childcare. Greg is no smarter than Meryl Dorey when it comes to numbers

It appears that Beattie has petitioned the Department of Health and Ageing for permission to post pertussis notification data for 2008-2010, and is trumpeting the data as proof that the pertussis vaccine is ineffective. This is one of Dorey's regular catchcries, and like Dorey, Beattie is unable to correctly interpret the data he claims supports his position.

So let's take it on critically, shall we?

Beattie's claim is as follows (quoted from his site under fair use doctrine)

For the years 2008-2010, there were a total of 9333 cases of whooping cough reported in children aged 0-4 years. Among them were 754 who were either too young or otherwise not eligible to have been vaccinated, and 1497 for whom vaccine status was not known.

Of the remaining 7082 cases whose vaccine status was known, and who were eligible to have received the vaccine, 5296 (75%) were fully vaccinated for their age. A further 986 (14%) were partly vaccinated. Only 800 (11%) were un-vaccinated.

This data is for a cohort with, as Dopey Dorey notes in her Facebook post, a ~95% vaccination rate.

So what does this tell us? Well, first, let's define what we'd expect to see if the vaccine was really ineffective. Given that 95% are vaccinated, and therefore 5% are not, an ineffective vaccine should give us roughly equivalent numbers - 95% of cases would be in the vaccinated group, and 5% in the non-vaccinated. All clear so far?

Well, is this what we see?

No. What we see, even by Beattie's numbers (of which more criticisms anon) is a rate of 11% for the unvaccinated population, or a little over twice what we'd expect to see if in fact Greg and Meryl's "vaccine doesn't work" assertion was correct.

Remember, these are Beattie's own numbers that show this.

So it doesn't look good for Meryl's interpretation so far. It looks, in fact, like the vaccine *does* in fact have a preventative effect, with roughly twice the likelyhood of acquiring an infection in the unvaxed group that we'd expect to see in the hypothetical scenario.

But wait, there's more. Beattie's wording also mentions the partly vaccinated, a group which, by definition, are not part of the fully vaccinated group. This adds another 14% to the notifications here, giving us a neat 25%. So an individual without complete vax coverage is actually more like five times more likely to contract an infection than we hypothetically expect. Remember, the 95% figure for vaccination quoted in this group is for complete vaccination. The incompletes are part of the five percent, and they concomitantly drive up the figures here.

But there's even more. Beattie has excluded the population who are unable to receive the vaccine. These people are, by definition, part of the 5% unvaccinated population. This adds yet another 754 cases to the five percent bucket. This drives up the figures even further - and Beattie has dishonestly (or incompetently) excluded these cases from his calculations.

In fact, the only cases which should be excluded from calculations are the unknown vaccination status cases.

This gives us an overall case pool of 7836 cases to play with, NOT Beattie's claimed pool of 7082. So we have to recalculate everything. So let's re-iterate.

  • 7836 pertusis cases for whom vaccination status can be determined
  • 5296 in the fully vaccinated population (95% of population)
  • 986 partly vaxed
  • 800 unvaxed
  • 754 ineligible (these last three categories form the 5% population)
    • total cases outside the fully vaxed population - 2450

We can already see that the unvax group comprises far more than the 5% of the cases that we'd expect to occur if, in fact, the vaccine was ineffective. In fact, cases in the unvaccinated group form 31.2% of notifications, a rate over six times what we'd expect for an ineffective vaccine. In fact, it falls fairly neatly between six and seven times more likely to contract an infection than they'd be if the vaccine really didn't work.

And Greg Beattie marvels that he's been called out as an "innumerate dingbat".

The maths here are not hard at all. They're barely high-school level, but they do require a little care, and Beattie has not taken that care any more than Dorey did when she declared the vaccine ineffective based upon them. Remember, we've just seen that unvaccinated or incompletely vaccinated individuals are more than six times more likely to contract symptomatic, lab-confirmed pertussis than we'd expect given the population figures.

For a vaccine, I'd call that fairly effective. Not perfect, but nothing is.

And we haven't even drilled into severity of infection - a factor which these numbers cannot reveal, but which is shown by strong studies to be reduced by vaccination - put simply, if you're vaccinated, you can still catch an infection, but it'll be far less severe than in a non-vaccinated individual. You'll most likely recover quicker, and you'll be less likely to develop severe complications, which in the case of pertussis include death.

Add in that factor and who knows where the dice will fall? One thing's for sure, they won't fall in Greg Beattie's favour.

Yes, Greg, you're an Innumerate Dingbat. You're just going to have to learn to live with it.


[10.29pm: I've updated some of the wording because I was not clear enough in stating that comparisons are against the hypothetical non-working vaccine scenario that Dorey and Beattie are spruiking]

[Update 10:47am the next day: THANKS REDDIT for a delicious couple of thousand page views]

[Update 8th Feb 10:25am: The dingbat has responded, so I've added a new post explaining why he's still a dingbat]

posted @ Sunday, February 05, 2012 6:54 PM


Comments on this entry:

# re: How not to use maths to support your assertions

Left by Laura at 2/5/2012 10:13 PM
ummm.. i feel like your maths is a fail. even including partial unvax and ineligble into unvaxxed proportion it is still less than 50% of infections, so you are actually proving their point another way.

Next issue, how does not being vaccinated give you a 5% chance of infection? that's so not how it works. i'd say infection chance is 100% when exposed to virus. better to look at numbers of infection based on the groups percentage of population.

ie if there are 2 million kids 0-4 and 95% of them are vaxxed then rate is 5296 in 1.9 million = 0.28% infection rate vs unvaxxed/partial vax group is 2540 in 100,000 = 2.54% infection rate ie TEN TIMES the rate of infection of vaxxed group. thus vaccine is clearly effective.

# re: How not to use maths to support your assertions

Left by Jason at 2/5/2012 10:43 PM
Thanks Laura,

I've updated some of the wording to make it more clear that my comparison is against the hypothetical "vaccine doesn't work" scenario.

There's no absolute risk here, because for that we need entire population figures and not just notifications, and I haven't included those.

At no point did I state a 5% chance of infection as in paragraph 2. What it means is that *if* the vaccine was ineffective as the antivaxers like to claim, then 5% of the notifications should come from the 5% unvaxed population, give or take a statistical deviation.

I do like the way your last paragraph converts to more real rates, but that's based on a hypothetical total population, which I haven't included.

# re: How not to use maths to support your assertions

Left by Laura at 2/5/2012 10:53 PM
Or another way to look at it is this.

there are 5296 cases in the vaccinated group which represents 95% of the population. this means that if the vaccine was completely ineffective we would expect that to make that up to 100% of the population, we would see infections at the same rate in the unvaccinated children.

so 1% = 5296/95 = 55.74 cases
so 5% = 278 cases

so even looking at just the unvaxxed kids you have 800 cases, so almost THREE TIMES the expected no of cases if the vaccine was ineffective. (800/278 = 2.8)

now if we look at the partially vaccinated group who fall into the same 5% classification of unvaccinated we have 986 cases. so 986/278 = 3.5 times the expected rate if vaccine was ineffective. so roughly the same, maybe slightly higher rate of infection. i would need numbers from several years to do stats to tell if it's the same as the completely unvaxxed group or not.

now, if we combine all three non fully vaccinated groups we have 2450 cases. 2450/278 = 8.8 times the expected number of cases if the vaccine was 100% ineffective.

thereore, proved by some very simple maths, getting your child fully vaccinated for whooping cough will decrease their chance of catching pertussis by almost 9 fold. i'd say that's a good enough reason to do it!

# re: How not to use maths to support your assertions

Left by jason at 2/7/2012 5:26 PM
The innumerate dingbat has responded, claiming he couldn't comment here.

I have no idea what buttons he was furiously mashing to get an error message, because as you can see, I'm able to comment as normal

Anyway, a rebuttal of his continued failure to comprehend simple mathematics is to follow, and will include a handy piece of software to crunch the numbers live, in the webpage, however you like.

# re: How not to use maths to support your assertions

Left by jason at 2/7/2012 5:28 PM
And no, I will not register at Greg's site to comment there. I have no wish to receive spam about his latest book. He'll just have to learn to push submit buttons properly.

# re: How not to use maths to support your assertions

Left by reasonablehank at 2/7/2012 5:35 PM
Is this thing on? Why can't I comment? Why are you mocking my freedom of expression?

# re: How not to use maths to support your assertions

Left by reasonablehank at 2/7/2012 5:37 PM
Please ignore my last comment damning you for making it impossible to comment in your comments.

# re: How not to use maths to support your assertions

Left by jason at 2/7/2012 5:39 PM
Where's the like button?

Oh, that's write, I haven't got round to coding one yet. Busy busy.
Comments have been closed on this topic.
Vaccination Saves Lives: Stop The Australian Vaccination Network
Say NO to the National School Chaplaincy Program