Apathy Sketchpad

Dr Briffa has updated his blog again. He says:

there are common (but lacking in substance) tactics that are used to discredit and refute my assertion that we don’t know if MMR causes autism or not.

Here, I think, are the main ones:

1. Claim that I should provide the evidence that MMR can cause autism (even when that it’s not my position that MMR causes autism).

His position is this (from the same post):

I maintain that with the state of the evidence as it is that we don’t know beyond reasonable doubt that MMR does not cause autism.

That’s wrong. If you replace the word “reasonable” with “all” then it’s true, but the phrase “reasonable doubt” was invented precisely to stop people claiming exactly this kind of nonsense. If he wants to suggest that there are reasonable grounds for doubt, then he should provide some, and given how many studies have failed to show any link between MMR and autism, I think those grounds will have to take the form of some studies that do. Essentially, evidence. Words aren’t going to cut it.

Meanwhile, he is arguing that ‘we’, as scientists who think that MMR is effectively vindicated with regard to autism, should provide further evidence to back up this claim, which was a near certainty even before the first study and is pretty-well unassailable now. He is using the very tactic he accuses us of.

2. Argue that because we don’t have evidence definitive evidence [sic] that MMR causes autism, then that MUST mean it doesn’t (this is illogical, but you’d be surprised how many times this card is played scientists who really ought to know better).

Well I think I’ve explained often enough already that nobody is doing this. (His response seems to be to find something I’ve said which, if cunningly misinterpreted and then fed through several steps of inference, appears to imply that I secretly think otherwise.) His basic accusation is that we say the evidence says something it doesn’t, but the fact of the matter is that he is saying we say something that we don’t. He is using the very tactic he accuses us of.

3. Misrepresent the strength of the science (this is actually the most common one, and my assertion is that the evidence used to vindicate MMR with respect to autism, from a scientific perspective, doesn’t amount to a hill of beans)

Dr Briffa says that

there’s a huge pile of anecdotal evidence and some experimental evidence too which supports the idea that MMR vaccination might cause autism.

There is “no credible evidence” to support this claim. He is using the very tactic he accuses us of.

4. Insult me (e.g. call me ‘wilfully ignorant’)

Dr Briffa told me

you have a very flimsy grasp of even the most simple logic, as well as the application of scientific principles and evidence in the real world. … you do not really know what you are talking about, and have no integrity either.

And he told a commenter on his blog

I suspect I won’t be the only person reading your assertion that the ball is red as an example of quite breathtaking stupidity.

“I’m a complete tool!”

Yes, and this comment appears to have come from an uncharacteristic moment of mental clarity for you. See, you can do rational thinking, after all.

…

If [you are] really not as bewilderingly stupid as I believe [you] to be, then [you] would have given the right answer – actually the ONLY answer one can honestly give.

…

Reading your last post caused a vision to flash through my mind. This is it:

A little boy told did a very stupid thing and needed to be corrected severely by his father. The boy, now upset, crying, snotty, red-faced with tears running down his cheeks then shouts at his father ‘I hate you!’. Where that came from is anyone’s guess…

…

“As my mother used to say ‘ask a stupid question, get a stupid answer.”

Or, as I prefer to put it, ‘ask someone stupid a question, get a stupid answer.’

All of the above was collected from his own comments underneath the post where he said we insult him. Against one person. He is using the very tactic he accuses us of.

5. Say nothing

Dr Briffa says nothing. He just hides that fact by talking a lot. He is using the very tactic he accuses us of.

The more astute of you will have spotted a pattern.

The other day I posted about Dr John Briffa’s rant against p-values. He has since then posted some responses, in the form of several comments under the original post and a whole new rant. Er, I mean, blog entry, of course. Not “rant”!

His thesis remains much the same: no matter what anyone does, since science can’t prove a negative, we can’t be sure MMR doesn’t cause autism. Which is true, but of course can be applied to any stupid hypothesis you care to come up with. In his recent post, which is called “Why the MMR-autism ‘war’ is far from over”, he says

What I am saying though is that there’s a huge pile of anecdotal evidence and some experimental evidence too which supports the idea that MMR vaccination might cause autism.

This really isn’t true. The Cochrane Collaboration examined 139 studies about MMR (not all about MMR-autism) and concluded that

No credible evidence of an involvement of MMR with either autism or Crohn’s disease was found.

In any case, this always goes the same way. There’s a bad study done that “suggests” something, in this case that MMR might cause autism, and a load of people latch onto this for some personal reason, then when someone points out that the research is rubbish they deny it. Eventually the weight of evidence becomes so great that the course of least resistance is to drop that one tiny part of their stance: their position switches to “that study was bad, yes, we can see that, but our theory is still right”. If you ask them to show some non-bad research that supports their hypothesis then they’ll go and do a literature search vast (if not rigorous) enough to put any PhD student to shame, before coming up with some bizarre study about giving vaccinations to chimps or something, and I always look at those and think “hang on, where the fuck did that come from? You’ve been ranting about how bad MMR is for years, and this is the first time you’ve mentioned that study. In fact, you were ranting about MMR for years before it was published! How do you expect to convince me that that’s influenced your opinion in the slightest? I want to see the evidence on which you’ve based your opinions, if there is any.” Of course that doesn’t invalidate their chimp-based study, but it does show that they’re starting with a conclusion and then collecting evidence to support it, when they should be starting with evidence and basing the conclusion on that. Once you’ve established that, the last thing you should do is to criticise their evidence — it’s much quicker for them to find more shaky evidence than it is for you to dismantle it, so they’ll always be a couple of steps ahead if you let yourself get drawn into that fight.

The evidence used to persuade us of the safety with regard to autism is simply inadequate. The fact is, I don’t know whether MMR causes autism or not. But then again, it seems neither do those who insist it is safe.

He also says

Now, I’ve got a sneaking suspicion that those of the pro-vaccine lobby will want to claim that this blog is scaremongering by making out that MMR vaccination causes autism.

I agree with jdc about that quote.

And while the reason that the debate rages on is usually put down to the likes of Dr Wakefield and the parents who believe their children were damaged by MMR, the real guilty parties here have been our Governments whose intransigence regarding proper, definitive research in the area has inevitably left a huge question-mark hanging over MMR.

That’s plain wrong. As I said in his blog comments (assuming that he hasn’t deleted them, although he’s been good to jdc’s, so I don’t want to imply that he will), it would be unethical to do that study: if the study group was large enough to show the effect (which even anti-MMR types claim is very rare, even when they’re demanding that all three of their children were hit by it) then you’re deliberately avoiding giving a potentially life-saving vaccination to at least hundreds of children, on the basis that a few ill-informed, untrained, tabloid-reading morons think there might be a risk. There’s no way that would ever get past an ethics committee.

You have to be a little bit detached and just accept that the so-called link between MMR and autism is, in fact, just made up. That doesn’t prove it’s false, but it puts its odds at much the same level as other made-up hypotheses, such as “cider causes shortness” or “MRI scans cause blindness”. (I just pulled those out of thin air.) Doing huge studies to attempt to disprove things you’ve made up would be a tremendous waste of time, and that doesn’t change just because they were made up a long time ago by someone else and then relentlessly repeated by bad journalists and angry but unqualified mothers.

I’ve just been pointed to a fantastically bad article about statistical significance on the blog of one Dr John Briffa, former natural health columnist for the Daily Mail, no less. He says:

This tells us, supposedly, whether there’s some real effect or change going on, or it’s merely something that’s most likely to be due to chance. Statistical significance in scientific studies is denoted by what is known as the P (or probability) value. A value of less than 0.05 is generally regarded as denoting ‘statistical significance’.

Sounds fine so far. Except, I do feel compelled to point out that the choice of 0.05 as a cut-off is utterly arbitrary. It’s a value that the scientific community agree on. It’s a consensus – it’s not carved in stone like some irrefutable scientific truth. If the scientific community decided that 0.01 was going to be a cut-off, then less things would be ‘statistically significant’. If the limit was set at 0.1 then many more things would be deemed significant. When we understand this, we begin to see just how arbitrary a lot of scientific ‘findings’ really are.

And that’s all true, although the fact that p<0.05 is a totally arbitrary choice isn’t exactly a secret. We all know it. Often, people will demand p<0.001. That’s why we quote p values rather than just printing “yes” or “no”. It’s a bit like saying “People over 6′2″ are considered ‘tall’. Except, I do feel compelled to point out that the choice of 6′2″ as a cut-off is utterly arbitrary” and then saying that therefore tallness isn’t a useful concept in real life.

One commenter said

yes totally agree
“science”.. statistics.. are black and white
life is grey, a vague misty kind of grey

so it’s clear that his plan is working. Medical science is anything but “black and white”. It’s all about measuring and weighing risks and benefits. About p-values and error bars. Confidence intervals and placebo effects.

There ends Briffa’s explanation of p-values. He hasn’t bothered to explain what they are and doesn’t plan to. (In fact, a p-value is roughly defined as the odds of getting a result that good if the hypothesis you’re testing is false: if you get a high p-value then there’s really no reason to think your hypothesis is true.) Instead, he charges straight into a badly thought out analogy:

Let’s imagine someone decided to do a big study on road safety. Let’s say they counted up the number of times someone, somewhere, crossed the road. And now, let’s imagine, they also count up the number of times someone gets run over (and hurt or killed) as a result of crossing the road. Now, I’m writing this on a plane and can’t even check if these statistics exist. But I think it’s reasonable to assume, that compared to the total number of road crossings, the number of people being knocked down is likely to be very small indeed.

Now imagine we applied some statistical ‘wizardry’ to this (with that arbitrary P value, remember) It’s not too difficult to imagine that one would turn up a result which shows: ‘crossing the road is not associated with a statistically significant increased risk of getting run over.’ Now, many doctors and scientists would interpret this finding as evidence that crossing the road is ‘safe’. However, we all know that while most of the time it is, sometimes it’s not.

He seems to be trying to implicitly conflate probability of an event occurring with statistical significance. He’s implying that because accidents are rare, the results of a study will be statistically insignificant. In fact, if a prospective study looked at people crossing the road (the tests), and the same number of other people sitting at home for the same length of time (the controls) then if even six of the test subjects got run over, your p value would be less than 0.05 (assuming the controls all survived). If eleven of the test group were hit by cars then you’d get p<0.001 level significance. You can test for very unlikely events and get good p-values; you just need a large sample space. In this case, you need to look at enough people that six die. That’s probably a much bigger number than is at all feasible, so in fact, you’d do a retrospective case-control study. You’d locate a group of people who were killed by being hit by cars, find out how many of them were crossing the road at the time, and compare that to the proportion of the general population crossing the road at any given time. I suspect you’d find a big (and statistically significant) difference there. That’s how we test for rare events. His analogy proves nothing at all.

That said, he’s right that one could do a study that shows no link. That’s pretty easy: we all know that crossing the road actually is very safe considering how many times most people do it. The point is that that study would not say “crossing the road isn’t linked to death by car accident”; it would say “we have not found a link between crossing the road and death in car accidents”. It certainly wouldn’t prove that no such link exists, and no competent scientist would ever claim it did.

The point that he is attempting to prove by this is as follows:

An example of where statistical significance appears to have got in the way of a constructive debate on the subject is vaccination. Our Government here in the UK, most doctors (I suspect) and many commentators would have us believe that vaccination, including the measles, mumps and rubella vaccination (MMR) is ‘safe’. Many will not even entertain the thought that there may be a problem with MMR.

That’s totally untrue. Of course injecting pathogens into people has associated risks. Nobody with any relevant knowledge is claiming otherwise. The claim is simply that the risks are tiny compared to the benefits, and that autism isn’t one of the risks.

Science hasn’t proven that there’s no risk. Science can’t do that: you can’t prove a negative. It’s possible that there’s exactly one person in the world whose body is set up in such a way that the MMR jab would cause them to become autistic. In that case, there would be a risk, but it would be impossible to detect it unless (indeed, even if) your study contained that one person (in which case the risk would go away when the study was done anyway). So we’re stuck? No, not really.

We know that all the ‘evidence’ that MMR causes autism is, for want of a better word, shit. It’s rubbish. It is insignificant. We also know that a number of better studies have found no risk at all. So yes, it’s still theoretically possibly, but why should it be true? Crossing the road causes car accidents? That figures. We don’t need epidemiology to convince ourselves that that’s pretty likely. But “MMR causes autism”? Why not “MMR causes tallness” or “MMR causes hair loss” or “hats cause kidney stones”? None of those have been absolutely disproven either. There’s absolutely no reason for it to be true, except that many people irrationally believe it to be true, and their beliefs can all be traced back to a load of scary stories published together in a right-wing, anti-scientific propaganda tome. And in Dr Briffa, it even has people standing up for it by harping on about “the limitations of science”. (He has form on this.)

What we have here, then, is a religion. Classic case.