Happy British Science Week, the UK’s largest grassroots celebration of science, technology, engineering and maths. The week runs from 11th to 20th of March, and we thought that it might be quite nice to remind you that we LOVE science here at 4th Estate. To kick things off, an extract from I think You’ll Find It’s a Bit More Complicated Than That by Ben Goldacre!
What Is Science? First, Magnetise Your Wine . . .
People often ask me (pulls pensively on pipe), ‘What is science?’ And I reply thusly: Science is exactly what we do in this column. We take a claim, and we pull it apart to extract a clear scientiﬁc hypothesis, like ‘Homeopathy makes people better faster than placebo,’ or ‘The Chemsol lab correctly identiﬁes MRSA’; then we examine the experimental evidence for that hypothesis; and lastly, if there is no evidence, we devise new experiments. Science.
Back in December 2003, as part of our Bad Science Christmas Gift series, we discovered The Perfect Sommelier, an expensive wine-conditioning device available in all good department stores. In fact there are lots of devices like this for sale, including the ubiquitous Wine Magnet: ‘Let your wine “age” several years in only 45 minutes! Place the bottle in the Wine Magnet! The Wine Magnet then creates a strong magnetic ﬁeld that goes to the heart of your wine and naturally softens the bitter taste of tannins in “young” wines.’
At the time, I mentioned how easy it would be to devise an experiment to test whether people could tell the difference between magnetised and untreated wine. I also noted how strange it was that none of these devices’ manufacturers seemed to have bothered, since it could be done in an evening with ﬁfty people.
Now Dr James Rubin et al. of the Mobile Phones Research Unit at King’s College London, have published that very study, in the esteemed Journal of Wine Research. They note the dearth of experimental research (quoting, chufﬁngly, the Bad Science column), and go on: ‘One retailer states, “We challenge you to try it yourself – you won’t believe the difference it can make.”’
‘A review of Medline, PsychInfo, Cinahl, Embase, Amed and the Web of Science using the search term “wine and magnet” suggested that, as yet, no scientists have taken up this challenge.’
Now, this study was an extremely professional operation.
Before starting, they did a power calculation: this is to decide how big your sample size needs to be, to be reasonably sure you don’t miss a true positive ﬁnding by not having enough subjects to detect a small difference. Since the manufacturers’ claims are dramatic, this came out at only ﬁfty subjects.
Then they recruited their subjects, using wine. This wine had been magnetised, or not, by a third party, and the experi-menters were blind to which wine was which. The subjects were also unaware of whether the wine they were tasting, which cost £2.99 a bottle, was magnetised or not. They received wine A or wine B, and it was a ‘crossover design’ – some people got wine A ﬁrst, and some people got wine B ﬁrst, in case the order you got them in affected your palate and your preferences.
There was no statistically signiﬁcant difference in whether people expressed a preference for the magnetised wine or the non-magnetised wine. To translate back to the language of commercial claims: people couldn’t tell the difference between magnetised and non-magnetised wine. I realise that might not come as a huge surprise to you. But the real action is in the conclusions: ‘Practitioners of unconventional interventions often cite cost as a reason for not carrying out rigorous assess-ments of the effectiveness of their products. This double-blind randomised cross-over trial cost under £70 to conduct and took one week to design, run and analyse. Its simplicity is shown by the fact that it was run by two sixteen-year-old work experience students (EA and RI).’
‘Unfortunately,’ they continue, ‘our research leaves us no nearer to an understanding of how to improve the quality of cheap wine and more research into this area is now called for as a matter of urgency.’
This extract is taken from I think You’ll Find It’s a Bit More Complicated Than That by Ben Goldacre, OUT NOW!