Last month there was a paper published about a correlation between chocolate consumption and Nobel Laureates.
EDIT: I now see the article may not be accessible to everyone. Here’s a summary: Messerli suggests that, because some flavinoids that are found in chocolate have been linked to improved cognition, one might expect a country that eats more chocolate on average to produce more Nobel Laureates on average. Indeed, Messerli finds a linear correlation between the two variables. While the tone of the short paper is not entirely serious, we’ve previously reported on many spurious correlations, and why they’re so easy to find between cultural variables. The chocolate/Laureate correlation looked like one of these spurious findings, so we set out to debunk it by showing correlations with some less expected variables. If this is the case, then papers like the one criticised here are dangerous because they give credence to this questionable method, while producing media-grabbing headlines.
Me and James wrote a response article, but it’s just been rejected, citing ‘lack of space’ (Dorothy Bishop has also posted a recent response). Here’s the 175 words we submitted. Amongst the 4 statistical tests, try spotting the 6 hidden references to chocolate:
Chocolate consumption (CC) correlates with the number of Nobel laureates (NL) per capita1. However, correlation studies are a rocky road, and it’s easy to fudge correlation and causation. Our data mars the previous inference. Average IQ2 does not correlate with CC (r=0.27, p=0.21). CC correlates with the (log) number of serial3 and rampage3 killers per capita (r = 0.52, p=0.02, fig. 1). NL correlates with the annual road fatalities per capita3 (r=-0.55, p=0.0066). Controlling for GDP3 and mean temperature4, CC is not a significant predictor of NL (F(1,19) = 3.6, p = 0.07). These correlations are unlikely to be causal, so why are they robust? Cultural phenomena diffuse in a way that leads to spurious correlations between independent variables5. While flavonoids may aid cognition, there is little evidence to suggest there is an isomorphic link between individual-level benefits and widespread population-level effects. The original work may have elicited snickers, but it is receiving media coverage. If researchers declare this to be a robust approach, then it’s a slippery slope to a world of pure imagination.
And here’s a longer version of the paper: ChocolateSerialKillers_WintersRoberts (including three more puns).
References
[1] Messerli, F.H. (2012). Chocolate consumption, cognitive function, and Nobel laureates. New England Journal of Medicine. DOI: 10.1056/NEJMon1211064
[2] Lynn, R. and Vanhanen, T. (2002). IQ and the wealth of nations. Praeger Publishers.
[3] Wikipedia:
http://en.wikipedia.org/wiki/List_of_serial_killers_by_number_of_victims;
http://en.wikipedia.org/wiki/List_of_rampage_killers; http://en.wikipedia.org/wiki/List_of_countries_by_traffic-related_death_rate
http://en.wikipedia.org/wiki/List_of_countries_by_GDP_%28nominal%29_per_capita
[4] Mitchell,T.D., Hulme,M., and New,M., 2002: Climate data for political areas. Area 34:109-112.
[5] Roberts, S. and Winters, J. (2012). Constructing knowledge: Nomothetic approaches to language evolution. In McCrohon, L., Fujimura, T., Fujita, K., Martin, R., Okanoya, K., Suzuki, R., and Yusa, N., editors, Five Approaches to Language Evolution: Proceedings of the Workshops of the 9th International Conference on the Evolution of Language. World Scientific. pp 148-157.
Messerli, F. (2012). Chocolate Consumption, Cognitive Function, and Nobel Laureates New England Journal of Medicine, 367 (16), 1562-1564 DOI: 10.1056/NEJMon1211064
Great post!
Lack of space is an annoying reason for NEJM to flake out, there are important points here b’twix the hilarious correlations. I’m sure it would have been a boost for the issue, and it’s sad to see a paper like this become a drifter.
> correlates with the (log) number of
You can fit any function to any data and have it correlate with tiny p-values.
@Rachael: An excellindt observation.
@Luke: That’s precisely our point! Although the use of log seemed justified to normalise the distribution.
And for the sake of another pun… did you count log as a hidden reference?
Ha ha, no I hadn’t though of ‘log’, that means there’s 7 references!
V. good indeed.
Lovely. And what a bummer, no space for those 175 words!
The BBC have just caught up… http://www.bbc.co.uk/news/magazine-20356613
could have been rejected due to rather poor grammar…..”Me andc James” etc
I wonder if the age of the laureates has been taken into account? I mean, as an achievement award, surely lifespan is an obvious candidate. I’m wondering if, given the negative correlation with road deaths,this is an example of survival of the fittest. Or, correlating with chocolate consumption, fattest.
If you’re going to criticise grammar then it’s best to begin your sentence with a capital letter, not make a spelling mistake when quoting the original poor use of grammar and also, end your sentence with a period, or full-stop.
Love the response Sean. Those curlywurly’s at the British Medical Journal should stop approving these correlations…anyone can data-mine for maltesers to find a significant correlation but what’s the model? If there is not one, then you have a thing called endogeneity. Thinking out-loud, countries with Nobels are those likely to have developed academic research facilities and open immigration program for researchers (get the best foreigners who may even become citizens), so most likely high GDP per capita, which like 1,000 other things may be loosely proxied by consumption of chocolate per capita or Microwave’s per capita or Lego pieces per capita….
Chris Blattman wrote a great blog post on a BMJ published article about brushing your teeth causing cardiovascular disease:
” “Toothbrushing is associated with cardiovascular disease, even after adjustment for age, sex, socioeconomic group, smoking, visits to dentist, BMI, family history of cardiovascular disease, hypertension, and diagnosis of diabetes.
…participants who brushed their teeth less often had a 70% increased risk of a cardiovascular disease event in fully adjusted models.”
The idea is that inflamed gums lead to certain chemicals or clot risks.
In the past five days I’ve seen this study reported in five newspapers, half a dozen radio news shows, and several blogs. These researchers know how to use a PR firm.
Sounds convincing. What could be wrong there?
OH WAIT. MAYBE PEOPLE WHO BRUSH THEIR TEETH TWICE A DAY GENERALLY TAKE BETTER CARE OF THEMSELVES AND WATCH WHAT THEY EAT.
I’m consistently blown away by what passes for causal analysis in medical journals.
Here’s a cruel and simplified guide to prevailing opinion and practice:
Medical research: Correlation never implies causation.
Epidemiological research: My correlation implies causation, because I controlled for socioeconomic status.
Economics: Confused and conflicted, but sometimes correlation implies causation. If you can find a cool instrumental variable we’ll surely publish you.
I side with the economists on this one. ”
http://chrisblattman.com/2010/06/01/does-brushing-your-teeth-lower-cardiovascular-disease/
@Alastair I agree! And the toothbrushing and cardivascular link is an interesting example. I guess part of the problem is that, if ANYTHING can be linked, what should we NOT control for? One way is to try and control for everything (see my post here: http://replicatedtypo.com/the-final-correlation-bayesian-causal-graphs-as-an-alternative-to-phylogenetics/5616.html).
Another approach is to use large-scale correlation studies as hypothesis-generating processes, which can only be backed up by experimental and modelling work. See the paper by James and I here: http://replicatedtypo.com/evolang-previews-constructing-knowledge-the-nomothetic-approach-to-language-evolution/4809.html
Perhaps, some of you are taking this a tad too seriously? I think if you read the original Messerli article carefully (or even casually?) you will find it is very ‘tongue in cheek’. Just look at his ‘Disclosures’ at the end! The very point he is making is we should be careful about assuming spurious correlations. He just makes it subtly and with a smile but sadly some people think he is being serious! (By the way, for those of you REALLY not paying attention – the Messerli article was in NEJM not the BMJ).
@imberhk: You just made me day 🙂 . Thanks for your useful comment and style guide advice.
Great article guys. I was looking to find just this sort of data when I came across your paper and so I just cross-referred to it in my own blog ( http://johncherrie.blogspot.co.uk/2012/11/does-eating-chocolate-make-you-serial.html ).
It seems ridiculous to me that the NEJM doesn’t have an electronic letters page where you coudl post your contribution, especially when they can run umpteen Facebook pages and blogs!
Pierre André and I also sent a comment the 25th of october top the NEJM. The response article was 2000 words and we submitted it the 25 of October. The NEJM told us that letters may number up to 175 words in length. We wrote a shorter document of less than 175 words and sent it version the 29 of October. We were also rejected at the end because of the lack of space. The paper of 2000 words can be found at https://9d558120-a-62cb3a1a-s-sites.googlegroups.com/site/fabgouret/research/messerlinote.pdf?attachauth=ANoY7cq4gh6p7OHjV34w8EkbTG6tY-SOhNsRH84knWAj8o5ptojxEXkAce226Kf_LdehylieZ5UCcBmozG6Vq2IEroOhfoCtXoytEYLZlQXw-UgRR1kP_FUws0WE_fO5-u_k4h43mly-CLCQCDmut88ctCzm0_LprO0H8YG2IHpjBATiqEPr3hmCcpjiRfjfbR-UeFBWKfJNara2TDyVW1e8z9wL4k-85KJJWc-FJD7J-mwXB2RxjF4%3D&attredirects=0