Drang naar Samenhang

As I indicated in my previous post , it is not easy to estimate beforehand to which situations your conclusions generalize. But it is important to at least  make an effort. Often, conclusions are wildly oversold, creating a paradoxical situation when the results are not replicated. Usually, a hidden moderator is invoked and all of a sudden the scope of conclusions previously advertised as far-reaching is drastically narrowed. An example of this arrived in my mailbox the other day. A while ago, our registered replication report of Hart & Albarracin (2011) came out. I already  blogged  about it. The first author, William Hart, has now written a response to our report. It will appear in the next issue of Perspectives on Psychological Science; I have an advanced copy of the response. Hart doesn’t raise substantive concerns about our report but he does suggest that maybe we didn’t replicate the original findings because the original study was ...

The other day, the economist Andreas Ortmann kindly put in a plug for my latest blog post on Facebook. This elicited a response from another researcher, who said: One of my friends tried to replicate one of Rolf Zwaan's findings on verb aspect and failed ages ago, except they did it in Cantonese and not Dutch. This isn't a perfect replication, but if you read the conclusions, then it should be. So part of the problem isn't even the strength of the data, it's also the fact that being over-certain and over-generalizing conclusions is the standard way psychology papers are written. The commenter then helpfully provided this link to the article (with which I was not familiar). I found the comment interesting, even though it incorrectly states that our study  (see also  this post  from January) was conducted in Dutch. (You can’t even perform the study in Dutch due to grammatical differences with English.) More importantly, though, the claim that our finding was not re...

The other night I was watching a Dateline episode on a false confession that landed someone in jail for a crime he didn’t commit. The story is quite similar to that of Brendan Dassey in Making a Murderer:  a learning-disabled boy being coerced by detectives into falsely confessing to having committed heinous crimes. These are very upsetting stories. Even more upsetting is that false confessions are quite common in the US  and have led to a great number of wrongful convictions. It occurred to me that the false confession debate provides an intriguing analogy with the replication debate, which was recently reignited after the publication of a critique in Science of the Reproducibility Project . Many people have written great blogposts about this latest controversy already (e.g., Uri Simonsohn , Simine Vazire , Sanjay Srivastava , Michael Inzlicht ,  Dorothy Bishop , Daniël Lakens, and David Funder ). This post approaches the debate from a different angle. I explore...