Drang naar Samenhang

The popular press as well as the psychological literature itself is abuzz with reports about the field trying to clean up its act, especially by demanding replications (though not everyone agrees ). Many reports do not fail to mention that this sudden sanitation urge was prompted by the Stapel fraude case. This is ironic, because it is logically impossible to replicate Stapel. His work was never “plicated” in the first place! Nevertheless, it is clear that the field is in a state of revolution. There are large-scale replication efforts under way, most notably the Open Science Framework . But replication starts at home. So here are some things we can do as authors, reviewers, and editors. Authors Incorporate direct replications into your workflow (I hesitate to use this buzzword but it seemed quite popular among those interested in replication   during a recent Google hangout).  I have reported direct replications in one published paper and I have r...

In our everyday experience time is continuous and chronological. In stories we can jump around in time. Just as with many other things, Aristotle had already given this discrepancy thought.  In his Poetics he declared that historians are to provide a blow-by-blow chronological account of events. Authors of fiction, on the other hand, are not bound by this directive. For them, plot is the constraining factor. If the plot calls for a jump ahead in time, the author should do so. No need for a detailed account of Odysseus’ daily bowel movements during the seven years that he was ensnared by Calypso. Time shifts are extremely common in stories. For example, we often encounter phrases like  an hour later , which force us to jump an hour forward in time from one sentence to the next. But how do we process such time shifts? Cognitive psychologists have begun to address this question. An early example is here . In a study published in 1996, I compared time shi...

In 2001 and 2002 , my students and I published two papers on mental simulation in Psychological Science. I warned my students that these experiments might draw a lot of attention and possibly criticism. I was about to make full professor, so I was not worried about myself but I was worried about my students. I was particularly concerned about a backlash from traditional psycholinguists. I was wrong. There was no backlash. Instead, other people started using our paradigm and made nice careers for themselves doing so. It stayed like this until 2008. Then a paper appeared co-authored by Mahon and Caramazza. They called our experiments “elegant and ingenious” (thanks for that) but argued that our results did not rule out an account in which the actual work was done by abstract symbols rather than perceptual representations, with activation cascading down from abstract representations to the perceptual and action systems.  (This hypothesis does not seem implausible ...

In a previous post I alluded to the fact that I had produced an amusing title a few years ago for an article that was published in Psychological Science (it was intended as a parody on the article titles in that journal). I also mentioned that that article won the Ig Nobel Prize for psychology last year. This prize is awarded for “research that first makes you laugh and then makes you think.” I thought it might be interesting to describe the creative process behind this paper. I’ll start by saying that I was not a creative force behind the paper, so I’m describing it as a participant observer. My favorite Beatles song is A Day in the Life . One of the things I like about this song is the sudden switch from the main theme to the middle part ( Woke up, fell out of bed… ). When I first heard the song, I was amazed. How on Earth did John Lennon and Paul McCartney come up with the idea of switching in the middle of the song to a totally different piece and then back? I ...

Many papers have more than one experiment. How do researchers string together experiments? When I was a beginning assistant professor, I would tell my students The difficult part is not Experiment 1, it is Experiment 2 (when saying this, I would always picture myself as a grizzled war veteran, smoking a pipe and using its stem to point out locations in the field of battle to my crop of young recruits.) Why was Experiment 2 so difficult? Well, it couldn’t stray too far from Experiment 1 or reviewers might expose a gap in the causal chain that leads from experiment to experiment. It also couldn’t be too close because then you’d have a direct replication of Experiment 1. (Nowadays we would see this as a virtue but back then it would probably have been viewed as unnecessary padding.) For many years I thought having a tight chain of experiments was normal for experimental research. But then I started reading articles in social psychology. I did this partly because...

In previous posts I have talked quite a bit about amusing titles (I promise that my next post will be on something else). I borrowed this term from an article that looked at the effects of such titles on citations and found that “highly amusing titles” were cited 30% less than other titles. The authors of the article were careful to point out that correlation does not imply causation. Maybe these articles had less to offer contentwise—thus receiving fewer citations—and tried to make up for it in packaging. We can establish causality in experiments. I reported initial evidence that amusing titles yield lower confidence in the associated results. Admittedly, this evidence is not very strong but it’s a first indicator. Whether or not titles are amusing is in the eye of the beholder, so “amusing” is not a useful label. I propose to speak of descriptive and elaborative titles . A descriptive title is “A Comparison of Treatment A and Treatment B.” An elaborative ti...

Good thing I called the previous update “preliminary results!” I discovered later that many subjects had “clicked through” the descriptions, merely reading the titles and spending only a second or so on the descriptions. In my preregistration I had said that I would exclude the data from subjects if their viewing times were < 30 sec for two or more abstracts. Well, it turned out I would have had to throw away data from more than half the subjects! Therefore I decided to rerun the study but now with a stern warning in the instruction that viewing times would be measured and that data from subjects with impossibly short viewing times would be unusable and that the subjects would not get paid. This seemed to help. In the second run, far fewer subjects had impossibly short viewing times, although there were still quite a few delinquents (they did not get paid). The lesson here: first run a sizeable pilot study before you pre-register, dummy! I overshot a little...