Drang naar Samenhang

There are quite a few comments on my previous post already, both on this blog and elsewhere. That post was my attempt to make sense of the discussion that all of a sudden dominated my Twitter feed (I’d been offline for several days). Emotions were runing high and invective was flying left and right. I wasn’t sure what the cause of this fracas was and tried to make sense of where people were coming from and suggest a way forward.  Many of the phrases in the post that I used to characterize the extremes of the replication continuum are paraphrases of what I encountered online rather than figments of my own imagination. What always seems to happen when you write about extremes, though, is that people rush in to declare themselves moderates. I appreciate this. I’m a moderate myself. But if we were all moderates, then the debate wouldn’t have spiralled out of control. And it was this derailment of the conversation that I was trying to understand. But before (or more likely a...

I missed most of the recent discussion on replication because I’m on vacation. However, the weather’s not very inviting this morning in southern Spain, so I thought I’d try to catch up a bit on the fracas, and try to see where both sides are coming from. My current environment induces me to take a few steps back from it all. Let’s see where this goes. Rather than helping the discussion move forward, I might, in fact, inadvertently succeed in offending everyone involved. Basically, the discussion is between what I’ll call the Replicators and the Replication Critics Reactionaries . I realize that the Replicators care about more than just replication. The Reactionaries are reactionary in the sense that they  Critics are opposing the replication movement. The Replicators and the Reactionaries  Critics are the endpoints of what probably is close to a continuum. Who are the Replicators? As best as I can tell, they are a ragtag group of (1) mid-career-to-senior methodologi...

Are we an intuitively cooperative species? A study that was published a few years ago in Nature suggests that indeed our initial inclination is to cooperate with others. We are only selfish if we are allowed to reflect. How did the researchers obtain these (perhaps counterintuitive) results? Subjects were given an amount of money and had to decide how much of this money, if any, they wanted to contribute to a common project. The subjects were told that they collaborated on this project with three other unknown players whose contributions were not known. They were told that each of the four players received a bonus that was calculated as follows: (additional money – own contribution) + 2*(sum of the contributions)/4. So you get the highest personal payoff by being selfish and contributing nothing to the common good, regardless of the total contribution of the other three players. A random half of the subjects were required to make a decision on the amount of their cont...

The gesture doesn't work all the time Why do we gesture? An obvious answer is that we gesture to communicate. After having just taken down his opponent in a two-legged flying tackle, the soccer player puts on his most innocent face while making a perfect sphere with his hands. This gesture conveys the following thought: “Ref, I was playing the ball! Sure, my opponent may be lying there writhing in pain and will soon be carried off on a stretcher but that’s beside the point. I do not deserve a red card.” But we also gesture when our conversation partner cannot see us. Years ago I saw a madwoman walking in the Atlanta airport. See seemed to be talking to no one in particular while gesticulating vehemently. For a moment I was worried she might pull out a machine gun and mow us all down. But when she got closer I noticed she was speaking into a little microphone that was connected to her mobile phone (a novelty at the time). Evidently, the person that was on the receiving end ...

A few months ago, I was asked to review a manuscript on social-behavioral priming. There were many things to be surprised about in this manuscript, not the least of which was that it cited several papers by Diederik Stapel. These papers had already been retracted, of course, which I duly mentioned in my review.  It has been said that psychology is A Vast Graveyard of Undead Theories . These post-retraction Stapel citations suggests that this cemetery might be haunted by various undead findings (actually, if they were fabricated, they weren’t really alive in the first place but let's not split semantic hairs). There are several reasons why someone might cite a retracted paper. The most obvious reason is that they don’t know the paper has been retracted. Although the word RETRACTED is splashed across the first page of the journal version of the article, it will likely be absent on other versions that can still be found on the internet. Researchers working with such ...

More and more people are involved in replication research. This is a good thing. Why conduct replication experiments? A major motivation for recent replication attempts appears to have been because there are serious doubts about certain findings. On that view, unsuccessful replications serve to reduce the initially observed effect size into oblivion. I call this replicating down . Meta-analytically speaking, the aggregate effect size becomes smaller with each replication attempt and confidence in the original finding will dwindle accordingly (or so we would like to think). But the original finding will not disappear from the literature.  No, I'm not Noam Chomsky Replicating down is definitely a useful endeavor but it can be quite discouraging. You’re conducting an experiment that you are convinced doesn’t make any sense at all. Suppose someone conducted a priming study inspired by a famous quote from Woody Allen’s Husbands and Wives: I can't listen to that much ...

One of my last actions as Editor-in-Chief of Acta Psychologica was to accept a manuscript for publication that is very timely given the current “crisis of confidence” in psychology. One of the paper's key points is that it is crucial to distinguish between hypothesis testing and exploratory research. The paper chimes in with many critics of current practices in psychology when it asserts that [i]t is essential that these hypotheses have been precisely formulated and that the details of the testing procedure (which should be as objective as possible) have been registered in advance. Several journals, such as Cortex , as well as special issues of journals already require authors to preregister their submissions and the Open Science Framework offers an extremely user-friendly platform to do just this. The paper makes a clear distinction between hypothesis-generating and hypothesis-testing research and argues that researchers regularly conflate the two, passing off expl...

During the investigation into the scientific conduct of Dirk Smeesters , I expressed my incredulity about some of his results to a priming expert. His response was: You don’t understand these experiments. You just have to run them a number of times before they work. I am convinced he was completely sincere. What underlies this comment is what I’ll call the shy-animal mental model of experimentation. The effect is there; you just need to create the right circumstances to coax it out of its hiding place. But there is a more appropriate model: the 20-sided-die model (I admit, that’s pretty spherical for a die but bear with me). A social-behavioral priming experiment is like rolling a 20-sided die, an icosahedron. If you roll the die a number of times, 20 will turn up at some point. Bingo! You have a significant effect. In fact, given what we now know about questionable and not so questionable research practices, it is fair to assume that the researchers are actually roll...