Are we Intuitively Cooperative (or are we Moving the Goalposts)?

Are we an intuitively cooperative species? A study that was published a few years ago in Nature suggests that indeed our initial inclination is to cooperate with others. We are only selfish if we are allowed to reflect.

How did the researchers obtain these (perhaps counterintuitive) results? Subjects were given an amount of money and had to decide how much of this money, if any, they wanted to contribute to a common project. The subjects were told that they collaborated on this project with three other unknown players whose contributions were not known. They were told that each of the four players received a bonus that was calculated as follows: (additional money – own contribution) + 2*(sum of the contributions)/4.

So you get the highest personal payoff by being selfish and contributing nothing to the common good, regardless of the total contribution of the other three players. A random half of the subjects were required to make a decision on the amount of their contribution within 10 seconds, whereas the other half of the subjects had to think and reflect at least 10 seconds before making their contribution.

The experiments showed an intuitive-cooperation effect. The mean contribution was significantly larger in the intuition condition than in the reflection condition. Hence the conclusion that we are selfish when given the opportunity to deliberate but cooperative when responding intuitively.

Enter my colleagues Peter Verkoeijen and Samantha Bouwmeester. (I wrote about another study by them in a previous post. Basically, the story is this. I have to walk past their office several times a day on my way to the coffee machine and when they have a paper coming out they won’t let me pass unless I promise to write a blog post about them.) They were surprised about these findings and decided to replicate them. They conducted several experiments but found no support for the intuitive collaboration scheme.

What did they do find? First of all, it turned out that only 10% of the subjects understood the payoff scheme. (Did you understand it right away?) This makes an interpretation of the original findings difficult. How can we say anything meaningful when the vast majority of subjects misunderstand the experiment?

Wait a minute! you might say. Perhaps the original study was run with a different subject pool. This is not the case however. One of the two original experiments that found the effect was run on Mechanical Turk. The replication attempts by my colleagues were also run on Mechanical Turk.

Verkoeijen and Bouwmeester were unable to find evidence for intituive cooperation in several experiments even ones that were very close to the original ones. An initial version of their manuscript was reviewed and an anonymous reviewer pointed out that the authors of the original paper, David Rand and his colleagues, in the meantime coincidentally had conducted studies in which they were also unable to replicate their own finding.

Rand and his colleagues had an interesting explanation for this. Mechanical Turk subjects have become familiar with this type of experiment and now will no longer act naively. The entire pool of subjects is now contaminated. There is no hope of finding the intuitive cooperation effect ever again in that crowdsourcing version of Chernobyl. Fortunately, the effect is still there if naïve subjects are used because the effect is moderated by naïveté.

To address the Chernobyl criticism, my colleagues conducted additional experiments. However, they found no evidence for the newfangled naïveté hypothesis. Turkers who classified themselves as not having participated in public-goods experiments before (they were told prior participation would not preclude them from getting paid this time around as well) showed no intuitive cooperation effect.

An anonymous reviewer of the second version of Verkoeijen and Bouwmeester’s manuscript moved the goalpost even a little further. The reviewer (was it the same one as before?) claimed that it is likely that the Turkers lied about having no experience with the experiment. Not only are the Turkers a heavily polluted bunch, they are also inveterate liars.

So in addition to the naïveté hypothesis, we now have the mendacity hypothesis. Such a line of reasoning opens the door to non-falsifiability, of course. Whenever you find the effect, the subjects must have been naïve and when you don’t they must have been lying about having no experience. The editor at PloS ONE  had the good sense not to let this concern block publication of Verkoeijen and Bouwmeester’s article.

The article includes a meta-analysis of the reported experiments. This analysis produced no evidence for the intuitive cooperation hypothesis. In fact, the aggregate effect is going in the opposite direction. In addition, there are several other unsuccessful replications of the intuitive cooperation effect performed by a Swedish group.

It looks like the discussion on intuitive cooperation has reached an impasse with some initial experiments by one group showing an effect while subsequent experiments from several groups have produced nonreplications. Where do we go from here?

Peter Verkoeijen and Samantha Bouwmeester have initiated a Registered Replication Report with Perspectives of Psychological Science. A number of labs will independently test the intuitive cooperation hypothesis according to a strict protocol to be developed in collaboration with the original authors. I cannot think of a better way to resolve the discussion and stop the goalposts from moving. And what's more important, I will be able to make it to coffee machine again.

Hello, old friend