The Undead Findings are Among Us

A few months ago, I was asked to review a manuscript on social-behavioral priming. There were many things to be surprised about in this manuscript, not the least of which was that it cited several papers by Diederik Stapel. These papers had already been retracted, of course, which I duly mentioned in my review.  It has been said that psychology is A Vast Graveyard of Undead Theories. These post-retraction Stapel citations suggests that this cemetery might be haunted by various undead findings (actually, if they were fabricated, they weren’t really alive in the first place but let's not split semantic hairs).

There are several reasons why someone might cite a retracted paper. The most obvious reason is that they don’t know the paper has been retracted. Although the word RETRACTED is splashed across the first page of the journal version of the article, it will likely be absent on other versions that can still be found on the internet. Researchers working with such a version, might be forgiven for being unaware of the retraction.

But citing Stapel??? It is not like the press, bloggers, colleagues at the water cooler, the guy next to you on the plane, and not to mention Retractionwatch haven’t been all over this case!

A second reason for citing a retracted article is obviously to point out the very fact that that paper has been retracted. Nevertheless, a large proportion of citations to retracted papers are still favorable, just like the Stapel citations.

"Don't expect any help from us."

Does this imply that retracted findings have a lasting pollutive effect on our thinking? A recent study suggests they do. The Austrian researcher Tobias Greitemeyer presented subjects* with findings from a now retracted study by Lawrence Sanna (remember him?). Sanna reported that elevated height (e.g., riding up escalators) led to more prosocial (prosocial being the antonym of antisocial) behavior than lowered height (e.g., riding down escalators). The findings were found to be fabricated, which is why the paper was retracted.

Greitemeyer formed three groups of subjects. He told the first two groups about the Sanna study but not the third group. All subjects then rated the relationship between physical height and prosocial behavior.

Next the subjects wrote down all their ideas about this relationship. At the end of the experiment, half of the subjects who had received the summary, the debriefing condition, learned that the article had been retracted because of fabricated data and that there was no scientific evidence for the relation between height and prosocial behavior. Subjects in the no-debriefing and the control condition did not receive this information. Finally, all three groups of subjects responded to the same two items about height and prosocial behavior that they had responded to earlier.

As you might expect, the subjects in the debriefing and no-debriefing conditions made stronger estimates about the relation on the initial test than did those in the control condition. More interesting are the responses on the second test, after the debriefing condition (but not the other two conditions) had heard about the retraction. On this test the subjects in the no-debriefing condition had the highest score.  But the crucial finding was that the debriefing condition still exhibited a stronger belief in the relation between height and prosocial behavior than did the control condition. So, the debriefing lowered belief in the relation but not sufficiently.

Greitemeyer provides an explanation for these effects. It turns out that the number of explanations that subjects gave for the relationship between height and prosocial behavior correlated significantly with post-debriefing beliefs. A subsequent analysis showed that belief perseverance in the debriefing condition appeared to be attributable to causal explanations. So retraction does not lead to forgetting and that this cognitive inertia occurs because people have generated explanations of the purported effect, which presumably lead to a more entrenched memory representation of the effect. 

But we need to be cautious in interpreting these results. First, it is only one experiment. A direct replication of these findings (plus a meta-analysis that includes the two experiments) seems in order. Second, some of the effects are rather small, particularly the important contrast between the control and the no-debriefing condition. In other words, this study is a perfect candidate for replicating up.

After a successful direct replication, conceptual replications would also be informative. As Greitemeyer himself notes, a limitation of this study is that the subjects only read a summary of the results and not the actual paper. Another is that the subjects were psychology students rather than active researchers. Having researchers read the entire paper might produce a stronger perseverance effect, as the entire paper likely provides more opportunities to generate explanations and the researchers are presumably more capable of generating such explanations than the students in the original experiment were. On the other hand, researchers might be more sensitive to retraction information than students, which would lead us to expect a smaller perseverance effect.

Greitemeyer makes another interesting point. The relation between height and prosocial behavior seems implausible to begin with. If an effect has some initial plausibility (e.g., meat eaters are jerks) retraction might not go very far in reducing belief in the relation.

So if Greitemeyer’s findings are to be believed, a retraction is no safeguard against undead findings. The wights are among us...

*The article is unfortunately paywalled