Drang naar Samenhang

A while back, Lorne Campbell wrote a blog post  listing the preregistered publications from his lab. This is a great idea. It is easy to talk the talk, but it’s harder to walk the walk. So under the notion that we don't want to be all hat and no cattle, I rounded up some replications and preregistered original papers that I co-authored. First the replications. I find performing replications very insightful. My role in two of the RRRs listed below (verbal overshadowing and facial feedback) was rather minor but the 2016 RRR and the issues surrounding it, on which I've blogged  before, felt like an onslaught. The 2012 replication study was used to iron out an inconsistency in the literature. An additional replication study is close to getting accepted and will be added to the list in an update. These days I use direct replications primarily when I want to build on work by others. As per Richard Feynman, before we move on we first need to attempt a direct replication of th

A short blog post that strings together 8 tweets that I sent out today about our new paper. Today our latest paper on grammatical aspect appeared in Collabra: Psychology . The article reflects the times we psychologists are living in. It does so not from the lofty perspective of the methodologist or statistician, but from the work floor on which the actual scientist (**ducks**) operates. Our first two experiments were inspired by Hart & Albarricin (2011). This research itself was inspired by some of our own work but took it from cognition into the realm of social psychology, as I described in this blog post . As the paper explains, these experiments were run in 2012, which is why they were not preregistered. Nobody was doing preregistration at the time. We were thinking to build on Hart and Albarricin (H&A) in what some would call a conceptual replication but which is better thought of as an extension. For the life of us, we couldn’t get an effect like that of H&A

Collabra: Psychology   has a submission option called streamlined review . Authors can submit papers that were previously rejected by another journal for reasons other than a lack of scientific, methodological, or ethical rigor. Authors request permission from the original journal and then submit their revised manuscript with the original action letters and reviews. Editors like me then make a decision about the revised manuscript. This decision can be based on the ported reviews or we can solicit further reviews. One recent streamlined submission had previously been rejected by an APA journal. It is a failed self-replication. In the original experiment, the authors had found that a certain form of semantic priming, forward priming, can be eliminated by working-memory load, which suggests that forward semantic priming is not automatic. This is informative because it contradicts theories of automatic semantic priming. When they tried to follow up on this work for a new paper, howeve

Today another guest post. In this post, Fernanda Ferreira and John Henderson respond to the recent and instantly (in)famous multi-authored proposal to lower the level of statistical significance to .005. If you want to discuss this post, Twitter is the medium for you. The authors' handles are @fernandaedi and @JhendersonIMB. Fernanda Ferreira John M. Henderson Department of Psychology and Center for Mind and Brain University of California, Davis The paper “Redefine Statistical Significance” (henceforth, the “.005 paper”), written by a consortium of 72 authors, has already made quite a splash even though it has yet to appear in Nature Human Behavior. The call to a redefinition of statistical significance from .05 to .005 would have profound consequences across psychology, and it is not clear to us that the broad implications across the field have been thoroughly considered. As cognitive psychologists, we have major concerns about the advice and the rationale for t

              A recurring theme in the replication debate is the argument that certain findings don’t replicate or cannot be expected to replicate because the context in which the replication is carried out differs from the one in which the original study was performed. This argument is usually made after a failed replication. In most such cases, the original study did not provide a set of conditions under which the effect was predicted to hold, although the original paper often did  make grandiose claims about the effect’s relevance to variety of contexts including industry, politics, education, and beyond. If you fail to replicate this effect, it's a bit like you've just bought a car that was touted by the salesman as an "all-terrain vehicle," only to have the wheels come off as soon as you drive it off the lot.*             As this automotive analogy suggests, the field has two problems: many effects (1) do not replicate and (2) are grandiosely oversold. D

I’m working on a paper with Alex Etz, Rich Lucas, and Brent Donnellan. We had to cut 2,000 words and the text below is one of the darlings we killed. I’m reviving it as a blog post here because even though it made sense to cut the segment from the manuscript (I cut it myself, the others didn’t make me), the notion of concurrent replication is an important one. The current replication debate has, for various reasons, construed replication as a retrospective process. A research group decides to replicate a finding that is already in the published literature. Some of the most high-profile replication studies, for example, have focused on findings published decades earlier, for example the registered replication projects on verbal overshadowing (Alogna et al, 2014) and facial feedback (Wagenmakers et al., in press). This retrospective approach, however timely and important, might be partially responsible for the controversial reputation that replication currently enjoys. A form of