tag:blogger.com,1999:blog-6322739827777311964.post2176139479865517247..comments2024-03-29T05:56:48.403+01:00Comments on Drang naar Samenhang: Replicating pencils and eagles but not steaksRolf Zwaanhttp://www.blogger.com/profile/07617143491249303266noreply@blogger.comBlogger11125tag:blogger.com,1999:blog-6322739827777311964.post-26319849798827906092013-01-18T10:37:46.331+01:002013-01-18T10:37:46.331+01:00I like JEP and similar precisely because you have ...I like JEP and similar precisely because you have the room to provide that context with the data. But after a few years blogging I now find myself writing papers and wanting to just link to a post I've written, so I know the feeling :)Andrewhttps://www.blogger.com/profile/16732977871048876430noreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-32115176086340776742013-01-16T15:45:09.007+01:002013-01-16T15:45:09.007+01:00The advantage of a blog over an article is that yo...The advantage of a blog over an article is that you can provide background. Otherwise people are likely to infer intentions that might be completely alien to the author of the paper.<br /><br />I guess this is what I was trying to articulate in my post on funny article titles. Scientific articles will likely become dry, workman-like descriptions of confirmatory research (the typical JEP paper;)). Blogs will provide more context and more flavor. The middle category (e.g., Psych Science) might disappear or become more blog-like.Rolf Zwaanhttps://www.blogger.com/profile/07617143491249303266noreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-70538062706077135052013-01-16T15:22:26.089+01:002013-01-16T15:22:26.089+01:00I've found it exceedingly useful too. The turn...I've found it exceedingly useful too. The turn around is nice a fast as compared with published work, which helps resolve all those little confusions before they turn into major fights :)Andrewhttps://www.blogger.com/profile/16732977871048876430noreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-2215178527970221452013-01-16T13:20:54.659+01:002013-01-16T13:20:54.659+01:00Actually, blogging and responding to comments like...Actually, blogging and responding to comments like these is useful for me as well in terms of articulating views.Rolf Zwaanhttps://www.blogger.com/profile/07617143491249303266noreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-67350475436805513622013-01-16T10:35:08.639+01:002013-01-16T10:35:08.639+01:00So I see embodiment as a means and not as an end.
...<i>So I see embodiment as a means and not as an end.</i><br />That's a useful context, actually. Thanks, it clarifies a few things nicely, as does the rest of your thoughts. Thanks :)Andrewhttps://www.blogger.com/profile/16732977871048876430noreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-13965434527887859432013-01-15T12:19:16.869+01:002013-01-15T12:19:16.869+01:00Indeed, you cannot rely on a single task. This is ...Indeed, you cannot rely on a single task. This is why we have used other tasks. We have used tasks where the pictures precede the sentences and are ostensibly unrelated to them. The dependent measures are memory, reading times, or the N400. So there are extensions in the literature already. Pictures were found to affect these dependent measures in predictable ways.<br /><br />With regard to the sentence-picture verification task, we're investigating right now whether making the object more or less action relevant modulates the effect. If the object is completely irrelevant to the action that is being described, people may not bother to represent it or its orientation. Is this an example of the kind of perturbation you have in mind?<br /><br />Many of our studies were attempts to find the limits of the effect. If you find it with orientation, do you also find it with shape? Do beginning comprehenders, who are not yet efficient at decoding, show the effect? Can a similar effect be found with different tasks and different dependent measures? And so on.<br /><br />The new study (that we're working on as we speak) is indeed more like probing. However, that study is more about language comprehension in general than it is about embodiment. It relates to my earlier work on situation models, although there are some embodiment aspects to it. <br /><br />I'm not interested in embodiment per se, to be honest. I'm interested in it only to the extent that it can help me explain the question how we understand language. That's what I'm really interested in. So I see embodiment as a means and not as an end.<br /><br /><br />Rolf Zwaanhttps://www.blogger.com/profile/07617143491249303266noreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-42263613750770796962013-01-15T10:29:52.687+01:002013-01-15T10:29:52.687+01:00I just seems like you think we have not thought of...<i>I just seems like you think we have not thought of the issues you raise ourselves. I cannot speak for others, but I have thought and spoken about them many times. </i><br />No, I don't think that, and I'm not trying to ride in with my wisdom :) <br /><br />That said, I don't see a lot of evidence of concern about this variation in the published literature. You're replicating results, sure, but you also need the extension and the probing of potential mechanisms, including ones that aren't from the conceptualisation embodiment hypothesis. For example, can you break the effect by experimentally perturbing some critical element of the task? What would that element be?<br /><br />The upcoming big 14 experiment paper sounds more like this, though, is that right? Lots of poking around?Andrewhttps://www.blogger.com/profile/16732977871048876430noreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-64214946036337113512013-01-14T14:36:35.687+01:002013-01-14T14:36:35.687+01:00This is something we're going to have to inves...This is something we're going to have to investigate, hopefully in collaboration with Louise Connell. Right now, I'd put my money on our results because of the high power.<br /><br />I don't think the sentence-verification shape effect is fragile at all. It's been replicated several times by several labs and twice here, with awesome power.:) The orientation effect is weaker but was also replicated twice. We're currently investigating why the orientation effect is smaller. The main point is that the shape and orientation effects did not bounce around.<br /><br /><br />It's important to stress that these experiments are very much pitted AGAINST finding an effect. Subjects are not forced to understand each sentence and are not questioned about the location (let alone orientation) of objects.<br /><br />We do this because otherwise we'd run into the standard psycholinguistic complaint that subjects are using "special strategies" (as if presenting them with dozens of ungrammatical sentences is somehow normal). Also, our subjects are reading impoverished materials (just like in 99% of language experiments). It is plausible that you'd find larger effects if the task involved connected discourse. This is also something we're going to investigate. <br /><br />I'd say it is pretty impressive that we got these effects against these odds. Moreover, Bayesian analysis guards against false positives. So we have very strong evidence--replicated several times in high-powered experiments--for shape and orientation.<br /><br />I appreciate your interest. These are important issues to discuss. I just seems like you think we have not thought of the issues you raise ourselves. I cannot speak for others, but I have thought and spoken about them many times. <br /><br />The replication study is an attempt to clarify the issue. As such, your comments describe pretty much our motivation for running the study in the first place. More studies are under way.Rolf Zwaanhttps://www.blogger.com/profile/07617143491249303266noreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-67011547036215221012013-01-14T14:16:47.221+01:002013-01-14T14:16:47.221+01:00To play devil's advocate for a moment: you say...To play devil's advocate for a moment: you say your paper solves the problem. Why isn't the score 1-all between results that make sense and results that don't? <br /><br />The reason I ask is that I'm not that impressed by small, fragile effects; I think they are a hint that you've asked the wrong question. I'm interested in what people actually doing this research feel about it; are you worried that the effect bounced around, or are you happy now it's going in the expected direction?<br /><br />Sorry if this comes off as a bit rude; I'm genuinely interested in the answer but the question is inherently fairly pointed :)Andrewhttps://www.blogger.com/profile/16732977871048876430noreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-79447470720774659282013-01-14T11:47:03.853+01:002013-01-14T11:47:03.853+01:00It would not be appropriate for me to be open abou...It would not be appropriate for me to be open about the review process (many years ago I was a reviewer on Connell's CogSci paper). But let's just say I was not at all happy that a finding that was the opposite of earlier findings was interpreted as supporting the same hypothesis, just like you are.<br /><br />In Connell's defense, the amodal view predicts a Null effect so the fact that she got an effect suggests that reading the sentence influenced judging the pictures. <br /><br />In her second paper on this topic (of which I was not a reviewer), Connell points out why color might be different from shape and orientation. I think this is an interesting hypothesis but as we say in our replication paper, we're not totally convinced. <br /><br />So our paper basically solves the problem and your and my concern: all patterns are in the same direction.Rolf Zwaanhttps://www.blogger.com/profile/07617143491249303266noreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-66808608612556880722013-01-14T11:27:05.760+01:002013-01-14T11:27:05.760+01:00You note in the paper that Connell was happy that ...You note in the paper that Connell was happy that her original results supported an embodied account, even though they were the reverse of the typical pattern. You then find the typical pattern, and are happy that this is evidence for embodied cognition. <br /><br />My question is, what are your thoughts about what this says about this style of embodied cognition? Two published papers reporting opposite patterns of results both claim to support the same hypothesis, and no one seems that worried (well, except me :). Do you think this is a problem? If so, how big a problem, and if not, why not?Andrewhttps://www.blogger.com/profile/16732977871048876430noreply@blogger.com