Friday, February 15, 2013

Catching Fly Balls is not Reading Dostoyevsky


In 2001 and 2002, my students and I published two papers on mental simulation in Psychological Science. I warned my students that these experiments might draw a lot of attention and possibly criticism. I was about to make full professor, so I was not worried about myself but I was worried about my students. I was particularly concerned about a backlash from traditional psycholinguists.

I was wrong. There was no backlash. Instead, other people started using our paradigm and made nice careers for themselves doing so. It stayed like this until 2008. Then a paper appeared co-authored by Mahon and Caramazza.

They called our experiments “elegant and ingenious” (thanks for that) but argued that our results did not rule out an account in which the actual work was done by abstract symbols rather than perceptual representations, with activation cascading down from abstract representations to the perceptual and action systems.  (This hypothesis does not seem implausible to me and even has a certain appeal but Mahon and Caramazza didn’t offer any direct evidence to support it.)

Mahon and Caramazza used our experiments (among others) to stake out their position that cognition involves interactive activation between abstract symbols and the systems of perception and action. In their assessment, our account was too embodied.

Now there is a paper by Wilson and Golonka who state that these same experiments are not embodied enough (there is no pleasing people, is there?). They use them (and others) to advance their claim that cognition does not need mental representations. That pretty much rules out all of cognitive science (including Mahon and Caramazza) but let’s roll with it.

What evidence do Wilson and Golonka provide for their claim? They cite a book by Pfeifer and Schreier. I used this book in a seminar I taught about 10 years ago and greatly enjoyed it. The students and I were fascinated by the descriptions of little robots that could display forms of intelligent behavior purportedly without having mental representations. It is debatable whether these robots really had no mental representations but let’s roll with it.

As the students and I worked through the book, we kept wondering when it was going to address issues cognitive scientists are actually interested in, like understanding language, solving problems, and reasoning. The robots had nothing to say about this. They were mostly rummaging around, industriously collecting empty coke cans.

Once in a while over the past decade, I kept checking back to this literature but it never seemed to come closer to addressing the questions of interest to cognitive scientists.

In fact, Rodney Brooks, the father of the subsumption architectures on which these robots were based, has left the field altogether to become CEO of a company that produces the roomba, a vacuum cleaner with a subsumption architecture. It hides under your couch when you are around but comes out when nobody’s there and cleans your carpet. Very useful of course but a far cry from being able to comprehend a story, write a scientific paper, or reminisce about the past.

Wilson and Golonka discuss some other evidence in support of their claim. They describe interesting research (by others) on how an outfielder catches a fly ball. Obviously, catching a fly ball involves action and perception in the environment.

They also discuss impressive work by Thelen on the A-not-B error. Again, though, the task is very much an “in-the-world task”; it involves perception of and reaching for an object.

Most tasks that cognitive scientists are interested in do not have such strong connections with the environment. The authors, of course, are aware of this and state This is the point where standard cognitive science usually jumps in and claims that conventional meaning requires representational support.

You can hardly blame the standard cognitive scientist for wanting to jump in here because you can drive a truck through the gap in their line of argumentation.

Wilson and Golonka acknowledge that language is a tremendous step up from the other examples they have discussed but are optimistic they can take this step. I can’t see the reason for this optimism because all they provide is a very loose discussion of conventions and situation semantics. I am worried that they are underestimating how tremendous the step really is. 

The authors talk about any cognitive task as solving a problem and leading to an overt response. But I can read Crime and Punishment, I can listen to it on tape, and blind people can read it using Braille. These tasks differ on the surface, but the differences are minor with respect to what cognitive scientists are most interested in in cases like this, namely the end result: a mental representation of the described situation. Reading Crime and Punishment is not the same as catching a fly ball. Calling it solving a problem seems a stretch and it does not lead to an immediately observable response.

The main problem with nonradical embodiment research, according to the authors, is the assumption that cognition is done in the head. Of course it is done in the head. Where else would it be done if you’re sitting on your couch reading Crime and Punishment?

Wilson and Golonka are critical of our 2001 experiment, saying that no task analysis was done. They could have leveled that criticism against pretty much any other cognitive experiment but it’s apparently especially problematic when it concerns ours. But do they have ideas how such an analysis ought to be performed? No. They point out that even Gibson himself (making him sound like the L. Ron Hubbard of ecological psychology) had a hard time coming up with something useful here. So they leave it at a vague critique of our experiment.

That said, I do think the authors have a good point that more attention should be given to task analysis. I just would have liked to get specifics.

The article is generally very low on specifics. Do the authors present any experiments themselves that can convince us that their approach can scale up from catching fly balls to reading Dostoyevsky or constructing an argument? No. Do they at least suggest the outlines of such experiments? Nope.

The article is a manifesto, a rehashing of old ideas that have already been shown to go nowhere (well, they’re hiding under your couch). It provides no roadmap of how we get from catching fly balls to something cognitive scientists care about. It also contains several mischaracterizations of the criticized literature. In fact, there is another recent proposal on the role of embodiment that seems a lot more promising.

That said, to each his own and I see nothing wrong with encouraging Wilson and Golonka to continue on their quest. To have real impact, though, they need an experiment: the kind of experiment that people from various theoretical persuasions will use a decade later as a reference point for framing their own theoretical claims.


9 comments:

  1. I think the main problem is that you've misread this paper as being our attempt to say everything is solved. This paper isn't the argument winner and was never intended to be; it's the starting point of a research programme that we want to convince people is worth their time.

    Some specific points:
    The main problem with nonradical embodiment research, according to the authors, is the assumption that cognition is done in the head. Of course it is done in the head. Where else would it be done if you’re sitting on your couch reading Crime and Punishment?
    As we make very clear, the problem is the assumption that cognition has to happen only in the head. The embodiment hypothesis is that cognition and behaviour emerge from the activity of dynamical systems which include components from the brain but also the body and the environment, coupled via perceptual information. This is equally true whether catching balls or reading a book. Our claim, the radical claim, is that once you allow for this, then the bit that the brain is responsible for is going to look a lot different than standard cognitive psychology thinks, so we need to back up and rethink a few assumptions.

    Wilson and Golonka are critical of our 2001 experiment, saying that no task analysis was done. They could have leveled that criticism against pretty much any other cognitive experiment but it’s apparently especially problematic when it concerns ours.
    This is true; pretty much any cognitive experiment falls down here, and this is the problem. I wouldn't take the fact you have two papers mentioned in ours too personally, though; I came across your Eiffel tower study because it was getting a lot of press and one of our reviewers suggested we look at sentence-verification stuff as another 'embodied' option that's out there. As I said on Twitter, we're really not that interested in your stuff per se, it's the field more generally we're worried about

    what cognitive scientists are most interested in in cases like this, namely the end result: a mental representation of the described situation
    This is a weird move, but surprisingly common; defining cognitive science as the hunt for representations, thus making non-representational cognitive science a nonsense idea from the start. But cognitive science is actually in the business of explaining behaviour, and if you can do so best without representations then this should worry no one. We've pointed to work where the non-representational account beats the representational one; you should also read Louise Barrett and Tony Chemero's books for more work in this vein, there's plenty of it.

    we kept wondering when it was going to address issues cognitive scientists are actually interested in, like understanding language, solving problems, and reasoning.
    This is a related weird move. Cognitive scientists are interested in a lot more than these kinds of things; many of us are very interested in perception and action, for example.

    They also discuss impressive work by Thelen on the A-not-B error. Again, though, the task is very much an “in-the-world task”; it involves perception of and reaching for an object.
    This shifting of the goal posts always cracks me up. Until Esther and Linda's work, the A-not-B was a paradigm case for representational cognitive developmental psychology. Now they've shown it emerges from the dynamics of infant reaching, people now move and say 'well it's just about action'. This used to drive Esther a bit nuts, and with good reason; this sort of slipperiness is a sign you're not fighting a good theoretically grounded attack.

    ReplyDelete
    Replies
    1. "But cognitive science is actually in the business of explaining behaviour, and if you can do so best without representations then this should worry no one. "

      Agreed. I just don't think this is going to work for things like language comprehension, which happens to be an important topic in cognitive science and one I happen to be interested in. And you have done nothing to convince me otherwise. So if your approach can solve your problems, wonderful. Just don't claim that it can solve mine.

      "this sort of slipperiness is a sign you're not fighting a good theoretically grounded attack."

      Unlike the sort of language comprehension that I described, the A-not-B error deals with action in the world. It just doesn’t scale up to the issues I’m interested in. So to me it looks like you’re shifting the goal-posts.

      Delete
    2. Language also involves action in the world! This means the dynamics of things like speech and writing and the information these dynamics create are there to play a role, just as they are in perception/action tasks. The hypothesis is that those dynamics will do the same kind of work in language, which will alter what internal mediation language requires. And there's the research pathway you were looking for; experimentally investigating that hypothesis.

      Delete
    3. Sure. There are many language tasks like that But I refer again to my Dostoyevsky example. Whether you hear or read the story is relatively trivial for your understanding. For example, I just don't see how your approach will tackle anaphoric resolution, grammatical aspect, global coherence, and a host of other issues.

      Delete
    4. The argument 'I don't see how it's possible', while a common response, is not much of an argument. What, precisely, are we lacking? Representations? If so, it's worth repeating (again) that when we reject representation we don't reject internal mediation; clearly the brain is up to something, we just don't think it's representing anything.

      The problem with representations has always been grounding their content; embodiment is supposed to help that but it only can if perception and action are sufficiently useful, and as soon as they are (we argue) then whatever the brain needs to be doing, it won't need to be representation. We've had some preliminary thoughts on the consequences of this new job description for the brain but quickly realised we need neuroscientists on board to help us out; hence our upcoming Frontiers topic.

      Delete
  2. The article is a manifesto, a rehashing of old ideas that have already been shown to go nowhere (well, they’re hiding under your couch).
    Manifesto is a rude word to use with a scientist. It's a position paper, laying out our reasons for doing things in a particular way. And going nowhere? You really haven't been keeping up with the literature; you should read more work by Louise Barrett, Randy Beer, Geoff Bingham, Linda Smith, etc etc.We haven't pulled this out of thin air, it's the structure underpinning the research we think is doing the best.

    It provides no roadmap of how we get from catching fly balls to something cognitive scientists care about.?
    We provide four steps and two detailed worked examples. It's up to interested readers to apply these to their work, we can't run every study. We're obviously happy to clarify anything that needs it if people want to apply it but don't quite see how; in fact, we've already had a couple of enquiries along these lines, so maybe we can start building on those.

    To have real impact, though, they need an experiment: the kind of experiment that people from various theoretical persuasions will use a decade later as a reference point for framing their own theoretical claims.
    This is the Psych Science model of psychology, and as I said on Twitter, what we really need is a good 20-30 years of theoretically structured hard work in collaboration with as many other people as possible. There is no slamdunk experiment that will convince anyone (partly because representations are so poorly defined that you can always whack one into your explanations); but our job now is to accumulate successes. You've got a head start on us; if we're still where we are now in 20 years, then we will have failed. But not yet.

    ReplyDelete
  3. "This is the Psych Science model of psychology"

    I didn't mean you ONLY needed one experiment. After all, it's not exactly like I have only one experiment either. But one would have been better than none. Right now there is no specific idea in the paper to tackle the issues that you you say other people aren't tackling well.

    As I said in the paper, I wish you well on your quest. I think your approach might prove very useful for the things you're interested in. I just don't think it's going to produce anything that's useful for researchers interested in topics like language comprehension. To think it would shows an ignorance of the complexity and subtlety of language.

    ReplyDelete
  4. In its unscientific sense, 'manifesto' seems a fair description of much of the position paper. It's a statement of intention.

    What frustrated me is that I got to the end of it without a clear idea of how they were going to deal with language in practice.

    And yet an embodied approach seems a natural one to explain our and behaviour.

    As far as I can see, research into artificial intelligence in various forms is finding that to try to emulate or understand humans is less about algorithms and more about organisms. And it's quite hard to start at the bottom and come up with anything like our intelligence. This is a problem of complexity.

    So I agree that although reading a novel seems a long way from catching a ball, it could be just a matter of getting to grips with the complexity, as with the wolf pack model. But it is a much harder problem.

    I had a look at http://www.academia.edu/749401/Situation_models_in_language_comprehension_and_memory and I could make sense of the examples - not least the inclusion of movies - how cutting conventions support an intelligible narrative.

    But I was disappointed that the fly-ball example wasn't explored more in relation to language. The relevant convention I would have thought was throwing and catching. The ball could just as well be anything else that happens to be to hand. JP de Ruiter seems to explore this stuff. Communication involves everything present in the scene, including whatever form our memory takes and our mental processes.

    Why should someone throw a ball at me, and why should I catch it? Why write a novel, why read it?

    How do we use the energy arrays? Say I'm at a party and I switch my attention between 2 conversation - what resources have I called on so that I can now understand the second conversation and not the first (although I may still respond to my name if it occurs in the first conversation)?

    You can include representation - whatever that is - in your models as you wish, so long as you come up with something that illuminates the human condition.

    Whether or not Rolf's model is ultimately better than Andrew and Sabrina's, he seems a lot further on with language. I think they have the potential to come up with strong insights, but I'd like to see some good examples of what they intend.

    By the way, I'm still trying to start a conversation about these things on my blog - http://pabloredux.wordpress.com/2014/01/14/art-of-the-talkie/ - not scientifically, but just conversation, so with room for digression, ignorance and generalisation. Visitors welcome!

    ReplyDelete