Tuesday, September 17, 2013

How to Cook up Your Own Social Priming Article


  1.  Come up with an idea for a study. Don't sweat it. It's not as hard as it looks. All you need to do is take an idiomatic expression and run with it. Here we go: the glass is half-full or the glass is half-empty.
    2.     Create a theoretical background. Surely there is some philosopher (preferably a Greek one) who has said something remotely relevant about optimists and pessimists while staring at a wine glass. Include him. For extra flavor you might want to add an anthropologist or a sociologist into the mix; Google is your friend here. Top it off with a few social psychology references. There, you have your theoretical framework. That wasn’t so hard, was it?
    3.     Think of a manipulation. Again, this is nothing to get nervous about. All you need to do is take the expression literally. Imagine this scenario. The subject is in a room. In the glass-full condition, a confederate comes in with an empty glass and a bottle of water. She then pours the glass half full and leaves the room. In the glass-half-empty condition, she comes in with a full glass and a bottle. She then pours half the glass back into the bottle and leaves.
    4.     Think of a dependent measure. This is where the fun begins. As you may know, the dependent measure of choice in social priming research is candy. You simply cannot go wrong with candy! So let’s say the subjects get to choose ten pieces of differently colored pieces of candy from a container that has equal numbers of orange and brown M&Ms. Your prediction here is that people in the half-full condition will be more likely to pick the cheery orange M&Ms than those in the half-empty condition, who will tend to prefer the gloomy brown ones.
    5.     Get a sample. You don’t want to overdo it here. About 30 students from a nondescript university will do nicely. Only 30 in a between-subjects design?, you worry. Worry no more. This is how we roll in social priming.
    6.     Run Experiment 1. Don’t fuss about issues like the age and gender of the subjects and details of the procedure; you won't be reporting them anyway.
    7.     Analyze the results. Normally, you’d worry that you might not find an effect. But this is social priming remember? You are guaranteed to find an effect. In fact, your effect size will be around .8. That’s social priming for you!
    8.     Now on to Experiment 2. Come up with a new manipulation. What’s wrong with the glass and bottle from Experiment 1?, you might wonder. Are you kidding? This is social priming research. You need a new dependent measure. Just let your imagination run wild. How about balloons? In the half-full condition, the confederate walks in with an inflated balloon and lets half the air out in front of the subject. In the half empty condition, she half-inflates a balloon. And bingo! You’re done (careful with the word bingo, by the way; it makes people walk real slow).
    9.     Think of a new dependent measure. Why not have the subjects list their favorite TV shows? Your prediction here is that the half-full condition will list more sitcoms like Seinfeld and Big Bang Theory than the half-empty condition, which will list more crime shows like CSI and Law & Order (or maybe one of those stupid vampire shows). You could also include a second dependent measure. How about having subjects indicate how much they identify with Winnie de Pooh characters? Your prediction here is obvious: the half full condition will identify with Tigger the most while the half empty condition will prefer Eeyore by a landslide.
    10. Repeat steps 5-7.
    11. Now you are ready to write your General Discussion. You want to discuss the implications of your research. Don’t be shy here. Talk about the major implications for business, health, education, and politics this research so evidently has.
    12. For garnish, add a quirky celebrity quote. Don’t work yourself into a lather. Just go to www.goodreads.com to find a quote. Here, I already did the work for you: “Some people see the glass half full. Others see it half empty. I see a glass that's twice as big as it needs to be.”  George Carlin. Just say something clever like: Unless you are like George Carlin, it does make a difference whether the glass is half empty or half full.
    13. The next thing you need is an amusing title. And here your preparatory work really pays off. Just use the expression from Step 1 as your main title, describe your (huge) effect in the subtitle and your done: Is the Glass Half Empty or Half Full? The Effect of Perspective on Mood.
    14. Submit to a journal that regularly publishes social priming research. They’ll eat it up.
    15. Wax poetically about your research in the public media. If it wasn’t a good idea to be modest in the general discussion, you really need to let loose here. Like all social priming research, your work has profound consequences for all aspects of society. Make sure the taxpayer (and your Dean, haha) knows about it.
    16. If bloggers are critical about your work, just ignore them. They’re usually cognitive psychologists with nothing better to do.
    17. Once you’ve worked through this example, you might try your hand at more advanced topics like coming out of the closet. Imagine all the fun you’ll have with that one!
    18. Good luck!


23 comments:

  1. I've been wanting to write an article like this for ages. I was thinking of researching whether people believe that something will happen "when pigs can fly", and submitting it to the Journal of Porcine Aviation Potential.

    ReplyDelete
    Replies
    1. +1! But oh boy, I wonder what the response will be. I'll just post this on the social psych facebook page and see what happens.

      Delete
    2. Great name for a journal! I learned about Porcine Automotive Behavior from Richard Scarry http://www.thingamababy.com/baby/2009/12/contest-5.html, so the sky is the limit.

      Given that this blog covers everything, I already have a post on flying pigs http://rolfzwaan.blogspot.nl/2013/05/fun-with-flying-pigs-importance-of.html.

      Delete
    3. The Journal of Porcine Aviation Potential just scored its first reference in the peer-reviewed literature: 10.1177/1948550616673876

      Presumably it now has an impact factor. Next stop: get it listed by the Web of Science.

      Delete
  2. Lol, very funny! On the serious side, blogging this may cause you to be scooped :)

    ReplyDelete
    Replies
    1. Somehow, I anticipate this will happen.

      Delete
  3. Sounds like fun, Rolf, I'd like to try it. But I have a couple of questions:

    Why should I call this social priming? There's nobody else there to make it social, and it's all about processes that are happening in someone's mind, so shouldn't it be cognitive priming?

    Also, I have a limited research budget and I've been following your advice about only running 30 subjects. It finally worked, but it took me 17 tries. Does that mean that my first 16 attempts were wrong and I finally got it right? Does that mean I don't need to report the first 16 attempts? In any case, it turned out to be pretty expensive and time consuming because I had to run over 500 subjects to get one study to work. I feel slightly cheated -- you made it seem so easy. Are there any shortcuts?

    p.s. I meta-analyzed the 17 studies and found no effect, is that bad, or did I just see so many half-empty glasses that I'm being a pessimist?

    ReplyDelete
    Replies
    1. You have a point there, Dave. The same goes for a lot of other studies labeled as "social priming." Maybe they should be called "environmental priming"?

      17 tries? Hmm, I based my recipe on the literature and there is no mention of multiple tries anywhere. So you cannot be right. Are you sure you did everything correctly?

      Depends on how those glasses got empty. Did you drink the contents?

      Delete
    2. What is considered priming in this case? Apart from it being a stimulus providing semantic/embodied information I'm not sure what it means when you call it social priming.

      Since we can distinguish between subliminal, supraliminal and overt priming, priming simply equals providing information. It seems that social priming is often seen as unconscious (Bargh style) priming directly affecting behavioral outcomes. This is very much different from the embodiment-methapor example above in which often the outcome rather than the prime is social. If I would come up with a version for this I would defintely follow your guideline :)

      (love this blog!)

      Delete
    3. I always mess up recipes, it's probably my fault. I most likely used the wrong kind of candy in Step 4. Also, I probably should have pre-heated the oven, but the subjects kept complaining. I give up -- I think I'm going to switch to neuroscience.

      Delete
    4. Ironically, I messed up a recipe a few hours ago. I wanted to make pasta alle vongole only to find out that I should have ordered the vongole in advance in order to get fresh ones. So I made Thai chicken instead.

      Ah, neuroscience! The Walhalla of researcher degrees of freedom.

      Delete
  4. Are you sure this only applies to social priming research?

    ReplyDelete
  5. Excellent - I am now thinking of starting up a collaboration with the dept. of veterinary medicine and work on the expression "to flog a dead horse"

    ReplyDelete
    Replies
    1. :) I only have one concern: how is candy going to figure into this? Sugar cubes for the horse are out, since it's already dead.

      Delete
  6. Great article Rolf! I think you forgot a few intermediate steps though that go something like...

    - Realize that your results are not significant with 10+ different dependent measures.
    - Find significance in 2 out of 20 experiments.
    - Report on those 2 and disregard the others as if they never happened.

    ReplyDelete
    Replies
    1. Thanks Neil! I didn't know this was part of the recipe.;) I merely told the aspiring cook to have confidence the batter will somehow will rise to the effect sizes we see in the literature of its own accord.

      Delete
    2. Oh yes, you need some good yeast to get those effect sizes.

      I did once have a colleague in a seminar who said that "some manipulations worked and some didn't." When asked by the professor if his social priming experiments worked about "1 out of 20" manipulations, he replied "yes" without any realization of the irony or why we all started laughing and staring aghast. *facepalm*

      Delete
    3. Rolf,
      Ironically, your half-empty/half-full manipulation (3) has been used in what I consider to be one of the best, and most solid papers in Cognitive Psychology. See Experiment 1, 2, and 3 here:
      http://psy2.ucsd.edu/~mckenzie/Sher&McKenzie2006Cognition.pdf
      My point here is that whether a specific manipulation or a dependent measure 'means' something, or seems 'smart' or 'silly', depends on its role in larger theoretical frameworks. For me, the above 2006 paper is all the better because it makes a profoundly interesting and novel theoretical point using such simple and ingenious methods.

      Delete
    4. Piotr, that's interesting. I agree with your point that the interestingness of a study depends on the context. Nevertheless, Step 1 seems to be an accurate description of many studies. As I've indicated in other fora, the main point is Step 7. These effect sizes are just unbelievable given the designs of these studies.

      Delete
  7. Rolf, of course I understand that your main point is the implausibility of a large effect size given a tenuous, indirect and fragile relation between an IV and DV in many of these studies (hence social psychologists often use massive amount of statistical control). But, again, somewhat ironically, the effect size in Sher and McKenzie glass-half-full/half-empty Study 1, is over .8. If you'd like, I can give you many cognitive priming studies with similar effect sizes -- a feat often accomplished by controlling the heck out of the stimuli, presentation times, distance, etc. In short, the effect size itself is not grounds for suspicions, only in the context of theoretically vague and implausible connection. I think there is no disagreement there. But I wanted to chime in since I am getting tired of people 'reflexively' criticizing a study if it reports a large effect.

    ReplyDelete
  8. Your own colleague Hal Pashler has a nice review on this, showing cognitive priming effects to be much smaller: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0042510. As he correctly points out, it is odd that social priming effects, which are much more indirect, should be so much bigger than cognitive effects. There is also this meta-analysis, which shows semantic priming effects to be between .10 and .30: http://www.ncbi.nlm.nih.gov/pubmed/11206202.

    Quite funny that I tried to make a parody and actually came up with something that makes more sense than the studies I was targeting.

    ReplyDelete
  9. Can we do something more constructive?

    ReplyDelete
  10. This post should integrate Neil's comments and it would be absolutely perfect.

    ReplyDelete