When I mentioned in yesterday’s post that Diederik Stapel
was contemplating writing a blog, the response was not as negative as he and I
(as well as one other person I had mentioned this to) had imagined.
It is clear that people are still angry at Stapel. But anger
is not a very useful emotion at this point. It’s a little bit like “Praying for
Boston.” Praying is not really helping anyone (as Richard Dawkins likes to
remind us) but it may make people feel good about themselves. By the same
token, rightful indignation about Stapel’s fraudulent behavior may make us feel
good about ourselves (why, aren’t we ethical?) but is not helping to improve
the way we do science.
Like everyone else, I was initially angry at Stapel. Well,
first I was bewildered at the depth and the scope of his fraud and then came
the anger. Because everyone else around me was also angry, I noticed that I was
starting to get a little less angry. (Maybe there is a social psychological
theory that can explain this phenomenon?)
I remember being asked by my university’s newspaper if I
thought there were more cases like Stapel’s. I
said I didn’t (because I didn’t). But only a month later, I found myself
chairing a committee charged with investigating alleged fraud committed by Dirk
Smeesters.
In my role of committee chair, anger was definitely not a useful emotion. The experience of serving on the Smeesters committee gradually made me take a more analytical perspective on the Stapel case. And what residual anger I might have felt was vented in lame jokes on Twitter at Stapel’s expense.
In my role of committee chair, anger was definitely not a useful emotion. The experience of serving on the Smeesters committee gradually made me take a more analytical perspective on the Stapel case. And what residual anger I might have felt was vented in lame jokes on Twitter at Stapel’s expense.
So back to the question at hand: Should Stapel write a blog?
Various people who have responded to my post—via the social media on via
email—have said they’d be interested to hear what he has to say.
If Stapel uses the blog to justify his prior actions, this
would again fan the flames and rightfully so. Obviously, presenting “empirical”
studies would also be an astoundingly bad idea.
But there are other things Stapel might write about. The
cognitive psychologist Dermot Lynott (@DermotLynott) suggested a good analogy
on Twitter. Stapel’s role could be similar to that of Frank Abagnale, a
convicted fraudster turned government consultant. An interesting thought.
I think Stapel should definitely write blogs.
BeantwoordenVerwijderenWhen news of Stapel's fraud broke, I felt a sort of sympathy for Stapel. Except for the blatant data fabrication, Stapel was only guilty of conducting research according to established conventions in the field. Consider this statement from the Stapel report:
An experiment fails to yield the expected statistically significant results. The experiment is repeated, often with minor changes in the manipulation or other conditions, and the only experiment subsequently reported is the one that did yield the expected results. It is unclear why in theory the changes made should yield the expected results. The article makes no mention of this exploratory method; the impression created is of a one-off experiment performed to check the a priori expectations. It should be clear, certainly with the usually modest numbers of experimental subjects, that using experiments in this way can easily lead to an accumulation of chance findings...
That statement (as well as the majority of section 5 from the report) describes verbatim how research proceeds in many social psychology labs I have observed. I felt that the discovery of Stapel's fraud was actually unfortunate because it detracted from what I believed to be the more harmful, subtle, and widespread practices in the field. (fortunately, it added to the simmering discontent which has sparked what some are calling "the revolution").
So I sympathize, and I am very interested to hear Stapel's perspective. I think you are right (in your previous blog) that although his empirical credibility is nonexistent, he may have valuable insight into how the widespread practices in the field produce a "flawed science." I love the analogy to Frank Abagnale.
However, I think Stapel should be prepared for resentment, skepticism, perhaps even verbal hostility. Unlike you, I question his theoretical knowledge (perhaps he is knowledgeable about the content of the field, but I don't know if that amounts to actual theoretical knowledge). But, if he can accept that such resentment and skepticism is probably warranted, and can contribute to the discussion of (1) an insider's view of the research culture that enabled his fraud, and the effect it might have on the theories in the field, and (2)how to create better research practices that deter not just blatant fraud, but the more subtle practices described by the quote above, than I think that could be very valuable.
In a way, he is in a unique position to comment on (1), because many people cannot do so without incriminating themselves or their colleagues. Obviously it would be difficult to Stapel to be incriminated further.
p.s., I also found it interesting that the European cases were investigated completely openly, whereas much secrecy surrounded the fraud discovered in the U.S.
This quote from the Stapel report is spot on. You're right that this seems to be a common practice. You can find a comment to the same effect in the Smeesters report by one of the people we interviewed. In fact, I was told (this is not in the report) that I "didn't understand the field" because this is how it works: you conduct multiple experiments "until you get the effect."
VerwijderenStapel seems to be very prepared for resentment and verbal hostility. He already has received a great deal of hostility as he describes in his book.
Your last two points are excellent. I also think Stapel is in a unique position to give an insider's view on the culture that gave rise to his fraud. In fact, he already gave me some examples of this, which I didn't put in my earlier post.
You're also right that the European cases were investigated completely openly, whereas there is a shroud of secrecy in the Sanna case. And the case of Hauser is also peculiar.
I think Sarah's comment is spot on: Stapel's blog could be very interesting, because he has inside knowledge that he can freely write about, since he has nothing to lose. Whether it is wise for him to do so is another matter, because it will probably invite an enormous amount of vitriol. The internet can be a very harsh place, and it's not just us gentle academics that will read his blog. Also, I don't think he can stay away from discussing issues that might be construed as self-justification: what made his book most interesting to me were not the 'technical' issues -- the questionable research practices he employed and observed around him, the way he fabricated data -- but rather his reconstruction of what drove him down that road, the way he justified it to himself, his ever more baroque self-deception, the gradual moral perversion if you will. Questionable research practices are not just tricks that researchers use to pump up their data, the way Abagnale used his con tricks, they're things they have to justify to themselves or avoid thinking about, justify to others or avoid talking about. I think that moral fog is part of the culture that sustains questionable research practices, but discussing it will easily sound like self-justification.
BeantwoordenVerwijderenI agree. I bet he is aware the internet is a harsh place. My sense is that he is genuinely trying to find out what drove him to do the things he did. This is in fact why he wanted to meet.
VerwijderenGreat point that his discussing of the moral fogmight be perceived as self-justification. Nevertheless, his comments can help us understand the culture better. And we really need to understand it better if we want to change it.
I kind of disagree. When a person makes up a data-set, you do not have outliers, weird conditions, nonsignificant results turning significant when adding a covariate, etc. If your data does not work, you just make up better data.
BeantwoordenVerwijderenSince Stapel basically made up all his data, I would argue that out of all social psychologists, he probably has the least experience with questionable research-practices.
Stapel created his own research-culture. One where the data were always perfect. He may give insight into that self-created, but not representative, culture. I doubt he has anything interesting, new and verifiable to tell about the more representative cultures where questionable research practices may (probably) happen to some degree.
Besides questionable research practices, Stapel could possibly give insight into his internal motivations. But there we have to be careful. His previous statements and interviews have not all been completely true to put it mildly. This makes sense, of course he is motivated to portray, and maybe even believe, a version of the story that makes him look good (or, less bad at least). However, that does mean that it will always be unclear how much value to attach to his explanation about internal motivations. It is unverifiable, and therefore will always be shrouded in uncertainty.
Finally, he could say something about the stats, how he made up data. This is more objective and can be verified. However, his failure to make up good data, makes me wonder how interesting that would be. Information about how he did it will not help us prevent new cases, only to catch others who have already done it, and since him being caught means that his system was not good enough, this will not help us catch people who are a.t.m. getting away with making up stats.
I am not saying he should not blog, and probably it would be interesting for some people to read, I just do not see where anything he would say about his fraud would be a valuable contribution to the academic discussion of fraud and questionable research practices. Regarding qrp's and detecting fraud, others have probably better knowledge, and regarding the internal motivation, the information will be too subjective and thus untrustworthy.
Your first two paragraphs are both humorous and insightful. I assume though that he underwent a transition from QRPs to outright fabrication.
BeantwoordenVerwijderenBesides the things you mention, Stapel could talk--as the other commenters have said--about the culture that allowed him to flourish.
Deze reactie is verwijderd door de auteur.
VerwijderenThe first assumption is easy to test; just look at his earlier work. Since fabrication has been determined to be extremely likely in the case of his dissertation, Stapel can not have dabbled in qrp's for long (if at all).
VerwijderenI agree with him being able to comment on the culture, but, I am afraid that claims about this culture are hard to verify. Assuming we still believe in classical social psychological findings, we should expect these comments to be biased.
(Not saying this to be judgmental btw, I am sure if you would ask me for my life-history it would be biased and self-serving as hell.)
But, hey, let's see. I am happy to be proven wrong. Anything that improves the system is a good thing in my book. I just wouldn't put my money on it.
(deleted previous comment for small edit)
Irrespective of outrage, we can view Stapel's case from tax-payers perspective. Basically he defrauded government. He asked for funding for experiments but made up the data and further wasted man-hours of phd students and post-docs who analyzed these data. What is the difference to some government contractor who obtains money, workforce and landscape property rights to build a highway in 5 years time and instead spends the resources on planting carrots in the fields? After 5 years we found out that the company developed it's own "construction culture". So we just dismiss the company. They are banned to apply for any future construction contracts. They write a book about how their plan to plant carrots was derailed and maybe they start blog. This is absurd.
BeantwoordenVerwijderenOf course there are differences between construction and science funding. The assessment of the science project and its impact whether before or after it was done is difficult. Peer-review is unreliable and in the case of non-applied research it is difficult to reason about the utility of research. While in highway construction there are standards for evaluating the final product, in science these are hazy and often arbitrary (e.g. impact, innovativeness).
Anyway, in Stapels case even the most hazy and loose standards were transgressed, so what should we do? From tax-payers perspective, the least thing Stapel might do is to belatedly provide the data, analyses and research he was payed for. This may be difficult because some of his research is incremental - it build on the top of previous fictional research. It is as if the construction company asked and obtained extra money for drilling a tunnel through a big mountain but neither the tunnel nor the mountain was there. And we should not forget other researchers who spent time drilling through the fictional mountain. The bottom-line is that funders should improve the evaluative standards. In fact, outright fraud is of little concern, sloppy research is much bigger concern and is causing a much bigger waste of tax-payers money.
What I really don't understand about Stapels case is why he gave in? Why did he confess to making up the data? He could just refuse to admit any misdeeds, take a hiatus and return again when the heat is off - just like smeesters, sanna and hauser did. My impression of stapel is that he is confused. He should figure out what he wants and how he wants to achieve it. Then he may start blogging.
I like your analogies. I think our science would be a lot better if we regarded ourselves as engineers building bridges. Would we be willing to cross our own bridges?
BeantwoordenVerwijderenYou make two important points. Outright fraud is disturbing but in the long run, sloppy research is a much bigger concern. Some people seem to think the problem has been solved after a few fraudulent researchers have been caught and some cosmetic changes have been applied.
I also don't understand why Stapel gave in where others did not. I'm glad he did because it made the Levelt committee's work that much easier. This is the kind of question I am interested in.
I can see why a Stapel Blog might be interesting and instructive. Still, I don't think it would be a good idea.
BeantwoordenVerwijderenBy writing such a blog, Stapel would in a way re-assert himself as a teacher; somebody from whom researchers can (and should) learn something important about their field. The problem with that is that he has shown himself to be an untrustworthy teacher who has no problems with abusing his position for selfish ends, at enormous cost to his pupils. If that does not disqualify him from taking on the teacher role, I'm not sure what would.
I don't think 'teaching' is the right word, he would be conveying his own experiences, impressions and interpretations of research culture. But no, I do not think his fraud has disqualified him from ever again being trusted on any topic whatsoever. As I've argued before, it's rather the reverse: he couldn't be trusted before, but now that he's been exposed it's worth listening to what he has to say for himself. I'm not saying one should believe everything he says, but to simply exclude him from all communication seems unwise to me.
VerwijderenSo I don't think one can argue that because he has lied before, he will never speak the truth again. As I acknowledged in the post, there will always be a suspicion of self-justification to what he says about research culture. But psychologists, and scientists in general, should also be wary of the fact that there might be an element of self-justification in their complete rejection of anything Stapel has to say. That's too convenient a way to avoid having to consider what might be inconvenient truths. I'm a bit suspicious of all the righteous indignation that's been poured on him.
Good points, of course. And I agree that, as a field, perhaps we cannot afford to not listen to what he might have to say, no matter how we may feel about it (or him). I did not really mean that we can never trust him to tell the truth, or that he should be ignored forever. That would be a simplistic view indeed. We need to allow ourselves to learn from his mistakes.
VerwijderenI do wonder, however, whether at this moment he can be considered sufficiently 'reformed' to be the one to teach us about these things (I do think it would be teaching, at least from his point of view). It's the same feeling I had about his book: I was expecting him to publish a book eventually, but I personally felt that it was inappropriate for him to do it so soon. I find it difficult to believe that somebody who has been behaving so dishonestly for such a long time can reform himself and give an honest public assessment of the whole affair within one year of being found out. That was my main reason for deciding not to read the book, at least not for now.
Of course I should (and do) question my own motives and convictions. I do not feel particularly outraged or indignant. I just have these gnawing suspicions. In the end, it's an empirical question whether such a blog would do any good. If it does, I would be all for it. But I don't think it actually will.