My former colleague Roy Baumeister famously said that replication
is a "career niche for bad experimenters.”* I like to use this quote in my talk. Roy
is wrong, of course. As anyone who has tried to conduct a replication study
knows, it requires a great deal of skill to perform replications. This leads to
the question Is there a career niche for
replicators?
----
* The link is to a paywalled article but I'm sure you can scihub your way to it.
I was asked this question yesterday when I gave a talk on
Making Replication Mainstream at the marvellous Donders Institute for Cognition, Brain, and Behaviour in Nijmegen. I get asked this question regularly. My standard
answer is that it is not a good career choice. Implicit in this answer is the
idea that in order to become a tenured faculty member, one has to make a unique
contribution to the literature. Promotion-and-tenure writers are always asked
to comment on the uniqueness of a candidate’s work. Someone who only conducts
replication studies would run the risk of not meeting the current requirements to become and remain faculty members.
During lunch, a group of us got to talking some more about
this issue, to which I hadn't given sufficient thought, as it soon turned out.
It was pointed out that there is a sizeable group of researchers who would like to remain in science, have excellent methodological skills but don’t necessarily have the ambition/creativity/chutzpah/temerity to pursue a career as faculty member.
It was pointed out that there is a sizeable group of researchers who would like to remain in science, have excellent methodological skills but don’t necessarily have the ambition/creativity/chutzpah/temerity to pursue a career as faculty member.
These researchers, was the thinking at our lunch table, are
perfectly suited to conduct replication research. The field would benefit
greatly from their work. If we truly want to make replication mainstream, there
ought to be a career niche for them.
If faculty member is not a viable option, then what would be a good career niche for replicators? It was suggested at our table that replicators should become staff members,
much like lab managers. They would not be evaluated on the originality or uniqueness of their
publications. In fact, maybe they would not even be on the
publications, just as lab managers often are not on publications. Faculty
members select studies for replication and replicators conduct them and by doing so make a value contribution to our science.
I think this is a fair summary of our discussion. I have no
strong opinions on this career niche for replicators yet but I wonder what ya'lls thoughts on this are.
----
* The link is to a paywalled article but I'm sure you can scihub your way to it.
I think this is an interesting idea. If we imagine that such a person might also want to teach some classses (maybe especially classes in research methods?) and the replication studies provide opportunities for undergraduate students to learn valuable research skills in a hands-on way, then the faculty member option might be viable. I agree that replications probably wouldn't meet the criteria for tenure, but perhaps this could be a non-tenure track option, or a strategy for teaching-oriented institutions that don't expect their faculty to do a lot of original research.
BeantwoordenVerwijderenI bring this up because there is limited funding and institutional support for replication research. In the framework I described, the research space, participant pool, IRB, etc. could come from the institution, the research staff could be undergrad students, and the funding could be a combination of funds from the department/institution (as a form of teaching) and research funds from (tenure-track) faculty who want to see a particular effect replicated because it is relevant to their own research.
Yes, the person could teach classes or labs. In my department, lab managers are sometimes involved in lab classes. I worry about non tenure-track positions, as this wouldn't really give these researchers much of a career prospect.
VerwijderenYour points about funding are well-taken and your suggestions in this regard are interesting.
With all due respect, I think this is a (very) bad idea. It institutionalizes the implicit, popular, and wrong assumption that a) being a good researcher primarily means having original and sexy ideas (a notion that has brought trouble upon us), and b) doing replication is somehow second rate and/or less important. As for a) it is obviously very important that a researcher doesn't only have original and sexy ideas but also that they are able to provide solid and reliable evidence for that or any other idea. With regards to b) performing a good, reliable, valid, and preferably collaborative replication requires *more* communicative, experimental, and statistical skills than finding evidence that (in the eyes of reviewers) justifies publishing a new idea.
BeantwoordenVerwijderen
VerwijderenI noticed you snuck "sexy" in there. Promotion and tenure letter writers are not asked to judge whether the research is sexy--in my experience at least. They are asked to consider whether the research is unique (often meaning separate from that of the graduate advisor). I also think you're introducing a false dichotomy. Original is not the opposite of solid and reliable. The best use of replication will be in the context of original research. Collaborative efforts such as PsyAccellerator will be crucial in this endeavor.
"Different from grad advisor" is indeed not sufficient for being considered "original" but it is necessary. Yep, you're right. You didn't introduce a false dichotomy. We do indeed agree that an original researcher must be able to conduct good replications. As I noted in the post, there is a large group of researchers who a comfortable performing replication studies but have difficulty coming up with their own ideas (this is not just my observation, it was raised by an ECR at the colloquium). Currently, there are no real career prospects for these individuals. A staff position might be an option.
VerwijderenI agree, replicators need a great deal of skill. However, it is not sufficient to be a skillfull methodologist, you also need to know the field in which you are operating. As "direct" replications are never identical with the original studies, minor deviations may exert serious psychological consequences. Of course, I am thinking of the camera in the pen-study replications. Had the replicators been knowledgeable about the impact of objective self-awareness, the would have hidden the camera to avoid the original effect to be suppressed. The same is true for replications in other areas. Unless replicators have a sufficient expertise in the area of interest, nonreplications have to be taken with great restraint. Thus, I am skeptical that replicators could make up a special branch of researchers.
VerwijderenI agree that there needs to be knowledge about the field one is operating in. In the proposal discussed in the post, this would be the bailiwick of the faculty member who supervising the replicator.
Verwijderen