tag:blogger.com,1999:blog-6322739827777311964.post1409300256780796162..comments2018-12-01T10:16:23.148+01:00Comments on Rolf Zwaan: Can we Live without Inferential Statistics?Rolf Zwaanhttps://plus.google.com/110098308161411677670noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-6322739827777311964.post-77875929463165577772015-03-22T16:19:56.480+01:002015-03-22T16:19:56.480+01:00typo: "... that any significant result ...&qu...typo: "... that any significant result ..." should be "... that every significant result ...".Eric Marishttps://www.blogger.com/profile/01359337132974282153noreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-21887615885079756892015-03-22T10:06:26.047+01:002015-03-22T10:06:26.047+01:00This is a radical editorial choice for which I see...This is a radical editorial choice for which I see no justification. The reproducibility crisis in science has nothing to do with the framework that you use for statistical inference. Instead, it has everything to do with selectivity (in participant selection, data analysis, and reporting). Any selection mechanism (including the Bayesian ones) through which you funnel noisy data will introduce bias.<br /><br />I'm curious to see what comes out of this experiment, but I do not expect any radical changes. At the end, scientists need a way to deal with the uncertainty (noise, randomness) in their data, and the different flavours of statistical inference are just different principled ways of data-based decision making under uncertainty. An editor may prohibit authors to report on the outcome of these decision-making tools, but he cannot prevent author or readers applying their own decision making tool on the data. They are then trying to solve the same problem (data-based decision making under uncertainty) that professional statisticians have tried to solve for us.<br /><br />In my own field (cognitive and systems neuroscience), we do need principled ways of data-based decision making under uncertainty. I can show you the most beautiful 2D maps of brain activity, free for interpretation like a Rorschach plate, but nothing more than smoothed noise. Of course, although correct application of the appropriate statistical techniques will identify these maps as pure noise, this thus does not imply that any significant result obtained with those techniques will also be scientifically revealing.<br /><br />Eric Eric Marishttps://www.blogger.com/profile/01359337132974282153noreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-4815387763489967932015-02-26T21:36:49.465+01:002015-02-26T21:36:49.465+01:00Regarding your questions:
"do the findings su...Regarding your questions:<br />"do the findings support the hypothesis?"<br />"can I cite finding X as support for theory Y?"<br />First, there are actually researchers who don't test hypotheses, but instead do research to answer theoretical questions. So the non-exclusive way to put it is "do the findings answer researcher's question?"<br /><br />Second, the usual way that the research is digested is this. Researcher presents results in the results section. Then in the discussion section she discusses how the results answer the question at hand and how well her answer is supported. The ban won't change anything about this. The additional argumentative step from results to conclusions has always been there and has always been necessary. <br /><br />Finally, as John Kruschke pointed out, the editorial does not discuss and exclude bayesian parameter estimation. So yes, there is inference beyond NHST...Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6322739827777311964.post-77048187849283606472015-02-26T18:19:34.520+01:002015-02-26T18:19:34.520+01:00As I've mentioned before, it's pretty stra...As I've mentioned before, it's pretty strange to accept submissions with p-values but to weed them out after acceptance. That's not so much an experiment in science as an prescriptivist editorial reflex in science reporting: p-values will still be part of the analytical process, and possibly even of the submission and review process.<br /><br />To try to take the editors' side (I don't really care since I've never heard of the journal in question before), decent descriptive statistics - both numerical and graphical - often render inferential statistics superfluous. One category is the 'so obvious it hits you between the eyes' one (cf. the intra-ocular trauma test). The other is the one where it's pretty clear that the predictions just don't pan out. In both cases, you don't really need inferential statistics - if inferential stats tells you something different from what a look at the data shows, it's usually the latter that right. For the other cases, it's usually straightforward to compute the inferential statistics yourself if you have to for most experimental designs.<br /><br />That said, I think that insistence on more or less any reporting requirement, be it standardised effect sizes, power levels or p-values, is fundamentally misguided as it presupposes that anything that works for one's own (often ANOVA-based) studies should work for analyses that are more left-field.Janhttps://www.blogger.com/profile/17765078332699225416noreply@blogger.com