Johann
New
Three days ago I wrote a long commentary on this very interesting blog post, written by Scott Alexander, pertaining to Bem's recent meta-analysis. Scott's opinion typifies the reaction I have seen, in the social science fields, to Bem's results—especially to the fact that his original experiments have now been substantially replicated. The discerning reader will notice that back when Ritchie, Wiseman & French (2012) conducted their small null meta-analysis, the talk was all about "the curse of the failed replications". Stephen Novella wrote that one important solution to the "researcher degrees of freedom" problem was replication. Why? I'm guessing it's because replication seemed to be the doom of Bem, at the time.
Now, however, Bem's positive results have cast that tenet of good science out the window—or, rather, the skeptics seem to have done that. Scott uses the principle of conservation of expected evidence to argue that, since science is so broken now, Bem could just as easily produce another meta-analysis with this degree of evidence, given enough time. Notice the extraordinary difference between this attitude and Novella's attitude: we can just assume Bem's research will be replicated! It's no longer even surprising that psi experiments replicate because parapsychology is the control group of science, according to these skeptics; it's robust results serve, overall, only to prove the ubiquity of methodological and statistical flaws in science.
Anyway, I made a number of observations on his piece and he graciously responded, so I thought it might interest members of this board to read the exchange. There's much to commend in his approach; in particular, I applaud his honesty in attributing to parapsychological research a high standard of quality, lamenting the fact that psi research seems to be ahead of his own social field in terms of its safeguards against biases and its policies for the publication of null results. His drive to improve science is also laudable, IMO, and a great positive consequence—even if not for all the right reasons—of Bem's studies on academia. But I question the very considerable assumption behind his approach; the white elephant in the room; and postulate that perhaps we should take the psi hypothesis seriously, in addition to questioning the scientific method.
I would be interested to see if there is any skeptic on this board who disagrees with the main thrust of my conclusions, as represented on Scott's blog.
Now, however, Bem's positive results have cast that tenet of good science out the window—or, rather, the skeptics seem to have done that. Scott uses the principle of conservation of expected evidence to argue that, since science is so broken now, Bem could just as easily produce another meta-analysis with this degree of evidence, given enough time. Notice the extraordinary difference between this attitude and Novella's attitude: we can just assume Bem's research will be replicated! It's no longer even surprising that psi experiments replicate because parapsychology is the control group of science, according to these skeptics; it's robust results serve, overall, only to prove the ubiquity of methodological and statistical flaws in science.
Anyway, I made a number of observations on his piece and he graciously responded, so I thought it might interest members of this board to read the exchange. There's much to commend in his approach; in particular, I applaud his honesty in attributing to parapsychological research a high standard of quality, lamenting the fact that psi research seems to be ahead of his own social field in terms of its safeguards against biases and its policies for the publication of null results. His drive to improve science is also laudable, IMO, and a great positive consequence—even if not for all the right reasons—of Bem's studies on academia. But I question the very considerable assumption behind his approach; the white elephant in the room; and postulate that perhaps we should take the psi hypothesis seriously, in addition to questioning the scientific method.
I would be interested to see if there is any skeptic on this board who disagrees with the main thrust of my conclusions, as represented on Scott's blog.