Questions about Questionable Research Practices

Authors

DOI:

https://doi.org/10.31156/jaex.27502

Keywords:

questionable research practices, QRP, meta-analysis, publication bias, fraud, ganzfeld, anomalous cognition, parapsychology

Abstract

Parapsychologists have made considerable gains over the decades. For example, they have an informative (and constantly growing) series of meta-analyses, out of which have developed ongoing debates and critiques over methodologies. Some critiques, well-intentioned though they may be, center on so-called questionable research practices (QRPs), and it has even been argued that QRPs alone may account for psi effects. In this article, I critique an article by Bierman et al. (2016) on QRPs that finds a much-reduced but still significant overall effect in a meta-analysis of ganzfeld studies by Storm et al. (2010). The series of “hypothetical” analyses undertaken by Bierman et al. are themselves argued to be questionable. Researchers are advised to be watchful of methodological oversights that misrepresent the available data and ultimately cast parapsychological research in a dim light.

References

Bancel, P. A. (2018). Simulating questionable research practices. Proceedings of the presented papers of the Parapsychological Association 61st annual conference (pp. 105-115). Parapsychological Association.

Bem, D. J., & Honorton, C. (1994). Does psi exist? Replicable evidence for an anomalous process of information transfer. Psychological Bulletin, 115, 4–18.

Bierman, D. J., Spottiswoode, S. J. P., & Bijl, A. (2016). Testing for questionable research practices in a meta-analysis: An example from experimental parapsychology, PLoS ONE, 11(5), 1–18.

Darlington, R. B., & Hayes, A. F. (2000). Combining independent p values: Extensions of the Stouffer and binomial methods. Psychological Methods, 5(4), 496–515.

Honorton, C. (1985). Meta-analysis of psi ganzfeld research: A response to Hyman. Journal of Parapsychology, 49, 51–91.

John, L. K., Loewenstein, G., & Prelec, D. (2012a). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532.

John, L. K., Loewenstein, G., & Prelec, D. (2012b). Measuring the prevalence of questionable research practices with incentives for truth telling (supplemental materials (pp. 1–17). https://journals.sagepub.com/page/pss/suppl/data-supplements

Palmer, J. (2016a). Hansel’s ghost: Resurrection of the experimenter fraud hypothesis in parapsychology [Editorial]. Journal of Parapsychology, 80, 5–16.

Palmer, J. (2016b). Statistical issues in parapsychology: Hypothesis testing—plus an addendum on Bierman et al. (2016) [Editorial]. Journal of Parapsychology, 80, 141–143.

Rosenthal, R. (1979). The ‘File Drawer Problem’ and tolerance for null results. Psychological Bulletin, 86(3), 638–641.

Scargle, J. D. (1999, September 17). Publication bias (The “File-Drawer Problem”) in scientific inference. https://arxiv.org/pdf/physics/9909033

Storm, L. (2025). A new approach to psi: Re-envisioning paranormal experience as meaningful coincidence. Routledge.

Storm, L., Tressoldi, P. E., & Di Risio, L. (2010). Meta-analyses of free-response studies 1992-2008: Assessing the noise reduction model in parapsychology. Psychological Bulletin, 136(4), 471–485. doi:10.1037/a0019457

Thalbourne, M. A. (2003). A glossary of terms used in parapsychology. Puente.

Downloads

Published

2026-03-31

How to Cite

Storm, L. (2026). Questions about Questionable Research Practices. Journal of Anomalous Experience and Cognition, 6(1), 92–1010. https://doi.org/10.31156/jaex.27502

Issue

Section

Theoretical and methodological papers