Anomalous Cognition: An Umbrella Review of the Meta-Analytic Evidence1

Objective: The aim of this study was to assess the results of all meta-analyses on anomalous cognition conducted between 1989 and 2021 in order to find moderators associated with greater effect sizes. Method: We included all meta-analyses of studies related to anomalous cognition published up to 2021. Results: Our dataset, accumulated over more than 80 years of investigation, refers to 11 meta-analyses related to six different states of consciousness. The evidence clearly shows that anomalous cognition seems possible and its effects can be enhanced by using a combination of some non-ordinary or altered states of consciousness (e.g., dreaming, ganzfeld, etc.), coupled with free-response procedures, or neurophysiological dependent variables. These conditions facilitate an alternative form of cognition seemingly unconstrained by the known biological characteristics of the sense organs and the brain. Conclusion: The accumulated evidence expands our understanding of the mind-brain relation and the nature of the human mind.

combined with an unconscious psychophysiological response protocol, are the best positive moderators.
The scientific study of the reality and characteristics of anomalous cognition (other terms used are anomalous perception, extrasensory perception, and nonlocal cognition) 22 , unconstrained by the known biological characteristics of the sense organs and the brain, has employed quantitative techniques under controlled conditions, popularized in the English-speaking world by the pioneering work of Joseph Banks Rhine in the early 1930s at the Psychology Department of Duke University . After Rhine's first studies, many other investigators, mainly in USA and Europe, have continued this line of investigation by using a variety of methodological and experimental procedures in order to offer solid evidence of the existence of this type of cognition. This type of research was later referred to as "proof oriented."; later, research became more "process oriented" in order to discover the environmental, physiological, and mental factors (including personality traits) that might elicit and enhance (i.e., moderate) this type of cognition (Stanford, 1974).
With the accumulation of experimental studies, it has been deemed necessary to summarize quantitatively the results using appropriate statistical tools. Even though the term meta-analysis was coined in 1976 by the statistician Gene V. Glass (1976), the first meta-analysis per se dates back to 1940 (Pratt et al., 1940), and consisted of 145 reports on extra-sensory perception experiments published from 1882 to 1939.
This meta-analysis also included an estimate of the required number of unpublished papers that would be required to reduce the overall significant effect to mere chance (referring to the publication bias problem). At that time, the statistical tools for the quantitative synthesis of the results were quite poor but, with the improvement of such tools, more sophisticated meta-analyses have been carried out by different authors.
Some five decades later, by way of a formal meta-analysis of forced-choice precognition studies (Honorton & Ferrari, 1989), evidence began to accumulate that suggested that an anomalous form of cognition seemed possible, primarily under controlled laboratory conditions. The forced-choice design is so named because the target-guess is "one of a limited range of possibilities which are known to [the participant] in advance" (Thalbourne, 2003, p. 44). Precognition is defined as "a form of extrasensory perception in which the target is some future event that cannot be deduced from normally known data in the present" (Thalbourne, 2003, p. 90). For the P A G E 5 7 period 1935 to 1987, including a total of 309 studies (50,000 participants and approximately two million individual trials), there was a weak albeit significant effect size of 0.02, with 92 studies (30%) showing significant hitting (p < .05) (Honorton & Ferrari, 1989). Another forced-choice meta-analysis by Stanford and Stein (1994) reported an association between hypnosis and ESP, reporting that there was "cumulative ESP-test significance for hypnosis" (p. 235). Steinkamp, Milton, and Morris (1998) also meta-analyzed forced-choice studies (from 1935-1997), but they compared clairvoyance with precognition in order to evaluate any difference between the two. Clairvoyance is defined as "paranormal acquisition of information concerning an object or contemporary physical event" (Thalbourne, 2003, p. 18). Steinkamp and colleagues hypothesized that clairvoyance studies would have a significantly higher effect size because precognition had an extra "calculational step," involving "real-time ESP" (clairvoyance) and then extrapolation from that information "to make an informed prediction about future events" (p. 193). Assessing 22 study-pairs with effectively similar procedures, effect sizes for both modalities were almost identical, with no significant difference between the two. They concluded that the burden of proof rested with those "who argue for a difference between effect sizes under real-time and future ESP" (p. 209). Storm, Tressoldi, and Di Risio (2012) continued where Honorton and Ferrari (1989) left off on forced-choice ESP. For the period 1987 to 2010, they formed a homogeneous dataset of 72 studies that yielded a weak, but statistically significant mean effect size of 0.014. There was no evidence that these results were due to low-quality design or selective reporting. They noted a linear incline in effects indicating that effect sizes increased over that period. Bem, Tressoldi, Rabeyron, and Duggan (2016) looked at the cumulative evidence related to so-called behavioral precognition in a normal state of consciousness. The main methodological characteristic of these studies was that participants were requested to predict future events randomly presented, using a forced-choice procedure. The overall effect size was .09, but the results showed that tasks requiring a fast response yielded a statistically significant effect size of 0.11. In contrast, those not requiring a fast response yielded an almost null effect of 0.03.
Turning to a different experimental design, Milton (1997)

meta-analyzed 78
free-response studies published over the period 1964 to 1993 (these studies included remote viewing studies, in which the percipient "attempts to describe the surroundings of a geographically distant agent"; Thalbourne, 2003, p. 107; although an agent is not always used). In all studies, participants were in a normal (waking) state of consciousness, but the task requires a good control of mental information similar to

P A G E 5 8
Journal of Anomalous Experience and Cognition (JAEX) that requested in some meditation practices, in order to distinguish between information related to the target and that deriving from inner mental activity. The term free response refers to "any test of ESP in which the range of possible targets is relatively unlimited and is unknown to the percipient" (Thalbourne, 2003, p. 44) and participants are requested to verbally describe without constraints their mental content (mentation) as it pertains to randomly preselected targets (usually photographs or videoclips), included among a set of decoys that are presented on-screen. Milton found a mean effect size of 0.16. Dunne and Jahn (2003) presented a total of 653 formal trials conducted "over several phases of investigation" (p. 207) during a 25-year period. Percipients had to "describe verbally an unknown remote geographical target where an agent is, was, or will be situated at a prescribed time" (p. 209), thus classifying these trials as remote viewing. The authors reported a significant effect size of 0.21. Baptista, Derakhshani, and Tressoldi (2015) followed up with their meta-analysis including all studies available up to 2014, comprising the SRI, the SAIC (Utts, 1996), the Milton (1997), and the Dunne and Jahn (2003) databases, obtaining an overall effect size of 0.38.
Mossbridge, Tressoldi, and Utts (2012) ventured into new territory when they assessed unconscious physiological anticipation effects in their meta-analysis of 26 studies (for the period 1987 to 2010). This anticipatory effect is also referred to as presentiment (sensing an event before it occurs). The overall significant effect size was 0.21. Duggan and Tressoldi (2018)  can be used to identify target materials correctly and more often than would be expected by chance.
Finally, we come to the ganzfeld design, which is a "special type of environment (or the technique for producing it) consisting of homogeneous, un-patterned sensory stimulation" to the eyes and ears of the participant who is usually in "a state of bodily comfort" (Thalbourne, 2003, p. 45). A number of investigators pioneered the technique in the 1970s (Braud et al.,1975;Honorton & Harper, 1974;Parker, 1975). The Ganzfeld technique can be used to test telepathy, as well as clairvoyance and precognition. Te-lepathy refers to the "paranormal acquisition of information concerning the thoughts, feelings or activity of another conscious being" (Thalbourne, 2003, p. 125).
In one of the earliest ganzfeld meta-analyses, Honorton (1985) found a hit rate of 38% in his database (N = 28), where 25% was expected by chance. Bem and Honorton (1994) conducted a second meta-analysis on ten computer-controlled autoganzfeld studies (in this design, targets are randomly-selected, presented, and scored). The hit rate did fall, but to a still significant 32%. Milton and Wiseman (1999) followed up with their assessment of 30 new studies (1987 to 1997), yielding a non-significant effect size of 0.013. However, an Exact Binomial test on trial counts produce a significant hit rate of 27% (Utts, 2008). (2010) meta-analyzed a database of 29 ganzfeld studies (1997 to 2008) and found a significant effect size of 0.14. The most recent meta-analysis by Storm and Tressoldi (2020) covered studies from 2008 to 2018, with an effect size of 0.13. The most comprehensive study to date is by Tressoldi and Storm (2021b) as it pulls together all valid ganzfeld studies (N = 113) from 1974 to 2020, obtaining a statistically significant overall effect size of 0.09.

Storm, Tressoldi and Di Risio
The main aims of this study are: (a) to assess the strength of evidence supporting the reality of anomalous cognition obtained from meta-analyses, and (b) to determine its moderators (i.e., the conditions that increase its efficiency).

Method Search procedure
For the present study, we collected all available meta-analyses conducted up to 2021 published in English-language peer-reviewed journals. (As a matter of expediency, we also use the online study by Tressoldi and Storm (2021b)-for details, see next section.) This synthesis represents an update to 2021 of previous reviews presented by Tressoldi (2011) and more recently by Cardeña (2018). We searched the Google Scholar, PubMed, and Scopus databases with the keywords: "meta-analysis and ganzfeld" or "anomalous" or "extrasensory perception" or "clairvoyance" in the title.

Inclusion and Exclusion Criteria
All meta-analyses should include studies related to different but specific phenomena suggestive of anomalous (nonlocal, extrasensory) cognition. Studies includ-

P A G E 6 0
Journal of Anomalous Experience and Cognition (JAEX) ed in the meta-analyses assessed here, used the following original inclusion criteria: appropriate randomization (using electronic equipment or random tables) of the target presentation; when appropriate, random target positioning during judgment (i.e., target was randomly placed in the presentation with decoys); masked response transcription or impossibility to know the target in advance; when appropriate, sensory shielding from sender (agent) and receiver (perceiver); when appropriate, target independently checked by a second judge; experimenters masked to target identity.
The 13 meta-analyses had to have been published in peer-reviewed English-language journals. Only ESP meta-analysis (i.e., telepathy, clairvoyance, and precognitive) were assessed, and they had to provide sufficient methodological and statistical information for the authors to prepare appropriate tables. We excluded older meta-analyses comprised of studies that were included in more recent meta-analyses.
For example, all meta-analyses related to anomalous cognition in a ganzfeld condition before 2020 (e.g., Bem & Honorton, 1994;Milton & Wiseman, 1999b;Storm et al., 2010) were not included here because all studies in those meta-analyses were analysed in the Tressoldi and Storm (2021b) meta-analysis. We also excluded the Milton and Wiseman (1999a) meta-analysis because it was related to mass participation without any control over recruitment and motivation of participants who were requested to predict masked targets, similar to the lottery guessing tasks. A partial overlap with the studies included in Honorton and Ferrari (1989) is presented in Steinkamp, Milton, and Morris (1998) who included only the studies that combined both a clairvoyance and a precognition task.
The general methodology adopted in most of the studies included in the meta-analyses required participants to identify concealed or future targets that were generated randomly and presented in either a forced-choice or a free-response condition. Other studies do not record overt choices, but instead focus on neurophysiological responses (e.g., EEG, heart rate) prior to target presentation. The states of consciousness during these tasks range from ordinary (normal) to non-ordinary (altered or modified under conditions such as hypnosis, ganzfeld, etc.).

Descriptive Statistics
Authors of the meta-analyses, number of studies included in each of them, the states of consciousness, response types, and overall effect sizes with corresponding with 95% confidence intervals (CIs), are presented in Table 1.

P A G E 6 2
Journal of Anomalous Experience and Cognition (JAEX) As reported on Table 1, we included a total of 11 suitable meta-analyses, reporting 16 overall effect sizes obtained from 928 studies. All effect sizes, but the Bem et al.
(2016) related to slow responses, are significant. As to their type, apart from the use of a proportion index in Stanford and Stein (1994), all other effect sizes are conceptually but not mathematically equivalent because they are Cohens' d equivalent, either mean z/√n or t/√n (as normal approximations to the binomial test), or Hedges's g, with sizes ranging from -∞ to +∞ and none deriving from correlation effect sizes. The two proportion index effect sizes were transformed into z/√n to compare them with the other effect sizes (see Table 1 and 4) applying Rosenthal and Rubin's (1989) formula. The effect sizes used measure how far the summary statistic (e.g., hit rate, mean) deviates from the null hypothesis value in terms of number of standard deviations; Hedges's g is used as a correction for potential small sample effects.
As presented in Table 1, the 11 meta-analyses cover anomalous cognition in six different states of consciousness from the normal state in a waking condition (i.e., non-altered stated of consciousness or non-ASC) to altered states of consciousness (ASC) such as the dream state, and three types of responses: (i) overt conscious free-response; (ii) overt conscious forced-choice; (iii) unconscious physiological measures.
In Figure 1, the effect sizes of all meta-analyses presented in Table 1 are shown in decreasing magnitude to enable a visual comparison of their differences.

Meta-Analyses Reporting Standards
In order to give an overall picture of the quality of all 11 meta-analyses, we checked whether four major meta-analysis reporting standards (MARS): studies selection criteria, effect size formulas, studies quality check, publication bias check, were followed as recommended by the APA (Appelbaum et al., 2018). Table 2 shows that most of the meta-analyses actually implemented these four standards before the publication of the MARS in 2018.

Questionable Research Practices and Publication Bias
We can also test the methodological standards of meta-analyses by calculating the percentage of studies affected by so-called "questionable research practices" (QRP; Banks et al. 2016;John et al., 2012). The tacit aim of QRPs in the first place, is to obtain a statistically significant result. If we observe that in most meta-analyses a high percentage of studies did not reach significance, either the authors of the stud-

P A G E 6 4
Journal of Anomalous Experience and Cognition (JAEX) ies applied QRPs unsuccessfully, or might not have applied them at all. It is worth to point out that in psi, researchers have been urged for decades to submit and publish non-significant results, way before this happened in psychology and other disciplines (Wiseman et al., 2019).
In Table 3, in the 9 meta-analyses that reported the number of significant studies, it was a simple matter to calculate the percentages of non-significant studies which ranged from 54% to 81% , with one exception at 23%. Thus, it can be argued that the majority of authors of the studies included in most of the meta-analyses probably did not use QRPs, or used them unsuccessfully arguing against the argument that the results can be explained away by publication bias. The high failure rates, however, indicate the difficulties in detecting anomalous cognition, which has been a consistent characteristic in psi research. Our finding, however, is actually a defense of the phenomenon, as it is unlikely that experimenters deliberately aimed to produce a majority of non-significant findings just to perpetuate the myth of an unreliable anomalous form of cognition.

Moderators Analyses
We now consider two possible moderators of anomalous cognition: State of consciousness and Response type. First, state of consciousness has been considered a key variable in parapsychology, this assumption is decades old and underpins the ganzfeld design (Parker, 1975;Cardeña & Marcusson-Clavertz, 2020). Second, as shown in Table 1, there are three response types (forced choice, free-response, and unconscious physiological anticipation). We see forced-choice, in both the normal and altered states of consciousness, as the least efficient strategy because the responses, manual or verbal, are consequently partially controlled by ordinary (local) cognitive activity. Free response would be the next best strategy, though it requires participants to filter out local mental activity, thus probably explaining why selected participants do better than naive (unselected) participants (see Storm & Tressoldi, 2020). Physiological anticipation, however, seems to be the best of the three options (confirmed by the two meta-analyses with the largest ESs; see Table 1, #4 & #9), arguably because the anomalous/nonlocal information bypasses conscious mental activity.
We considered whether it was possible to predict the overall effect size, taking into account the state of consciousness and the type of response as the main moderators. For each meta-analysis, we assigned a rank score from 1 to 2 according to the state of consciousness with these criteria: normal (waking) state of consciousness = 1; modified (altered) state of consciousness during the response = 2. Similarly, we assigned a rank score to the type of response with these criteria: forced-choice = 1; free-response = 2; physiological responses = 3. Applying the above criteria to each meta-analysis presented in Table 1, we produced rank scores and corresponding effect sizes listed in Table 4.

P A G E 6 6
Journal of Anomalous Experience and Cognition (JAEX)

Discussion
A first point to be made in regard to the quality of the meta-analyses, even of the older ones, is that they are supported by a high level of adherence to the reporting standards guidelines, which militate against an explanation of questionable research practices. Given the range of effect sizes presented in Figure 1 and Table 1, we can reach a few conclusions. It is evident that the effect sizes (ESs) were stronger (above .30) in two meta-analyses that featured altered states of consciousness (ASC) and Free Response (FR) protocols-Remote Viewing (meta-analysis #6) and Dream-ESP (meta-analysis #8). We note that two meta-analyses also had relatively strong ESs (.21 and .28), though they did not feature AsCs (#4 & #9), but used physiological responses as dependent variables. The Ganzfeld condition, however, seeks to induce an ASC and is invariably an FR protocol, although the ESs were somewhat weaker, falling just below .10 (#11). Turning to the non-ASC meta-analyses, these generally yielded the weakest ESs of all (#1, #2, #3, #5, & #10), and tended to feature forced-choice protocols, so we conclude that the combination of ASCs and FR conditions gives the experimenter a clear advantage when relatively strong ESs are sought.
It is important to consider that each of these 11 meta-analyses includes more information than is reported in Table 1. For example, in Honorton and Ferrari (1989), delays in feedback of milliseconds yielded a mean effect size almost three-times as large as the mean when delays were in months. Furthermore, selected participants (e.g., previously tested, or trained in meditation or relaxation techniques) yielded larger effect sizes up to three times greater than non-selected (first-time or non-trained) participants (Storm et al., 2010, Storm & Tressoldi, 2020Tressoldi & Storm, 2021b). We acknowledge the importance of participant type, but the observed strong correlations between the overall effect sizes and the combination of State of Consciousness plus Response Type suggest that the manipulation of these two variables are most critical for the emergence of anomalous/nonlocal cognition. Our advice to researchers interested in this area is that the best methods to use include free-response involving selected participants in modified or controlled states of consciousness, or studies with physiological measures as dependent variables.

Study limitations
The updated standards for the best scientific evidence require registered meta-analyses (see Tressoldi & Storm, 2021a, as an example) of registered reports (Chambers, 2013), or multi-laboratory studies with preregistered methods and data analyses (e.g., Open Science Collaboration, 2015; Protzko et al., 2020) that limit not only the use of the QRPs, but also the degrees of freedom in the experimental designs and data analyses.
None of our meta-analyses satisfy such criteria. Most of the included meta-analyses were carried out applying the standards available and agreed by the scientific community at the time of their completion. As a consequence, our interpretations of what they can say about the evidence and the moderators of an anomalous cognition must be taken with caution and see if they will hold with the results of new meta-analyses carried out with modern standards.

Conclusions
Notwithstanding the limitations of this study, we can provisionally state that the overall picture is that anomalous cognition manifests its potentialities by bypassing normal waking consciousness, either by modifying it or using implicit (unconscious) physiological mechanisms. It seems then that humans (and probably also animals; see Alvarez, 2010Alvarez, , 2018 possess two alternative ways of obtaining information: first, by using their physiological functions, sensory organs, and brain, and second, by using an anomalous/nonlocal mental capacity that might be used as a complement to the ordinary local perceptual abilities, which therefore pushes for a modified interpretation of mind and consciousness in general. It is evident that this anomalous perceptual capability requires a revised theoretical interpretation of the nature of the human mind-brain relation (for an overview of the different hypotheses and theories, see Cardeña, 2018). Although it is clear that the proposal of an anomalous human cognition is incompatible with, say, a physicalist or an eliminative materialist interpretation (Ramsey, 2020), it is compatible with some Western and Eastern philosophical interpretations that may be familiar to the reader such as idealism (Kastrup, 2018), dual-aspect monism (Walach, 2020), and Advaita Vedanta (Sedlmeier & Srinivas, 2016). All these philosophical interpretations support the view that Consciousness and Mind contents are primary and not a by-product of physical and biological matter such as the brain.