Mind-Matter Interaction at Distance: Effects on a Random Event Generator (REG)

I think I have indeed misinterpreted what's happening. Looking at the figures, the location of the cut-off is increasing with time, so that 20% figure will be much too low. Quite how it's increasing with time isn't clear - it doesn't appear to be growing in proportion to the square root of t curves shown in the graphs.

Or perhaps it is growing in proportion to the square root of t - so that the criterion is just that the trace crosses the parabola shown in the graphs - BUT they aren't counting such crossings when they occur soon after the start of the session - which happens up to about #5 in the first figure but apparently isn't counted as a "hit" for that reason. (Not that I can see anything about that in the text.)

The puzzling thing is that whatever exactly they are doing, and whatever the resulting probability of success for each session, surely the difference between the 82.3% success rate and 13.7% in the control runs, in 102 sessions, must be hugely significant?
 
The puzzling thing is that whatever exactly they are doing, and whatever the resulting probability of success for each session, surely the difference between the 82.3% success rate and 13.7% in the control runs, in 102 sessions, must be hugely significant?

I imagine we'd need to know precisely what they are measuring to figure out whether the distribution we see represents expected variance fluctuations over a relatively small sample or whether the results are significant?

Did they do the calculations or did they let the machine do it? Do we know that the machine does it correctly?
 
I imagine we'd need to know precisely what they are measuring to figure out whether the distribution we see represents expected variance fluctuations over a relatively small sample or whether the results are significant?

Did they do the calculations or did they let the machine do it? Do we know that the machine does it correctly?

I don't have time to elaborate at the moment, but the result for their control group, 13.7%, is impossible if the experiment was conducted as described; it can only be due to a massive systematic error.
 
I don't have time to elaborate at the moment, but the result for their control group, 13.7%, is impossible if the experiment was conducted as described; it can only be due to a massive systematic error.

Thanks Jay, no hurry - look forward to hearing more!
 
I haven't had a chance to go through the study in detail yet, but a couple of points.

What they are calling a "control group" seems to be a proof-of-concept type of measurement instead, to get an idea of how often their set-up produces false-positives. In that case, the results in both "controls" show that their proof-of-concept seems to fail. How can you get 48% one time and 13.7% the next?

The study wasn't all that "pre-registered". The preliminary study had already been finished, and the experimenters were already a third of the way into the second study before they registered any of it.

This is yet another study where there is no attempt to validate that they are measuring "psi", content merely with producing "anomalous" results and assuming it has something to do with human intent and something we regard as "psi", even though their "control" measurements already suggest that wonky results are being produced regardless.

Linda
 
This post concerns the confirmatory study. As I said in my previous post, the result for the control group is impossible if the experiment was conducted as described in the paper. Here's why.

The experiment comprised 102 experimental and 102 control sessions. In each session a (supposedly) random bit stream was generated at a rate of 200 bits per second for at least 61 seconds, for a total of at least 12,200 bits/session. The objective in the experimental group was for the subject to try to influence the bit stream such that the cumulative number of 1s generated would, at some point during the session, be more extreme than the 5th or the 95th percentile of the cumulative binomial(n, .5) distribution for the total number of bits n generated at that point in the session. Call such a result a "success." The proportion of experimental sessions that resulted in a success were then to be compared with the analogous proportion from a control group of sessions in which there was no subject.

To see how impossible the control group results are under the experiment as described in the paper, we need to determine what the expected proportion of successes is under the null hypothesis; that is, we need to determine the probability that the cumulative number of 1s in a stream of 12,200 Bernoulli(.5) bits would be more extreme than the 5th or the 95th percentile of the cumulative distribution at some point in the stream. This probability should be substantial, as random walk theory informs us that the probability approaches 1 as n increases without bound, and 12,200 is a pretty big n.

To determine the above probability, I conducted a Monte Carlo simulation comprising 10,000 sessions of 12,000 random Bernoulli(.5) bits each, and computed the proportion of sessions that were successes as defined above. That proportion was .797 ± .004 (p ± SE). That is, if the null hypothesis were true (and the study actually conducted as described), then we would expect, on average, that 79.7% of sessions would be successes. Note that my estimate slightly underestimates the expected proportion in the experiment, because each of the actual sessions comprised more than 12,000 bits, some quite a few more. So my results are conservative. Nonetheless, the results in the experimental group appear utterly unremarkable. The experimental result of 82.3% is completely consistent with the theoretical value of 79.7% (2-tailed p-value=.59).

In contrast, for the control group's result, 13.7%, the p-value is 9.5 × 10^(–61). In other words, the finding for the control group is impossible if the experiment was conducted as described in the paper. The result must be due to massive systematic error.
 
Last edited:
Thanks Jay!

Do they say anything that contradicts your results in the paper, or are they simply silent on it? They don't clearly state their conclusions on what they think is significant but can we infer it from the stats they posted?

Anyone see it differently from Jay?
 
Thanks Jay!

Do they say anything that contradicts your results in the paper, or are they simply silent on it? They don't clearly state their conclusions on what they think is significant but can we infer it from the stats they posted?

I'll say they do! Read the first paragraph of the Discussion section.
 
Thanks Jay!

Do they say anything that contradicts your results in the paper, or are they simply silent on it? They don't clearly state their conclusions on what they think is significant but can we infer it from the stats they posted?

Anyone see it differently from Jay?

I would just repeat that the first graph shown in the paper demonstrates that the software is not in fact counting all excursions beyond ± 1.65 standard deviations, because there's clearly an excursion below the lower limb of the parabola near the start, between about #3 and #5, but the minimum z value has been logged as -1.461 at #12.

Either there's an error in the software, or possibly they are excluding some excursions near the start and have forgotten to mention this in the paper (!). I'm only guessing at what might be happening. But I think that would have a large effect on the "hit rate", because, if I understand correctly, a substantial number of excursions can be expected near the start.
 
A friend of mine did a similar, slightly less sophisticated Monte Carlo approach to Jay's and came to the same conclusion that the control condition was far from truly random.

It is not uncommon in parapsychology for an RNG control to show deviant behavior; at the PA convention, engineer Garrett Moddel told us about how he had hoped to find psi without living things (or, as he called them, "glorified bags of salt water") and set up an experiment to see whether an RNG would show anomalous behavior "in anticipation" of being strongly disturbed; he reported a strong, clearly distinguishable spike of activity. However, this disappeared in the replication experiment. Moddel's conclusion was that, unfortunately, his positive results had been the result of an experimenter effect. Similarly, Hideyuki Kokubo and Takeshi Shimizu actually made an entire presentation about how the physical bio-pk effects they observed in their lab were still marginally present in dummy control periods both prior to and after a formal experimental session was conducted, but (importantly) not for formal control periods on different days. Their subjects had a tendency to think about their targets after and before their trials.

While the temptation is to conclude experimental error in a psi experiment when the controls don't behave precisely like the experimental targets, one really should take note of the fact that controls aren't there to behave predictably; they're there to serve as an observation from which a difference can be deducted that rules out the influence of confounding variables. Here, we very clearly have an example of 204 sessions (the vast majority of which were pre-registered), 102 of which differed from the others only by not being subject to active intention. That we observe an actual difference in the response variables for these two conditions is the anomaly. And it occurred for both the pilot and the repeat experiment.

What is certain is that these experiments require replication. IMO, at least ten RNGs should be used, and if the same patterns can be independently observed in each of them then that would be pretty compelling. Additionally, control periods should be administered both directly after an experimental session by an assistant and also randomly scheduled on different days by a computer.
 
A friend of mine did a similar, slightly less sophisticated Monte Carlo approach to Jay's and came to the same conclusion that the control condition was far from truly random.

It is not uncommon in parapsychology for an RNG control to show deviant behavior; at the PA convention, engineer Garrett Moddel told us about how he had hoped to find psi without living things (or, as he called them, "glorified bags of salt water") and set up an experiment to see whether an RNG would show anomalous behavior "in anticipation" of being strongly disturbed; he reported a strong, clearly distinguishable spike of activity. However, this disappeared in the replication experiment. Moddel's conclusion was that, unfortunately, his positive results had been the result of an experimenter effect. Similarly, Hideyuki Kokubo and Takeshi Shimizu actually made an entire presentation about how the physical bio-pk effects they observed in their lab were still marginally present in dummy control periods both prior to and after a formal experimental session was conducted, but (importantly) not for formal control periods on different days. Their subjects had a tendency to think about their targets after and before their trials.

While the temptation is to conclude experimental error in a psi experiment when the controls don't behave precisely like the experimental targets, one really should take note of the fact that controls aren't there to behave predictably; they're there to serve as an observation from which a difference can be deducted that rules out the influence of confounding variables. Here, we very clearly have an example of 204 sessions (the vast majority of which were pre-registered), 102 of which differed from the others only by not being subject to active intention. That we observe an actual difference in the response variables for these two conditions is the anomaly. And it occurred for both the pilot and the repeat experiment.

What is certain is that these experiments require replication. IMO, at least ten RNGs should be used, and if the same patterns can be independently observed in each of them then that would be pretty compelling. Additionally, control periods should be administered both directly after an experimental session by an assistant and also randomly scheduled on different days by a computer.

What about Jay's conclusion that the experimental results were right around expectation (ie: the results were not significant)
 
What about Jay's conclusion that the experimental results were right around expectation (ie: the results were not significant)

Oops, looks like I missed that. But given that result, either Jay or my friend made an error simulating the data; the percentage of control sessions meeting the cutoff for my friend's simulation was ~ 52%. I'll need to check with him more closely.
 
What Tressoldi et al write in their paper about the number of bits they used...

Sessions lasted from 60 to 200 seconds with setting of “Sample Size 10 bit” and “2 Per Second” on the Psyleron™ REG-1

But Jay wrote this:

In each session a (supposedly) random bit stream was generated at a rate of 200 bits per second for at least 61 seconds, for a total of at least 12,200 bits/session.

I think that's a possible source of error. How I interpret Tressoldi et al is that about 20 bits were generated per second (which makes sense given the jagged shape of their plots).

But I will make sure to message Tressoldi to ask exactly how many bits were used, since I was having trouble figuring this out myself, earlier. Because English is not the first language of these researchers, the pre-print version of their paper seems to lack some detail. They're also looking for feedback.
 
What Tressoldi et al write in their paper about the number of bits they used...



But Jay wrote this:



I think that's a possible source of error. How I interpret Tressoldi et al is that about 20 bits were generated per second (which makes sense given the jagged shape of their plots).

That was for the pilot study. For their second study, they specifically say they increased the bits to 200 from 20.

Linda
 
But I will make sure to message Tressoldi to ask exactly how many bits were used, since I was having trouble figuring this out myself, earlier. Because English is not the first language of these researchers, the pre-print version of their paper seems to lack some detail. They're also looking for feedback.

It may be worth pointing out that anomaly in the first graph, where the initial excursion below -1.65 standard deviations doesn't appear to have been logged.
 
I just ran my own Monte Carlo simulation for 10,000 iterations and the result is 54%. I think the key parts of the code are the following:

Code:
posline <- 1.65*sqrt(0.5*0.5*200*x)
a <- which(abs(CumDev) > posline)
if (length(a) > 0) {
            1
} else {
            0
}

I want to know if Jay used the same calculation for the upper bound of the cumulative deviation plot: standard deviation = sqrt(npq), where n in my code = 200x. The code was instructed to compare the absolute values of the cumulative deviation vector with a vector of values 1.65 standard deviations above the mean; if at any point x in the sequence (x being itself a vector of numbers 1-60), the former vector's element was greater than the latter's corresponding element, the session was marked a 1. A sum of "1 sessions" determined the total number of simulations where the cutoff was reached, and this was then divided by the total number of sessions.

Here's an example plot from the simulation where the cutoff was achieved:
RplotSimulation.jpeg
The code output seems to verify that things are running smoothly (I did this for a number trial runs):

Code:
PseudoPlotCumDev()
[1] "Locations where 1.65 above or below"
[1] 3

As for how the psyleron software calculated the cutoff, looking at one of Tressoldi et al's example plots, their cutoff bounds are equal to my cutoff bounds, so I assume they used the same function as I did.

exampleSession.jpg
 
Last edited:
Thanks Johann - could you possibly give a "for dummies" explanation of how to read those graphs?

No problem; both of the graphs display the same thing: for each trial, 200 coins are flipped that come up "1" or "0", and then the total number of 1s is added and 100 subtracted from it (100 is the expected number of 1 bits in 200 flips). This value is a deviation. For instance, if 97 1s come up, the deviation for that trial is -3. Deviations are then added, cumulatively, and plotted on the graph.
So if the deviations come up like this for four trials: 1, -6, 2, -1, then the resulting cumulative values are: 1, -5, -3, -4 (i.e. 1, 1+-6, 1+-6+2, etc). The red trace on my simulation is just these cumulative values plotted against the trial #.

As for the smooth upper and lower curves, those plot the values of deviations that are 1.65 binomial standard deviations above or below the mean. A binomial standard deviation = sqrt("probability of getting a 1" x "probability of not getting a 1" x # of flips at that point). 1.65 binomial standard deviations for trial # 5 is about 26 1s above or below the mean.
 
Last edited:
It may be worth pointing out that anomaly in the first graph, where the initial excursion below -1.65 standard deviations doesn't appear to have been logged.

Good catch. First I'd like to see what we come up with, then I'll make note of anything that we all agree is important to ask about.

EDIT: From my simulation and my friend's, it looks like the computed percentage of sessions that should have reached the cutoff is about 54% (at variance with Jay's reported value of 79%), meaning that the experimental result for Tressoldi et al is about 28% above the theoretical, while the control is about 40% below.

Again, I agree that the control result is practically impossible under the null; if we take 54% to be the probability of a "hit", the likelihood of observing 14 or fewer hits in 102 trials is p = 2.58*10^-17. The experimental result of 84 hits in 102 trials is also very unlikely; p = 1.69^-9.
 
Last edited:
Back
Top