Anomalous cognition, dissociation, and motor automatisms.
ANOMALE KOGNITION, DISSOZIATION UND MOTORISCHE AUTOMATSMEN
COGNICION ANOMALA, DISOCIACION, Y AUTOMATISMOS MOTRICES
The general hypothesis tested in the experiment is that psi is facilitated by dissociated states of consciousness. Underlying this hypothesis is the more basic premise that the relative absence of overt manifestations of psi comes not from the lack of psi information in the unconscious mind, but rather by awareness of this information being blocked by psychological defense mechanisms. One vehicle by which these defense mechanisms might be circumvented is a dissociated state of consciousness, as exemplified by Ernest Hilgard's (1977) "hidden observer," which under hypnosis can recall mentation that is inaccessible to normal waking consciousness.
The application of the concept of dissociation to psychic phenomena can be traced back to F. W. H. Myers (1903). Even though he did not use the term much himself, dissociation is clearly what he had in mind when he introduced the notion of the secondary self, which can function independently of the primary personality or supraliminal consciousness. The secondary self often manifests through what Myers called sensory or motor "automatisms." Myers included under motor automatisms automatic writing, automatic speaking, automatic drawing, and use of the pendulum. 1 consider motor automatisms to represent a purer form of dissociation than sensory automatisms, because motor automatisms in their unadulterated form do not involve any conscious cognition.
Seeking an anomalous cognition (AC) test related to motor automatisms that participants (Ps) could relate to easily, I decided to employ the Ouija board concept (Palmer, 2011). However, because use of the Ouija board for conjuring spirits has been shown to have adverse psychological effects in some cases (e.g., Palmer, 2001), 1 decided to make the association to the Ouija board somewhat more distant. As part of this effort, I substituted an alphabet board consisting of the 26 letters arrayed on the four outer sides of the board. The target words were one-syllable homographs. Ps were instructed to repeatedly move a pointer or planchette randomly around the board until they felt the impulse to stop on a particular letter, at which time they recorded the letter on a notepad. In the meantime, a sender in another room on a different floor of the building was asked to transmit the target.
Prior to the session, participants filled out a trait measure of dissociation, consisting of Persinger's Complex Partial Epileptic Signs scale (Persinger& Makarec, 1993), with scores on the Tellegen Absorption Scale (Tellegen, 1978) partialled out. At the end of the session, Ps were shown the five words in the target set and asked to rate each word on a 0-20 scale based on its correspondence to letters they had gotten from the board, as well as mental imagery they had spontaneously experienced at any point during the session. They also completed a rating scale on their experiences during the session, the key item of which was what percentage of the movements of their hand they felt had been guided by an outside force (OF), a state measure of dissociation. To allow for assessment of a linear trend, the sample was divided into three roughly equal groups: none of the time, 1-40% of the time, and 40-100% of the time. The 11 of 40 Ps who reported that their hand was so guided 1-40% of the time demonstrated a strong AC effect, whereas Ps in the other groups averaged close to chance. I speculated that the subjective experiences of dissociation in the 1-40% group tended to differ qualitatively from those in the 41-100% group, but I did not interview the Ps about their experiences. The trait dissociation measure correlated significantly positive with the OF question but not with the AC scores.
The present experiment was an extension of Palmer (2011) with the following specific objectives: (a) to attempt to replicate the OF/AC correlation and to understand it better by interviewing those Ps who answer the OF question positively about the precise nature of their experience of the outside force; (b) to assess a possible relationship between trait dissociation and AC using a more direct measure of dissociation; (c) to explore the effects of additional procedures to facilitate state dissociation during the task; and (d) to explore possible differences in scoring as a function of whether Ps use the right or left hand for their AC responses. The experiment also differed methodologically from Palmer (2011) in that the task was forced choice rather than free response, and there was no sender.
Watson's (2003) Dissociative Processes Scale (DPS) was chosen as the measure of trait dissociation. One reason is that it is more discriminative within the normal range than the standard but highly clinical Dissociative Experiences Scale. Second, it contains items that seemed to reflect the particular kind of dissociative process I was interested in, which can be represented by the metaphor of one part of one's mind keeping a car on the road while the other part is engaged in conversation with the person in the passenger seat. The DPS has three factors or subscales: Obliviousness, Imagination, and Detachment. The items that struck me as most relevant to the present experiment are in the subscale measuring Obliviousness, which Watson (2003) defines as "the tendency to engage in mindless and automatic behaviors and to enter into naturally occurring trance states" (p. 300). Thus, I based my hypothesis on the Obliviousness subscale.
Although parapsychological studies comparing motor performance with the right and left hand have led to rather ambiguous results (Broughton, 1976; Maher & Schmeidler, 1977), left-hand performance was found to be superior in a psychology experiment in which Ps had to match by touch the directional orientation of stainless steel rods they could not see to other rods with diverse orientations (Benton, Varney, & Hamsher, 1978). On this basis, along with the general tendency for AC to be associated with right-hemisphere processing (Braud, 1975), I gingerly predicted superior performance with the left hand.
The major hypotheses were as follows:
Hypothesis 1: Performance on an AC task providing for motor automatisms will be better for Ps who have the experience that the associated hand is being moved by an OF during the task than for other Ps.
Hypothesis 2: Performance on an AC task providing for motor automatisms will be positively related to Ps' tendencies toward dissociation as measured by the Obliviousness subscale of the Dissociative Processes Scale.
Hypothesis 3: Among right-handed Ps, performance on an AC task providing for motor automatisms will be better if the AC responses are made with the left hand than with the right hand.
Hypothesis 4: Dissociative tendencies as measured by the Obliviousness subscale of the DPS will be greater in Ps who feel their hand was being moved by an OF during the AC task than in other Ps.
The study was approved in advance by the Rhine Research Center (RRC) Institutional Review Board.
Eighty mentally healthy adult volunteers were recruited to serve as Ps. An announcement was sent out to the RRC mailing list of more than 3,000 persons who had contacted the RRC in the past, but almost all the respondents were from the local area. The letter indicated that I was especially interested in testing people who found the rationale behind the experiment congenial. Ps were contacted, usually by email but occasionally by phone, in the order I received their responses. Once a session time had been established, I emailed them the DPS as well as the RRC's Participant Information Form, which they either returned by email or brought with them to the session.
Dissociative Processes Scale (DPS). The DPS consists of 33 items (Watson, 2003). Responses are made on a 5-point scale from strongly agree to strongly disagree. Cronbach's alpha was .93 at original testing and .94 at retest. The scale consists of three factorially derived subscales: Obliviousness (14 items), Detachment (6 items), and Imagination (7 items). The other six items are presumably buffers. Cronbach alphas for the scales range from .85 to .89.
Post-experiment questionnaire. Following the test session, I (the experimenter) asked the Ps a series of questions concerning their experience during the session and recorded their responses on a rating scale. The questions were presented in a semistructured interview format, which allowed me to better understand Ps' experiences and to detect any misunderstandings of the questions by providing for follow-up questions, such as requests for concrete examples of relevant experiential elements. The most important question was taken directly from Palmer (2011): "Did you have the impression at any time during the session that the pen was being guided by an outside force? (yes; no). If yes, about what percentage of the movements were caused in this manner? (1-20%; 21-40%; 41-60%; 61-80%; 81-100%)." Other questions concerned mainly Ps' expectations of success in the task and how well they were able to implement the task procedure.
The AC Task
Apparatus. The AC task used a Dell GX240 desktop computer and monitor, a computer writing tablet (Addesso Cyber Tablet 1200), and a pen that came with the tablet. The basic idea is that Ps move the pen over the surface of the tablet and stop at a particular location to register their "guess."
Target board. The tablet is 13.5" x 16.5" and has a writing area of 9" x 12." The writing area is covered by a sheet of flexible transparent plastic attached to the back side of the writing area. A 4" square piece of writing paper was taped under the plastic cover so as to cover the middle of the writing area. This target area is referred to hereafter as the grid. Three strips of soft wood (5 cm high) were attached to the outside of the plastic cover along the right, left, and back sides of the grid. The purpose was to keep P from moving the pen outside the target area. For the same purpose, a 1 mm thick piece of picture-hanging tape was placed along the front side of target area. It was not as steep as the other barriers, to allow P to reach the bottom of the grid with the pen without making it uncomfortably vertical. The tablet and grid are illustrated in Figure 1.
The grid was formatted as a 4 x 4 square matrix consisting of 16 smaller squares, each 1" x 1". The matrix was divided into four quadrants, with each quadrant consisting of four squares. In each square was printed a single digit from 1 to 4, such that within each quadrant each number appeared once and only once. The numbers were distributed in a quasi-random fashion such that they were balanced with respect to their locations in the matrix.
For each trial, P explores the grid by moving the pen over the surface of the cover, in contact with it at all times. P makes a response by stopping the pen at the chosen location for 1 s. The identity of the square under the tip of the pen is then sent to the computer as P's response for the trial.
Target sequences. The targets were multiple-aspect, in that each could identified by square (P = 1/16), quadrant (P = 1/4), and number (P = 1/4).
At the beginning of the experiment, John Kruth, Executive Director of the RRC, generated 80 successive random sequences of 36 square targets (1 to 16). To do so, he used an algorithm developed by Marsaglia and Zaman (1987), which has been thoroughly tested for randomness. To avoid intuitive decisions, Kruth was instructed to use 1 and 2 as the seed numbers. This sequence was emailed to the programmer, Daniel Ruszaj, who incorporated it as a file in the software package he was developing to create the AC scores. This program, written in Java, compared the targets with P's responses to calculate how many square, quadrant, and number hits each P obtained. For example, if the target was the square in the upper left corner, and P chose the square next to it on the right, P would obtain a quadrant hit but not a square or number hit. This procedure allowed the experimenter to remain blind to the targets throughout the experiment.
Attempts to Facilitate a Dissociative State
Task manipulation. The purpose was to disengage the conscious mind from the AC task.
Eyes-closed. In this condition, Ps were to blank the mind with eyes closed and wait for an impulse to stop the hand movement and make a response. It was emphasized that this impulse should be a feeling in the hand, not a mental or cognitive image.
Quotations. In this condition, Ps were asked to distract the conscious mind by reading a series of quotations on the computer screen.
A pool of 100 quotations was created from Bartlett's Familiar Quotations (Bartlett, 2002). In the book, the quotations are arranged by the birth year of the author of the quote. I started at 1900 and worked forward. The list was long enough so that no P would read any quote twice. A separate list of six quotations was created for the practice sessions. These quotes did recycle. The long list was randomized before testing began.
The 19th P indicated during the post-task interview a marked dissociative feeling related to her hand movements after reading quotes she found humorous. This struck me, as the participant in one of the early 20th century automatic writing experiments mentioned that she felt the text she was reading as a distraction task had to be entertaining to be effective (Solomons & Stein, 1896). This P's experience changed my mind about the best quotes to use to achieve the experimental objective. Because I was not keeping track of the AC scores, one reason being that I would not be biased by them if I wanted to make a methodological adjustment after the experiment began, and because I considered the experiment exploratory, I decided to immediately create a new pool of quotes taken from the Dictionary of Modern Humorous Quotations (Metcalf, 2009). I chose the ones that were most humorous to me. The same adjustments for length were made as with the original quotes, but overall the new quotes were shorter than the original ones. To compensate, I increased the number of quotes in the task list to 144. Some of them are sexual in nature (but not pornographic; they resemble jokes one finds on late-night television talk shows in the U.S.). Ps were given the opportunity to view instead the original quotes upon indication that they might be offended by the humorous ones. No one requested the original quotes. A separate list of humorous quotes was also created for the practice trials.
The quotes were presented on the computer screen one at a time. As it would be distracting to require P to press a button to make the next quote appear, I had the computer replace each quote with the next quote after a fixed number of seconds. The intent was for each quote to disappear as soon as P had finished reading it. I took two steps to achieve this. First, I tried to make all the quotes of approximately the same length. To accomplish this, I would often choose only some of the sentences from multi-sentence quotations and occasionally omit trivial words within sentences. Second, before the task, I had Ps time themselves reading a typical quote (not in the task list) using a stopwatch. I then set the duration of each quote to match that number. Ps were given the option to change this number after the practice session. Ps were told that if they finished a quote before it changed, they should begin rereading it. If, on the other hand, the quote changed before they finished reading it, they should not worry about it but just move on to the next quote.
Whenever an AC response was registered, the screen briefly flashed. This single flash was intended to assure P that a response had registered, without distracting P from reading the quotes.
During the instruction period, Ps were told that immediately following the AC task they would take a short quiz to measure how well they remembered the quotes they saw. The quiz was a recognition task in which they were shown 10 quotes that had been presented during the AC task and 10 unseen control quotes, all in random order. Ps indicated by a mouse click whether they recognized the quotation from the AC task. It was given solely to motivate the Ps to pay attention to the quotes during the AC task. All Ps got a very high score on it.
Progressive relaxation. The second method used to facilitate a dissociated state of consciousness during the AC task was to induce a mild altered state of consciousness immediately beforehand. The induction consisted of an abbreviated form of the Jacobson progressive relaxation technique, frequently given at the beginning of AC ganzfeld tests. Progressive relaxation is particularly appropriate for a task that provides for motor automatisms, because it includes commands to alternatively tense and relax various muscle groups.
The relaxation suggestions were followed by a review of the instructions for the AC task and suggestions that P would succeed in the task. The entire exercise was recorded on tape by the experimenter and played over speakers to the Ps.
The experimental design was a 2 x 2 factorial with Method (eyes-closed vs. quotations) and Hand (left vs. right) as the independent variables. Hand refers to whether Ps were asked to hold the pen with the right or left hand during the AC task. Twenty Ps were assigned to each of the four cells, abbreviated ER (eyes-closed, right hand), EL (eyes-closed, left hand), QR (quotations, right hand), and QL (quotations, left hand). Condition assignments were made by Ruszaj, who performed a random permutation of the numbers 1-80, with 20 of the numbers coded as representing each of the four conditions. The list was then sent to me so that I could be informed immediately before a session which condition a given P should be in.
Lab Room Layout
P was seated in front of a table in a padded chair that could be tilted backwards and adjusted for height. In the left-hand conditions, the tablet was located to the left of the computer monitor, whereas in the right hand conditions these positions were reversed. Two small audio speakers were located at the back of the table facing P. Before the AC test, P was given the choice to have the room illuminated by the overhead lights or a small table lamp with a 40W bulb located at the far right end of the table.
After collecting any questionnaires the Ps brought with them, I gave them a more detailed description of the procedure than was provided in the solicitation letter. I illustrated how to take the AC task by moving the pen around the grid and stopping for 1 s. In an effort to minimize response biases, 1 emphasized that they should explore the entire grid before making a selection. I also described the three methods of scoring (square, quadrant, and number) and told them that they would receive feedback of their scores at the end of the phase. Finally, I described the task manipulation (quotations or eyes-closed).
Next, I set up the practice session on the computer. I had Ps adjust the location and orientation of the tablet as well as the height of the adjustable chair for maximum comfort and ease in moving the pen around the grid. Ps in the quotation conditions then timed their quotation reading speed as described above. The practice session had two stages. In the first stage, I began by having Ps look at the grid while moving the pen around. When I was comfortable they could do this easily, I asked them to continue with their eyes closed for a few more trials. To get them adjusted to how long they needed to stop the pen for a response to register, a beep sounded after the pen was motionless for 1 s. In the second practice stage, Ps practiced the procedure to be used in the formal experiment, which did not include the beeping sound. I asked them to continue practicing until they and I agreed that they could do the movements correctly and effortlessly. This practice was particularly important in the quotation conditions, where Ps had to learn to make their AC responses independent of where they were in the process of reading the quotations; the common tendency at the outset is to make the AC response at the time one finishes reading a quotation. Ps varied widely in how long they practiced, and the number of practice trials usually exceeded the 36 trials in the formal experiment.
After the practice was completed, I set up the main experiment, asking Ps in the quotation conditions if they wanted to change the length of time each quotation appeared on the screen. I told Ps in the eyes-closed conditions that a beep would inform them when the 36 trials were done but that they would not hear a sound after each AC response. During the task I was seated in the adjacent room doing professional reading. I closed the door between this room and the test room as I left. Ps were instructed at the end of the AC task (eyes-closed conditions) or the quotation recognition quiz (quotation conditions) to knock on the door to retrieve me.
When I returned, I immediately had Ps fill out the post-task questionnaire, and I discussed their answers with them afterwards. I focused especially on their subjective experience while taking the AC task. I then described the AC feedback screen, which consisted of trial-by-trial scores, total scores, and number of hits needed for a statistically significant AC score for all three scoring modalities (square, quadrant, and number). I told them that because I did not want to see the scores myself, after I left the room they could see their AC score (and quiz score, if applicable) by pressing the <enter> key on the computer keyboard. They should then press the <enter> key again to clear the screen. A paper tablet was available on which they could write down their AC scores; they could take the sheet home with them if they wished. After asking if they had any further questions about the experiment, I paid them $15 in cash and had them sign a receipt. Finally, I thanked them for their participation and left the room.
Eight pilot sessions were conducted with six individuals to determine if it was necessary or desirable to make refinements in the procedures and materials as described above. The Ps were drawn from people closely associated with the RRC, and none were included in the formal experiment.
Although there were three separate AC scores, to obviate multiple analysis issues I decided to compute a single AC z-score to test the hypotheses. I wanted a composite score but I wanted it to reflect the application of a motor automatism as opposed to a strictly cognitive mechanism. For that reason, I omitted the number scores from the conglomerate. Specifically, for each run, the two remaining component scores (quadrant and square) were combined to produce a single z-score, the computation of which is described in Results.
The statistical tests used to test the hypotheses included ANOVAs, t tests, chi-squares, and Pearson correlations. The need for nonparametric alternatives for the parametric tests was eliminated by applying normalization procedures to highly skewed distributions. The statistical analyses were performed using SPSS software.
Left-handed Ps were removed from analyses involving the Hand variable. I was not interested in hemisphere dominance, just which hand the P customarily uses or is comfortable using. Ambidextrous Ps were retained.
All p values are two-tailed expect for tests of the outside-force-AC relationship, which was considered a replication of Palmer (2011)
Performance errors are one of the hazards of asking Ps to perform a task over which they are supposed to have minimal conscious control. In this case, the concern was mechanical errors in moving the pen around the tablet. To minimize these errors is why Ps were given extensive practice before the official task. However, I was never convinced this learning would completely carry over to the formal session. One reason is that Ps were now trying to use psi to obtain a good score. The second reason is that they were presumably in a different state of consciousness as a result of the intervening relaxation exercise.
As I did not observe Ps performing the formal task, I could only get information about possible errors by looking at the data, which I did not do until data collection had been completed. Useful information would be reaction times (the time from the end of trial t to beginning of trial t+1), so I had the computer record this information in milliseconds.
After the experiment, I printed out a sheet for each P with two columns of data. The first column gave the 36 square responses for the run and the second column gave the corresponding reaction times. I noticed that for many Ps some of the reaction times were quite low, as little as 750 ms. In most of these cases the response square was the same for this trial and the preceding trial; in almost all other cases the two response squares were adjacent. During the practice sessions, Ps were encouraged to "explore the grid" before each response by moving the pen randomly around the tablet, so I expected the reaction times to be at least several seconds. On the other hand, my feedback during debriefing suggested that during the actual task Ps would often go directly from one response to the next, which could be done quite quickly. But if that were going on, I would expect the response squares for the two trials to be nonadjacent much more frequently than they in fact were.
Instead, I concluded that most of these responses were inadvertent jiggles of the pen in the area of the first response (despite my efforts to warn Ps to keep the pen still between trials). This would cause an unintended response to register. I had noticed this behavior during the early stages of the practice sessions, but by the time the session ended it had vanished. However, as noted above, there was reason to fear that it might re-emerge during the formal task.
In the quotation conditions, the screen flashed after each response, so Ps knew, consciously or unconsciously, that a trial registered and they could use this information, consciously or unconsciously, to correctly identify the proper target for the next trial by jumping one target ahead. However, this information was not available to Ps in the eyes-closed conditions. I had considered continuing the beeps after each response in the eyes-closed conditions, which would have functioned like the flashes in the quotation conditions. I decided against this, partly because I was afraid the beeps would bring them out of the meditative state that I expected to be optimized in the eyes-closed conditions, but also because Ruszaj was unable to get the software to produce the beeps reliably.
I decided to make up a set of alternate response sequences that would take this bias into account. Specifically, I wanted to capture the response sequences the Ps intended, not necessarily the ones that were sent to the computer. To do so, I perused each sheet individually to detect trials on which the bias was likely to have occurred. Because the sheets did not include the target sequences, I could do this with no sensory information about hits, and thus I was not biased. I assumed that the quotation Ps adjusted for the bias and the eyes-closed Ps did not. Thus, for quotation Ps, I simply removed the response, meaning that the suspect trial was declared missing. For eyes-closed Ps, however, I needed to adopt a more complicated strategy. Starting at the top of the run, for each suspect trial, I removed the response for that trial and moved all the remaining responses up one space; for example, if Trial 6 was suspect, the response to Target 6 would now become the response to Target 5, the response to Target 7 would become the response to Target 6, and so forth. This left a number of missing (unscored) trials at the end of the run corresponding to the number of suspect trials, as the sequence of registered responses had been prematurely exhausted. This exercise resulted in marked changes in the AC scores in the eyes-closed conditions but trivial changes in the quotation conditions.
I also decided to remove from subsequent analyses the run scores of 9 Ps who made a large number of such errors. All these Ps were in the eyes-closed condition; 5 used the right hand and 4 used the left. I did this because I felt the chances were high that at least one of my guesses about whether a response was erroneous would be wrong, rendering the resulting sequence especially unlikely to be a valid representation of the P's intent. (I fully expected that some of my guesses were incorrect even in the other cases, but I believed that overall the revised sequences would be closer than the original sequences to Ps' intended responses.) I also questioned whether the removed Ps were performing the task competently overall. Changes were made to the response sequences of 40 of the remaining 71 Ps (56%).
To create for each P hit totals adjusted for the number of trials, 1 applied the formula Y = X*36/N, where Y is the adjusted score, X is the original score, and N is the number of valid trials. I then converted all scores to z-scores using the formula (X-M)/SD, where X is the number of hits, and M and SD are the mean and standard deviation of the distribution of hits for all valid Ps. These z-scores were obtained for the square, quadrant, and number hits as recorded by the computer for all trials of all Ps (Model 1) and for the corresponding hits obtained using the revised response sequences as described above (Model 2). The hit scores used to test the predictions were an unweighted combination of square and quadrant scores using the Stouffer formula ([z.sub.s]+[z.sub.q])/[square root of 2], where [z.sub.s] is the z-score for squares, [z.sub.q] is the z-score for quadrants. Hereafter, these scores will be referred to as "location zs." They were predesignated as the main dependent variable and the variable used to test the hypotheses.
Normalization. All the hit distributions with the exception of number hits were positively skewed due to the presence of outliers on the right tail. I am strongly opposed to removing outliers just because they are outliers, as doing so grossly violates the integrity of the distribution. To remove the excess skewness, I developed a procedure for systematically moving the outliers toward the mean. Specifically, I find the largest distance between any two adjacent outliers and move this value and all more extreme values on that tail half this distance toward the mean. I repeat this process until the value of the skewness statistic is less than 2.0. This method was applied to all the skewed distributions and the resulting distributions were used for all analyses.
Comparison of models. Model 1 is represented by use of the original 36-trial scores of all 80 Ps. This model is preferable if either of two assumptions is valid. The first assumption is that even in the eyes-closed conditions, Ps were not set off course by their invalid responses, because either (a) they knew, perhaps by psi, that their last registered response was not intended and adjusted their responses accordingly to match the real targets for all future trials, or (b) through some less mechanical means they simply "made things turn out right." The latter has been referred to as psi being "goal-oriented." Often cited as evidence for goal-orientation is a random event generator (REG) experiment in which results were equally strong regardless of the internal complexity of the REG (Schmidt & Pantas, 1972). Model 1 is also valid if the observational theories (Millar, 1978) were operative, because the Model 1 scores were the ones shown to the Ps at the end of the session. Model 2 is preferable if Ps in the eyes-closed condition were not aware that they had registered an unintended response and attempted to match their response to the target of the preceding trial.
I decided to choose between the models empirically. The first test one normally does in evaluating data is a global analysis (e.g., an ANOVA) that registers the effects of the manipulated independent variables, in this case. Method (eyes-closed vs. quotations) and Hand (right vs. left), on the dependent variable, location zs. I decided to do the same ANOVA for Model 1 and Model 2 and compare the results. If the total Ffor one or both models is significant, I would choose for subsequent analyses the model that is most significant (highest [R.sup.2]). If neither model is significant, I would choose the model that yields the highest magnitude mean z-score (hitting or missing direction) across all Ps in the model. If there is no psi in the data, it doesn't matter which model I choose; all I will get from either one are Type-I errors, and there is no reason to expect more of these from one than from the other. If there is psi in the data, then the best guess is that the model that shows the stronger results at this stage is the more valid. The only hypothesis tested by these ANOVAs is the handedness hypothesis (Hypothesis 3). All the other hypotheses involve individual-difference measures as predictors and are not dependent on the ANOVA. These were to be tested using only one model, thus eliminating the multiple-analysis problem insofar as it relates strictly to the dependent variable.
The ANOVAs represented a 2 x 2 factorial, between-subjects design, with Method (eyes-closed vs. quotations) and Hand (right vs. left) as the independent variables. The four cells are ER (eyes-closed, right hand), EL (eyes-closed, left hand), QR (quotations, right hand), and QL (quotations, left hand). Location z was the dependent variable. Because one of the independent variables represented which hand was used for the task, the five left-handed Ps were removed from the ANOVAs.
The means and standard deviations for location z in each cell, left-handers removed, are reported in Table 1 for both Model 1 (N = 75) and Model 2(N = 68). Only seven disqualifies were removed, because two were left-handed and removed on that basis. The Model 1 ANOVA was significant overall, F(3,71) = 3.56, p = .018 ([R.sup.2] = .13). The only significant effect was the Method x Hand interaction, 7r(1,71) = 6.66,p = .012. Inspection of the means reveals that the interaction consisted of positive scoring (psi hitting) in the EL and QR conditions, offset by negative scoring (psi missing) in the ER and especially the QL conditions.
Model 2 was also significant overall, F(3,64) = 5.34, p = .002 ([R.sup.2]= .20). Significant main effects for Method, F(1,64) = 5.38,p = .024, and Hand, F(1,64) = 5.08,p = .028, were superseded by a significant Method x Hand interaction, F(3,64) = 3.56, p = .018. As with Model 1, the interaction reflected positive scoring in the ER, EL, and QR conditions offset by negative scoring in the QL condition. Because [R.sup.2] :was higher in Model 2 than in Model 1, Model 2 was chosen for the subsequent analyses. Model I is not discussed further and in fact was never analyzed.
For subsequent analyses of Model 2, conditions ER, EL, and QR were pooled to form condition EQR. The reason for isolating the QL condition is as follows. In an earlier paper (Palmer, 1975), I argued that when an AC mean is below chance, one should expect a negative relationship between AC and a predictor, and this is what I found in a series of OBE/ganzfeld experiments I conducted many years ago (e.g., Palmer, Bogart, Jones, & Tart, 1977). The rationale for this prediction is that the Ps with the lowest scores are producing the most psi, with Ps scoring low on the predictor averaging around chance. Thus, as I had hypothesized positive relationships between AC scores and dissociation, it is at least highly questionable to apply that prediction to the QL condition, where there was significant overall psi-missing. Three lefthanders were also removed from the EQR condition, reducing the total N from 71 to 68. The mean number of location hits for the 49 Ps in the EQR condition was 0.42 (SD =1.12), t(48) = 2.62, p = .012. The mean for the 19 Ps in the QL condition was -0.76 (SD = 0.94), t(18) = 3.53, p = .002.
It is obvious from these results that Hypothesis 3, better performance with the left hand, was not supported.
Number z. Because number hits are independent of location hits, the cell means were added to Table 1. The overall model for number zs is nonsignificant, F(3,64) = 0.58, p = .63. Neither the main effects nor the interaction were significant. Because the QL mean was the lowest of the four AC cell means, I compared the EQR and QL conditions on number hits. The result is nonsignificant, t(66) = 0.92.
State Dissociation (Outside Force) and AC
Change in hypothesis. The state measure of dissociation was the question "Did you have the impression that at any time during the session the pen was being guide by an outside force?" Based on the results of Palmer (2011), it seems that the prediction should be that performance would be significantly better for Ps who experienced the OF from 1 to 40% of the time during the task than for other Ps (Hypothesis 1).
Based on data I had observed prior to looking at the AC scores, but before I observed any AC scores in the present study, I decided to modify the hypothesis to state simply that performance would be significantly better for Ps who have the experience that the associated hand is being moved by an outside force (for any percentage of the movements) during the task than for other Ps. There were two reasons for the switch. First, based on the results of Palmer (2011), I decided for the present study to interview Ps at the end of the session about their experience during the session. I had expected that the reason for the curvilinear effect in Palmer (2011) is that those who experienced the outside force < 40% of the time were in a qualitatively different state of consciousness than those who experienced it > 40% of the time. I found no evidence of such a difference in the interviews in the present study.
Second, a smaller percentage of the Ps said they had experienced an outside force in the present study (44%) than in the Palmer (2011) study (65%). This finding affected my decision to change the hypothesis for the following reason. Many of the Ps in the Palmer (2011) study were college students, whereas almost none were college students in the present study. Even though I was not the students' professor, I had a similar enough role as an "authority figure" that some of them might have wanted to please me, a common malady in mainstream psychology research called "demand characteristics." A good way to accomplish this objective would be to assure me that they had the "expected" and "hoped for" experience of their hand being moved by an outside force, even when it was not. To do this fully, they would not only want to acknowledge having the experience but to have it a high percentage of the time. I think more likely than an outright lie is adoption of an overly liberal conception of what the experience of an outside force would be like. This hypothesis can explain why 21% more Ps claimed the OF effect in Palmer (2011) than in the present study. If those Ps in the > 40% OF condition who presumably acquiesced to demand characteristics were shifted from that condition to the 0% force condition, better reflecting their true experience, the relationship would conform to the revised hypothesis for the present study.
Location z-scores. In the EQR condition, the 19 Ps who answered the OF question "yes" scored significantly above chance (M= 0.78, SD= 1.22), f(18) = 2.79, p = .006, one-tailed, and significantly higher than the 30 Ps who answered it "no" (M= 0.19; SD = 1.01); for the difference, t(47) = 1.83, p = .037, one-tailed. Thus, Hypothesis 1 was confirmed, but only for the EQR condition. On the other hand, there was a negligible, nonsignificant difference in the QL condition, where the six Ps who answered the OF question "yes" had an AC mean of-0.64 (SD = 1.22) compared to a mean of-0.82 (SD = 0.84) for the 13 who answered it "no," t(17) = 0.37, p = .72.
Number z-scores. How Ps answered the force question had no effect on the number hits. The 28 Ps who answered the OF question "yes" had a mean z of -0.06 (SD = 0.84) compared to a mean of -0.03 (SD = 1.10) for the 43 "no" responders. The difference is nonsignificant, t(69) = 0.10.
Trait Dissociation and AC
The Dissociative Processes Scale (DPS) used to measure trait dissociation has three subscales: Obliviousness, Imagination, and Detachment. Descriptive data for the three subscales are as follows. Obliviousness (M= 43.62, SD = 11.10); Imagination (M= 23.02, SD = 6.90); Detachment (M= 15.35, SD = 5.44). In our sample, the intercorrelations among the scales ranged from .44 to .63. The intention from the outset was to analyze the three scales individually. All the scales had acceptable skewness and kurtosis. The correlations of these scores with the AC measures are summarized in Table 2.
Location z-scores. Obliviousness. Hypothesis 2 predicted a significant positive correlation between location hits and scores on DPS Obliviousness in the EQR condition. This hypothesis was not confirmed, r(47) = -.07.
Imagination. There was a significant positive correlation between location hits and DPS Imagination in the EQR condition, r(Al) - .35, p = .01. The correlation was in the same direction and of comparable magnitude, although not significant, in the QL condition, r(17) = .30,p= .21.
Detachment. There was a significant positive correlation between location hits and DPS Detachment in the EQR condition, r(47) = 30, p = .03. The correlation was in the same direction and of comparable magnitude, although not significant, in the QL condition, r(17) = .25,p = .30.
Number z-scores. Obliviousness. The correlation between DPS Obliviousness and number hits is nonsignificant in the EQR condition, r(47) = -.03, but surprisingly significant in the QL condition, r(17) = .46, p = .05.
Imagination. There was a significant positive correlation between number hits and DPS Imagination, r(69) = .26,p = .03. The correlation is nonsignificant in the EQR condition, r(47) = .19, but significant in the QL condition, r(17) = .56, p = .01.
Detachment. The correlation between number hits and DPS Detachment is nonsignificant in the EQR condition, r(47) = .23,p = .11, but significant in the QL condition, r(17) = .62,p = .005.
Trait Dissociation and OF Ratings
There is no significant difference between Ps who did and did not claim they felt their hand being moved by an outside force during the AC task on DPS Obliviousness. Thus, Hypothesis 4 was not supported. However, there is a substantial and highly significant difference in their scores on the Detachment subscale. The numbers are presented in Table 2.
Independence of Predictors
I wanted to know if each of the three significant predictors of location hits in the EQR condition (OF question, DPS Detachment, DPS Imagination) could be demonstrated to have an effect independent of the other two. It is obvious from Table 2 that OF and Detachment are not independent, and that was confirmed by a regression analysis I performed with all three predictors. However, I wouldn't expect them to be independent, because they are respectively state and trait measures of the kind of dissociation that the OF responses were intended to reflect. Therefore, I created a new variable, Dissociation, that combines the two measures as if they were two items on a single scale. I coded a "yes" response to the OF question as + 1 and a "no" response as -1, the dummy codes I used in the previous regression analysis. I then generated standard scores for OF and Detachment, added them, and divided by 2. This is the same way I created the location z-scores. For consistency, I also converted Imagination scores to standard scores. I then performed a new linear regression analysis with Dissociation and Imagination as the sole predictors. R for the model was .42, F(2,46) = 4.97, p = .011. The two betas were suggestively significant: Dissociation: [beta] = .24, p .093; Imagination: [beta] = .28,p = .052. The same analysis gave a significant correlation between Dissociation and Imagination, r(Al) = .30, p = .034. Finally, Dissociation is significantly correlated with location hits, r(47) = .35, p = .013.
The initial analysis revealed significant psi-missing in the QL condition (AC responses made with the left hand while reading quotations on the computer screen) offset by significant psi-hitting in the other three conditions, which were pooled to create the EQR condition. The significant psi-missing in the QL condition makes sense in that reading quotations is more cognitively demanding than blanking the mind, and any psychomotor task is more difficult to perform with the nondominant hand. Combining the two added yet another layer of difficulty. Although I attempted to mitigate this problem by giving Ps liberal practice under the conditions they would experience in the formal task, as I explained earlier, I never expected that it would be enough to completely solve the problem. These difficulties can easily lead to frustration, and such a negative mindset has long been associated with psi-missing in parapsychology.
The OF question was intended as a measure of a dissociated state during the AC task. Its validity in this regard was assessed by determining whether it correlates significantly with an appropriate trait measure of dissociation capacity, for which I chose the DPS. Specifically, I expected the OF question to correlate with the Obliviousness subscale of the DPS, and this expectation was reflected in Hypotheses 2 and 4.
However, there was not a significant relationship between the OF question and DPS Obliviousness. Instead, there was a quite strong positive relationship between the OF question and another of the DPS subscales, Detachment. Although not predicted, this relationship makes sense. Watson (2003, p. 300) defines the Detachment subscale as measuring "feelings of depersonalization and derealization," which doesn't seem very relevant. However, greater relevance can be found if one examines the six items that compose the subscale. At least three of these reflect separation from the body, very much like an out-of-body experience. For example, Item 17 reads "At times I have felt disconnected from my body." It is not a huge inferential leap to extend this concept to one's hand being outside of, and not controlled by, the rest of one's body and the mind that governs it. It is striking that the correlations between the OF question and the other two DPS scales, Imagination and Obliviousness, are nowhere near significant, despite the fact that the three subscales correlated with one another in the .4 to .6 range. Finally, DPS Detachment scores were directly and positively related to location hits in the EQR condition. In short, the present study confirms that trait dissociation is indeed reflected by the tendency to experience one's hand being moved by an OF during an AC motor task.
The reinterpreted-as-positive relationship I found in my previous motor automatism study (Palmer, 2011) between AC hits and perception of the hand as being moved by an outside force at least some of the time during the AC task was found in the present study, confirming Hypothesis 1, albeit only for the EQR condition. Surely there is no reason for the effect to hold in the QL condition, where there was significant overall psi-missing. If anything, one would predict a reversal of the OF/AC relationship in this condition, for a reason I discuss further below. One could say that the significant correlation between location hits and the Dissociation measure created for the regression analysis provides a conceptual confirmation of Hypothesis 1, because the combination of OF and DPS Detachment (the components of Dissociation) is a better measure of the dissociation construct that the OF question was intended to capture than the OF question alone. Also, this correlation is significant by a two-tailed test.
The significant correlation of AC hits with DPS Detachment in the EQR condition was matched by a significant correlation of AC hits with DPS Imagination. Also, both DPS scales correlated positively and significantly with number hits in the entire sample. For number z-scores there was not the sharp difference between total scores in the EQR and QL conditions found with location z-scores. For both predictors, the correlation with number z-scores was actually stronger in the QL condition than in the EQR condition.
An Overall Interpretation: The Two-Process Model
So, what might this complex pattern of results be telling us? First, note that accessing a number as such requires cognitive activity in the strong sense. Whereas such cognitive processes could also be used to access square and quadrant, they are not necessary. If Ps were able to follow the instructions properly, any such cognitive activity should have been unconscious. Second, recall that Watson (2003, p. 300) describes the Imagination subscale as assessing "absorption, imaginativeness, and fantasizing." This describes activity that is more cognitive than for the other two scales, even to the point of reflecting imagery.
My interpretation from all this is that there were two kinds of psi processes contributing to the AC test outcomes, and I will label what follows as my two-process model. One process was almost purely motor (which can be expressed metaphorically as the hand performing the AC task) and the other was primarily cognitive. The low-magnitude results parapsychologists consistently find in AC tests suggests that most individual responses to targets are not psi-mediated; they are simply "wild guesses." This leaves plenty of room for different processes to mediate the responses on different trials in a forced-choice AC test such as the one used in this experiment. In the present case, that would mean that within a 36-trial run exhibiting some psi, most trials would be chance, a few would be motor-psi-mediated and a few others cognition-psi-mediated. It also assumes, as noted above, that the variance associated with DPS Detachment can be divided into two components: that which significantly predicted responses to the OF question (Component A) and that which correlated significantly (r = +.52) with DPS Imagination (Component B).
The motor process was active only in the EQR condition and affected only location hits. It was predicted by Component A of DPS Detachment and was experienced by Ps who felt their hand being moved by an OF. This model is supported by the following specific findings: (a) the significant positive relation between DPS Detachment and the OF question, along with the lack of a significant relation between the OF question and DPS Imagination; (b) the significant correlation between the OF question and location z-scores exclusively in the EQR condition; (c) the significant correlation of DPS Detachment with location z-scores only in the EQR condition; (d) the lack of a significant relation between number z-scores and either DPS Detachment or the OF question in the EQR condition.
The cognition process was operative in all the conditions, including QL. It was predicted primarily by DPS Imagination, and only by DPS Detachment insofar as that was correlated with DPS Imagination (Component B). The model is supported by the following specific findings: (a) a significant correlation between DPS Imagination and location z-scores in the EQR condition; (b) a significant correlation between DPS Imagination and number z-scores in the entire sample. Although either the motor or the cognitive process could be used for location hits, only the cognitive process can achieve number hits; the ability of the "hand" to point to the correct spot on the grid is useless for identifying numbers per se.
The two-process model is also supported by the regression analysis. In fact, seeing if such support could be found was my main motive for performing this analysis. I have a philosophy about the respective roles of bivariate and multivariate relationships that is the reverse of what seems to be the norm in psychology and to some degree in parapsychology: In my view, bivariate relationships should be primary, and targeted (as opposed to global) multivariate analyses should be used to interpret their meaning. I think what follows provides a good example of why this approach makes sense. First, the suggestively significant betas for Dissociation and Imagination provide support for the proposition that these variables made independent contributions to the prediction of location hits in the EQR condition. On the other hand, the significant correlation between Dissociation and Imagination, combined with the significance of the full model, provides support for the complementary proposition that these variables also made joint contributions to the prediction. The suggestive confirmation of the independent contributions of Dissociation and Imagination is more impressive than it might seem on the surface because it was found despite the .3 correlation between them. Given the notoriously poor reliability of forced-choice ESP scores, a suggestive result is the best that one could reasonably expect if the two-process model is correct. (2)
Some additional comments are needed about number hits in the QL condition. The correlation between number z-scores and DPS Imagination was particularly strong in the QL condition. However, what is most surprising is that the direction of the relationship was positive, because, as I noted earlier in the paper, one would expect a negative correlation when the AC mean is below chance (Palmer, 1975). My explanation of the positive correlations between the DPS scales (even Obliviousness) and number z-scores in the (psi-missing) QL condition draws on my theory of different processes operating on different trials. Specifically, high-Imagination Ps were able to overcome on some trials the apparent frustration manifested on other trials by the difficulty of the QL procedure, but only for cognitive responding. In other words, the high-Imagination Ps were able to partly offset or cancel the psi-missing trials caused by frustration with psi-hitting trials mediated by their relatively good cognitive psi abilities, bringing the mean number z-score obtained by these high-Imagination Ps close to chance. This offset would not apply to low-Imagination Ps, so their scores on number hits would remain low, the net result being a positive correlation between DPS Imagination and number hits in the QL condition. Why Ps could not do that using the motor process remains unclear.
Finally, the two-process model has caused me to recognize that a cognitive process was probably involved in producing the results of the Palmer (2011) study, something I had previously been in denial about. Specifically, the results in Palmer (2011) were best when participants based their ESP guesses on a combination of their alphabet board responses and mental imagery they might recall from the session.
Any psi researcher is required in my view to at least acknowledge the possibility that their results were contaminated to at least some degree by investigator psi, or I-psi. (I use this term rather than the more customary E-psi because I think the potential psi sources should include the Principal Investigator, who may not be interacting with the participants.) Although I-psi can never be ruled out, it can be minimized by eliminating as much as possible intuitive or random investigator decisions. This is why I had Kruth use the seed numbers 1 and 2 to activate the algorithm for target decision. It was my original intent to have the condition assignments made by a continuation of that number sequence, but due to logistical complications it was in fact done by my programmer Ruszaj using another algorithm. He had no stake in the outcome of the study and showed no particular interest in its conceptual aspects, so I doubt he had the motivation I consider necessary to be an implicit psi source. In theory I could have influenced the process myself, but at the time I had no idea when Ruszaj would perform the task or even that he was going to be the one to do it.
By far the most likely source of I-psi in the study was my adjustment of the response sequences for some Ps in the eyes-closed conditions. Even though I did my best to make these decisions rationally, more than one option could occasionally be defended on rational grounds, and it is possible that psi could have tilted my decision process in a favorable direction in some cases. Clearly, the results in these conditions were better for Model 2 than for Model 1. However, there were equally strong results in the QL condition, where the impact of my decision-making was minimal. On the other hand, my general expectation in studies such as this is that both the participants and the investigator have input into the results (Palmer & Millar, 2015), and I see no reason to make an exception in this case.
First, to meet my own requirement for publication of empirical reports in the JP, I hereby declare that all significant results from this exploratory study must be considered tentative unless or until they are replicated, unless they are replications themselves. The only relationship that meets the latter standard is that confirming Hypothesis 1, but only if one accepts my modification the hypothesis. (3)
On the other hand, although only one of the four hypotheses was confirmed by the data (state-dissociation-AC), and in this case the hypothesis had to be reworded (for reasons I consider justified), there was a surprisingly large number of significant effects that were not hypothesized, most of which make sense both individually and in relation to one another, and they could have been justifiably hypothesized were I more astute. I maintain that more attention should be paid to whether an effect is hypothesizeable than whether it is hypothesized, (4) By this standard, and granted that little if anything was established by this study, it seems to me that the likelihood is sufficiently great that the psychological processes functioning as described in the two-process model contributed to the production of weak but genuine psi effects in this study that the model could profitably be taken into account in designing future process-oriented research of relevance to the model. For my part, I have completed a followup study in which 1 attempted to train the best individual Ps in the present study to develop proficiency in the procedure through extensive practice with subliminal auditory feedback of hits. The purpose was to increase the reliability of their AC scoring. The results will be reported in a separate paper.
Bartlett, J. (2002). Bartlett's familiar quotations: A collection of passages, phrases, and proverbs traced to their sources in ancient and modern literature (17th ed.). Boston, MA: Little, Brown.
Braud, W. G. (1975). Psi-conducive states. Journal of Communication, 25, 142-152.
Hilgard, E. (1977). Divided consciousness. New York, NY: Wiley.
Maher, M., & Schmeidler, G. R. (1977). Cerebral lateralization effects in ESP processing. Journal of the American Society for Psychical Research, 71, 261-271.
Marsaglia, G., & Zaman, A. (1987). Toward a universal random number generator (FSU-SCR1-87-50). Gainesville, FL: Florida State University.
Metcalf, F. (2009). Dictionary of modern humorous quotations (3rd ed.). New York, NY: Penguin.
Millar, B. (1978). The observational theories: A primer. European Journal of Parapsychology, 2, 304-332.
Myers, F. W. H. (1903). Human personality and its survival of bodily death (2 vols.). London, UK: Longmans, Green.
Palmer, J. (1975). Three models of psi test performance. Journal of the American Society for Psychical Research, 69, 333-339.
Palmer, J. (2001). A mail survey of Ouija board users in North America. International Journal of Parapsychology, 12, 67-93.
Palmer, J. (2011). Motor automatisms as a vehicle of ESP expression. Journal of Parapsychology, 75, 45-60.
Palmer, J. (2013). JP publication policy: Statistical issues [Editorial], Journal of Parapsychology, 77, 5-8.
Palmer, J. (2016). Statistical issues in parapsychology: Hypothesis testing--plus an addendum on Bierman et al. (2016) [Editorial], Journal of Parapsychology, 80, 141-143.
Palmer, J., Bogart, D. N., Jones, S. M., & Tart, C. T. (1977). Scoring patterns in an ESP ganzfeld experiment. Journal of the American Society for Psychical Research, 71, 122-145.
Palmer, J., & Millar B. (2015). Experimenter effects in parapsychological research. In E. Cardena, J. Palmer, & D. Marcusson-Clavertz (Eds.), Parapsychology: A handbook for the 21st Century (pp. 293-300). Jefferson, NC: McFarland.
Persinger, M. A., & Makarec, K. (1993). Complex partial epileptic signs as a continuum from normals to epileptics: Normative data and clinical populations. Journal of Clinical Psychology, 49, 33-45.
Schmidt, H., & Pantas, L. (1972). Psi tests with internally different machines. Journal of Parapsychology, 36, 222232.
Solomons, L. M., & Stein, G. (1896). Studies from the Psychological Laboratory of Harvard University. II. Normal motor automatism. Psychological Review, 3, 492-512.
Tellegen, A. (1978). Differential Personality Questionnaire. Minneapolis, MN: University of Minnesota.
Watson, D. (2003). Investigating the construct validity of the dissociative taxon: Stability analyses of normal and pathological dissociation. Journal of Abnormal Psychology, 112, 298-305.
Rhine Research Center
2741 Campus Walk Ave., Bldg. 500
Durham, NC, USA
I am grateful to the Bial Foundation for their support of this research.
(1) As I am Editor of the JP, to avoid self-serving bias I followed my policy in such cases and had this paper peer-reviewed under the auspices of Dr. Richard Broughton of the University of Northampton.
(2) The more general proposition that underlies my model for this experiment, namely that different psychological processes mediate psi in different trials of the same run, is to my knowledge novel in parapsychology. Thus, I think it is worth presenting even if it cannot be considered to have been conclusively confirmed.
(3) This standard is my (more demanding) substitute for multiple analysis corrections, to which I have strong objections (Palmer, 2013).
(4) For a further defense of this approach, see Palmer (2016).
Caption: Figure 1. Target board with pen
Table 1 Mean (n) for Model 1 (N = 75) Location, and Model 2 (N = 68) Location and Number, z scores as a Function of Method and Hand Location z (Model 1) Location z E Q E R -0.07 (18) 0.38 (19) 0.46 (14) L 0.14 (19) -0.75 (19) 0.41 (16) (Model 2) Number z (Model 2) Q E Q R 0.39 (19) -0.01 (14) 0.21 (19) L -0.76 (19) -0.11 (16) -0.21 (19) Note. E = Eyes-closed. Q = Quotations. R = Right hand. L = Left hand. Left-handers were removed. Table 2 Means (Standard Deviations) Comparing Yes and No Responders to the OF Question on the DPS Subscales for the Entire Sample Subscale Yes (N = 35) No (N = 45) t P Obliviousness 45.26 (10.96) 42.71 (11.21) 1.02 .31 Imagination 23.34 (7.57) 22.78 (6.41) 0.36 .72 Detachment 17.86 (5.38) 13.40 (4.68) 3.96 .0002
|Printer friendly Cite/link Email Feedback|
|Publication:||The Journal of Parapsychology|
|Date:||Mar 22, 2017|
|Previous Article:||Bem's "feeling the future" (2011) five years later: its impact on scientific literature.|
|Next Article:||Experimenter fraud: what are appropriate methodological standards?|