Binocular virtual reality displays: when problems do and don't occur.
It is fashionable to label any computer representation of three-dimensional (3D) space as "virtual reality." We have previously suggested (Wann & Mon-Williams, 1996) that the term virtual reality (VR) should be reserved for displays in which observers perceive that they are within, or interacting with, a structured 5D environment as opposed to perceiving the display as a 2D projection of a 3D animated model. (We also suggested that the term reality is unsatisfactory, given that systems neither do, nor will, provide unconstrained realism. Despite this, the term virtual reality is now so ingrained in current terminology that we will use it here.)
Some VR systems utilize standard computer screens, whereas others use large-screen projection or head-mounted displays (HMDs) to enhance the user's sense of immersion. In all cases the display may be binocular or biocular. A binocular display can present stereoscopic depth information using various approaches, such as field-sequential single-screen displays with shutter spectacles, single-screen polarized displays, or dual-screen HMDs. A number of studies have looked at the human factors involved in such stereoscopic displays (e.g., Patterson, Moe, & Hewitt, 1992; Yeh & Silverstein, 1990). Although the vast majority of early HMDs were designed to display stereoscopic depth, not all VR systems provide such information. Some VR systems present the two eyes with images that, when fused, do not present intraimage disparities; these are referred to as bi-ocular displays.
If an observer wishes to change fixation from a far object to a near object (or vice versa), the retinal image of the target object is initially defocused, and an error occurs between the target and the angle of ocular vergence. We use the term vergence disparity to refer to this error of fixation. In order to overcome vergence disparity, the eyes must change vergence angle to maintain single vision; and to bring clarity to the retinal image, the eyes must focus in a process known as accommodation. In natural vision the accommodation and vergence responses are neurally cross-linked so that accommodation produces vergence eye movements (accommodative vergence) and vergence causes accommodation (vergence accommodation). Under normal viewing conditions, accommodation and vergence vary synkinetically and are dependent on object distance.
All currently available binocular displays create an illusion of depth by combining dual 2D images generated from screens with a fixed focal plane. Hence appropriate vergence disparity may be created in a VR display through the presentation of separate images to the right and left eyes, but the creation of appropriate blur information is problematic because the blur should respond to the user's accommodative effort. Current VR systems do not provide appropriate blur cues and therefore require a constant accommodative effort. A VR system that permitted introducing blur through focal depth adjustment would also need to be able to detect changes in accommodation, but extant technology does not allow for the monitoring of accommodation and the alteration of blur cues. The requirement for constant accommodation with changing vergence angle causes problems for the visual system because of the cross-links between vergence and accommodation (Wann, Rushton, & Mon-Williams, 1995). This means that the normal accommodation-vergence relationship is disrupted, and the visual system is presented with demands that do not occur in natural settings [ILLUSTRATION FOR FIGURE 1 OMITTED].
Mon-Williams, Wann, and Rushton (1993) demonstrated that a stereoscopic VR display can produce deficits of binocular vision, and these early findings have been confirmed by more recent investigations (Howarth & Costello, 1996). However, Rushton, Mon-Williams, and Wann (1994) established that a bi-ocular HMD does not cause problems for the visual system when assessed in the same manner as the stereoscopic display. In contrast to a binocular system, a bi-ocular display presents a virtual image equivalent to that of a cinema screen at some optical distance. A bias may still exist between the cues for accommodation and vergence, but this bias will remain constant and, within limits, is unlikely to cause problems to the visual system (Rushton et al., 1994). Although this may seem to provide indirect evidence that a conflict between accommodation and vergence produces visual stress, too many engineering differences exist between the systems to allow such a conclusion to be drawn.
The purpose of this study was to discover whether conflicting vergence and accommodative stimuli are sufficient to cause visual stress and, if so, to determine the conditions and thresholds of conflict that may cause problems.
In order to avoid the complicating factors involved in HMD design, field-sequential shutter glasses were used to address the research question. This technology is an increasingly popular method for presenting stereoscopic images in commercial and research use. The 3D-MAX shutter goggles were manufactured by Kasan Electronics Co., Ltd. (Tokyo, Japan) and were run on a Research System Pentium. All of the displays were presented on a color monitor of 1024 x 768 pixels with a 120-Hz display refresh. The shutter goggles were worn in all conditions to ensure that they were not a confounding factor. All experimental conditions were of 10 min duration. This period was selected because it is representative of normal arcade exposure and because it has been found to induce problems (Mon-Williams et al., 1993).
Participants were informed about the nature of the experiment and that they might experience some "visual fatigue" after viewing the display, but that they would be screened for any problems following the experiment. Their visual status was established using the battery of tests detailed later. They were then asked to enter the VR environment for 10 min before being reassessed on the same ophthalmic battery. They were informed that they could withdraw from participation at any time during the experiment.
Four conditions were explored. In Condition 1, participants were shown a bi-ocular display. This condition effectively worked as a control for the other three conditions. The biocular display consisted of a set of anatomy photographs taken of the meninges and blood vessels of the central nervous system. Participants were required to identify a series of anatomical features within the slide by clicking on appropriate features using a mouse. Following correct identification of the relevant structure, the slides were advanced and the procedure repeated.
Condition 2 used the same slides as the first condition, but the display presented stereoscopic photographs of the sections with disparities that provided the anatomical features with an impressive sense of 3D depth. Although the sense of depth was compelling, the maximum distance that existed between distant and far objects were in the range of only 5 to 10 cm. The demands on the vergence system were therefore relatively small. The order of presentation for the four stimulus conditions was pseudo-randomly distributed.
In Condition 3 a game task (Starfield, Kasan Electronics Co.) was employed that required participants to scan and attend to display detail. Starfield is a "shoot and destroy" game centered on a starship traveling through space. A wider range of disparities was presented within this condition as images of meteors approach from optical infinity ([greater than] 6 m) to pericorporeal space (40 cm).
Condition 4 presented the same disparities as in Condition 3 but involved the constant fixation of a cross that oscillated from infinity to 40 cm in a sinusoidal fashion at a frequency of 0.3 Hz. (It should be emphasized that the software for this final condition was created in house and that it was not supplied by the manufacturer of the shutter glasses.) A control condition was also explored in Condition 4. We controlled for the tracking component of Condition 4 by assessing three participants who tracked a real object in space over the same disparity range while wearing the shutter goggles.
In all conditions, participants sat 40 cm from the screen, producing an accommodative demand of 2.5 diopters (D). All of the disparities within the displays were presented behind the accommodative depth of the screen (i.e., they required a relative divergent effort).
Participants were 28 healthy young adults (15 men and 13 women, mean age 22 years, range 19-25 years) from the Psychology Department at the University of Reading. Criteria for exclusion from the study were a history of previous ocular disorder or any medical condition (e.g., asthma, diabetes, or epilepsy), abnormal binocular vision, or reduced ([less than] 6/12 or 20/40) vision/visual acuity. Only one person was excluded prior to the study, and this was because of the presence of strabismic ambylopia (a common developmental ophthalmic problem).
All participants readily understood the tasks and had full binocular vision, good stereopsis, and normal amplitudes of accommodation and convergence. The degree of heterophoria in the participants corresponded to the norm for the general population (Pickwell, Kaye, & Jenkins, 1991). Vision/visual acuity of all participants was 6/12 (20/40) or better, and all had good (NS) near acuity. Participants were randomly allocated to one of four groups of seven. Each group participated in only one of the four conditions. There were no differences in visual status among groups, and each group contained esophoric and exophoric participants (see the next section).
An initial change in vergence angle or accommodative state is initiated by a phasic controller within the vergence or accommodation system, respectively. A tonic controller in the vergence and accommodation system then adapts to reduce the demands being placed on the phasic response component (see Schor, 1980, for a full description). The constant resting point (or bias) of the vergence tonic controller is known as heterophoria and may be measured by opening the normal feedback loop to vergence. The most common clinical method for opening the vergence loop is to place a Maddox rod in front of one eye. This is a high-powered cylindrical lens that prevents fusion between the eyes and creates a thin red line from a point source of white light. If the visual axes are convergent, the bias is described as esophoria, and if divergent, exophoria.
A steady-state error known as fixation disparity is often present when maintaining vergence in a closed-loop environment (Schor, 1980). The amount of prism that neutralizes an individual's fixation disparity is described as associated heterophoria. Associated heterophoria is commonly measured using the Mallett unit in a test involving a central fixation target visible to both eyes and two monocular markers. The test relies on subjective reports regarding the alignment of the monocular markers. In a clinical setting the fixation disparity measures are generally recorded in prism diopters (1 prism diopter, [Delta] = the angle with a tangent of 0.01).
A battery of ophthalmic tests was undertaken to assess the binocular status of the participants. All participants understood and readily completed the tests involved in the assessment. The tester (the first author) performed a pretest on each participant and then left the laboratory until recalled by the second author. This ensured that the tester was blind to which condition (bi-ocular or binocular) had been completed. Full details of the tests and the rationale behind their selection are given elsewhere (Mon-Williams et al., 1993). The order of the tests within the battery was randomized across participants and between premeasures and postmeasures. The battery consisted of the following five tests.
History and symptoms. A general medical and ocular history was recorded, as were any symptomatic complaints. Participants were asked to score the following symptoms on a 10-point scale: headache, diplopia, blurred vision, sore eyes, and eyestrain.
Vision/visual acuity. The Glasgow Acuity Cards (McGraw & Winn, 1993) were used to record the minimum angle of resolution (MAR) obtained with binocular vision. These data are recorded on a log scale so that the unit of measure is the logMAR.
Heterophoria. Once the presence of binocular vision was established, an assessment of heterophoria was carried out using a Maddox rod and a variable-powered prism (Risley) at 6 m.
Associated heterophoria. The presence or absence of associated heterophoria was established using Mallett units for distance (6 m) and near (40 cm) viewing.
Near heterophoria. In order to determine near (40 cm) heterophoria, participants viewed a white LED placed in front of the right eye, and a red Maddox rod was placed in front of the left eye. Mounted in the same plane as the target was a tangent scale with thin vertical black lines separated by 0.5 cm, below numbers that subtended 5 arcmin. A small white LED was positioned in the center of the tangent screen, and this was placed in line with the right eye. A thin, high-contrast vertical line (i.e., a good accommodative stimulus) acted as a fixation target on either side of the LED. Participants were asked to maintain focus on this line. The participant focused on the central line for 15 s and then the left eye was uncovered for approximately 0.25 s. When the left eye was uncovered, the participant observed a thin red vertical line (created by the Maddox rod) on the tangent scale. The participant reported the position of the red line by stating which number the line passed through.
Conditions 1 through 3. Preplanned contrasts using Dunn's procedure (Keppel, 1982, p. 146) were carried out on the means of interest. No statistically significant changes were found after immersion for 10 min in Conditions 1 through 3. However, statistical tests may mask changes in binocular vision that are important on an individual basis. Of greater interest, therefore, is whether any of the participants showed clinically significant changes in their visual status. The data were inspected to determine whether participants showed individual changes that would be considered significant in a clinical setting. Significant changes were selected on a clinical basis as a 4 [Delta] shift in heterophoria; a decrease in vision/visual acuity of 0.1 logMAR; a 0.5 [Delta] shift in distance-associated heterophoria; a 1 [Delta] shift in near-associated heterophoria.
On the basis of these criteria, no participant showed a clinically significant change in visual status after Condition 1. In Condition 2, two participants showed an esophoric shift in the distance-associated heterophoria. In Condition 3, one participant showed a significant increase in exophoria. No significant changes occurred in any other aspect of visual function for these three participants. None of the three participants with clinically significant changes reported any adverse visual symptoms.
Condition 4. When the cross was moved at 0.3 Hz, two from the group of seven participants had problems fusing the cross. One estimated that he had a single target for only 25% of the immersion period, whereas the other maintained fusion for approximately half the exposure time. All the other participants in Condition 4 managed to obtain and maintain fusion of the stimulus target.
Table 1 provides a summary of the changes found in Condition 4. Preplanned contrasts using Dunn's procedure were carried out on the measures of interest. These tests revealed significant changes in all of the measures [TABULAR DATA FOR TABLE 1 OMITTED] except near heterophoria. A significant increase occurred in symptomatic complaints (p [less than] .007); participants reported 38.5% of maximum discomfort in any one category. Vision/visual acuity decreased by 0.04 logMAR units (p [less than] .01), distance heterophoria changed by 3.7 [Delta] (p [less than] .01) in the exophoric direction, distance-associated heterophoria shifted by 1.2 [Delta] (p [less than] .03) in the esophoric direction, and near-associated heterophoria changed by 0.6 [Delta] (p [less than] .03) in the esophoric direction. Shifts in heterophoria and associated heterophoria were consistent regardless of whether the participant was esophoric or exophoric prior to immersion.
These data were also inspected to determine whether participants showed individual changes that would be considered significant in a clinical setting. On this basis, six participants showed clinically significant changes. The participant who did not show a clinically significant change was the same one who maintained fusion for only 25% of the immersion time.
Three participants showed a clinically significant exophoric shift in distance measurements of heterophoria. In contrast, these three participants demonstrated a clinically significant esopboric shift in both distance- and near-associated heterophoria measurements. Two other participants showed a clinically significant esophoric shift in distance-associated heterophoria, although no significant change in distance heterophoria was measured. One of these participants also showed a significant esophoric shift in associated heterophoria at near distances. None of the participants underwent a significant change in vision/visual acuity or near heterophoria. No deficits of binocular vision were found in the three participants exposed to the control tracking condition.
These results demonstrate that stereoscopic information produced in a VR display will not necessarily cause problems for the visual system. Over relatively short exposures (e.g., 10 min) the visual system seems robust to normal displays that incorporate stereoscopic disparity. However, it has been demonstrated that presenting a continual conflict between accommodation and vergence, and thus stimulating the visual system to continually adapt, does seem to lead to difficulties over this period. Hence when a user is continually switching his or her fixation point in depth, a conflict between accommodation and vergence may cause deficits of binocular vision.
It is notable that only the problematic display caused the report of adverse visual symptoms accompanied by measurable changes in ophthalmic status. This suggests that symptomatic complaint may be a reasonable method of quickly evaluating the performance of an individual display. It is obviously desirable, however, for any study to confirm and extend symptomatic reports with measures of ophthalmic and physiological change. Associated heterophoria is commonly regarded as one of the most efficacious signs of whether the binocular system is being placed under stress and has a high correlation with symptomatic complaint (Pickwell et al, 1991). The results reported here and in previous work (Mon-Williams et al., 1993; Rushton et al., 1994) confirm that associated heterophoria is a reliable sign that has a good agreement with adverse symptoms. We would therefore recommend this measure to those concerned with human factors research into display technology.
Significance of Change
The production of associated heterophoria, together with subjective reports of ocular discomfort, suggests that the visual system is being placed under some stress. Such stress may produce short-term problems of eyestrain and headaches. Stress may also produce short-term changes in visual function that may be problematic to some users if they attempt visually demanding tasks (e.g., driving) following immersion within a display. Given the relatively "hard-wired" nature of the mature adult visual system, the possibility of causing long-term problems through repeated immersion for short periods appears unlikely (though no studies have directly addressed this issue). That binocular headsets have been in use in amusement arcades for a long time, without reports of long-term problems, appears to provide some indirect evidence that longer-term changes do not occur. Questions still remain, however, regarding long-term immersion, the effects of immersion for young users (with immature visual systems), and the effects of immersion for adults with binocular dysfunction.
It is worth considering the response of the visual system to the stress it is undergoing. In pilot trials two participants were exposed to a condition in which the cross oscillated at 1 Hz. Neither was able to fuse the target, nor did either show any changes in visual status following immersion. It therefore appears that for some stimuli, the system may not be able to adapt.
In Condition 4, however, five participants were able to fuse the slower (0.3 Hz) moving cross, and this resulted in an adaptation response. Where adaptation does occur, it is unclear whether it is the conflict itself or the process of adaptation that causes visual stress. If it were the conflict that caused the problems, then the individuals may have required only a longer period in which to fully adapt, or, alternatively, they may have reached the limits of their adaptive capabilities. In either case, increasing the adaptive responses of the visual system should decrease the problems that an individual experiences on entering a virtual environment. However, the more readily the system adapts upon entering the virtual environment, the longer it may take to adapt back to the natural world following exposure unless the system is capable of dual adaptation. It is impossible to determine these factors without further experimentation.
The successful design of virtual environments requires human factors research. It has been predicted that binocular VR displays may cause problems for users because of the normal relationship between accommodation and vergence. Work reported in this paper has confirmed that this may occur in settings in which a continual pressure is placed on the relationship. Binocular displays do not necessarily produce stress to the visual system, however, and this study has shown that binocular vision is robust to short exposures of displays in which stereoscopic information is present.
Our investigation used relatively short exposures (10 min) in adult participants. We recommend that any binocular system be judged on its own merits, given its planned usage and user population. The measures reported here are robust tools for the assessment of vision and visual stress, and such appraisal is desirable for any display system that may become commercially available.
The problematic software reported in this study is not commercially available. The effects that we report from a short exposure to a continually varying stimulus, however, may occur following longer exposure periods with displays that present large disparity ranges (Condition 3). Furthermore, we have observed stimuli comparable to Condition 4 occurring through unintended design flaws in a commercially available system. In this case a popular game presented field-sequential disparities in a stereo display. This was not a bundled system, so users buy the software and a "compatible" display from different suppliers. Unfortunately, because of a minor software-hardware incompatibility, the frame synchronization signal was intermittently lost and the two fields left-right reversed every 3 to 5 s. This was apparent as mere "jitter" in the game image, which a user might attribute to a noisy head-tracker. We discovered, however, that this "jitter" was in fact caused by an intermittent switch between crossed and uncrossed disparities, causing the stereo depth of the image to alternate from behind to in front of the screen, requiring a repeated switch in vergence disparity. These kinds of errors should seldom occur with a bundled system, in which each component is designed to fit within the whole. In a marketplace where users buy "compatible" components off the shelf, however, it is predictable that field-sequential displays may be prone to the errors we have highlighted because of either poor manufacture of a single component or a minor incompatibility between well-manufactured components.
The authors express their gratitude to Puja Kochhar for her help in running the experiments and Simon Rushton for providing technical support. Research reported in this paper was supported by EPSRC grant L16125. The shutter goggles used in this experiment were provided by Kasan Electronics Co., Ltd., but the research was not supported by this company, and the authors do not have any proprietary interests in 3D-MAX or Kasan Electronics Co., Ltd.
Howarth, P. A., & Costello, P. J. (1996). Visual effects of immersion in virtual environments: Interim results from the UK Health and Safety Executive study. In Society for Information Display International Symposium Digest of Technical Papers (Vol. 27, pp. 885-888). Santa Ana, CA: Society for Information Display.
Keppel, G. (1982). Design and analysis: A researcher's handbook. Englewood Cliffs, NJ: Prentice-Hall.
McGraw, P. V., & Winn, B. (1993). Glasgow acuity cards: A new test for the measurement of letter acuity in children. Ophthalmic and Physiological Optics, 13, 400-403.
Mon-Williams, M., Wann, J., & Rushton, S. (1993). Binocular vision in a virtual world: Visual deficits following the wearing of a head-mounted display. Ophthalmic and Physiological Optics, 13, 387-391.
Patterson, R., Moe, L., & Hewitt, T. (1992). Factors that affect depth perception in stereoscopic displays. Human Factors, 34, 655-667.
Pickwell, L. D., Kaye, N. A., & Jenkins, T. C. A. (1991). Distance and near readings of associated heterophoria taken on 500 patients. Ophthalmic and Physiological Optics, 11, 291-296.
Rushton, S., Mon-Williams, M., & Wann, J. (1994). Binocular vision in a bi-ocular world: New generation head-mounted displays avoid causing visual deficit. Displays, 15, 255-260.
Schor, C. (1980). Fixation disparity: A steady state error of disparity-induced vergence. American Journal of Optometry and Physiological Optics, 57, 618-631.
Wann, J. P., & Mon-Williams, M. (1996). What does virtual reality need?: Human factors issues in the design of 3D computer environments. International Journal of Human Computer Studies, 44, 829-847.
Wann, J. P., Rushton, S. K., & Mon-Williams, M. (1995). Natural problems for stereoscopic depth perception in virtual environments. Vision Research, 19, 2731-2736.
Yeh, Y. Y., & Silverstein, L. D. (1990). Limits of fusion and depth judgment in stereoscopic color displays. Human Factors, 32, 45-60.
Mark Mon-Williams is a research fellow at the University of Reading. He received his Ph.D. in psychology from the Glasgow Caledonian University in 1994.
John P. Wann is a reader in developmental psychology at the University of Reading. He received his Ph.D. in psychology in 1988 at Cambridge University.
|Printer friendly Cite/link Email Feedback|
|Author:||Mon-Williams, Mark; Wann, John P.|
|Date:||Mar 1, 1998|
|Previous Article:||Guidelines for presenting quantitative data in HFES publications.|
|Next Article:||Sound localization: information theory analysis.|