Visual accommodation and virtual images: do attentional factors mediate the interacting effects of perceived distance, mental workload, and stimulus presentation modality?
Pilots are required to integrate a considerable amount of information in order to operate effectively. This requires combining their perception of the outside world with information derived from on-board systems. Such information is presented via a variety of displays, which may include head-up, head-down, and helmet-mounted displays. Most information has previously been provided via the visual modality, but there is a trend toward exploring modes of interaction using other sensory channels. In particular, the auditory channel has been investigated as a means of providing information (e.g., warnings) that may improve a pilot's situation awareness - including the use of auditory head-up displays that can present auditory information spatially localized in three dimensions.
Enriching interaction in this way, however, also creates new design problems. In particular, a system designer now needs to consider not only the most appropriate sensory modality in which to convey a particular piece of information but also how this may interact with information presented in other modalities. For instance, it is important to establish whether new methods of information presentation will improve or worsen problems that may already exist with visual displays. An established area of concern with visual displays, particularly head-up displays, is that their use may lead to an inappropriate visual accommodation (focusing) response under some circumstances. This paper will briefly review some of the studies that have investigated accommodation problems with HUDs and describe a study that was conducted to examine whether or not changing the presentation modality of information to be processed would differentially affect the visual accommodation response.
Visual head-up displays (HUDs) have been a feature of many military aircraft cockpits for a number of years. More recently, helmet-mounted displays (HMDs) have been introduced. The basic principle of HUDs and HMDs is similar, and so although the following discussion will refer primarily to HUDs, the issues are also likely to be relevant to HMDs. HUDs and HMDs present symbology and/or imagery, such as forward-looking infrared (FLIR), to the pilot as a virtual image by reflecting images off a combiner glass or the helmet visor placed in the pilot's line of sight. The imagery is usually collimated so that it lies at or near infinity. Thus in principle, the images presented on the HUD should appear to overlay and be in the same plane as the outside world. One theoretical advantage of presenting information in this way is that the pilot can remain head-up and can view and attend to the HUD imagery and the outside world at the same time without having to change the level of accommodation (focus) of the eye.
Unfortunately, the situation appears to be more complicated than the foregoing description implies. A number of studies (Edgar, Pope, & Craig, 1994; Iavecchia, Iavecchia, & Roscoe, 1988; Kotulak & Morse, 1995b; Norman & Ehrlich, 1986; Roscoe, 1979, 1984, 1987a, 1987b, 1987c; Roscoe, Corl, & Couchman, 1994) have suggested that virtual image displays such as HUDs may actually contribute to an anomalous accommodation response, although other investigators (Leitner & Haines, 1981; Newman, 1987; Silverstein & Wilbert, 1987; Weintraub, 1987; Weintraub & Ensing, 1992) have questioned the existence, extent, or impact of the effect.
The impact of different methods of information presentation on the visual accommodation response is complicated by research that indicates that the accommodation response may be influenced by a number of visual and cognitive factors, including image blur (Fincham, 1951), workload (e.g., Edgar et al., 1994; Malmstrom, Randle, Bendix, & Weber, 1980; Winn, Gilmartin, Mortimer, & Edwards, 1991), and perceived target size or proximity (e.g., Heath, 1956; Hull, Gill, & Roscoe, 1982; Kotulak & Morse, 1995a; Rosenfield & Ciuffreda, 1990). Furthermore, there is probably a complex interaction among some or all of these factors. For instance, a study by Winn et al. (1991) found a differential effect on the accommodation response depending on whether a mental processing task was stimulus dependent (i.e., performing the task required information from a visual source) or stimulus independent (i.e., the information to be processed came from a nonvisual source - e.g., auditory presentation). These data have obvious implications for the use of aural information sources inside the cockpit.
Accommodation was measured in a number of conditions in an attempt to examine the effects of and possible interactions between such factors as workload, modality of stimulus presentation, and distance cues provided by an occluding virtual image. Given that other factors, such as stimulus distance, may also have an effect, an attempt was made to make these other factors as realistic as possible in the laboratory situation. Thus the visual stimuli were provided by an optically distant virtual image overlaid on a distant real image.
The experimental setup is illustrated in Figure 1. Accommodation was measured using a laser optometer (Hennessy & Leibowitz, 1972). The laser speckle was presented for 0.5 s at random intervals in each of the conditions described in this section. The level of accommodation was determined using a method-of-limits procedure. Four measurements were made for each participant in each condition, and all results were corrected for the effect of chromatic aberration (Owens & Leibowitz, 1975) and for the plane of stationarity of the laser speckle not lying at the surface of the optometer drum (Charman, 1974). The order of the trials was randomized for each participant. An effort was made to provide a good, naturalistic stimulus to accommodation, which in this case was an outside world scene of a building surrounded by trees. The building was 30 m from the participant, and a light bracket on the wall of the building served as a fixation target. This scene was viewed through an open window (window panes were folded back out of the line of sight), providing a square 340 (of visual angle) aperture 1.4 m from the participant.
Ten participants (five men and five women) were tested. The mean age of the participants was 29 years, with a range between 20 and 39 years. Viewing in all conditions was monocular with one eye lightly occluded. This is not the usual situation in aircraft, but the aim of this study was to investigate the differential effects on the visual accommodation response of different stimulus presentation conditions without the results being influenced by vergence eye movements. All participants were functional emmetropes with an acuity with normal correction of 6/6 or better. No participant had any history of ocular problems.
Mean levels of accommodation were measured under the following conditions:
1. Viewing real world only. Participants were asked to look at the lamp bracket on the wall of the building and to try to keep it in focus.
2. Viewing real world and overlaid image. A virtual image was superimposed on the outside world image and was adjusted to lie at the same optical distance as the wall. The participant's task was to view the images and to try to keep both the outside world and the overlaid image in focus. The image was an array of hashes (see the following for further details) presented on a beam splitter placed immediately in front of one eye of the participant; this beam splitter was present even in conditions in which no overlaid visual image was presented. This configuration gave a good stimulus to accommodation, but the virtual image overlaid the real world, and the strong occlusion cues provided by the hash array would suggest that it was closer, though optically it was at the same distance.
3. Processing information presented visually in the image overlaid on the real world. In this condition the visual stimulus was the same as in Condition 2, except that a series of numbers was presented in the center of the hash array. The participant's task was to subtract seven from a two-digit number and give the response orally. The two digits constituting each number were presented sequentially (to make this condition as close as possible to the aural condition) in a 1.5 s interval. The digits overlaid the fixation object (the lamp bracket). There was then a 1-s interval before the presentation of the next pair of numbers. In addition to the occlusion cues present in the previous condition, this condition imposed a mental processing task on the participant.
4. Processing information presented aurally while viewing the real world and overlaid image. Everything in this condition was the same as in Condition 3, except that the numbers to be processed were presented aurally rather than visually. The sound source was located 1 m from the participant at right angles to his or her line of sight. The digit in the visual image was replaced by another hash symbol. Thus in this condition there were occlusion cues and a mental processing task, but they were essentially independent.
5. Processing information presented aurally while viewing the real world. This condition was the same as Condition 4, except that the participants were viewing only the real world. This condition is also identical to Condition 1 but with the addition of aural information to be processed. Thus in this condition there were no occlusion cues, but there was a mental processing task.
The components of each condition are summarized in Table 1. The overlaid image consisted of an array of hash symbols generated by a laboratory computer on a monitor with white phosphor (type unknown), in the center of which the digits to be processed (or the laser speckle) could be presented. The format of the hash symbols is illustrated in Figure 2. The pattern of hashes was square and subtended 23.4 [degrees] of visual angle. Each symbol, including the digits, subtended 0.95 [degrees] of visual angle with a limb size of 0.1 [degrees] (Snellen equivalent, 6/36). The luminance of the symbols against a dark background, measured using an LMT L1009 digital luminance meter, was 11.5 cd/[m.sup.2]. The background scene was clearly visible through the hash array.
The efficacy of the hash symbols as a stimulus to accommodation was assessed in a pilot study using six participants. Accommodation level viewing the outside world (as described previously) was 0.21 D (SD 0.13), and the accommodation level viewing the hashes in a darkened room was 0.37 D (SD 0.35). These data suggest that the bashes provide a reasonable stimulus to accommodation, but unfortunately both sets of results may be influenced by cognitive factors to a greater or lesser extent. This is particularly true when viewing the hashes in darkness. Participants had no external reference for the position of the hashes and were not informed of the optical distance of the hashes from themselves. Informal questioning revealed that most participants perceived the hashes as lying "within" the laboratory, which may influence the results. Although a previous study (Edgar et al., 1994) found that changes in accommodation in different conditions tended to be consistent, there was considerable individual variation in the absolute level of accommodation. For these reasons the discussion in this paper will concentrate on changes in accommodation rather than on absolute levels.
TABLE 1 Components of Each Condition Used in This Study Condition 1 2 3 4 5 Viewing a real-world scene * * * * * (blur cues) Viewing an overlaid virtual image * * * (occlusion cues) Processing information presented visually (mental effort) * Processing information presented aurally (mental effort) * *
An attempt was made to obtain measurements of tonic accommodation or dark focus (i.e., the resting level of accommodation in complete darkness) for each of the participants. Unfortunately, the dark focus position for 4 of the 10 participants was outside of the measurable range of the laser optometer, and thus no analysis of the dark focus data is included in this paper. Given that there appears to be a relationship between the effect of mental effort and dark focus on the overall accommodation response (e.g., Jaschinski-Kruza & Toenies, 1988), this is clearly an area worthy of further study.
Using a real-world image made it difficult to control the luminance level of the background image. The luminance level may have some effect on the accommodation response, because apart from anything else, a change in luminance will lead to a change in pupil size and a consequent change in the depth of focus of the eye (Charman & Whitefoot, 1977; Hennessy, Iiada, Shina, & Leibowitz, 1976). Reductions in luminance have been shown to affect the accommodation response function (Alpern & David, 1958; Campbell, 1954), but the effects become pronounced only below about 5 cd/[m.sup.2] (Johnson, 1976). The luminance level of the outside world scene was kept within the range of 15 to 150 cd/[m.sup.2] by the addition of neutral density filters to the beam splitter used to present the overlaid image. The luminance of the wall forming part of the realworld view was measured at the beginning and end of each run using an LMT L1009 digital luminance meter. The average luminance (measured through the combiner) was 47 cd/[m.sup.2] (range 15-141 cd/[m.sup.2]). The average change in luminance from the beginning to the end of a session was 29 cd/[m.sup.2] (SD 22). For 6 of the 10 participants, luminance at the end of the run was greater than at the beginning, and for the remaining 4 participants it was the opposite. Thus, although luminance level may have affected the absolute level of accommodation, it is extremely unlikely that consistent changes in accommodation in a particular direction were caused by changes in luminance, given that luminance increased over the course of the experiment for some participants and decreased for others.
The mean accommodation levels of the 10 participants in each condition are shown in Figure 3. The data were analyzed using a one-way analysis of variance with repeated measures. The effect of viewing condition was significant, F = 14.325, p [less than] .001. A priori pairwise comparisons were made using Dunn's test. The results of these comparisons are shown in Table 2.
The highly significant effect of viewing condition on the overall level of accommodation suggests that the accommodation response was affected by more than simply the response to image blur. If blur was the only factor determining the level of accommodation, then none of the conditions reported in this paper should differentially affect the accommodation response. The finding that there were shifts in accommodation in the different conditions suggests that other factors, particularly cognitive, may have been mediating the blur-driven response and thereby influencing the overall level of accommodation. For instance, adding an occluding stimulus or increasing workload in a particular condition led to consistent changes in the level of accommodation. These changes were often small in magnitude but appear to be cumulative; adding an occluding [TABULAR DATA FOR TABLE 2 OMITTED] stimulus and increasing workload leads to a bigger change in the accommodation level than introducing either independently.
An interesting comparison is found between Conditions 3 and 4, which differ only in the method of information presentation (visual or aural presentation of information). When viewing an array of hashes superimposed on the outside world, if information to be processed was presented visually, then there was a significant shift in accommodation. If the information to be processed was presented aurally, then there was no significant shift in the level of accommodation. This effect is small but consistent; there is no significant difference between the level of accommodation in the two processing conditions (3 and 4). Eight of the 10 participants showed a greater change in the level of accommodation when processing information presented visually as opposed to aurally. The remaining two participants showed only very small differences (less than 0.05 D) between the two processing conditions.
It is interesting to note that there is a greater effect on the accommodation response when the information to be processed is presented visually. This is not what one might intuitively expect, as an inappropriate change in accommodation in the visual condition might lead to increased blur and thus make the image containing the information to be processed more difficult to read. One participant spontaneously reported that this was the case, though for others the change may have been within the individual's depth of focus, and so it may not have been noticeable. In the aural presentation, there is no need to extract information from the visual stimulus, so it wouldn't matter if the stimulus was blurred. Thus the result is somewhat counterintuitive.
It is possible that despite attempts to match the workload in Conditions 3 and 4, workload may have been higher in one condition, perhaps as a consequence of the effort required to recode (e.g., the visual information) and may have caused the different effects on the accommodation response. Another possible - and perhaps more plausible - explanation is that the interaction between different influences on the accommodation response may be mediated by attentional factors. Studies of aircraft HUDs have suggested that attention may be drawn to the overlaid virtual image (Brickner, 1989; Fischer, Haines, & Price, 1980; Foyle, McCann, & Sanford, 1993; Hart & Brickner, 1987; Wickens & Long, 1995), a phenomenon usually referred to as attentional or cognitive tunneling.
Whether or not participants were attending to the overlaid virtual image might have had an impact on the level of accommodation. The virtual image overlaid and occluded parts of the outside world, and this may have acted as a depth cue, giving the appearance that the virtual image was closer, even though optically it was at the same distance as the outside world. Indeed, studies that have assessed the perceived distance of overlaid imagery (Kotulak & Morse, 1995b; Schor & Task, 1996) have found that all participants perceived the overlaid image as closer than the background scene. Kotulak and Morse (1995b) looked at oculomotor responses with the Apache HMD and found that the change in the level of accommodation was greater if participants attended to the overlaid symbology as opposed to the background scene. It has previously been suggested (Malmstrom et al., 1980) that attentional factors may influence the overall accommodation response, though these authors suggested that attentional factors could not provide a complete explanation for their data. However, attentional factors may provide a more reasonable explanation if it is assumed that they serve to mediate the interaction of other influences on accommodation.
Thus in the present study the smaller effect of the aural processing task (Condition 4) compared with the visual one (Condition 3) may have been attributable to the fact that although the participants were still looking toward the occluding stimulus, they were attending primarily to the aural processing task and so were less influenced by the implied depth cues provided by the overlying virtual image. The complexity of the interacting effects on the overall accommodation response can be illustrated by considering two other studies. Francis, Thompson, and Owens (1995) reported that responding to an auditory stimulus had a greater effect on the accommodation response to a distant target than did responding to a visual stimulus. This is the opposite effect to that found in this study, but there were no occlusion cues in the Francis et al. study, and the accommodation level in Condition 5 of this study illustrated that processing of aural information may have some effect on accommodation response. Schor & Task (1996) looked at accommodation responses linked to explicit attentional shifts in a simulated night vision goggle (NVG) system with overlaid imagery. These investigators found that although participants were asked to shift attention between the background scene and the overlaid imagery, there were only very small changes in the accommodation response (less than 0.1 D on average), which are considerably smaller than those found in this study.
A number of differences between the two studies may explain this difference. For instance, the processing task in the present study required sustained attention to the overlaid image, whereas the Schor and Task (1996) study required only transient shifts of attention and recognition rather than processing of data on that image, and the latency of the accommodation response tends to be longer than that for eye movements (Campbell & Westheimer, 1960). Also, in the Schor and Task study, participants were required to actively monitor information in the background image, which was not the case in the present study. Finally, Schor and Task used binocular viewing of the background, which would provide vergence cues to accommodation, whereas the present study used only monocular viewing.
Summary and Conclusions
Increasingly diverse ways of presenting information to the pilot in the aircraft cockpit are being considered. These include visual and auditory head-up displays, which can present the pilot with a visual image overlaying the outside world, combined with three-dimensional sound. The results of this study suggest that the method of information presentation may influence the visual accommodation response and that this response may be mediated by attentional factors. Furthermore, different components of a complex task, such as using a HUD (e.g., presence of an occluding virtual image or presence of a mental processing task), appear to have a cumulative effect on the visual accommodation response. This suggests that attempting to study any isolated component of a task may lead to equivocal results, because even though any single factor may not lead to a significant change in accommodation, when a number of factors are combined, the effects can be greater. Furthermore, the differential effects of presenting information either visually or aurally has obvious implications when considering how to present different kinds of information in the cockpit. It should be emphasized that all viewing in this study was monocular, and the overt effects may be less pronounced with binocular viewing (see, e.g., Leibowitz, Gish, & Sheehy, 1988). Even if the influences discussed do not lead to overt changes in accommodation when using HUDs and HMDs, there is the interesting possibility that the visual accommodation response may act as a sensitive indicator of changes in cognitive or attentional factors and might, therefore, be useful as a tool for investigating these factors.
The authors are grateful to the late Roger Green (Centre for Human Sciences, DERA, Farnborough, UK) for his helpful comments on this work. A preliminary report of these data was presented at the Eighth International Symposium on Aviation Psychology, Columbus, Ohio, 1995.
Alpern, M., & David, H. (1958). Effects of illuminance quantity on accommodation of the eyes. Industrial Medicine & Surgery, 27, 551-555.
Brickner, M. S. (1989). Apparent limitations of head-up displays and thermal imaging systems. In Proceedings of the Fifth International Symposium on Aviation Psychology (pp. 703-707). Columbus, OH: Ohio State University.
Campbell, F. W. (1954). The minimum quantity of light required to elicit the accommodation reflex in man. Journal of Physiology (London), 123, 357-366.
Campbell, F. W., & Westheimer, G. (1960). Dynamics of accommodation responses of the human eye. Journal of Physiology, 151, 285-295.
Charman, W. N. (1974). On the position of the plane of stationarity in laser refraction. American Journal of Optometry & Physiological Optics, 51, 832-837.
Charman, W. N., & Whitefoot, H. (1977). Pupil diameter and the depth-of-field of the human eye as measured by laser speckle. Optica Acta, 24, 1211-1216.
Edgar, G. K., Pope, J. C. D., & Craig, I. R. (1994). Visual accommodation problems with head-up and helmet-mounted displays? Displays, 65(2), 68-75.
Fincham, E. F. (1951). The accommodation reflex and its stimulus. British Journal of Ophthalmology, 35, 381-393.
Fischer, E., Haines, R. F., & Price, T. A. (1980). Cognitive issues in head-up displays (Tech. Paper 1711). Moffett Field, CA: National Aeronautics and Space Administration.
Foyle, D. C., McCann, R. S., & Sanford, B. D. (1993). Attentional effects with superimposed symbology: Implications for head-up displays (HUD). In Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting (pp. 13401344). Santa Monica, CA: Human Factors and Ergonomics Society.
Francis, E. L., Thompson, T. C., & Owens, D. A. (1995). Ocular accommodation varies with visual vs. auditory attention. Investigative Ophthalmology and Visual Science, 36(4), S458.
Hart, S. G., & Brickner, M. S. (1987). Helmet-mounted pilot night vision systems: Human factors issues. In Proceedings of the Spatial Displays and Spatial Instruments Conference (pp. 13-1-13-21). Moffett Field, CA: National Aeronautics and Space Administration.
Heath, G. G. (1956). Components of accommodation. American Journal of Optometry and Archives of American Academy of Optometry, 33(11), 569-579.
Hennessy, R. T., Iiada, T., Shina, K., & Leibowitz, H. W. (1976). The effect of pupil size on accommodation. Vision Research, 16, 587-589.
Hennessy, R. T., & Leibowitz, H. W. (1972). Laser optometer incorporating the Badal principle. Behavioural Research Methods & Instrumentation, 4, 237-239.
Hull, J. C., Gill, R. T., & Roscoe, S. N. (1982). Locus of the stimulus to visual accommodation: Where in the world or where in the eye? Human Factors, 24, 311-319.
Iavecchia, J. H., Iavecchia, H. P., & Roscoe, S. R. (1988). Eye accommodation to head-up virtual images. Human Factors, 30, 698-702.
Jaschinski-Kruza, W., & Toenies, U. (1988). Effect of a mental arithmetic task on dark focus of accommodation. Ophthalmic and Physiological Optics, 8, 432-437.
Johnson, C. A. (1976). Effects of luminance and stimulus distance on accommodation and visual resolution. Journal of the Optical Society of America, 66(2), 138-142.
Kotulak, J. C., & Morse, S. E. (1995a). The effect of perceived distance on accommodation under binocular steady-state conditions. Vision Research, 35, 791-795.
Kotulak, J. C., & Morse, S. E. (1995b). Oculomotor responses with aviator helmet-mounted displays and their relation to in-flight symptoms. Human Factors, 37, 699-710.
Leibowitz, H. W., Gish, K. W., & Sheehy, J. B. (1988). Role of vergence accommodation in correcting for night myopia. American Journal of Optometry and Physiological Optics, 65, 383-386.
Leitner, E. F., & Haines, R. F. (1981). Magnitude of visual accommodation to a head-up display (Tech. Paper 1796). Moffett Field, CA: National Aeronautics and Space Administration.
Malmstrom, F. V., Randle, R. J., Bendix, J. S., & Weber, R. J. (1980). The visual accommodation response during concurrent mental activity. Perception and Psychophysics, 28, 440448.
Newman, R. L. (1987). Responses to Roscoe, "The trouble with HOODs and HMDs." Human Factors Society Bullletin, 30(10), 3-5.
Norman, J., & Ehrlich, S. (1986). Visual accommodation and virtual image displays: Target detection and recognition. Human Factors, 28, 135-131.
Owens, D. A., & Leibowitz, H. W. (1975). The fixation point as a stimulus for accommodation. Vision Research, 15, 11611163.
Roscoe, S. N. (1979). When day is done and shadows fall, we miss the airport most of all. Human Factors, 21, 721-731.
Roscoe, S. N. (1984). Judgments of size and distance with imaging displays. Human Factors, 26, 617-629.
Roscoe, S. N. (1987a). The eyes prefer real images. In Proceedings of the Spatial Displays and Spatial Instruments Conference (pp. 21-1-21-9). Moffett Field, CA: National Aeronautics and Space Administration.
Roscoe, S. N. (1987b). The trouble with HUDs and HMDs. Human Factors Society Bulletin, 30(7), 1-3.
Roscoe, S. N. (1987c). The trouble with virtual images revisited. Human Factors Society Bulletin, 30(11), 3-5.
Roscoe, S. N., Corl, L., & Couchman, D. H. (1994). Eye accommodation: Use of vernier optometers in behavioral research. International Journal of Aviation Psychology, 4, 359371.
Rosenfield, M., & Ciuffreda, K. J. (1990). Proximal and cognitively induced accommodation. Ophthalmic and Physiological Optics, 10, 252-256.
Schor, C. M., & Task, H. L. (1996). Effects of overlay symbology in night vision goggles on accommodation and attention shift. Aviation, Space, and Environmental Medicine, 67, 1039-1047.
Silverstein, L., & Wilbert, J. (1987). Responses to Roscoe, "The trouble with HUDs and HMDs." Human Factors Society Bulletin, 30(10), 5-6.
Weintraub, D. J. (1987). HUDs, HMDs and common sense: Polishing virtual images. Human Factors Society Bulletin, 30(10), 1-3.
Weintraub, D. J., & Ensing, M. (1992). Human factors issues in head-up display design: The book of HUD. Wright-Patterson Air Force Base, OH: CSERIAC.
Wickens, C. D., & Long, J. (1995). Object vs. space-based models of visual attention: Implications for the use of head-up displays. Journal of Experimental Psychology: Applied, 1, 179-183.
Winn, B., Gilmartin, B., Mortimer, L. C., & Edwards, N. R. (1991). The effect of mental effort on open- and closed-loop accommodation. Ophthalmic and Physiological Optics, 11, 335-339.
Graham K. Edgar received a Ph.D. in visual psychophysics from the University of Keele in 1989. He is currently employed in the Human Factors Department of British Aerospace, where he specializes in the sensory and cognitive issues associated with advanced displays.
Christopher A. Reeves received an M.Sc. in ergonomics from University College London in 1993. He is currently employed as an Evaluation Officer at the Royal National Institute for the Blind.
|Printer friendly Cite/link Email Feedback|
|Author:||Edgar, Graham K.; Reeves, Christopher A.|
|Date:||Sep 1, 1997|
|Previous Article:||The effects of spatial frequency on binocular fusion: from elementary to complex images.|
|Next Article:||Multiaccommodative stimuli in VR systems: problems & solutions.|