AMES, SHELLY L. PhD, BSc (Optom); WOLFFSOHN, JAMES S. PhD, MCOptom, FAAO; MCBRIEN, NEVILLE A. PhD, MCOptom, FAAO
As technological advances lead to cost reductions, virtual reality devices, including virtual reality head-mounted displays (HMDs), are becoming more accessible to the general public. However, side effects associated with the use of virtual reality devices such as adverse symptoms are well documented.1–9 Investigation of the symptoms that occur after viewing with virtual reality HMDs could lead to improved designs and enable the link, if one exists, between oculomotor changes and symptoms to be determined. To date, however, a standard questionnaire has not been designed specifically for examining the symptoms that occur after the use of virtual reality HMDs. Some of the limitations with the questionnaires used in previous investigations include a lack of questions related to ocular discomfort, the use of questions not relevant to viewing in virtual reality environments, or the use of scales that provide only limited information. Therefore, a questionnaire designed specifically for the investigation of symptoms resulting from virtual reality viewing with a HMD is warranted.
There are many factors that can contribute to symptoms when using an HMD. These include the weight and fitting of the HMD10–12 and the postural demands imposed by the equipment. Cobb et al.11 found that it is better to have an environment that requires a shift in posture than an environment that requires prolonged static postures. Visual comfort can be affected by the angle of gaze, relative to the primary horizontal gaze position, imposed by the position of the screens. Heuer et al.13 found the closer the stimulus, the lower the preferred viewing angle. Discomfort can be induced through the perception of flicker in CRT displays14 and is likely to have similar adverse effects in virtual environments. Factors such as low illumination and low spatial resolution, which are characteristic of most commercially available HMDs, can result in accommodation drifting toward its resting state15, 16 and may contribute to ocular discomfort when viewing with a HMD.
Sensory conflict between the signals received by the three major spatial senses—the visual system, the vestibular system, and nonvestibular proprioception—can also lead to symptoms.17 One of the sources for this conflict arises from the abnormal nature of movement of the images on the retina when viewing in an HMD. Normally when a person moves his or her head when fixating an object that is moving in the same direction as the head, the images of stationary objects in the visual field will move on the retina. When using an HMD, if the head moves, the image moves with the head so stationary objects do not move on the retina as they normally would. This gives rise to conflicting visual–vestibular information and is thus a source for sickness3 and could account for why subjects were observed to minimize their head movements in the study by Cobb et al.11 Head-tracking devices can reduce this conflict by imitating the normal movement of stationary objects on the retina by updating the scenery as the head moves. However, the delays involved in updating the display after movements by the user can be detrimental and have been proposed as one of the major causes of side effects.6, 18 Finally, the conflicting cues to the accommodation and vergence systems in stereoscopic displays, as a result of the fixed image plane distance and changing disparity cues, have been proposed as a cause of visual stress18 and ocular discomfort.19 Because there are many sources of visual discomfort, it is likely that problems will be idiosyncratic to the display being used.
Examples of standard questionnaires that have been used to investigate symptoms after viewing in virtual reality environments include the Pensacola Motion Sickness Questionnaire (MSQ), which was developed by Navy scientists to study the effects of motion sickness,20 the Simulator Sickness Questionnaire (SSQ),20 and the International Standard for Visual Display Units (VDU's) proposed by the International Standards Organization.21 The Pensacola MSQ has been used quite extensively to investigate the symptoms that result from viewing in a virtual environment.22–25 However, there are differences between the factors that induce motion sickness and those that produce sickness/symptoms resulting from simulator exposure or virtual reality viewing. Consequently, some of the symptoms in the MSQ are either never reported or occur at a level below baseline under simulator exposure.26 The SSQ was developed to overcome the deficiencies of the MSQ for examining the effects of exposure to flight simulators;26 it consists of a reduced set of questions adapted from the Pensacola MSQ. The SSQ or adapted versions of the SSQ have also been used in studies investigating the effects of virtual reality viewing.6, 27 Although the SSQ is more suitable than the MSQ for investigating the symptoms that result from viewing in a virtual environment, it does not adequately cover the range of possible ocular symptoms. Although Peli28 used the International Standards for VDUs questionnaire, it does not adequately cover the range of nonocular symptoms that have been reported to occur with viewing virtual imagery using an HMD. The International Standard for VDUs consists of just six symptoms, namely postural discomfort and headache and four others related to the eyes.
The length of time to conduct the questionnaire also needs to be considered as the duration of symptoms after virtual reality viewing is uncertain. Mon-Williams et al.1 and Mon-Williams and Wann29 found symptoms dissipated by 5 min postvirtual reality viewing, whereas Regan and Ramsey24 found certain symptoms lasted up to several hours in a small percentage of subjects. In many other studies, symptoms were only assessed immediately after viewing, and hence no information about the duration of symptoms was reported.3, 6, 10, 30 If symptoms after virtual reality viewing do decline within 5 min, studies investigating them require a questionnaire that can be performed within a 5-min window.
An important issue with respect to administration of a questionnaire for use in investigating symptoms that result from viewing virtual reality is whether to administer the questionnaire before and after viewing or only after viewing. It has been suggested that exposure to questionnaires before viewing may elevate subjective reports of symptoms by causing subjects to dwell on their internal states.22, 28 Peli28 found subjective reports of symptom levels that were substantially lower than those found by Mon-Williams et al.1 and Howarth and Costello.6 A possible explanation is that Peli28 and Mon-Williams et al.1 used different HMDs. Furthermore, although Peli28 and Howarth and Costello6 used the same HMD, Peli's subjects engaged in a less active task. Peli28 suggested the discrepancy in symptom levels between his and the studies by Mon-Williams et al.1 and Howarth and Costello6 was possibly the result of subjects being primed to expect at least some of the symptoms presented in the questionnaire administered before viewing. It is important to assess whether priming subjects elevates symptom levels, because ideally, it is advantageous to present a questionnaire before viewing so that changes from previewing to postviewing can be calculated.
Therefore, the primary aim of this investigation was to design a suitable questionnaire, which we have named the Virtual Reality Symptom Questionnaire (VRSQ), for the investigation of symptoms induced through viewing virtual reality imagery with an HMD. To achieve this, it was necessary to design a pilot questionnaire using information from previous studies, run a trial using the pilot questionnaire, and optimize the pilot questionnaire using the results of the trial.
Further aims were to determine the duration of symptoms postviewing and to ascertain whether symptom reports are elevated by priming subjects (i.e., having some subjects complete the questionnaire before viewing).
Symptom Selection for the Pilot Questionnaire
The first step in choosing the symptoms for the pilot questionnaire was to examine the individual questions that have been included in questionnaires previously used in virtual reality research. To date, a total of 47 different symptom questions, 32 nonocular and 15 ocular, have been examined in questionnaires on virtual reality viewing. If all of these questions were included in a new questionnaire, it would certainly take too long to administer, particularly if the questionnaire is to be administered several times over a short postviewing period. In addition, there is a paucity of information on verification of the applicability of assessing particular symptoms.
The first step in reducing the extensive list of previously investigated symptoms was to examine each symptom for content relevance, that is, its relevance to the area under investigation. To do this, the frequency with which each symptom has been reported to occur, after viewing virtual reality imagery, in studies that have presented subjects with structured symptom questionnaires or symptom checklists was examined. The most frequently reported ocular symptom is eyestrain1, 2, 5, 6, 22, 24, 27, 29 followed by blurred vision,1, 2, 5, 6, 29, 32 double vision,1, 2, 5, 6, 29 sore/aching eyes,1, 2, 5, 6, 29 and tired eyes.1, 2, 5, 6, 29 Other ocular symptoms reported to occur are dry eyes,19 difficulty focusing,19 watery eyes,6 hot/burning eyes,6 irritated eyes,6 and discomfort from eyes.6 The most frequently reported nonocular symptoms are nausea1, 5, 6, 22–24, 31 and headache,1, 5, 6, 22, 24, 31 followed by dizziness5, 6, 22, 24, 31 and disorientation.2, 5, 6, 24 Other reported nonocular symptoms include general discomfort,5, 6, 31 difficulty concentrating,5, 6, 31 drowsiness,5, 6, 24 stomach awareness,5, 6, 24 fatigue,6, 31 boredom,5, 6 sweating,5, 6 claustrophobia,5 and exhilaration.5
After compiling this list of the most relevant symptoms, 12 nonocular (claustrophobia and disorientation combined) and 11 ocular symptoms were identified. Content coverage was examined to ensure that adequate numbers of symptoms were included to fully examine both the nonocular and ocular symptoms and that the number of questions also reflected the importance of each area. Because only three symptoms in this list relate to quality of vision, a further symptom, vision discomfort, was added. This additional symptom question allows subjects to indicate that their vision is not comfortable even if they are unable to classify the discomfort as difficulty focusing, blurred vision, or double vision. The symptom “discomfort from eyes” was removed because it is ambiguous; subjects may be unsure whether it refers to a general feeling of discomfort because the eyes are not comfortable or whether it refers to eyes themselves feeling uncomfortable. The final selection of symptoms for the pilot questionnaire consisted of the most frequently reported symptoms mentioned here, with the exclusion of “discomfort from eyes,” and the inclusion of vision discomfort resulting in 12 nonocular (claustrophobia and disorientation were combined) and 11 ocular symptoms (see Table 1). Other symptoms that have been investigated following virtual reality viewing were excluded on the basis of overlap or that they have not previously been reported to occur.
Once the symptoms were chosen for the pilot questionnaire, it was then necessary to determine the scale for rating them. The simplest option is a dichotomous yes/no scale, as used by Mon-Williams, Wann and Rushton,1 and Mon-Williams and Wann,29 which determines the absence or presence of a symptom. However, symptoms generally lie on a continuum, and using a dichotomous response scale is likely to be less informative than one with more response alternatives. A scale with more response categories may be more efficient at evaluating changes in symptoms and detecting important correlations with fewer subjects.32 To maximize reliability, a seven-category scale, which is considered to discriminate between severity but remain manageable to subjects,33 was chosen for the pilot questionnaire. The scale was unipolar (0–6) because a negative rating for a symptom is difficult to conceptualize. Qualitative descriptors were also used: “none” was placed above the zero, “slight” above 1 and 2, “moderate” above 3 and 4, and “severe” above 5 and 6. An example of the scale is shown in Table 2.
The pilot questionnaire, which consisted of 23 symptoms, 12 nonocular and 11 ocular symptoms, was administered to four adult students before conducting the experimental trial and was found to take less than 2 min to complete. It could therefore be administered at 2-min intervals.
The subjects were final-year optometry and vision science students ranging in age from 21 to 28 years. They were divided into two groups of eight, balanced for age (mean age of group 1 was 22.75 ± 2.31 years and group 2 was 22.38 ± 1.60 years), gender (four females and four males in each group), and refraction (five myopes and three emmetropes in each group; average refraction was -2.91 ± 2.62 D for group 1 and -2.06 ± 1.99 D for group 2). Age and refraction for both groups were normally distributed (as determined by Kolmogorov-Smirnov test).
The research carried out in this investigation followed the tenets of the Declaration of Helsinki. Informed consent was obtained from subjects after they read the information sheet provided, which outlined the nature and possible consequences of the study. The investigation was approved by the Institutional Human Ethics Committee of the Department of Optometry and Vision Sciences, The University of Melbourne.
The Virtual Reality System
The HMD used to display the images was a V6 HMD (Virtual Research, Santa Clara, CA). The V6 HMD has dual 1.3-in diagonal active matrix LCDs with a resolution per eye of 640 by 480 pixels (102,400 triads), which, at a viewing distance of approximately 3 cm, gave a resolution equivalent of 6/30 Snellen. It has a contrast ratio of 200:1, field of view of 60 deg diagonally, and weight of 0.821 kg. The interscreen separation (ISS) is adjustable from 52 to 74 mm and was set at the subject's distant viewing interpupillary distance. The convergence angle of the screens is fixed at 4.3 deg, which is equivalent to a convergence distance of 80 cm for an interpupillary distance of 60 mm. The image plane is at approximately 90 cm.
The viewing material was a 20-min stereoscopic video, entitled “Eye to Eye,” similar to the style of a “current affairs” program. The imagery was filmed using dual cameras mounted 60 mm apart. Therefore, the range of disparities, and thus depth differences, was equivalent to what an individual with an interpupillary distance of 60 mm would normally experience. Individuals with larger interpupillary distances would see slightly less depth and those with interpupillary distances <60 mm would see slightly larger depth differences.
Subjects in group 1 were primed by having them complete the questionnaire before viewing the imagery in the HMD, thus informing them of the symptoms that may occur as a consequence of viewing virtual reality imagery with an HMD.
For all subjects, the questionnaire scale and corresponding descriptors were explained before completing the questionnaire for the first time (this was before viewing for subjects in group 1 and immediately after viewing for subjects in group 2). The questionnaire was administered by having subjects read and answer the questions independently of the examiner.
The subjects who were not primed before viewing (group 2) were asked the general question, “Are you experiencing any symptoms at the present time?” before the start of the experimental run. Although this question can also be considered as “priming,” it does not indicate any particular symptoms to the subjects. In addition, it enabled any subjects who were experiencing symptoms at the start of the trial to be rebooked for a later date so as not to confound the results. The ethical requirement to make all subjects aware of potential undesirable symptoms could also be considered as priming. Therefore, the “nonprimed” subjects were actually primed but to a lesser degree than the “primed” subjects. However, they are referred to as the nonprimed subjects throughout this article.
During viewing, subjects were seated in a stationary (no wheels) swivel chair with a 3-m radius of empty space to the front and sides; there were minimal head or body movements. They wore their habitual refractive corrections (spectacles or contact lenses). All subjects, primed and nonprimed, completed the questionnaire immediately after the 20 min of viewing the stereoscopic imagery and at 2-min intervals for a further 10 min postviewing.
The data obtained from the administration of the questionnaire is strictly categorical data; therefore, nonparametric statistics were used for the analysis of the symptoms. The previewing low-level symptom ratings for group 1 were not subtracted from the postviewing levels to give change values because the same previewing data were not available for group 2. Baseline was taken as zero for all symptoms for both groups for comparison purposes.
To determine whether administering the questionnaire before viewing (i.e., priming) resulted in significantly higher symptom ratings, nonparametric equivalency testing using the method described by Munzel and Hauschke34 was applied to the data obtained immediately after viewing. The equivalence tests were performed on the data obtained through pooling the ratings for all symptoms and on the individual symptom data. The maximum irrelevant difference (δ), where δ ε [0, 1/2), was set at 0.2, which is considered a reasonable value according to Wellek and Hampel.35
To determine the duration of symptoms, the data for the primed and nonprimed subjects were pooled, and a one-sample sign test was used to test the data for significant increases above baseline; each symptom was tested at each postviewing time point.
The data for the primed and nonprimed subjects were also pooled to assess the frequency of endorsement for each symptom. The frequency of endorsement refers to the proportion of subjects that responded to each alternative of an item. For a yes/no scale, this is simply the proportion of people who respond yes and the proportion who respond no. For the purpose of this analysis, all ratings above zero were taken as a yes response. The frequency of endorsement for each symptom was determined by adding the number of yes responses at each of the postviewing time points up to and including 10 min (total of six time points) and dividing by the total number of responses. For each symptom, there were six responses from each subject (one at each time point), so for the 16 subjects in the combined groups, the total number of responses for each symptom was 96 (6 by 16). For each symptom, the total number of yes responses was divided by the total number of responses (96), resulting in one frequency of endorsement for each symptom on the questionnaire.
Item-total correlation was used to test the homogeneity, the degree to which the questions relate to one another, of the questionnaire.32 Item-total correlation is the correlation of the individual item with the scale omitting that item. The following formula given by Nunnally36 can be used to determine an item's contribution to the total score:
where ri(t –1) is the correlation of item i with the total score (with the effect of item i removed), rit is the correlation of item i with the total score, ςi is the standard deviation of item I, and ςt is the standard deviation of the total score. The data from both the primed and nonprimed subjects groups were pooled and a Pearson product moment correlation was used to determine rit.
Priming of Subjects
There was no evidence, using the nonparametric equivalency testing method described by Munzel and Hauschke,34 that priming subjects resulted in higher symptom levels immediately after viewing for all symptoms combined (p < 0.001, for correctly accepting equivalence). The total symptom ratings for each group (primed and nonprimed) after viewing are shown in Fig. 1. Because the symptom ratings for the primed and nonprimed subjects were found to be equivalent (see previously), the data from the two groups were pooled to assess the duration, frequency of endorsement, and homogeneity of the symptoms.
Duration of Symptoms
For the pooled data (i.e., primed and nonprimed groups combined), 16 of the 23 symptoms were significantly higher than baseline immediately after viewing. Eight symptoms were still significantly higher than baseline at 2 min postviewing, compared with three symptoms at 4 min postviewing and two symptoms at 6 min postviewing. By 8 min postviewing, there were no symptoms that were still significantly above baseline. The significance levels for each symptom at each time point are shown in Table 3.
Frequency of Endorsement
The frequency of endorsement for each symptom was determined from the pooled data (i.e., primed and nonprimed groups combined) and is shown in Table 4. A low frequency of endorsement, <0.2, means that few subjects (<20%) experienced that symptom, which suggests that the five nonocular and five ocular symptoms with frequencies of endorsement of <0.2 could possibly be discarded from the final questionnaire. These 10 symptoms coincided well with the symptoms that did not increase significantly; all seven of the symptoms that did not increase significantly had frequencies of endorsement of <0.2.
Homogeneity of the Questions
The item-total correlations, which are used to assess the homogeneity of each symptom, were determined for all of the symptoms in the pilot questionnaire using the combined data from both the primed and nonprimed groups. The general rule is that items should have an item-total correlation (ri(t –1)) of at least 0.2 and any items with correlations below 0.2 should be discarded.37 The majority of symptoms, 15 in total, met the requirement of having an item-total correlation of 0.2 or higher for at least the first three time points monitored after viewing.
The four ocular symptoms that did not meet the minimum item-total correlation requirement of 0.2 for at least the first three postviewing time points monitored correspond well with the ocular symptoms that had frequencies of endorsement of <0.2. Irritated eyes was the only exception, because it did not meet the frequency of endorsement minimum requirement but did meet the minimum requirement for homogeneity.
The three nonocular symptoms that had item-total correlation values below 0.2 for at least the first three postviewing time points also had low frequencies of endorsement. Two symptoms, nausea and stomach awareness, had low frequencies of endorsement but satisfactory item-total correlation values.
After discarding most of the symptoms indicated to be less useful from the frequency of endorsement data (nausea was retained and vision discomfort was discarded; see “Discussion”), the item-total correlations were calculated again. The remaining 13 symptoms met the minimum value of 0.2 for the item-total correlation for at least the first three postviewing time points (see Table 5) except dizziness, which had an item-total correlation of 0.16 immediately after viewing but satisfactory homogeneity at subsequent time points.
Frequency of endorsement data obtained from the trial of the pilot questionnaire indicated that approximately half of the symptoms could be discarded as <20% of subjects experienced them. The results of the trial also demonstrated that the majority of symptoms, after viewing virtual reality imagery with an HMD, dissipate within 6 min postviewing. Some symptoms such as blurred vision and sore/aching eyes dissipate even more rapidly, within 2 min postviewing. Priming subjects by administration of the questionnaire before viewing was found not to elevate symptom ratings.
The seven nonocular symptoms (general discomfort, fatigue, boredom, drowsiness, headache, dizziness and difficulty concentrating), indicated to be the most useful according to the frequency of endorsement data, correspond well with the most frequently reported nonocular symptoms in previous investigations.1, 2, 5, 6, 22, 24, 27, 29 However, the frequency of endorsement data did not indicate that the most commonly reported nonocular symptom, nausea, is useful. This is most likely a consequence of the imagery used in this trial, which was of a relatively sedate nature. Many previous studies have used more dynamic imagery.10, 24, 38 Therefore, retaining nausea in the final questionnaire is likely to be beneficial for investigations using more dynamic imagery.
The ocular symptoms indicated to be the most useful according to the frequency of endorsement data also correspond well with the most frequently reported ocular symptoms in previous investigations. The exceptions were double vision and vision discomfort. Double vision has been reported after HMD use in several previous studies,1, 2, 6 but in this investigation, double vision had a low frequency of endorsement (0.01). This may be because the ISS of the HMD used in this study was adjusted to each subject's interpupillary distance (IPD), thus reducing prismatic effects. In contrast, Mon-Williams et al.1, 6 used a fixed ISS for all subjects and, like the VPL Eyephone LX HMD used in their study, has an image plane distance of <50 cm (as determined by the authors), significant amounts of prism could have been induced. In the study by Rushton et al.,2 subjects were allowed to adjust the ISS themselves, and the authors discuss how mismatches between ISS and IPD could have induced prism; however, no information is available regarding the distance of the image plane in the HMD used (Visette 2000, Virtuality Entertainment Ltd, LEI). Induced prism does not explain why double vision occurred in the study by Howarth and Costello,1, 6 because the Virtual i-glasses used in that study had an image plane distance close to infinity and thus induced prism resulting from discrepancies between IPD and ISS would have been negligible. One significant difference between the current investigation and each of the three studies that found double vision after viewing with an HMD is that a head-tracking device was not used in this investigation. The delay in updating the display after a head movement in systems using head tracking has been proposed as a major cause of adverse effects18, 6 and thus could account for double vision. Therefore, the addition of double vision to the symptom list may be warranted when testing virtual reality systems that include head tracking. The other symptom mentioned here as an exception, vision discomfort, has not been reported in previous investigations, but in this study, it met the frequency of endorsement criteria for inclusion. However, because 89% of the time it was accompanied by reports of blurred vision and/or difficulty focusing, it can be discarded to force subjects to be more specific about the source of their vision discomfort. This is likely to be more informative in identifying a link between ocular symptoms and oculomotor changes.
The homogeneity of the questionnaire consisting of the remaining 13 symptoms is good, as indicated by item-total correlation values of >0.2.37 Each symptom, except dizziness, had an item-total correlation of at least 0.2 for the first three postviewing time points measured, which corresponds well to the duration of most symptoms. It is important to retain dizziness, because it increased significantly for the first two postviewing time points and it has been reported after the use of virtual reality equipment in numerous studies.5, 6, 22, 24, 31 Only two symptoms were at significant levels beyond the third postviewing time point.
The rapid dissipation of symptoms, found in this study, is in general agreement with previous findings.1, 2, 29 Symptoms may be more severe and of longer duration for less optimal HMD designs and more dynamic imagery, but the questionnaire must be able to determine symptoms in more optimized viewing conditions. This result indicates that studies investigating the effects of viewing virtual reality imagery only have a short timeframe in which to make relevant measurements, and thus studies that have used lengthy test batteries may have failed to capture important information because the effects may have dissipated before being measured. If the relationship between symptoms and oculomotor changes is to be investigated, both the symptom questionnaire and the oculomotor measures must be performed within the first 5 min postviewing.
One way to reduce the time taken to assess symptoms postviewing is to administer the questionnaire before viewing so that subjects are familiar with the questionnaire postviewing. Administering the questionnaire before viewing has the added advantage of enabling the changes in the symptom ratings from pre- to postviewing to be calculated, eliminating the need to make assumptions about the previewing symptom ratings. There was no evidence from the results of this study that administering the questionnaire before viewing elevates symptom ratings. However, this finding may have been undermined by the small sample size and the fact that the nonprimed subjects were still primed to some degree before viewing.
The selection of symptoms for the VRSQ, which is specifically related to virtual reality viewing, as determined by this investigation should consist of the following eight nonocular symptoms: general discomfort, fatigue, boredom, drowsiness, headache, dizziness, difficulty concentrating, and nausea; and five ocular symptoms: tired eyes, sore/aching eyes, eyestrain, blurred vision, and difficulty focusing. The final questionnaire is shown in Fig. 2. The VRSQ has approximately half as many questions as some of the other questionnaires used for the same purpose such as that used by Howarth and Costello6 and the MSQ. It will therefore take less time to complete, approximately 1 min, allowing symptoms to be assessed before they may dissipate. The VRSQ is also likely to be useful for investigating the symptoms that occur with other virtual reality systems such as visual display unit with shutter goggles and IMAX theater. However, for benefit in the assessment of symptoms after the use of displays used in the workplace such as a visual display unit, the VRSQ would require refinement to remove symptoms that are likely to be irrelevant (e.g., nausea) or susceptible to influences other than the display itself (e.g., boredom).
This study has developed and refined a questionnaire specifically for virtual reality viewing. Although statistical approaches were used to optimize the questions, it requires further validation before it can be considered optimal. In particular, it needs to be tested on a larger subject group and possibly in a more diverse age range. Also, it should be validated on several virtual reality systems, including those that involve active head movements.
This investigation was supported by a grant from the Independent Television Commission UK (NMcB).
Neville A. McBrien
Department of Optometry and Vision Sciences
The University of Melbourne
Victoria, 3010 Australia
1.Mon-Williams M, Wann JP, Rushton S. Binocular vision in a virtual world: visual deficits following the wearing of a head-mounted display. Ophthal Physiol Opt 1993;13:387–91.
2.Rushton S, Mon-Williams M, Wann JP. Binocular vision in a bi-ocular world: new-generation head-mounted displays avoid causing visual deficit. Displays 1994;15:255–60.
3.Oohira A, Ochiai M. Influence on visual function by a stereoscopic TV programme with binocular liquid crystal shutter and Hi-Vision TV display. Ergonomics 1996;39:1310–4.
4.Hasebe H, Oyamada H, Ukai K, Toda H, Bando T. Changes in oculomotor functions before and after loading of a 3-D visually-guided task by using a head-mounted display. Ergonomics 1996;39:1330–43.
6.Howarth PA, Costello PJ. The occurrence of virtual simulation sickness symptoms when an HMD was used as a personal viewing system. Displays 1997;18:107–16.
7.GanNichida T, Ervin A, Gemmill MC, Drack A, Holt PH, Lynn M. Ophthalmic changes associated with symptom reports after use of head-mounted display (HMD). Invest Ophthalmol Vis Sci 1999;40:S58.
8.Howarth PA, Finch M. The nauseogenicity of two methods of navigating within a virtual environment. Appl Ergon 1999;30:39–45.
9.GanNichida T, Ervin A, Gemmill MC, Lynn M, Drack A, Holt PH, Stulting R. Are ophthalmic changes after HMD use associated with age? Invest Ophthalmol Vis Sci 2001;42:S393.
10.Hoshino M, Takahashi M, Oyamada K, Ohmi M, Yoshizawa T. Body sway induced by 3D images. In: Fisher SS, Merritt JO, Bolas MT, eds. Stereoscopic Displays and Virtual Reality Systems IV: Proceedings of the SPIE, vol 3012. Bellingham, WA: SPIE Press, 1997:400–7.
11.Cobb SV, Nichols S, Ramsey A, Wilson JR. Virtual Reality induced symptoms and effects (VRISE). Presence Teleop Virt 1999;8:169–86.
12.Nichols S. Physical ergonomics of virtual environment use. Appl Ergon 1999;30:79–90.
13.Heuer H, Bruwer M, Romer T, Kroger H, Knapp H. Preferred vertical gaze direction and observation distance. Ergonomics 1991;34:379–92.
14.Isensee SH, Bennett CA. The perception of flicker and glare on computer CRT displays. Hum Factors 1983;25:177–84.
15.Leibowitz HW, Owens DA. Night myopia and the intermediate dark focus of accommodation. J Opt Soc Am 1975;65:1121–8.
16.Charman WN, Tucker J. Dependence of accommodation response on the spatial frequency spectrum of the observed object. Vision Res 1977;17:129–39.
17.Reason JT. Motion sickness adaptation: a neural mismatch model. J R Soc Med 1978;71:819–29.
18.Yeh YY, Silverstein LD. Limits of fusion and depth judgement in stereoscopic color displays. Hum Factors 1990;32:45–60.
19.Wilson JR. Effects of participating in virtual environments—a review of current knowledge. Safety Sci 1996;23:39–51.
20.Kennedy RS, Jones MB, Lilienthal MG, Harm DL. Profile analysis of after-effects experienced during exposure to several virtual reality environments. In: Proceedings of the 76th Aerospace Medical Panel Meeting on Virtual Interfaces: Research and Applications 18–22 October 1993; Lisbon, Portugal; Lisbon: AGARD, 1993:2.1–2.9.
21.International Organization for Standardization. ISO 9241–3. Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs). Part 3: Visual Display Requirements. Geneva: International Organization for Standardization, 1992.
22.Regan EC, Price KR. The frequency of occurrence and severity of side-effects of immersion virtual reality. Aviat Space Environ Med 1994;65:527–30.
23.Regan E. Some evidence of adaptation to immersion in virtual reality. Displays 1995;16:135–9.
24.Regan EC, Ramsey AD. The efficacy of hyoscine hydrobromide in reducing side-effects induced during immersion in virtual reality. Aviat Space Environ Med 1996;67:222–6.
25.Krebs WK, Kaiser JP, Darken RP, Sinai MJ, McCarley JS. Visual deficits following prolonged exposure in a virtual environment. Invest Ophthalmol Vis Sci 1999;40:S433.
26.Kennedy RS, Lane NE. Simulator Sickness Questionnaire: an enhanced method for quantifying simulator sickness. Int J Aviat Psychol 1993;3:203–30.
27.Kolasinski EM, Gilson RD. Ataxia following exposure to a virtual environment. Aviat Space Environ Med 1999;70:264–9.
28.Peli E. The visual effects of head-mounted display (HMD) are not distinguishable from those of desk-top computer display. Vision Res 1998;38:2053–66.
29.Mon-Williams M, Wann JP. Binocular virtual reality displays: when problems do and don't occur. Hum Factors 1998;40:42–9.
30.Ijsselsteijn W, de Ridder H, Hamberg R, Bouwhuis D, Freeman J. Perceived depth and the feeling of presence in 3DTV. Displays 1998;18:207–14.
31.Kolasinski E. Prediction of simulator sickness in a virtual environment [PhD Thesis]. University of Central Florida; Orlando, Florida; 1996.
32.Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to their Development and Use, 2nd ed. Oxford: Oxford University Press, 1995.
33.Miller GA. The magical number seven plus or minus two: some limits on our capacity for processing information. Psych Rev 1956;63:81–97.
34.Munzel U, Hauschke D. A nonparametric test for proving noninferiority in clinical trials with ordered categorical data. Pharm Stat 2003;2:31–7.
35.Wellek S, Hampel B. A distribution-free two-sample equivalence test allowing for tied observations. Biometrical J 1999;41:171–86.
36.Nunnally JC. Psychometric Theory, 2nd ed. New York: McGraw-Hill, 1978.
37.Kline P. A Handbook of Test Construction: Introduction to Psychometric Design. London: Methuen, 1986.
38.Regan E. Some side effects of immersion virtual reality. Virtual interfaces. In: Proceedings of the 76th Aerospace Medical Panel Meeting on Virtual Interfaces: Research and Applications; October 18–22, 1993; Lisbon, Portugal. Lisbon: AGARD, 1993.