Online Circular Contrast Perimetry: A Comparison to Standard Automated Perimetry

Abstract

Purpose: 

The aim was to validate and compare the diagnostic accuracy of a novel 24-degree, 52-loci online circular contrast perimetry (OCCP) application to standard automated perimetry (SAP).

Design: 

Prospective cohort study.

Methods: 

Two hundred and twenty participants (125 normal controls, 95 open angle glaucoma patients) were included. Agreement, correlation, sensitivity, specificity, and area under receiver operating curves (AUC) were compared for parameters of OCCP, SAP, and optical coherence tomography (OCT) for the retinal nerve fiber layer and macular ganglion cell complex inner plexiform layer.

Results: 

Pointwise sensitivity for OCCP was greater than SAP by 1.02 log units (95% CI: 0.95–1.08); 95% limits of agreement 0.860 to 1.17. Correlation and agreement for global indices and regional zones between OCCP and SAP were strong. OCCP mean deviation (MD) AUC was 0.885±0.08, similar to other instruments’ parameters with the highest AUC: SAP MD (0.851±0.08), OCT retinal nerve fiber layer inferior thickness (0.908±0.07), OCT ganglion cell complex inner plexiform layer inferior thickness (0.849±0.08), P>0.05. At best cutoff, OCCP MD sensitivity/specificity were comparable to SAP MD (90/74 vs 94/65%).

Conclusions: 

OCCP demonstrates similar perimetric sensitivities to SAP and similar AUC to SAP and OCT in distinguishing glaucoma patients from controls. OCCP holds promise as a glaucoma surveillance and screening tool, with the potential to be utilized for in-clinic and at-home perimetry and expand community testing.

Glaucoma is among the leading causes of blindness worldwide, yet remains undetected in a significant proportion of the population.^1,2 Early detection and lifelong surveillance are essential for delaying disease progression and improving patient outcomes.³ Perimetry is used in clinical practice to assess patients with visual field defects, diagnose glaucoma, and monitor disease progression.⁴ Until recently, perimetry could only be operated using precalibrated, dedicated machines, such as those that provide standard automated perimetry (SAP) or frequency doubling perimetry (FDP) and only by trained staff in optometry and specialist ophthalmology practices.^5,6

Most health care systems are confronting an aging population with a growing burden of chronic eye disease.^7,8 Most public ophthalmology providers are already overexpended and patients face barriers to care including long public waitlists and high out-of-pocket costs for private clinics.⁸ Developing low-cost perimetry technologies available on personal computers, tablets, or other devices can offer promising alternatives to the current clinic-based machines that can reduce the number of in-person clinic visits, thereby promoting resource conservation and lower health care costs.^5,9–12 An added advantage of reducing in-clinic visits may also be preventing the spread of coronavirus disease 2019 (COVID-19), which has already become apparent with telehealth and at-home care services.^12–14

Perimetry has been developed using computers,¹⁵ tablets,^11,16,17 smartphones,¹⁸ and virtual reality headsets.^19,20 Perimetry on these devices can offer patients several advantages over conventional perimetry machines, including increased accessibility, convenience, opportunities for more frequent monitoring between clinical visits, lower costs expended on clinic visits, and possibly earlier detection of disease or disease progression.¹²

An ideal computer-based or device-based perimetry test should be user-friendly and reliable, particularly outside of a controlled, supervised environment.¹² To optimize accessibility, ideally, such a perimetry test requires no additional hardware beyond which the user already has convenient access, such as a personal computer. To enhance adherence to surveillance programs, ideally, the test should be both enjoyable and practical. One of the limitations of older perimetry machines is that some patients find them uncomfortable, tiring, and anxiety provoking.^21,22 In recent years, clinic-based perimetry has been optimized for improved speed including the development of Swedish Interactive Threshold Algorithm (SITA) Faster, which can offer significantly shorter testing times and the potential for more frequent clinic monitoring.²³ Although a shorter test offers practical benefits, SITA Faster does not improve many of the disadvantages linked to the perimetry machines themselves, namely their expense, the need for technical maintenance and trained operating staff, the inability for home monitoring and for some patients, ergonomic discomfort and anxiety associated with using perimetry machines.²¹ Ultimately, a digitized test, accessible from any standard computer, which is intuitive, comfortable, and accurate, may offer an improved user experience, promoting both adherence and patient satisfaction.

Online circular contrast perimetry (OCCP) has recently been developed, specifically as a means of providing perimetry on computer screens. It has been found already to have robust diagnostic metrics for glaucoma with users showing a preference for it over SAP in terms of convenience and usability.²⁴ A study of a normal cohort performing OCCP demonstrated consistent results sufficient to determine parameters for a normative data set.²⁵ As an online system accessed via the web browser on any computer, OCCP offers easier and more convenient access to perimetry, with a more enjoyable and comfortable user experience; these have the potential to improve the provision of care and improve disease surveillance outcomes.

The purpose of this study was to validate OCCP in a larger cohort by comparing its diagnostic capabilities with conventional glaucoma tests, and by assessing the agreement and correlation between OCCP and SAP in normal controls and patients with glaucoma.

METHODS

Participants

Participants enrolled in the study were recruited from a subspecialist ophthalmology practice in Melbourne. The study conformed to the tenets outlined in the Declaration of Helsinki. All participants provided written, informed consent before agreeing to participate. Ethical approval was obtained from the Royal Australian and New Zealand College of Ophthalmology Human Research and Ethics Committee (90.18), with local site governance.

Eligibility for the study included: ability to read and understand English; provision of informed written consent; open anterior chamber angle in one or both eyes; best-corrected visual acuity ≤0.7 logarithm of the minimum angle of resolution (logMAR) (for both glaucoma and control groups); satisfactory optical coherence tomography (OCT) image quality; reliable SAP and OCCP test results.

Exclusion criteria were: secondary causes of glaucoma; angle abnormalities; papillary anomalies; ametropia >±5 diopters; large peripapillary atrophy; neurological disorders; medication that could alter visual field results (ie, chloroquine, vigabatrin, pilocarpine, etc.); ocular laser or surgery in the previous 3 months; media opacities preventing good image scans; unreliable SAP and OCCP tests.

Participants with any ocular pathology other than glaucoma (such as visually significant cataract defined by Lens Opacities Classification System III greater than Grade 2,²⁶ nonglaucomatous optic neuropathy, retinal or macular pathology) or significant cognitive impairment were excluded.

We used traditional parameters for assessing test reliability for both OCCP and SAP tests: false-negative >33%; false-positive >15%; fixation losses >20%, based on the Heij and Krakau method.²⁷ Using the approach outlined by Wu and Medeiros,²⁸ visual fields were evaluated for related artifacts including eyelid or rim artifacts, inattention, improper fixation or fatigue and were excluded if any of these were present.

OCT scans were reviewed for appropriate centration; those with signal strength lower than 8/10 or segmentation errors were excluded.

Assessment of Clinical Parameters

All participants and clinical examinations were assessed by the study’s chief investigator (consultant ophthalmologist Dr Simon Skalicky) to identify factors that determined their suitability for the study. Participants underwent a comprehensive ophthalmic examination, where the following clinical parameters were collected: refractive correction for distance, best-corrected visual acuity, Cirrus OCT of the optic nerve head and macula (Carl Zeiss Meditec Inc., Dublin, CA), central corneal thickness (CCT), intraocular pressure using the Goldmann applanation tonometer (Haag-Streit International, Bern, Switzerland), SAP with the Humphrey Field Analyzer (HFA) SITA standard 24-2 test (Zeiss) and OCCP test. Before undergoing perimetric assessment, participants were thoroughly briefed on the process of visual field testing, test expectations and requirements, and that they would be supervised by a trained orthoptist throughout both tests. Each eye was tested sequentially with SAP followed by OCCP. For clinical and perimetric data analyses, 1 eye was selected randomly per participant using simple randomization.²⁹

Glaucomatous optic neuropathy was defined as optic nerve damage resulting in characteristic visual field deficits (on SAP) as defined by the American Academy of Ophthalmology.³⁰ The following optic nerve and OCT features were also considered consistent with glaucoma: diffuse or focal narrowing, or notching of the optic disc rim, progressive neuroretinal rim atrophy associated with an increase in cupping of the optic disc, diffuse or localized thinning of the parapapillary retinal nerve fiber layer (RNFL), optic disc hemorrhage involving the disc rim, parapapillary RNFL or lamina cribrosa, optic disc neural rim asymmetry of the 2 eyes consistent with loss of neural tissue, beta-zone parapapillary atrophy, thinning of the RNFL and/or macula on imaging.³⁰

Participants were classified into 2 groups:

1. Controls (normal intraocular pressure, RNFL, and optic nerve head appearance and SAP results; no other ocular pathologies).
2. Open angle glaucoma (open angle on gonioscopy; characteristic disc appearance and visual field changes).

The OCCP Test

OCCP utilizes circular flickering targets to assess 52-loci over 24 degrees of peripheral vision (Fig. 1). The OCCP test was delivered using a python-based web application, hosted on Azure (Microsoft, Redmond, WA), with tailored high-security architecture and database integrity. 24-2 testing strategy was provided using interactive JavaScript code.

FIGURE 1: Online circular contrast perimetry test settings. A, Flickering test target. B, Moving fixation: star begins at the screen top (inferior hemifield testing loci shown here); it later moves to the bottom of the screen for superior hemifield testing. C, Sequence of target presentation: targets appear for 3 on/off cycles lasting 360 ms, contrast is graded at the start/end of target presentation. D, Fixation target: spinning golden star. E, Blind spot localization optimizes viewing distance. B and E, The dark grey homogenous circles are a diagrammatic representation of where test targets may appear and are not present during the live test. Figure adapted from Alawa et al.¹⁸ Adaptations are themselves works protected by copyright. So in order to publish this adaptation, authorization must be obtained both from the owner of the copyright in the original work and from the owner of copyright in the translation or adaptation.

Targets consisted of concentric alternating dark and light rings (Fig. 1A). Targets were similar to Pulsar perimetry (Haag-Streit International) with the same level of contrast in all directions to avoid unintended stimulation of cells that selectively respond to a given orientation, as this has been shown to affect perimetric sensitivity.^31–33 Targets were comparatively smaller in size (3.5 degrees of visual angle in OCCP vs 5 degrees used in Pulsar perimetry), providing for a more granular and detailed spatial distribution of visual field loss.^31,32 Targets also retained consistent contrast throughout the spatial extent of the target, except for a contrast gradient at the edge. Increased light scatter is one of the considerations with hard-edged contrast stimuli, as this can inadvertently stimulate more distant ganglion cells.³² Spacing between the target centers was 6 degrees of visual angle, consistent with conventional perimetry machines.^33,34 Using simple trigonometry to correct for flat plane viewing, test loci were spaced on the monitor relative to fixation. Initially, the inferior hemifield is tested with the fixation target at the top of the screen (Fig. 1B); subsequently, the fixation target moves to the bottom of the screen and the superior hemifield is tested.

The duration of each target flicker was 60 ms, over 3 on/off cycles, lasting a total of 360 ms. Similar to conventional Frequency Doubling Perimetry (FDP; Welch Allyn, Skaneateles, NY and Carl Zeiss Meditec), targets had sinusoidal contrast with spatial frequency 0.5 cycles/degree and temporal counter phase flickering at 18 hertz (Hz).^35,36 In addition, for 50 ms contrast was ramped up and down linearly at the beginning and end of target presentation to prevent temporal transients and saccades (Fig. 1C).^37,38 Unlike FDP, in which both light and dark bands vary around a mean of background luminance, in OCCP light rings were set to the background screen color (light grey) while the intensity of dark rings varied to achieve the desired target contrast.

OCCP uses a testing protocol similar to SITA strategies, based on a priori probability density functions with a 4/2 decibel (dB) staircase.³⁹ The test algorithm uses iterative maximum posterior probability calculations in real time to assess when testing can stop at each point. Two reversals are required at primary test points, after which only a single reversal is required for termination at each point.

Output ranged from pure white (255, 255, 255), as 100% relative luminance percentage, to black (0, 0, 0) as 0%. Relative luminance was calculated for each 256-greyscale level, based on the Web Content Accessibility Guidelines standards for relative luminance calculation.⁴⁰ Contrast of targets were calculated using the Michaelson formula by comparing peaks and troughs of targets:⁴¹ where RL₁ is the light band maximum (same as background) and RL₂ is the dark band minimum relative luminance. Background screen luminance was set at 224 candela per square meter (cd/m²) output. Similar to FDP, the following formula was used to convert contrast to relative decibels:³⁵

$\mathrm{Contrast}=\left({\mathrm{RL}}_{\mathrm{1}}-{\mathrm{RL}}_{\mathrm{2}}\right)/\left({\mathrm{RL}}_{\mathrm{1}}{+\mathrm{RL}}_{\mathrm{2}}\right)$

$\mathrm{Relative\; decibel}\left(\mathrm{rdB}\right)=-\mathrm{20log\; (Contrast\; Sensitivity)}$

Sample spot intensity was chosen for a minimum and maximum range of 0 and 36 rdB. This is a similar dynamic range to HFA as well as other portable perimetry devices, and sufficient for assessing human threshold estimates across the visual field.^17,42

Participants were instructed to fixate on a continuously spinning golden star (3.5 degrees of visual angle, Fig. 1D) and respond when a target appeared in their peripheral vision by clicking the mouse. When the user clicked in response to visualizing a target, a sound is produced; to help guide the user, this sound differs depending on whether the click is in the accepted response window or outside; clicks outside the accepted response window were recorded as false-positives. False-negatives were determined by a similar method utilized in SAP.⁴³ To account for the variability in participant reflex rates, target presentations were time-adapted to the patient’s response time (ie, the accepted latency of response time was determined by the user’s previous response times to ensure the optimal test tempo and user comfort, Fig. 1C).⁴⁴ There was also an additional, inbuilt random delay to avoid rhythmic responses.

Participants were guided through the test by the web application’s preprogrammed verbal instructions. All voice prompts were provided in English, however, OCCP is currently being developed for delivery in several languages.

Correct viewing distance was determined by the size of the monitor screen (which is assessed by the web application). On a 24-inch screen, users were instructed to complete the OCCP at a comfortable viewing distance of 40 cm.

User position monitoring was aided by blind spot localization at the test outset and webcam monitoring of head position for the test duration. The user’s blind spot was estimated at 15 degrees temporal and 0.5 degrees inferior. Using a 4×10-degree grid overlying the proposed area, the blind spot was then mapped out by testing small, nearby spots. Should the blind spot be detected too far from fixation, the user is instructed to move closer; likewise, if detected too close to fixation, the user is advised to move backward (Fig. 1E). In this way, the user’s position relative to the screen can be optimized for consistent testing. Monitoring of head position was provided by facial detection via the computer’s webcam with a refresh rate of one second. OCCP’s facial detection software utilized a freely validated algorithm based on a convolutional architecture for fast embedding model.⁴⁵ Head deviations of up to 15% in 4 planes (forward/backward, left/right) were permitted, and if there were any deviations beyond this threshold, the test was paused and the participant was instructed to correct their head position, after which the test would resume.

Testing Procedure

Participants completed OCCP via a browser-based web application on a computer in a controlled clinical environment for each eye separately, monitored by trained orthoptist and investigator Lazar Busija (L.B.) to ensure strict protocol adherence. Environmental conditions were standardized including background noise and lighting. The room lighting was kept dark, and the computer was turned on for 15 minutes before testing to ensure consistency of adaptation and screen brightness. All monitors were cleaned before testing to minimize any glare effect, which can reduce contrast sensitivity.¹⁶ Before commencement, participants were provided with a thorough explanation of the test and important test instructions. Participants were then comfortably seated at a desk, facing a computer monitor and were allowed 10 minutes to adapt to the ambient lighting before commencing the test. Although the web application guides users to set their position accurately before OCCP testing, for this clinical study, head height and position were optimized by L.B. The viewing distance (40 cm) was measured immediately pretest. No specialized equipment was used to support head position and participants completed OCCP without physical constraints.

Six different computers (in different testing rooms) were used in this study. Prior to perimetric testing, screen calibration for this study was performed using a SpyderX screen photometer (Datacolor, Lucerne, Switzerland) on each computer to ensure that the luminance range of output was consistent as predicted and that there was consistency of display across different monitors. However, for routine clinical practice, the user is guided through an interactive screen calibration process as a built-in feature of OCCP and provides a consistency of display across different monitors. Gamma was set at 2.2 and white temperature 6500 K. All tests were performed on 24-inch diagonal screens with a resolution of 1920×1080 pixels. All computers and testing rooms adhered to the same criteria outlined above, with consistency ensured by L.B.

Main Outcome Measures

These were: mean deviation (MD), pattern standard deviation (PSD), visual field index, mean sensitivity per point and per eye for OCCP compared to SAP, OCT measurements of average, superior, and inferior RNFL thickness, vertical cup-disc ratio, and mean, superior, and inferior macular ganglion cell complex inner plexiform layer (GCC) thickness. Calculation of OCCP global indices was based on data from a normative dataset in a normal cohort performing OCCP.²⁵

Statistical Analysis

Data were analyzed using GraphPad Prism 9.0 (GraphPad Software Inc., San Diego, CA), Statistical Package for Social Sciences (SPSS Inc., Chicago, IL) and Real Statistics in Excel 2016 (Microsoft 365). Statistical significance was set at P<0.05, with adjustment by the Bonferroni method. Normality was assessed using the Shapiro-Wilkes statistic. Assessment of intergroup differences was performed using the Mann-Whitney U analysis of ranks or the Student t test for parametric data.

To compare sensitivity thresholds across devices, SAP decibels were converted to logCS, where CS is the reciprocal of contrast threshold.³⁵ Contrast is defined as (peak-background luminance)/background luminance, which is equivalent to Weber contrast for luminance increments. For white-on-white perimetry using Goldmann size III stimuli, decibels are reported as 25+10×logCS.³⁵

Pointwise retinal CS thresholds were compared to SAP across increasing eccentricities for normal eyes and glaucomatous eyes and were subdivided by SAP MD deficit into mild (MD >–6.0 dB), moderate (MD –12.0 to –6.0 dB) and severe (MD < –12.0 dB) groups.¹⁷ Bland-Altman analyses were used to analyze the agreement and estimate the 95% limits of agreement (LoA) between the 2 tests’ MD, PSD, mean sensitivity per eye and per point. For Bland-Altman plots comparing mean CS thresholds, we presented the difference in CS thresholds for OCCP and SAP in log units with linear regression analyses.³⁵

To assess for regional agreement between OCCP and SAP, pointwise retinal threshold estimates were divided into 8 visual field zones (Fig. 2).¹⁷ Linear regression was performed to calculate slopes of the best-fitting regression lines and Pearson coefficients measured the strength of association. Intraclass correlation coefficients (ICC) were calculated to assess intertest reliability and were defined as: poor (<0.5); moderate (0.5–0.75); good (0.75–0.9) or excellent (≥0.90).⁴⁶ Because of the established bias between SAP and OCCP sensitivities, OCCP sensitivities were rescaled to match SAP sensitivities for accurate ICC analysis.^24,25 Reporting ICC and the Pearson statistic is consistent with past studies comparing modalities of perimetry.^10,17,47,48

FIGURE 2: Fifty-two-loci testing grid for the right eye used for both standard automated perimetry and online circular contrast perimetry. For regional analysis, loci were grouped into 8 distinct zones corresponding to visual field sectors, and their pointwise sensitivities were compared for online circular contrast perimetry and standard automated perimetry. Each locus is represented by a black-colored diamond.

The parameters from OCCP, SAP, and OCT were compared in terms of area under the receiver operating characteristic curve (AUC), sensitivity and specificity at different cutoffs to evaluate their diagnostic accuracy. For these parameters, we calculated the best cutoff point (defined as the value dividing healthy from glaucomatous eyes maximizing sensitivity+specificity), sensitivity at 80% and 90% specificity and AUC for detecting glaucoma. The sensitivities and specificities of the parameters with the greatest AUC were then compared for statistical significance using the χ² test; the AUCs were compared using the Hanley-McNeil method.⁴⁹ Agreement between the best parameters at best cutoff was evaluated with Cohen Kappa statistic,⁵⁰ defined as: excellent (>0.81);good (0.61–0.80); moderate (0.41–0.60); fair (0.21–0.40); and poor (<0.20).

RESULTS

Table 1 presents patient characteristics. In total, 220 participants (95 glaucoma patients, 125 controls) were included. Eight individuals were excluded because of failing to meet the perimetry reliability criteria. The mean age was 64.6 (±13.7) years. The mean testing time for OCCP (5:29±1:07) was shorter than SAP (6:04±1:12) in the glaucoma group (P<0.0001), but similar in the control group (5:11±1:33 vs 5:01±0:42, P>0.05). Figure 3 presents 2 examples of OCCP and HFA reports for patients with mild and moderate visual field defects.

TABLE 1 - Patient Characteristics and Perimetric Test Results: Glaucoma Versus Control Groups

Variables	Control Group	Glaucoma Group	P Value
Gender (F/M)	76/49	44/51	0.056
Disease severity, n. (%)
Mild	—	61 (64.2)
Moderate	—	13 (13.7)
Severe	—	21 (22.1)
Abnormal ONH (% eyes)	0	100	—
Age (y)	62.3±14.0	67.4±13.4	0.0019
LogMAR visual acuity	0.00±0.08	0.07±0.13	<0.0001
Corrected IOP (mm Hg)	16.05±3.81	13.34±3.82	<0.0001
CCT (µm)	564.19±40.44	543.82±41.99	0.0004
Spherical equivalent (D)	−0.21±2.44	−0.53±2.23	0.247
Instrument
OCCP
MD (dB)	0.30±1.57	−5.80±5.50	<0.0001
PSD (dB)	2.33±0.66	4.54±2.44	<0.0001
VFI (%)	98.48±1.61	84.11±16.76	<0.0001
Duration (minutes:seconds)	5:11±1:33	5:29±1:07	0.170
SAP
MD (dB)	−0.28±1.07	−6.19±6.94	<0.0001
PSD (dB)	1.66±0.46	5.41±4.37	<0.0001
VFI (%)	99.11±1.04	83.87±20.90	<0.0001
Duration (minutes:seconds)	5:01±0:42	6:04±1:12	<0.0001
OCT RNFL
MT (µm)	88.25±8.97	70.29±12.60	<0.0001
ST (µm)	108.74±14.87	83.17±17.63	<0.0001
IT (µm)	112.65±15.69	79.49±20.96	<0.0001
VCDR	0.51±0.18	0.68±0.16	<0.0001
OCT GCC
MT (µm)	77.84±7.18	66.67±9.04	<0.0001
ST (µm)	78.30±7.20	68.49±10.16	<0.0001
IT (µm)	77.42±7.43	64.84±9.60	<0.0001

Values given are mean±SD unless otherwise specified.

CCT indicates central corneal thickness; D, diopters; dB, decibel; GCC, ganglion cell complex inner plexiform layer; IOP, intraocular pressure; IT, inferior thickness; logMAR, logarithm of the minimal angle of resolution; MD, mean deviation; MT, mean thickness; OCCP, online circular contrast perimetry; OCT, optical coherence tomography; ONH, optic nerve head; PSD, pattern standard deviation; RNFL, retinal nerve fiber layer; SAP, standard automated perimetry; ST, superior thickness; VCDR, vertical cup-disc ratio; VFI, visual field index.

FIGURE 3: Representative visual field outcomes in 2 eyes having mild (A–B) and moderate (C–D) visual field defects. Eyeonic Online Circular Contrast Perimetry outcomes are shown in the left panels. Humphrey Field Analyzer Swedish Interactive Threshold Algorithm (SITA)-standard thresholds and total deviation plots are shown in the right panels. MD indicates mean deviation; PSD, pattern standard deviation; VFI, visual field index.

Influence of Eccentricity

Figure 4 displays mean pointwise log CS thresholds as a function of eccentricity for OCCP and SAP. Generally, thresholds increased with increasing eccentricity in normal eyes for both OCCP and SAP (Fig. 4A). Both OCCP and SAP significantly distinguished controls from mild, moderate, and severe glaucoma (Figs. 4B, C).

FIGURE 4: Contrast sensitivity thresholds across eccentricity. A, Online circular contrast perimetry (OCCP) versus standard automated perimetry (SAP) for normal controls. B, OCCP. C, SAP comparison of normal eyes from glaucomatous eyes subgrouped into mild [mean deviation (MD) >–6.0 dB, n=61], moderate (MD –12.0 to –6.0 dB, n=13) and severe (MD <–12.0 dB, n=21). Data points represent log contrast sensitivity thresholds, error bars represent SE. *P<0.05; **P<0.01; ***P<0.001; ****P<0.0001.

Agreement and Correlation Between OCCP and SAP

Figure 5 displays Bland-Altman plots comparing OCCP to SAP. Pointwise sensitivity for OCCP was greater than SAP by 1.02 log units (95% CI: 0.95–1.08); 95% LoA ranged from 0.860 to 1.17 (Fig. 5A). Similarly, OCCP per-eye sensitivity was greater than SAP by 1.02 log units (95% CI: 0.94–1.10); 95% LoA ranged from 0.37 to 1.66 (Fig. 5B). CS thresholds of OCCP reduced compared to SAP with increasing sensitivity threshold, both for per-point (Fig. 5A) and per-eye (Fig. 5B) analyses.

FIGURE 5: Bland-Altman plots (A–D) and area under receiver operating curves (AUC) (E). For Bland-Altman plots, the continuous horizontal line represents the mean differences (Bias) between the 2 tests. The dashed and dotted horizontal lines represent the 95% limits of agreement (Bias±1.96SD). A, Mean sensitivity (per point) with outliers shown in grey on visual field loci map (inset). B, Mean sensitivity (per eye). Data points represent log contrast sensitivity thresholds, the linear regression curve (solid line) and 95% CI curves (dashed lines) are shown. C, Mean deviation (MD). D, Pattern standard deviation (PSD). B–D, black-colored circles represent controls, white circles the glaucoma group. E, Instrument parameters with the highest AUC are shown. GCC indicates ganglion cell complex inner plexiform layer; IT, inferior thickness; RNFL, retinal nerve fiber layer; OCCP, online circular contrast perimetry; SAP, standard automated perimetry.

Table 2 displays the ICC, linear regression, Pearson correlation, and Bland-Altman agreements between MD, PSD, and sensitivities divided into regional zones for OCCP versus SAP. Except for Zone 1, the ICC reliabilities ranged from good to excellent.

TABLE 2 - Comparison of Mean Deviation, Pattern Standard Deviation, and Threshold Estimates According to Location: Online Circular Contrast Perimetry Versus Standard Automated Perimetry

	ICC	P Value	Pearson Correlation (Linear Regression Slope)	P Value	Bias (95% LoA) (dB)
MD	0.89	<0.001	0.90 (0.8)	<0.0001	0.51 (−4.20, 5.22)
PSD	0.76	<0.001	0.87 (0.51)	<0.0001	0.01 (−3.78, 3.80)
Zone analysis
Zone 1, superior central	0.74	0.002	1.00 (1.22)	0.002	−0.07 (−0.11, −0.04)
Zone 2, inferior-central	0.89	0.0015	1.00 (1.19)	0.002	−0.04 (−0.07, −0.01)
Zone 3, superior-peripheral-nasal	0.99	<0.001	0.99 (1.01)	<0.0001	0.01 (−0.02, 0.05)
Zone 4, superior-nasal	0.90	0.006	0.91 (0.83)	0.03	−0.02 (−0.09, 0.05)
Zone 5, superior-temporal	0.88	0.007	0.96 (0.67)	0.01	−0.02 (−0.01, 0.06)
Zone 6, inferior-temporal	0.86	0.019	0.90 (0.62)	0.04	0.00 (−0.04, 0.05)
Zone 7, inferior-nasal	0.90	0.002	0.95 (0.85)	0.01	0.02 (−0.02, 0.06)
Zone 8, inferior-peripheral-nasal	0.97	<0.001	0.98 (0.89)	0.0002	−0.01 (−0.03, 0.01)

dB indicates decibel; ICC, intraclass correlation coefficient; LoA, limits of agreement; MD, mean deviation; PSD, pattern SD.

Sensitivity, Specificity, and AUC

Table 3 displays the AUC, best cutoff point, sensitivity, and specificity for glaucoma diagnosis for each device’s parameters. OCCP MD, SAP MD, RNFL inferior thickness, and GCC inferior thickness were the parameters that demonstrated the greatest AUC for each device type in distinguishing healthy from glaucomatous eyes.

TABLE 3 - AUC, Best Cutoff Point, Sensitivity, and Specificity for Each Instrument in Discriminating Between Glaucomatous and Control Eyes

Instrument	Parameter	AUC (SE)	P Value	Best Cutoff Point	Se/Sp at Best Cutoff Point (%)	Se at 80% Sp (%)	Se at 90% Sp (%)
OCCP	MD (rdB)	0.885 (0.08)	<0.0001	−1.80	90/74	77	65
	PSD (rdB)	0.799 (0.09)	<0.0001	2.63	80/70	58	33
	VFI	0.856 (0.08)	<0.0001	96	90/68	68	50
SAP	MD (dB)	0.851 (0.08)	<0.0001	−1.76	94/65	70	44
	PSD (dB)	0.841 (0.09)	<0.0001	1.87	86/75	75	48
	VFI	0.838 (0.08)	<0.0001	98	81/76	66	45
OCT RNFL	MT (µm)	0.883 (0.08)	<0.0001	80	84/83	84	62
	ST (µm)	0.863 (0.08)	<0.0001	94	84/74	78	63
	IT (µm)	0.908 (0.07)	<0.0001	96	86/79	83	71
	VCDR	0.798 (0.09)	<0.0001	0.64	80/66	64	49
OCT GCC	MT (µm)	0.826 (0.08)	<0.0001	71	85/68	66	45
	ST (µm)	0.785 (0.09)	<0.0001	71.4	86/61	58	38
	IT (µm)	0.849 (0.08)	<0.0001	71.7	82/75	64	54
Parameter	Se (%) at best cutoff	Sp (%) at best cutoff	Se at 80% Sp	Se at 90% Sp	AUC
	P values (P c)				Mean±SE	P h	Kappa statistic at best cutoff
OCCP MD vs SAP MD	0.297	0.668	0.262	0.011	0.034±0.04	0.353	0.58
OCCP MD vs OCT RNFL IT	0.384	0.404	0.289	0.0008	0.023±0.03	0.482	0.45
OCCP MD vs OCT GCC IT	0.412	0.871	0.175	0.452	0.036±0.04	0.327	0.45
SAP MD vs OCT GCC IT	<0.0001	0.491	0.367	0.629	0.002±0.04	0.959	0.44
OCT RNFL IT vs OCT GCC IT	0.440	0.502	0.009	0.013	0.059±0.04	0.094	0.53
SAP MD vs OCT RNFL IT	0.237	0.110	0.120	0.0004	0.057±0.04	0.104	0.50

Parameters with the highest AUC per device are shown in bold with statistical comparison of percentage of sensitivity, specificity, AUC, and Kappa statistic

AUC indicates area under the receiver operating characteristic curve; GCC, ganglion cell complex inner plexiform layer; IT, inferior thickness; MD, mean deviation; MT, mean thickness; OCCP, online circular contrast perimetry; OCT, optical coherence tomography; P^c, P calculated with χ² test; P^h, P calculated with Hanley-NcNeil method; PSD, pattern SD; rdB, relative decibel; RNFL, retinal nerve fiber layer; Se, sensitivity; Sp, specificity; ST, superior thickness; SAP, standard automated perimetry; VCDR, vertical cup-disc ratio; VFI, visual field index.

The AUCs of these parameters were then compared for significance and levels of agreement using Cohen kappa statistic, and in Figure 5E for shape. At best cutoff, OCCP MD sensitivity/specificity were comparable to SAP (90/74 vs 94/65%) for detecting glaucoma (P>0.05). Using Cohen Kappa there was a moderate agreement for glaucoma diagnosis between all instruments’ best parameters; however, the agreement between OCCP MD and SAP MD was greater than all other interinstrument agreements (Kappa statistic at best cutoff=0.58).

DISCUSSION

This study found overall similar metrics between SAP and OCCP. There was a high level of correlation and agreement between global indices and regional sensitivities for OCCP and SAP. As a diagnostic tool for glaucoma, OCCP demonstrated strong AUC features, similar to SAP, OCT of the RNFL and GCC. As demonstrated in Figure 4, OCCP reliably distinguished mild disease from normal eyes.

We observed close agreement between MD results for OCCP and SAP, with similar trends for PSD. Similar observations were also reported by Kong et al,¹⁷ when comparing their tablet-based perimetry application to HFA.

Bland-Altman analysis revealed a bias between OCCP and SAP sensitivity results that widened as the sensitivity threshold increased. This relationship is consistent across the range of data, consistent with the findings from our previous studies,^24,25 and consistent with previous studies performing a similar comparison between SAP and contrast perimetry using sinusoidal targets such as FDP.⁵¹ Despite the bias in sensitivities, OCCP models a physiological hill of vision similar to SAP and reliably diagnoses glaucoma as shown by its high AUC characteristics and agreement with other test parameters across a range of glaucoma severity levels.²⁵

These results suggest OCCP holds promise as a glaucoma perimetry tool, both to complement existing clinic perimetry and expand community testing, particularly in populations with limited access to conventional perimetry machines. Furthermore, its minimal hardware requirements and the browser-based web platform make OCCP a widely accessible and cost-effective test that can be operated at home by the patient, although more work and data are required to assess its feasibility unsupervised in the home environment; this will be the subject of future studies.

These findings are consistent with other studies evaluating perimetry delivered on various devices.^{9,11,12,15–20,52} Computer-based, achromatic perimetry was reported to reasonably discriminate between control and glaucomatous eyes, demonstrating good concordance with HFA SITA standard perimetry.¹⁵ Tablet-based perimetry has good reliability and test-retest consistency, although there are limitations with gaze tracking and spatiotemporal precision.^12,17 OCCP incorporates many features to minimize the need for additional hardware and barriers to uptake. In contrast, devices such as virtual reality headsets may present additional hardware costs,^19,20 and the advantage of OCCP is its operability on the patient’s computer providing that there is a stable internet connection.

A reliable, cost-effective, and accessible perimetry test offers several advantages to both patients and providers over conventional machines, with the versatility to be used either supervised in-clinic or for home monitoring. Current perimetry machines often cause patient anxiety, and this has been shown to adversely affect adherence and test performance.²¹ Offering patients an improved user experience may serve to promote patient engagement, adherence, and satisfaction.²⁴ Improved access to care might help increase early detection rates (eg, in younger patients) or allow for more frequent monitoring (eg, in glaucoma patients at high risk of disease progression).¹² Frequent at-home monitoring reduces the mean absolute error of serial field tests, allowing earlier and more precise detection of progression.^11,12 Home-based perimetry also allows for clinic visits to be streamlined, thereby reducing commute times, waiting room numbers, and the expenditure of health care resources associated with clinic visits. Current guidelines recommend that patients should be regularly assessed within the first 2 years of diagnosis to monitor disease progression and optimize therapy, with 6-monthly or annual reviews thereafter.⁵³ Achieving these targets has been challenging due to overburdened service providers, limited access, and insufficient health care resources, especially during the COVID-19 pandemic.^12,13 COVID-19 has dramatically changed the landscape of clinical care, and in the postpandemic era, the provision of safe and effective glaucoma care should increasingly utilize portable services including perimetry.

There are several logistical challenges to consider for web-based perimetry. Inconsistencies in screen size and luminance between patients’ monitors might influence test reliability.⁵²

All monitors used in this study were the same size and resolution. Although variations in screen size can alter the extent of the visual field sampled, this can be appropriately compensated by adjusting the viewing distance: smaller screens require a closer viewing distance to maintain a constant visual angle. The web-based application automatically detects the monitor size and advises the user of the optimal viewing distance; it then positions the test targets accordingly to assess the test loci at the correct visual angle. User positioning is further optimized by continuous webcam monitoring and blind spot localization. Certain devices such as some smartphones are too small to complete the test and to address this, the application will detect use on a smartphone and will instruct the user to change to a device with a larger screen.

The variability of luminance between monitors has also been considered. This web application compensates by guiding the user through a screen calibration process before testing to ensure consistency of results, adjusting contrast, brightness, gamma, and white temperature to standardize the test. For scientific purposes, we used an external phoropter for calibration for this study, but this may not be necessary for routine clinical use and for at-home monitoring; future clinical studies will evaluate the robustness of testing on a variety of screens using the inbuilt calibration process. Reflections from light sources and screen tilt with reference to the viewing plane may also influence the above parameters and must be avoided.⁵²

In OCCP, peripheral targets were placed more widely apart (determined trigonometrically) than central targets to simulate the even 6-degree distribution of the Ganzfield bowl screen used by conventional perimeter machines.^33,34

Head positioning and posture, which are otherwise maintained on dedicated perimetry machines, can alter the mapping of loci and test-retest reliability. Supplying patients with a supportive viewing hood has been suggested,¹¹ although we aim to maintain the OCCP’s minimal hardware requirements. The consistency of head position and viewing distance was maintained by the combination of (1) blind spot localization at test outset, and (2) OCCP’s inbuilt facial detection software that detects small head movements via the monitor’s webcam, which then redirects the participant to adjust their position when required. Fixation losses are also assessed, and when detected the viewer is prompted in real time to refocus on the target.

For this study OCCP was performed under a closely monitored clinical environment by trained staff, such as conventional perimetry, to ensure optimal positioning and use of the web application. For such a test to be successful for home monitoring, thoroughly educating individuals before the test is important, including initially performing the test in a supervised clinical environment. In addition, appropriate messaging within the web application is imperative. Future studies are planned to evaluate OCCP at home in the absence of clinical supervision.

OCCP mean sensitivities for normal eyes differed from those with mild glaucoma with statistical significance (Fig. 4B). This suggests that OCCP might have the capability to detect early glaucomatous disease, however, we did not specifically evaluate patients with early disease or preperimetric glaucoma and should be considered as a future directive. We also recognize that using MD as the global index of visual field function has diagnostic limitations in early disease and should be interpreted alongside the patient’s other clinical parameters.⁵⁴ Similar to conventional SAP, OCCP assessed the central 24 degrees of the visual field, and it is conceivable that including a wider sampling field, such as the 30-2 protocol, might detect earlier glaucomatous damage. It is well known that significant ganglion loss must occur to produce detectable visual field abnormalities, including FDP; therefore, OCCP should be used in combination with other structural and functional tools to optimally assess patients with suspected or early disease.⁵⁵

This study has limitations. All participants were recruited from a single, subspecialist ophthalmology practice, which may introduce selection bias, although the larger cohort size promotes interpatient variability. Participants performed OCCP shortly after SAP, which might have some performance implications from changes in concentration, motivation, and fatigue.⁵⁶ More studies are required to assess test-retest consistency and the feasibility of the OCCP test on other device types (such as tablets).

There are also limitations regarding the use of reliability parameters. The OCCP used traditional and conservative measures of reliability, however, much of the use of these metrics has been questioned in recent literature.^23,56 False-negative responses have been correlated more strongly with visual field damage rather than poor patient vigilance.^43,57 In contrast, false-positive responses have been associated with artifactually elevated threshold sensitivity values including higher MD scores,^58–60 though excluding tests based on these responses is controversial as the effects are small.⁶⁰ While many modern strategies have abandoned some reliability parameters,²³ we believe it is important to assess them all for newer perimetry devices. Future studies, including those evaluating OCCP as a home monitoring device will assess the clinical significance of reliability indices. OCCP’s inbuilt feedback sounds on clicking are designed to guide and provide reassurance for the user, but for some users they might be a distraction, affecting attention and performance.^61,62 In this study, participants with very poor visual acuity (logMAR >0.7) were excluded; in future studies it may be interesting to know if OCCP is useful to detect perimetric changes for individuals with lower visual acuity.

In this present study, environmental testing conditions such as ambient lighting and background noise were standardized to facilitate reliable data collection. Future studies will evaluate the efficacy of OCCP as a home monitoring tool for glaucoma. Glaucoma home monitoring studies have shown promising results with other perimetry applications.^11,12 Although maintaining standardized environmental testing conditions might be more difficult at home, the design features of the OCCP have been considered to provide improved testing consistency despite small variations in screen and background lighting. Some participants might perform better outside the potentially stressful clinic environment. Other participants might have difficulties with the setup, maintaining the correct positioning, or operating the test due to technical inexperience, particularly the elderly, physically disabled, or cognitively impaired. We did not collect data on participant comorbidities or cognition, which may have also influenced test performance. Ultimately, suitability for home-based perimetry will require that patients are self-motivated, appropriately trained, physically able, and have access to a personal device and stable internet connection.

OCCP metrics show similar AUCs to SAP in distinguishing glaucoma patients from controls, offering a promising, cost-effective, and accessible perimetry alternative. Hopefully, OCCP can assist in expanding community glaucoma screening and surveillance programs. This may help providers meet the growing health demands of chronic eye disease, facilitate earlier disease detection, and improve patient outcomes.