Observer variation in quantitative assessment of retinal shortening with ultrasound in patients of total rhegmatogenous retinal detachment

Abstract

Purpose: 

To evaluate the interobserver variation in the assessment of retinal length to choroidal length ratio (RCR) as a marker for proliferative vitreoretinopathy (PVR) in cases of rhegmatogenous retinal detachment (RRD).

Methods: 

This was a double-masked, prospective study at a tertiary center. Ultrasound was used to calculate RCR in 50 eyes with total RRD by two observers. Both observers were trained after the first round of calculations, and all the calculations were repeated as before. Difference between the RCR values was stratified into four categories (<0.01, 0.01–0.05, 0.06–0.1, and >0.1) for descriptive analysis. A difference of 0.05 was set as the maximal limit for defining interobserver agreement. Correlation between RCR and interobserver difference was assessed.

Results: 

The mean interobserver difference in RCR values was found to be 0.06 ± 0.0 (P = 0.41) and was reduced to 0.04 ± 0.02 (P = 0.81) following training. The interobserver difference was <0.1 in 82% of the cases before training and in 98% of cases after training. The worst interobserver agreement was noted in cases with RCR < 0.8, and there was a good negative correlation between RCR and interobserver difference (r = −0.6, P ≤ 0.001).

Conclusion: 

There is good interobserver agreement in assessing RCR with ultrasound in eyes with RRD, which improves further with training. RCR needs careful assessment in eyes with very low RCR. This technique may be useful in prognostication of surgical outcomes in cases with advanced PVR.

Rhegmatogenous retinal detachment (RRD) is one of the most common indications for vitreoretinal surgery. Yet, technical advances notwithstanding, surgical outcomes are very frequently compromised by the occurrence of proliferative vitreoretinopathy (PVR).[¹²] This is a very important cause of concern for developing nations, where descriptive studies have detected delayed presentation and severe grades of PVR in comparison to developed nations.[³⁴] Recent research has focused on pharmacological prevention of PVR, but the perfect regimen remains elusive.[⁵⁶]

Conventionally, PVR has been known to be preretinal or subretinal and to result in retinal shortening.[⁷⁸⁹] Further understanding has revealed the importance of intraretinal PVR (iPVR), and suggestions have been made to include it as a separate entity.[¹⁰] Recently, we used ultrasound to calculate the retinal length to choroidal length ratio (RCR) in patients of retinal detachment and objectified retinal shortening.[¹¹] In a later study, we used a combination of ultrasonic and clinical examination to prove occurrence of retinal shortening in the absence of other forms of PVR (pre- or sub-retinal) and thus proved occurrence of iPVR in the absence of other forms of PVR.[¹²] Both these studies were indicative of possibility of a critical threshold of preoperative quadrantic and mean RCR, below which poor anatomical outcomes may be imminent. However, in both these studies, RCRs were calculated by a single observer only.[¹¹¹²]

The current study aims to identify interobserver variation during the assessment of RCR with ultrasound in patients of total RRD. Further, we also assess impact of degree of retinal shortening on observer variation.

Methods

Design

This is a double-masked, prospective, investigative study conducted at a tertiary eye care center of Northern India over a period of 6 months. The study was conducted in accordance with the tenets of the Declarations of Helsinki, and ethical clearance was obtained from the institutional review board. Informed written consent was obtained from the patients for all the investigative procedures involved. The methodology of this study has been described in details in the previous studies.[¹¹¹²] Briefly, ultrasonic findings of the eyes undergoing primary vitrectomy for total RRD were evaluated in this study. Patients with a history of other ophthalmic disorders (apart from cataract) in the eye undergoing surgery and those with giant retinal tears, media opacity, and choroidal detachment were excluded from the study.

Ultrasonic measurement and calculation of retinal length to choroidal length ratio

Preoperative ultrasound B-scan of the involved eye was performed in 34 patients by a single surgeon and using a single device (EZ Scan AB5500+, Sonomed, NY, USA). Longitudinally sectioned images were obtained for four quadrants, and a section of the optic nerve head was included in each image. Of the 136 images available, 50 were randomly selected and assigned an identification code by another surgeon. Both patients with or without PVR were included. These images were then separately evaluated by two observers for the assessment of RCR who were masked to each other's findings throughout the study.

All the measurements were done using “freehand line tool” of the freely available ImageJ software (http://imagej.nih.gov/ij/). With the help of the software, distance between two endpoints on a structure can be measured while moving the cursor's locus along the contour of the structure.[¹¹¹²] As in the previous studies, the retina was measured starting from the optic nerve head till its point of fusion with the choroid, while the choroid was measured as the distance between these two endpoints.[¹¹¹²] A perpendicular was drawn from the retina to the choroid to identify these endpoints in all the images where the ora serrata had not been imaged in the sonogram.[¹¹¹²] Both the observers repeated the measurements five times for each image and the arithmetic mean was taken after excluding measurements deviating more than ½ mm from the median. RCR was hence obtained for each image by the two observers.

Analysis

Data were entered into Microsoft Excel worksheets and statistical analysis was performed by another author using SPSS Software, Version. 20 (SPSS Inc, Chicago, Illinois, USA). Independent t-test was used to analyze the difference between the RCR values calculated by the two observers. The extent of variation between RCR values of the two observers was distributed into four categories (RCR value difference of <0.01, 0.01–0.05, 0.06–0.1, and >0.1). Further analysis was done to assess the impact of degree of retinal shortening on interobserver agreement. Three groups were formed on the basis of mean of the RCR values calculated by the two observers. Group 1 had RCR of <0.8, Group 2 had 0.8–0.89, while Group 3 had RCR >0.9. These groups were analyzed for limits of agreement between the two observers [Table 1]. Correlation between continuous variables has been assessed using Spearman's rank (R) correlation coefficient. Hereafter, all the confidence intervals (CIs) described are true for 95% of the values, and only the results with a two-tailed P < 0.05 have been described to be statistically significant.

Table 1: Impact of degree of retinal shortening on interobserver variation

Results

Of the 50 images provided, both observers identified a single image as unfit for the assessment of RCR with ultrasound. Hence, 49 images were included for analysis and evaluating interobserver variation. The mean RCR calculated by observer 1 was 0.86 ± 0.07, while that calculated by observer 2 was 0.85 ± 0.07. The difference in mean RCR calculated by both the observers was found to be statistically insignificant, P = 0.41 (CI = −0.16–0.04). The mean interobserver variation was found to be 0.06 ± 0.0. In nearly a quarter of the cases, the difference in RCR measurements was <0.01, while it was more than 0.1 in only 18% of the cases [Fig. 1]. In most of the cases (more than one-third), the difference was between 0.01 and 0.05 [Fig. 1].

Figure 1: Histogram showing interobserver variation in the assessment of retinal shortening with ultrasound

On analysis of the three groups, the mean RCR value calculated by observers 1 and 2 was 0.77 ± 0.08 and 0.78 ± 0.07, respectively, in Group 1, 0.86 ± 0.05 and 0.84 ± 0.05, respectively, in Group 2, and 0.92 ± 0.04 and 0.93 ± 0.03, respectively, in Group 3. The mean difference between the two observers was 0.12 in Group 1, 0.07 in Group 2, and 0.04 in Group 3. Hence, best agreement was noted in Group 3, while it was worst in Group 1. When a difference of 0.05 was set as the maximal limit of difference for agreement, best agreement was seen in Group 3 (80%) while the worst was noted in Group 1 (14.3%) [Table 1] (P = 0.028). The Spearman's Rho coefficient of correlation between mean RCR and interobserver difference was found to be −0.6 (P ≤ 0.001).

At this point in time, two observers were trained again regarding calculation of RCR and were also allowed to discuss the methodology. Calculations for RCR were repeated by both in a blinded fashion as done previously. Following training, the mean RCR determined by observer 1 was 0.86 ± 0.5, while that assessed by observer 2 was 0.85 ± 0.5. The difference between them was statistically insignificant, P = 0.81 (CI = −0.005–0.04). The mean interobserver variation was determined to be 0.04 ± 0.02. This time the interobserver variation was found to be <0.05 in more than 80% of the cases, while it was more than 0.1 in only 2% [Fig. 2].

Figure 2: Histogram showing interobserver variation in assessment of retinal shortening with ultrasound after training of the two observers

In two cases, the interobserver variation was found to be >0.1 despite training. The images of these cases have been compiled in Fig. 3. It can be seen that in both cases, the endpoint of assessment of retinal length was not clear as ora serrata had not been imaged in the sonogram. As explained in the Materials and Methods section and in the previous studies,[¹¹¹²] in such cases, a perpendicular has to be drawn to identify the endpoint of measurement.

Figure 3: (a and b) Ultrasound images where interobserver was >0.1 despite training. The encircled areas showing that the anterior limit of detached retina or the ora serrata had not been imaged in these two cases. The white-hashed lines depicting the method of drawing perpendiculars to determine the endpoint for measuring choroidal length in such cases

Discussion

In the earlier studies, we have discussed the utility of this technique in the preoperative judgment of retinal shortening and its severity.[¹¹¹²] Determination of a critical threshold for retinectomy was considered in those studies for cases of RRD with advanced PVR. However, in both those studies, RCRs were assessed by a single observer. The current study was designed to assess observer variation in this technique as reproducibility is crucial before practical usage is envisaged.

In this study, we found that there was good agreement between two-blinded observers in the determination of RCR with ultrasound, which further improved after training. The difference between the two observers despite training may possibly be related to inability of the sonographer to image the ora serrata in some cases which entails the requirement of manual interpretation of endpoints of measurement [Fig. 3]. We also found that the best agreement was present in eyes with more RCR and there was a good negative correlation between mean RCR and interobserver difference [Table 1]. Considering the previously defined limit of 0.8 in determining severe retinal shortening,[¹¹¹²] which was also set as the maximum limit for Group 1, this technique should be considered more accurate for ruling out severe retinal shortening than otherwise. However, as this is a pilot technique of quantifying retinal shortening, no comparisons can be made with documented literature. Further, the positive impact of training reflects probability of lesser observer variation in real-world environment where the examiner, the sonographer, and the analyst may be a single observer. As the study involved measuring retinal length and choroidal length five times before using the mean values to calculate the RCR, intraobserver variation was not a concern in this study.

The presence of iPVR can be devastating toward successful management of RRD.[¹⁰] As mentioned earlier, gold standard method for establishing the precise relationship between RCR value and presence of iPVR would be a prospective study that evaluates histopathological and/or immunohistochemical changes in specimens obtained following retinectomy. However, obtaining such samples and externalizing them in toto in the era of minimally invasive surgery are challenging.[¹²] A more useful approach would be creating scores for preoperative prognostication of cases with advanced PVR in terms of anatomical success following surgery. Considering the encouraging results of this and the previous studies, these scores can be calculated on the basis of preoperative qualitative clinical evaluation and quantitative ultrasonic determination of RCR.

Conclusion

The results of this study indicate low interobserver variation in the determination of RCR with ultrasound in patients with RRD, which is further minimized by training. Cases with very low RCR may be prone to observer variation and should be assessed carefully. The utility of this technique may be explored by future studies for prognostication of outcomes following surgery for RRD with advanced PVR.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.