Sjogren syndrome is a common autoimmune disease best known for causing dry eye and dry mouth but also features more widespread extraglandular manifestations including neuropathy, renal disease, and systemic vasculitis.1–5 Dry eye is a hallmark feature of Sjogren syndrome (SS), yet SS is underdiagnosed in patients with dry eye disease.6–9 When Henrik Sjögren described a specific type of dry eye, that of keratoconjunctivitis sicca (KCS), he highlighted its features of corneal and conjunctival staining with rose bengal.10–12 Other measures for identifying dry eye include Schirmer testing and newer methods including assessing tear osmolarity and inferior tear meniscus height.13–21 Use of any diagnostic test or sign should accurately differentiate between patients with a disease state and those without. For dry eye, ideally there would be a single test that would serve as the reference or gold standard. In reality, however, no such gold standard exists.
Instead, there are a variety of ocular signs and tests used to identify dry eye either alone or in combination with each other. These measures include the assessment of corneal staining with fluorescein, conjunctival staining with lissamine green, the ocular staining score (OSS), tear breakup time (TBUT), Schirmer testing, and tear osmolarity.22 In addition, patient-reported symptoms of dry eye are included in the assessments for identifying patients with dry eye disease.23,24 By using different reference standards, the classification of patients with dry eye may be haphazard between different studies. As a result, the quantitative assessment of the performance of different index tests and estimates of disease prevalence may be biased.25
In this study, we applied latent class analysis (LCA) to data from participants in the Sjögren's International Collaborative Clinical Alliance (SICCA) to estimate the sensitivity and specificity of some of the most commonly used clinical signs and tests to diagnose KCS. We used the term KCS intentionally because we applied LCA to diagnostic tests and signs to the SICCA cohort, which was constructed, in part, to assess the relationship of aqueous deficiency to Sjogren syndrome. In the absence of a gold standard KCS classification, this model-based approach provides a preliminary sense of relative diagnostic performance of these measures.
Study Design and Population
The SICCA cohort represents a cross-sectional study of participants enrolled from 9 international research sites. Participants (≥21 years of age) met at least one of the following conditions: 1) complaint of dry eye or dry mouth, 2) previous diagnosis of primary or secondary Sjogren syndrome (SS), 3) abnormal serology (positive anti-SSA, anti-SSB, or elevated ANA and RF), 4) bilateral parotid gland enlargement, or 5) multiple cervical/incisal dental caries.
At the baseline SICCA visit, participants had completed an interview and questionnaires, had oral/ocular/rheumatologic examinations, had blood and saliva samples collected, and had a labial salivary gland biopsy among other procedures.26,27 Ethical clearance was obtained from the institutional review board at each clinical site, and the study adhered to the tenets of the Declaration of Helsinki.
Variables and Measures
Independent variables that were recorded included OSS (abnormal ≥ 5), Schirmer I (abnormal ≤ 5 mm in 5 minutes), TBUT (abnormal < 10 seconds), and tear osmolarity (abnormal > 316 mOsm/L) as measured with the TearLabTM system (TearLab, San Diego, CA).
Levels of sensitivity and specificity for each relevant clinical sign and diagnostic test (variable) were estimated using LCA. This analysis provides a model-based clustering of participants into a specified number of “disease” classes based on the observed patterns of a series of binary predictor variables representing the presence or absence of important diagnostic features. The resulting classes can then be related to the disease status based on the class-specific patterns of diagnostic features. Our modeling was based on 4 or 5 predictor variables that relate to signs of KCS. Models incorporating different numbers of classes were compared using the Bayes information criterion (BIC). In addition, we also fitted the LCA models incorporating a random effect to relax the conventional assumption of conditional independence of within-class outcomes. Finally, sensitivity and specificity of individual predictors were estimated using the model-based classification with an underlying, unobservable “gold standard.26,28,29 Analyses were performed with the R package randomLCA (R Core Team, R version 3.3.2 and R Studio 1.0.136, R, Boston, MA).
A total of 3514 participants from 9 international sites were enrolled in SICCA. Most participants were women [3185 (91%) women vs. 309 (9%) men]. Gender was missing for 20 participants. Sjogren syndrome as defined by the ACR/EULAR criteria was diagnosed in 1541 participants (52.9%) and 116 participants (3.3%) could not be classified because of the missing data.
With LCA, we found a best-fit model with 2 groups. Incorporation of a random effect did not noticeably improve the model fit. The gold standard–positive group having an abnormal TBUT, OSS, and Schirmer I had a sensitivity of 99.5% (95% CI: 94.9%–100%), 91.0% (95% CI: 84.2%–95.1%), and 47.4% (95% CI: 44.6%–50.4%), respectively (Table 1). The gold standard–negative group having an abnormal TBUT, OSS, and Schirmer I had a specificity of 32.0% (95% CI: 28.6%–35.5%), 84.0% (95% CI: 79.0%–87.9%), and 88.5% (95% CI: 85.0%–91.2%), respectively (Table 1).
Only 79 participants had tear osmolarity tested because of the late addition of this test during the study period. When we included tear osmolarity in the model, we found that the gold standard positive–group having an abnormal TBUT, OSS, Schirmer I, and osmolarity had a sensitivity of 99.5% (95% CI: 94.9%–100%), 91.1% (95% CI: 84.2%–95.1%), 47.5% (95% CI: 44.6%–50.4%), and 45.7% (95% CI: 29.6%–62.7%), respectively (Table 2). The gold standard–negative group having an abnormal TBUT, OSS, Schirmer I, and osmolarity had a specificity of 32.0% (95% CI: 28.6%–35.5%), 83.9% (95% CI: 79.0%–87.9%), 88.5% (95% CI: 85.0%–91.2%), and 72.1% (95% CI: 53.2%–85.4%), respectively (Table 2).
Because the OSS comprises corneal staining with fluorescein and conjunctival staining with lissamine green, we wished to determine whether staining of the cornea or conjunctiva was able to distinguish the 2 groups from each other. LCA revealed that for the gold standard–positive group having an abnormal TBUT, fluorescein, lissamine, and Schirmer I had a sensitivity of 98.9% (95% CI: 97.7%–100%), 82.6% (95% CI: 78.9%–85.7%), 90.5% (95% CI: 87.2%–93.0%), and 49.8% (95% CI: 46.7%–52.9%), respectively (Table 3). In the gold standard–negative group, the specificity for an abnormal TBUT, fluorescein, lissamine, and Schirmer I was 29.1% (95% CI: 26.5%–31.7%), 88.8% (95% CI: 85.3%–91.6%), 73.0% (95% CI: 69.6%–76.2%), and 88.1% (95% CI: 86.1%–89.8%), respectively (Table 3).
By using LCA, we inferred that both TBUT (<10 seconds) and OSS (≥5) had a high sensitivity in identifying 2 groups of individuals within the population of patients with dry eye in the SICCA cohort—“dry” and “not dry”. However, although OSS exhibited a high specificity, TBUT did not. We found that the OSS's constituent components (lissamine green staining of the temporal and bulbar conjunctiva and fluorescein staining of the cornea) also exhibited high sensitivity and high specificity. Tear osmolarity, on the other hand, was neither sensitive nor particularly specific in distinguishing between 2 groups of patients with dry eye, although few participants in this cohort had tear osmolarity tested, which prompted us to include this variable as a sensitivity analysis. Although Schirmer I testing was not particularly sensitive, it was specific. Although Schirmer I has been criticized previously for exhibiting variability,23,30,31 the Schirmer I should be considered as complementary when combined with the use of the OSS. Taken together, our results suggest that the OSS or its constituent components are particularly sensitive and specific in identifying participants with keratoconjunctivitis sicca in our cohort. In addition, Schirmer I testing is complementary in that it is a specific test that can corroborate aqueous deficiency that typifies lacrimal gland dysfunction in Sjogren syndrome.
Currently, the application of classification criteria for dry eye disease varies depending on the study, making comparisons difficult. It is essential to have classification criteria for KCS for clinical trials and epidemiological studies to provide internal study consistency and to allow for comparisons between studies.
The OSS is composed of 2 types of stains used to identify the desiccated areas of the cornea or bulbar conjunctiva. Because concentrated lissamine green 1% solution is not commercially available, it must be prepared either by a specialty compounding pharmacy (which is not always available or an option at some centers because of differing compounding laws) or may be too time consuming to make for each patient, although protocols for making it using lissamine green–impregnated strips exist.32 Using components of the OSS (fluorescein and lissamine green staining) yielded a relatively high sensitivity for corneal fluorescein staining using a cutoff of 80%. Therefore, the assessment of corneal fluorescein staining alone may be a useful way to identify patients with KCS. This is reassuring because fluorescein strips are readily available and used in most ophthalmology clinics. In addition, this study shows that other tests, such as Schirmer I, can be complementary when combined with the OSS or with fluorescein staining alone when identifying patients with KCS.
In contrast to our results, independent component analysis (ICA) was used to suggest that tear osmolarity might be the best marker for identifying dry eye severity.33 ICA is a signal processing technique in which signals are assumed to be statistically independent.34 If used for diagnostic test studies, ICA would reward tests that are uncorrelated to other tests. Although tear osmolarity was identified as independent—that does not imply importance—a random number generator would have been identified as even more independent and would presumably play no role in dry eye detection.35 In one study, mean osmolarity of patients with blepharitis and SS was higher than in controls, although not significantly so.20 Some have suggested that corneal and conjunctival staining are inherently insensitive, but our findings suggest quite the opposite.36 We do stress the importance, however, of using concentrated lissamine green 1% in staining the conjunctiva because using single lissamine green–impregnated strip on conjunctiva does not adequately stain.32
A limitation of this study, similar to other studies of keratoconjunctivitis sicca is the use of expert clinical judgment in the absence of an objective “gold standard” for defining keratoconjunctivitis sicca. However, although expert opinion has driven the use of dry eye clinical signs and tests including conjunctival staining, corneal staining, Schirmer I testing, TBUT, and tear osmolarity, we applied a latent class model-based approach to allow us to cluster SICCA participants for keratoconjunctivitis sicca. Limitations of LCA include its unsupervised approach. In addition, LCA makes assumptions; in particular, it assumes that each diagnostic test is conditionally independent. Nevertheless, LCA can be a robust statistical method that analyses categorically-scored data particularly when 3 or more diagnostic tests or signs are being evaluated as in the present study's case.
Using the uniform diagnostic criteria for KCS may be useful for future studies of Sjogren syndrome. One of the contributions of SICCA has been the development of standardized diagnostic criteria for SS. Our analyses presented herein demonstrate that the OSS, which harkens back to Henrik Sjögren's original description of keratoconjunctivitis sicca, is a hallmark and distinguishing feature of aqueous deficiency in his eponymous syndrome.
1. Kidder D, Rutherford E, Kipgen D, et al. Kidney biopsy findings in primary Sjogren syndrome
. Nephrol Dial Transpl. 2015;30:1363–1369.
2. McCoy SS, Baer AN. Neurological complications of sjogren's syndrome: diagnosis and management. Curr Treatm Opt Rheumatol. 2017;3:275–288.
3. Maripuri S, Grande JP, Osborn TG, et al. Renal involvement in primary Sjogren's syndrome: a clinicopathologic study. Clin J Am Soc Nephrol. 2009;4:1423–1431.
4. Garcia-Carrasco M, Ramos-Casals M, Rosas J, et al. Primary Sjogren syndrome
: clinical and immunologic disease patterns in a cohort of 400 patients. Medicine (Baltimore). 2002;81:270–280.
5. Scofield RH. Vasculitis in sjogren's syndrome. Curr Rheumatol Rep. 2011;13:482–488.
6. Akpek EK, Klimava A, Thorne JE, et al. Evaluation of patients with dry eye for presence of underlying Sjogren syndrome
. Cornea. 2009;28:493–497.
7. Akpek EK, Bunya VY, Saldanha IJ. Sjogren's syndrome: more than just dry eye. Cornea. 2019;38:658–661.
8. Bunya VY, Fernandez KB, Ying GS, et al. Survey of ophthalmologists regarding practice patterns for dry eye and sjogren syndrome
. Eye Contact Lens 2018;44(suppl 2):S196–S201.
9. Henrich CF, Ramulu PY, Akpek EK. Association of dry eye and inflammatory systemic diseases in a tertiary care-based sample. Cornea. 2014;33:819–825.
10. Sjögren H. Keratoconjunctivitis sicca
. Hygiea. 1930:82;829.
11. Sjögren H. Keratconjunctivitis sicca. (in tran ophthalmol section Swedish med assoc. 1929-1931). Acta Ophthalmol. 1932;10:403–409.
12. Sjögren H. Zur Kenntnis der Keratoconjunctivitis sicca
. Keratitis filiformis bei Hypofunktion der Tränendrüsen. Acta Ophthalmol. 1933;1:1–151.
13. Savini G, Barboni P, Zanini M. Tear meniscus evaluation by optical coherence tomography. Ophthalmic Surg Lasers Imaging. 2006;37:112–118.
14. Zhou S, Li Y, Lu AT, et al. Reproducibility of tear meniscus measurement by Fourier-domain optical coherence tomography: a pilot study. Ophthalmic Surg Lasers Imaging. 2009;40:442–447.
15. Wang Y, Zhuang H, Xu J, et al. Dynamic changes in the lower tear meniscus after instillation of artificial tears. Cornea. 2010;29:404–408.
16. Chan HH, Zhao Y, Tun TA, et al. Repeatability of tear meniscus evaluation using spectral-domain Cirrus(R) HD-OCT and time-domain Visante(R) OCT. Cont Lens Anterior Eye. 2015;38:368–372.
17. Benelli U, Nardi M, Posarelli C, et al. Tear osmolarity
measurement using the TearLab Osmolarity System in the assessment of dry eye treatment effectiveness. Cont Lens Anterior Eye. 2010;33:61–67.
18. Versura P, Profazio V, Campos EC. Performance of tear osmolarity
compared to previous diagnostic tests for dry eye diseases. Curr Eye Res. 2010;35:553–564.
19. Bunya VY, Langelier N, Chen S, et al. Tear osmolarity
in Sjogren syndrome
. Cornea. 2013;32:922–927.
20. Bunya VY, Fuerst NM, Pistilli M, et al. Variability of tear osmolarity
in patients with dry eye. JAMA Ophthalmol. 2015;133:662–667.
21. Pepose JS, Sullivan BD, Foulks GN, et al. The value of tear osmolarity
as a metric in evaluating the response to dry eye therapy in the clinic and in clinical trials. Am J Ophthalmol. 2014;157:4–6.e1.
22. Whitcher JP, Shiboski CH, Shiboski SC, et al. A simplified quantitative method for assessing keratoconjunctivitis sicca
from the Sjogren's Syndrome International Registry. Am J Ophthalmol. 2010;149:405–415.
23. Lemp MA. Report of the national eye institute/industry workshop on clinical trials in dry eyes. CLAO J. 1995;21:221–232.
24. Craig JP, Nichols KK, Akpek EK, et al. TFOS DEWS II definition and classification report. Ocul Surf. 2017;15:276–283.
25. Begg CB. Biases in the assessment of diagnostic tests. Stat Med. 1987;6:411–423.
26. Shiboski CH, Shiboski SC, Seror R, et al. 2016 American college of rheumatology/European league against rheumatism classification criteria for primary sjogren's syndrome: a consensus and data-driven methodology involving three international patient cohorts. Arthritis Rheumatol. 2017;69:35–45.
27. Shiboski CH, Shiboski SC, Seror R, et al. 2016 American College of Rheumatology/European League against Rheumatism classification criteria for primary Sjogren's syndrome: a consensus and data-driven methodology involving three international patient cohorts. Ann Rheum Dis. 2017;76:9–16.
28. Baughman AL, Bisgard KM, Cortese MM, et al. Utility of composite reference standards and latent class analysis in evaluating the clinical accuracy of diagnostic tests for pertussis. Clin Vaccin Immunol. 2008;15:106–114.
29. See CW, Alemayehu W, Melese M, et al. How reliable are tests for trachoma?-a latent class approach. Invest Ophthalmol Vis Sci. 2011;52:6133–6137.
30. Versura P, Frigato M, Mule R, et al. A proposal of new ocular items in Sjogren's syndrome classification criteria. Clin Exp Rheumatol. 2006;24:567–572.
31. Versura P, Frigato M, Cellini M, et al. Diagnostic performance of tear function tests in Sjogren's syndrome patients. Eye (Lond). 2007;21:229–237.
32. Stock M, Salvay D, Shoss B, et al. Obtaining lissamine green
1% solution for clinical use. Cornea. 2015;34:1523–1525.
33. Sullivan BD, Whitmer D, Nichols KK, et al. An objective approach to dry eye disease severity. Invest Ophthalmol Vis Sci. 2010;51:6125–6130.
34. James CJ, Hesse CW. Independent component analysis for biomedical signals. Physiol Meas. 2005;26:R15–R39.
35. Acharya DP, Panda G. A review of independent component analysis techniques and their applications. IETE Tech Rev. 2008;25:320–332.
36. Sullivan BD, Crews LA, Sonmez B, et al. Clinical utility of objective tests for dry eye disease: variability over time and implications for clinical trials and disease management. Cornea. 2012;31:1000–1008.