|Year : 2019 | Volume
| Issue : 1 | Page : 32-36
Limbal ischemia: Reliability of clinical assessment and implications in the management of ocular burns
Ka Wai Kam1, Chaitali N Patel2, Neda Nikpoor3, Marco Yu4, Sayan Basu2
1 Department of Ophthalmology and Visual Sciences, Prince of Wales Hospital; Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Sha Tin, Hong Kong
2 Brien Holden Eye Research Center, LV Prasad Eye Institute, Kallam Anji Reddy Campus, Hyderabad, Telangana, India
3 Himalayan Cataract Project, Stanford University, Stanford, CA, USA
4 Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong; Department of Mathematics and Statistics, Hang Seng Management College, Sha Tin, Hong Kong
|Date of Submission||05-Jun-2018|
|Date of Acceptance||21-Aug-2018|
|Date of Web Publication||21-Dec-2018|
Dr. Sayan Basu
Tej Kohli Cornea Institute, LV Prasad Eye Institute, Kallam Anji Reddy Campus, Hyderabad, Telangana
Source of Support: None, Conflict of Interest: None
Purpose: Limbal ischemia is an important prognostic factor in the management of ocular burns. In this study, we evaluated the reliability of clinically assessing limbal ischemia among ophthalmic professionals. Methods: This study included 111 ophthalmic professionals who were shown 12 diffuse illumination color slit-lamp photographs of eyes with recent chemical injuries. Respondents were asked whether the photos were assessable and if yes, then to indicate the presence, location, and grade of limbal ischemia in each case. The responses were collected using a standard data collection sheet and the inter-observer agreement was calculated. Results: All participants responded to every question. Of the 1,332 responses, images were deemed assessable in 1,222 (91.7%) instances. The overall agreement (Fleiss' kappa) for the presence of limbal ischemia and severity of limbal ischemia was 0.106 and 0.139, respectively (P < 0.012). Among the four groups of observers, practicing cornea specialists displayed significantly (P < 0.003) higher kappa values (0.201–0.203) when compared to residents (0.131–0.185), fellows (0.086–0.127), and optometrists (0.077–0.102). All indicated a poor level of inter-rater consistency. Conclusion: The results indicate that clinical assessment of limbal ischemia is highly subjective and there is lack of reliability even among cornea specialists who regularly manage patients with ocular burns. A non-invasive, standardized, objective, accurate, and reliable modality for ocular surface angiography is desperately needed for proper assessment and prognostication of ocular burns.
Keywords: Chemical injury, imaging, limbal ischemia, limbus, ocular burns
|How to cite this article:|
Kam KW, Patel CN, Nikpoor N, Yu M, Basu S. Limbal ischemia: Reliability of clinical assessment and implications in the management of ocular burns. Indian J Ophthalmol 2019;67:32-6
|How to cite this URL:|
Kam KW, Patel CN, Nikpoor N, Yu M, Basu S. Limbal ischemia: Reliability of clinical assessment and implications in the management of ocular burns. Indian J Ophthalmol [serial online] 2019 [cited 2019 Jan 17];67:32-6. Available from: http://www.ijo.in/text.asp?2019/67/1/32/243760
Ocular burns and other forms of ocular trauma are common causes of visual impairment and blindness in the developing world. Chemical or thermal burns to the eyes can irreversibly damage the limbal stem cells leading to chronic corneal vascularization and scarring. Once blinding limbal stem cell deficiency develops, these eyes then require complex reconstructive procedures for visual rehabilitation.,, It is well recognized that appropriate management in the acute stage of ocular burn injury is crucial to preventing long-term visual morbidity. One of the key factors that play a decisive role in the management of acute ocular burns is the presence and extent of limbal ischemia., However, an accurate clinical assessment of limbal ischemia is difficult not only because limbal ischemia may be superficial and not correlate with the amount of surviving limbal stem cells but also because apparently healthy limbus may slough off during subsequent post-injury period.
Currently, there are no standard criteria to diagnose limbal ischemia. Clinicians rely on the subjective appearance of the vascularity and color of the limbus and peri-limbal region and compare that with a mental image of a healthy eye. Unlike the retina, fluorescein or indocyanine green angiography of the ocular surface is neither available nor practiced as a tool to definitively diagnose limbal ischemia. Therefore, although clinical assessment still is the gold standard in defining limbal ischemia, the reliability of clinical assessment itself remains questionable. In this study, we tried to quantify the reliability of clinically assessing limbal ischemia among a large group of ophthalmic professionals who are routinely involved in the care of patients with ocular burns.
| Methods|| |
This study was approved by the local Ethics committee. We recruited ophthalmologists and optometrists who participated in an international ocular surface workshop from both local and international centers and who possessed clinical interests in ocular surface diseases. We obtained basic information including their age, sex, country of origin, job position/title, major sub-specialty, years of experience in general ophthalmology/optometry, and years of subspecialty experience for subgroup analyses. Those who did not wish to participate in the study were excluded.
Outcome measures and sample size calculation
The hypothesis of the study was that there is a lack of consistency among ophthalmic professionals regarding the identification (present or absent) and grading (extent/severity) of limbal ischemia, therefore inter-observer agreement was the primary outcome measure of this study. Secondary outcome measure was the variation in agreement between different sub-groups of ophthalmic professionals. For statistically valid agreement analysis, we needed at least 10 gradable photographs and 100 respondents. Assuming a 15% chance of photos being of inadequate quality and 10% non-response rate, we selected photographs of 12 different eyes with ocular burns and interviewed 111 ophthalmic professionals ranging from optometrists to ophthalmology residents, fellowship trainees, and cornea and external disease consultants.
Slit-lamp photographs of 12 eyes which had prior acute chemical ocular injury were captured by a single experienced slit-lamp photographer with a camera-mounted slit-lamp (BX900, Haag-Streit Diagnostics, USA; Canon EOS 40D, Canon Inc., Japan) with diffuse and assisted illumination. All color photographs were captured with the built-in image software (EyeCap Digital, version 5.0) without application of fluorescein or utilization of cobalt blue light.
The photographs were displayed to all participants for sufficient amount of time for assessment and completion of the response. For each photograph, each participant had to first answer whether the photo was of gradable quality. If the photo was deemed not assessable, the rest of the questions were not recorded. If the photo was deemed assessable, each subject would need to determine whether limbal ischemia was present or absent; to grade the severity of limbal ischemia into mild, moderate, or severe; and to indicate the location of ischemic areas in terms of clock hours. Participants were instructed to put down additional comments or reasons if they found a photo to be not gradable on the questionnaire.
Univariate analyses were presented with mean, percentage, standard deviation, and 95% confidence intervals (CIs) where appropriate. The presence or absence of limbal ischemia was set as a binary outcome; severity of limbal ischemia was classified into mild, moderate, and severe as a categorical variable. The total number of clock hours of limbal ischemia was analyzed by its median and inter-quartile range (IQR) to look at the distribution. In evaluating the inter-rater agreement regarding the presence or absence of limbal ischemia among 111 raters, percent agreement was measured among raters and the Fleiss's kappa was calculated to assess the agreement for this binary variable. Fleiss's kappa was also used to assess the agreement of severity as there are more than two categories of options in the response. Fleiss' kappa coefficients were compared between different positions by empirical bootstrap with 1,000 replicates. According to Fleiss (1981) kappa benchmark scale, a kappa of <0.4 was considered as “poor agreement,” 0.40–0.75 as “intermediate to good agreement,” and >0.75 as “excellent agreement.”,
| Results|| |
A total of 111 eye care professionals participated in this study. Fifty-seven (56.4%) were male and the mean age was 31.6 ± 6.5 years old. Sixteen participants were ophthalmology residents, 29 were fellowship trainees, 42 were cornea and external disease consultants, and 11 were optometrists. Eighty-seven participants (86.1%) were from India, while 14 were from overseas institutions. Among all participants, 50 participants (49.5%) had joined ophthalmology training for 0–5 years, while 31 (30.7%), 10 (9.9%), and 6 (5.9%) had 6–10, 11–15, more than 15 years of experience in this field, respectively. Among the 90 ophthalmologists, 60 of them (66.7%) were receiving or had received sub-specialty training in cornea sub-specialty. Forty-one of them (68.3%) had been working in cornea sub-specialty for 0–5 years, while 10 (16.7%), 4 (6.7%), and 5 (8.3%) had been working as a cornea sub-specialist for 6–10, 11–15, and more than 15 years, respectively. All optometrists were receiving or had received training in cornea sub-specialty.
Assessability of photos
Of the 1,332 responses, images were deemed assessable in 1,222 (91.7%, range 84–100) instances. The entire set of 12 images was deemed gradable in 83.4% of respondents [Figure 1]. The photos [Figure 2] and [Figure 3] with the lowest rates were images P7 and P5, where 18 (16.2%) and 14 (12.6%) respondents, respectively, considered them to be ungradable.
|Figure 1: Distribution of raters indicating the presence and grading severity of limbal ischemia in the 12 images|
Click here to view
|Figure 2: Images with highest agreement in terms of the presence of limbal ischemia among 111 raters|
Click here to view
|Figure 3: Images with lowest agreement in terms of the presence of limbal ischemia among 111 raters|
Click here to view
Presence of limbal ischemia
The distribution of responses for the presence or absence of limbal ischemia is presented in [Table 1] and [Figure 1]. Extent of limbal ischemia ranged from 49.5% (image P7 in [Figure 3]) to 100% (image P2 in [Figure 2]). The Fleiss' kappa was 0.106 (95% CI 0.035–0.178, P = 0.008) indicating poor overall agreement for individual images among respondents.
|Table 1: Proportions of observers in grading assessability, presence, severity, and location of limbal ischemia|
Click here to view
Grading severity of limbal ischemia
The distribution of responses for severe limbal ischemia is also presented in [Table 1] and [Figure 1]. The proportion of respondents who thought that limbal ischemia was severe ranged from 1% to 89.1%. The Fleiss's kappa was 0.139 (95% CI 0.037–0.242, P = 0.012) indicating poor overall agreement for individual images among respondents.
Total amount of limbal ischemia
The median total number of clock hours of limbal ischemia indicated by all respondents for each image was presented in [Table 1] along with the 25th and 75th quartiles. The INRs ranged from 2 to 7 clock hours and a mean INR of 4.4 ± 1.7 clock hours.
Comparison between different groups of observers
Among four groups of observers, all displayed poor inter-rater agreement with kappa values below 0.4. Consultants had the highest kappa values in terms of indicating the presence of limbal ischemia and grading the severity of ischemia, with corresponding values of 0.201 and 0.203, respectively. The group with the second highest kappa values in indicating the presence and grading severity of ischemia were the residents (0.131, 0.185), followed by the fellows (0.086, 0.127) [Figure 2]. The optometrists had the poorest inter-rater agreement among all groups with a kappa of 0.077 when asked to indicate the presence or absence of limbal ischemia, and a kappa of 0.102 when grading the severity of ischemia [Figure 3].
When comparing the groups with highest and lowest kappa values, consultants displayed significantly higher inter-rater agreement when compared to optometrists with P values of 0.003 and <0.001, respectively [Table 2]. Moreover, when compared to their fellows, consultants also displayed statistically higher inter-rater agreement when indicating the presence and grading severity of ischemia with P values of 0.001 and 0.003, respectively. On the other hand, comparison between fellows and residents did not yield any statistically significant difference in values of kappa. Even after excluding the two images (Image 5 and 7) with the lowest rate of assessability, the kappa values and the level of significance did not change significantly [Table 2].
|Table 2: Fleiss' kappa values of different grades of observers and comparison between groups when all images were analyzed|
Click here to view
| Discussion|| |
We conducted this study to assess the uniformity of diagnosing limbal ischemia among ophthalmic healthcare professionals. Our findings showed an overall low inter-rater consistency in a large sample of 111 raters, regardless of their training and experience, in both indicating the presence of limbal ischemia and in grading the severity of ischemia. Our sub-group analyses revealed that consultants had the highest inter-rater agreement when compared to residents, fellowship trainees, and optometrists; however, the extent of agreement among consultants was still poor, reflecting the limitations of subjective assessment even among specialists who had rich experience in managing cornea and external eye diseases. Among the four groups, optometrists had the lowest inter-rater agreement which might be a result of a different nature of training.
The limbus, which contains epithelial stem cells, plays a vital role in maintaining the renewal of the corneal epithelium. Although the importance of diagnosing limbal ischemia and its association with prognosis following chemical ocular injuries is well established in the literature, there is no clinical scale available for clinicians to refer to when diagnosing limbal ischemia or objectively measuring the extent of ischemia. Hence the clinical assessment of limbal ischemia is largely subjective and the variability in diagnosing or detecting ischemia between different raters might contribute to misclassification of cases, initiation of excess or inadequate treatment, and variations in clinical outcomes.,, Clinical assessment of ischemia can neither differentiate between full thickness or partial loss of limbal vasculature nor is it always associated with irreversible loss of limbal stem cells. Dua et al. attempted to overcome this issue by incorporating additional features such as extent of loss of conjunctival epithelium in their classification system, and replacing the words “limbal ischemia” with a broader term known as “limbal involvement.” However, this too is often found to be inadequate because the extent of limbal de-epithelialization may not necessarily correlate with the extent of ischemia.
Nonetheless, slit-lamp biomicroscopy remains as the most commonly employed method when evaluating the limbus following chemical injury. Our study has clearly shown the poor reliability of this subjective evaluation. As all these clinicians would plan their treatment regimen according to their perceived severity, the inconsistency in clinical assessment implies that each of these eyes shown in the pictures would have received very different therapy should they present to the participants in the clinics. Because of this high variability in making a diagnosis of limbal ischemia, additional ancillary investigations such as anterior segment angiography with or without using optical coherence tomography (OCT) technique might benefit clinicians in defining more precisely the area of ischemic limbus, hence guiding subsequent therapy and proper counseling of the prognoses of the patients. This also highlights the statistical paradox of evaluating the sensitivity and specificity of future approaches such as OCT angiography when the gold standard (clinical slit-lamp evaluation) itself is unreliable. Unfortunately, current OCT angiography platforms are not optimized for ocular surface imaging and there is need for standardizing the image acquisition and post-processing protocols.
This is the first inter-rater study looking at the consistency in diagnosing limbal ischemia among a large cohort of ophthalmic professionals who regularly handle patients with ocular chemical burns. The large variation in terms of diagnosis, grading of severity, and determination of location of ischemia reflects that clinical assessment alone was much prone to personal bias and more objective ways of assessing the limbal vasculature are warranted. Since this study assumed that photographic examination was almost equivalent to a slit-lamp biomicroscopic examination of the eye, one might argue that the lack of a three-dimensional perspective or absence of fluorescein staining assessment might have hindered accurate clinical assessment. However, it was logistically impossible for all these ophthalmic professionals to examine all these patients physically within a short amount of time, particularly when such patients were often in discomfort because of their injuries. Therefore, to reduce such bias and difference between photos, we have adopted a standard way of capturing slit-lamp photographs by the same experienced photographer and each participant was shown the same set of photos.
| Conclusion|| |
We conducted a simple scientific study to evaluate the inter-rater variability among ophthalmic professionals in diagnosing limbal ischemia. Our results have clearly shown that a wide variability exists in identifying the presence and severity of limbal ischemia, even among consultants who had rich experience in managing corneal diseases. Additional objective measurement of the limbal vasculature such as non-invasive OCT angiography, when available for ocular surface vasculature, may help define the location and amount of limbal ischemia more precisely in the future.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Oliva MS, Schottman T, Gulati M. Turning the tide of corneal blindness. Indian J Ophthalmol 2012;60:423-7. [Full text]
Vazirani J, Nair D, Shanbhag S, Wurity S, Ranjan A, Sangwan V, et al.
Limbal stem cell deficiency-demography and underlying causes. Am J Ophthalmol 2018;188:99-103.
Basu S, Sureka SP, Shanbhag SS, Kethiri AR, Singh V, Sangwan VS, et al.
Simple limbal epithelial transplantation: Long-term clinical outcomes in 125 cases of unilateral chronic ocular surface burns. Ophthalmology 2016;123:1000-10.
Basu S, Mohamed A, Chaurasia S, Sejpal K, Vemuganti GK, Sangwan VS, et al.
Clinical outcomes of penetrating keratoplasty after autologous cultivated limbal epithelial transplantation for ocular surface burns. Am J Ophthalmol 2011;152:917-240.
Wagoner MD. Chemical injuries of the eye: Current concepts in pathophysiology and therapy. Surv Ophthalmol 1997;41:275-313.
Roper-Hall MJ. Thermal and chemical burns. Trans Ophthalmol Soc U K 1965;85:631-53.
Dua HS, King AJ, Joseph A. A new classification of ocular surface burns. Br J Ophthalmol 2001;85:1379-83.
Kuckelkorn R, Remky A, Wolf S, Reim M, Redbrake C. Video fluorescein angiography of the anterior eye segment in severe eye burns. Acta Ophthalmol Scand 1997;75:675-80.
Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull 1971;76:378-82.
McHugh ML. Interrater reliability: The kappa statistic. Biochem Med (Zagreb) 2012;22:276-82.
Davanger M, Evensen A. Role of the pericorneal papillary structure in renewal of corneal epithelium. Nature 1971;229:560-1.
Gupta N, Kalaivani M, Tandon R. Comparison of prognostic value of Roper Hall and Dua classification systems in acute ocular burns. Br J Ophthalmol 2011;95:194-8.
Brodovsky SC, McCarty CA, Snibson G, Loughnan M, Sullivan L, Daniell M, et al.
Management of Alkali Burns: An 11-year retrospective review. Ophthalmology 2000;107:1829-35.
Oie Y, Nishida K. Evaluation of corneal neovascularization using optical coherence tomography angiography in patients with limbal stem cell deficiency. Cornea 2017;36 Suppl 1:S72-5.
[Figure 1], [Figure 2], [Figure 3]
[Table 1], [Table 2]