Algorithm of automatic identification of diabetic retinopathy foci based on ultra-widefield scanning laser ophthalmoscopy

Jie Wang, Su-Zhen Wang, Xiao-Lin Qin, Meng Chen, Heng-Ming Zhang, Xin Liu,Meng-Jun Xiang, Jian-Bin Hu, Hai-Yu Huang, Chang-Jun Lan

1Aier Eye Hospital (East of Chengdu), Chengdu 610051,Sichuan Province, China

2Department of Ophthalmology, Chengdu First People’s Hospital, Chengdu 610095, Sichuan Province, China

3School of Computer and Artificial Intelligence, Southwest Jiaotong University, Chengdu 610097, Sichuan Province, China

4Chengdu Aier Eye Hospital, Chengdu 610041, Sichuan Province, China

Abstract

· KEYWORDS: diabetic retinopathy; ultra-widefield scanning laser ophthalmoscopy; intelligent diagnosis system

INTRODUCTION

Diabetes mellitus (DM) is a chronic disease that poses a significant threat to human health.According to the International Diabetes Federation, there were 370 million DM patients worldwide in 2019, and this number is projected to increase to 700 million by 2045[1-2].Diabetic retinopathy(DR) is a prevalent microvascular complication of DM and the leading cause of blindness in patients with DM[3].The prevalence of DR is as high as 34.6% among DM patients,and 7% of DR cases are associated with vision-threatening complications.Although the clinical treatment technology for DR has reached a relatively advanced stage, there still exists a significant number of patients who suffer from irreversible visual impairment due to inadequate early diagnosis and treatment.Therefore, early screening, diagnosis and timely treatment of DR patients are crucial for effective management of the disease[4-5].

In recent years, the continuous advancement of artificial intelligence (AI) diagnostic technology has led to a competition among ophthalmic AI research teams striving towards the development of DR AI diagnosis technology based on ordinary color fundus photography.Currently, numerous domestic and overseas research teams have developed diverse deep learning models for DR screening, all of which exhibit exceptional detection performance with an area under the curve ranging from 0.89 to 0.99, sensitivity between 87.2% and 100%, and specificity from 87.0% to 100%[5-12].

Figure 1 Image cropping and cutting.

When detecting DR lesions using color fundus photographic(CFP), there are several challenges.First, the fundus area displayed in CFP images is limited and even with 7-orientation splicing, only about 65%-70% of the retina area can be shown[13-14], which increases the risk of missing eye lesions.Second, the clarity of CFP can be easily affected by various factors such as dioptric medium clarity, shooting environment and operator skills.

The ultra-widefield scanning laser ophthalmoscopy (SLO)utilizes scanning laser ophthalmoscope technology and elliptical mirror technology, employing the elliptical bifocal principle to capture retinal imaging up to 200° in a mere 0.25s while maintaining high resolution[15].The device employs red(633 nm) and green (532 nm) lasers to simultaneously scan the retina and choroid, enabling acquisition of a “false color image” of the fundus, non-red light image, and choroid image through its dual-channel laser imaging capabilities, thereby facilitating lesion localization[16-17].

In a collaborative effort, our hospital and the Southwest Jiaotong University team conducted a study on “Research on algorithm for automatic identification of diabetic retinopathy lesions based on ultra-widefield scanning laser ophthalmoscopy”.The intelligent detection of hemorrhagic spots, cotton wool spots,exudates, and microaneurysms in ultra-widefield SLO provides valuable assistance to clinicians in the diagnosis of DR.

MATERIALS AND METHODS

Ethical ApprovalThe Institutional Medical Ethics Committee at our hospital meticulously reviewed and granted approval for the study (DQAIER202203001).All investigations were conducted in strict adherence to pertinent guidelines and regulations.The ethics committee waived informed consent,as none of the ultra-widefield SLO contained any personal information about the patients included.

Figure 2 Labeling of lesions.

DatasetsThe present study retrospectively amassed 1076 fundus photographs of DR patients who underwent ultrawidefield SLO examination at Aier Eye Hospital (East of Chengdu) between 2020 and 2022.All images were meticulously acquired by proficient ophthalmic technicians,and the dimensions of the photographs were 2600 mm×2048 mm.The dataset comprised a total of 253 proliferative diabetic retinopathy (PDR) images and 823 non-proliferative diabetic retinopathy (NPDR) images.

Datasets ProcessingRedundant information, such as eyelids and eyelashes, commonly present in ultra-widefield SLO can increase the computational burden of training and interfere with DR lesion detection.When using the entire ultra-widefield SLO as input for convolutional networks, local lesions can be difficult to focus on due to high resolution and large pixel value differences.Therefore, at the start of algorithm training,we crop and divide the image into N different images (Figure 1).

Labeling of LesionsUnder the guidance of the deputy chief physician from the Department of Fundus Diseases at our hospital, we manually labeled hemorrhagic spots, cotton wool spots, exudates, and microaneurysms in ultra-widefield SLO using LabelImg software (including coordinate information and name; Figure 2).

Model Establishment and TrainingOwing to the frequent conjunction with fundus hemorrhage, PDR often presents with less distinct information regarding hemorrhagic spots, cotton wool spots, exudates, and microaneurysms.Consequently, the trained model directly outputs results for PDR images, sans subsequent lesion recognition.Automated algorithm-based detection and lesion identification were conducted on NPDR images.The ultra-widefield SLO processing flow is delineated in Figure 3.

Model TrainingWe standardized the processed images and partitioned the dataset into training, test, and validation sets in accordance with the 6:2:2.The model was trained using a combination of ResNet50, FasterRCNN, and FPN (Feature Pyramid Networks).During the training process, the number of iterations was set to 100, with a batch size of 1.The optimizer employed was Stochastic Gradient Descent (SGD), and the initial learning rate was set to 0.001.In the event that the loss value for the training set failed to decrease during the training process, the learning rate was adjusted to half of its original value.

Improved ResNet50This study Improved ResNet50 by altering the size of convolution kernels to augment the model's feature extraction capacity.Concurrently, a Convolutional Block Attention Module (CBAM) is incorporated into the final layer of the ResNet50 architecture to facilitate model attention towards salient regions.Irrelevant image data is thereby mitigated.The Improved ResNet50 demonstrates the ability to extract subtle features from ultra-widefield SLO, effectively minimizes the number of network parameters, and exhibits a higher level of accuracy in distinguishing NPDR and PDR.

Faster Regions with CNN FeaturesThe first module of Faster Regions with CNN Features (Faster RCNN)[18]is the feature extraction mechanism, which utilizes an Anchor-based approach to generate candidate frames.This network integrates feature extraction, candidate frame selection, frame regression and classification into a single system that effectively detects target objects within images.A feature extraction network with a greater number of layers can extract features at various levels, and precise target features are beneficial for enhancing the recognition rate of targets[19].In ultra-widefield SLO, the dimensions of hemorrhages spots and microaneurysms targets are minimal.After undergoing feature extraction network processes such as convolution and pooling, the amount of information present is diminished.As a result, adjustments to the Anchor are necessitated in this study, and a clustering method based on means is employed to determine the optimal anchor size.

Feature Pyramid NetworksHemorrhagic spots and microaneurysms in ultra-widefield SLO images of DR patients exhibit uneven distribution, significant size variations, and reduced pixel information.Utilizing a single high-level feature for detection may result in information loss during the sampling process.To address the challenge of detecting significant differences between objects, this paper employs feature pyramid networks (FPN)[20], which can integrate feature maps with strong semantic information at low resolution and those with weak semantic information but rich spatial details at high resolution while minimizing computational costs.This approach effectively resolves the issue of deep residual lesion detection.

Figure 3 Image processing flow SLO: Scanning laser ophthalmoscopy;DR: Diabetic retinopathy; Faster RCNN: Faster regions with CNN features.

Table 1 Test results

Vessel SegmentationIt is challenging to differentiate between hemorrhagic spots and microaneurysms, which often leads to misdiagnoses.In this study, Res_Unet was employed to vessel segmentation, aiming to further delineate hemorrhagic spots and microaneurysms based on their proximity to blood vessels,generally within 20 pixels.Additionally, the accuracy of lesion detection was enhanced by correcting the results with lesions extracted using the Faster RCNN network.

RESULTS

Evaluation IndexTo assess the performance of the constructed multi-classification model, two evaluation metrics—precision and sensitivity—are utilized.Precision is computed as follows: Precision=TP/(TP+FP).The formula for calculating sensitivity is: Sensitivity=TN/(TN+FP).

Precision, also known as positive prediction value, is the proportion of true positive samples to the positive samples determined by the algorithm; Sensitivity (Sen), also known as Recall (R), is the proportion of true positive samples to all positive samples; True positive (TP): The number of samples that are actually positive and correctly predicted to be positive;False positive (FP): The number of samples that are actually negative and incorrectly predicted as positive; True negative(TN): The number of samples that are actually negative and correctly predicted to be negative[21].

Experimental ResultsAfter training and testing the model, we achieved an accuracy of 54.94% in detecting microaneurysms, 83.57% for cotton wool spots, 86.75% for exudates, and 87.23% for hemorrhages spots.The verification lesion detection results are shown in Table 1, and the lesion labeling results are shown in Figure 4.

Figure 4 Results of lesion labeling.

Based on the results of lesion detection, the algorithm proposed in this paper demonstrates high accuracy and sensitivity in detecting hemorrhagic spots, cotton wool spots and exudates in DR ultra-widefield SLO images.However,its performance is limited when it comes to microaneurysms detection due to their resemblance to deep retinal hemorrhages.Based on the detection results, distinguishing microaneurysms from hemorrhagic spots can be challenging, and identifying microaneurysms in ultra-widefield SLO images is difficult.Typically, the green channel image is selected for further analysis due to the limited number of microaneurysms in our dataset.Therefore, future studies should aim to design alternative algorithms for combined detection and conduct a range of experiments to compare and analyze the efficacy of lesion detection in ultra-widefield SLO images of DR.

DISCUSSION

With the progression of AI diagnostic technology, an increasing number of sophisticated diagnostic models have been developed.Studies comparable to the model presented in this research include the following.Wanget al[22]developed a deep learning-based DR Classification model.This model employed the optimization of the cyclic generative adversary network(CycleGAN) and convolutional neural network (CNN)model classifiers, and utilized ultra-widefield angiography to facilitate DR Classification, which is dependent on ischemic index and leakage index.Sunet al[23]developed an ultrawidefield SLO-based intelligent diagnostic model for various diseases.They assessed the performance of three deep learning networks (EfficientNet-B7, DenseNet, and ResNet-101),and the aggregate accuracy of the deep learning model for DR diagnosis averaged 93.00%.The two aforementioned algorithm modes do not extract features of hemorrhagic spots, cotton wool spots, exudates, and microaneurysms,instead, they diagnose the diseaseviaalternative features.Abitbolet al[24]employed a DenseNet121 CNN to develop a diagnostic model for various diseases, achieving disease diagnosis through enhanced image separation.The model demonstrated an accuracy of 85.2% for DR.Ohet al[5]recently developed a novel SLO image diagnosis model utilizing the ResNet-34 residual network.This model effectively extracted the region of interest from both the optic disc and the macula’s center, resulting in a DR detection accuracy of 83.38%±0.47%.The two aforementioned algorithms employ a single model for feature extraction.As SLO images possess more lesion information, it becomes challenging to detect small lesions.

In light of the dearth of lesion information in PDR and the conspicuous disparity between PDR and NPDR images, this study employs ResNet50 convolution kernel size substitution to directly generate PDR images, aiming to enhance the detection rate of PDR.For NPDR, an improved Faster RCNN is proposed.During the feature extraction phase, a residual network integrated with a feature pyramid is employed to extract lesion information, while simultaneously adjusting the FRN candidate box size, effectively addressing the challenges associated with extracting small lesions.In this research,Res_Unet was utilized for retinal blood vessel segmentation and the extraction of retinal blood vesselsviaan algorithm.Concurrently, the lesion detection outcomes of the improved Faster RCNN were optimized to augment the detection precision of microhemangioma and bleeding points.The primary limitation of our study is the small sample size,which precludes the inclusion of images of patients without DR for learning, resulting in suboptimal detection accuracy and sensitivity for arterioles.In the near future, we plan to increase the sample size for both learning and detection, and optimize the algorithm to enhance the lesion recognition rate.Furthermore, the presence of redundant information, such as eyelashes and eyelids in stereoscopic fundus photography,can impede lesion recognition.Consequently, we aim to develop an automatic segmentation method for retinal images to improve the intelligence of the system.At present, our algorithm is based on the entire fundus map.In the future, we intend to utilize the position of the optic disc and the center of the macula as reference points to identify lesions in different areas of the image.In the subsequent step, we will validate the model using additional data sets and optimize the model to achieve DR grading diagnosis.

The objective of this research is to facilitate the advancement of DR intelligent diagnosis studies, supply efficient supportive tools for clinicians, facilitate early diagnosis for patients, and decrease the blindness rate associated with DR.

ACKNOWLEDGEMENTS

Foundation:Supported by Hunan Provincial Science and Technology Department Clinical Medical Technology Innovation Guidance Project (No.2021SK50103);

Conflicts of Interest:Wang J,None;Wang SZ,None;Qin XL,None;Chen M,None;Zhang HM,None;Liu X,None;Xiang MJ,None;Hu JB,None;Huang HY,None;Lan CJ,None.