3D Semantic Deep Learning Networks for Leukemia Detection

Javaria Amin,Muhammad Sharif,Muhammad Almas Anjum,Ayesha Siddiqa,Seifedine Kadry,Yunyoung Nam and Mudassar Raza

1University of Wah,Wah Cantt,Pakistan

2COMSATS University Islamabad,Wah Campus,Pakistan

3National University of Technology(NUTECH),IJP Road Islamabad,Pakistan

4Faculty of Applied Computing and Technology,Noroff University College,Kristiansand,Norway

5Department of Computer Science and Engineering,Soonchunhyang University,Asan,31538,Korea

Abstract:White blood cells(WBCs)are a vital part of the immune system that protect the body from different types of bacteria and viruses.Abnormal cell growth destroys the body’s immune system,and computerized methods play a vital role in detecting abnormalities at the initial stage.In this research,a deep learning technique is proposed for the detection of leukemia.The proposed methodology consists of three phases.Phase I uses an open neural network exchange(ONNX)and YOLOv2 to localize WBCs.The localized images are passed to Phase II,in which 3D-segmentation is performed using deeplabv3 as a base network of the pre-trained Xception model.The segmented images are used in Phase III,in which features are extracted using the darknet-53 model and optimized using Bhattacharyya separately criteria to classify WBCs.The proposed methodology is validated on three publically available benchmark datasets,namely ALL-IDB1,ALL-IDB2,and LISC,in terms of different metrics,such as precision,accuracy,sensitivity,and dice scores.The results of the proposed method are comparable to those of recent existing methodologies,thus proving its effectiveness.

Keywords:YOLOv2;darknet53;Bhattacharyya separately criteria;ONNX

1 Introduction

Blood is a fluid that transports oxygen,providing energy to body cells that then produce carbon dioxide.It also plays a pivotal role in the immune system;blood circulating in living organisms contains 55% plasma,40% red cells,4% platelets,and 1% white blood cells(WBCs)[1].The five primary types of WBCs are acidophilus,lymphocytes,monocytes,basophils,and neutrophils.These blood cells contain nuclei that differ from those of other cells[2].WBC abnormalities are diagnosed by a blood smear test.Peripheral blood analysis is utilized for detection of diseases,such as malaria,leukemia,and anemia[3,4].Such disorders are revealed by an increase and decrease the number of WBCs in the human body.Variations occur in the morphological structure of blood cells in terms of color,shape,and size,and such variations aid in the diagnosis of abnormalities in the WBCs[5].Thus,segmentation and classification methods are used for the detection of WBCs.The manual evaluation of WBCs is laborious and time consuming[6],and computerized methods are a useful alternative that also minimize the workload of hematologists[7].Segmentation and classification of WBCs are performed using conventional and deep learning methodologies.In conventional approaches,features are extracted manually;however,in deep learning,images features are learned automatically through a pipeline to improve efficiency[8].In this study,an automated approach based on deep learning is proposed to segment and classify WBCs more accurately.The foremost contributions of the proposed work are as follows:

?The Open Neural Network Exchange(ONNX)is applied with a YOLOv2 model,which detects the different types of WBCs.The features are extracted using activation-5 of the ONNX model.The extracted features are fed to the YOLOv2 model.The proposed framework accurately detects the region of interest(ROI).

?The features are extracted using darknet-53,and the prominent features are selected based on Bhattacharyya separately criteria and fed to the shallow classifiers for the classification of WBCs.

2 Existing Literature

In the literature,significant work has been done for the detection of WBCs,and some of the recent works are discussed in this section[9,10].The detection of WBCs comprises four primary steps:pre-processing,localization/segmentation,extracting discriminant features,and classification.Pre-processing is a crucial step that is performed for noise removal and eradicating unwanted distortion to enhance the lesion region used in the subsequent segmentation step[11].Segmentation is another vital step;it is used to group the homogeneous pixels and segment the required region from the input images.WBC cells are difficult to segment because of variations in their appearance[12].Traditionally,WBCs were detected manually by pathologists,which is timeconsuming and can be inaccurate[13].Recently,automated approaches have been used for the detection of WBCs.Unsupervised clustering methods[14],thresholding approaches[15],shapebased approaches[16],and saliency-based models[17]are commonly used to localize WBCs.Watershed and histogram orientation approaches are used for the segmentation of WBCs.A large amount of data is presented into a set of vectors in the feature extraction process[18].Selection of the optimum diagnostic features is an important task for the detection of WBCs[19].Several types of features with different classifiers were used to differentiate the types of WBCs[20].Supervised methods,such as SVM,Bayesian,random forest[21],and Bayesian[22],are used for the classification of WBCs.However,even the best feature extraction and selection methods struggle with accurate classification” or something similar[23].Deep learning(DL)approaches are used widely to extract high-level information automatically[24]for the detection of ROIs,such as in WBC detection and classification[25].Contour aware neural networks are used to segment the WBCs.Pixel by pixel classification is performed using a fully convolutional neural network(FCN)[26].Mask R-CNN exhibits better classification as compared with other DL techniques[27].

3 Proposed Methodology

The proposed approach comprises localization,segmentation,high-level feature extraction/selection,and classification steps for the analysis of WBCs.In the proposed approach,WBCs are detected/localized using ONNX as the backbone of YOLOv2.The localized cells are segmented using the proposed 3-D semantic segmentation model.Finally,the WBCs are classified using multi-SVM.An overview of the proposed method is presented in Fig.1.

Figure 1:Proposed method architecture for WBCs localization and the segmentation

3.1 Localization of the WBCs

In this research,WBCs are recognized by the suggested WBC-ONNX-YOLOv2 model,as shown in Fig.2,where features are extricated from activation-5 LeakyReLU of the ONNX model.The extracted features are further fed to the YOLOv2 architecture.The proposed model has 26 layers in the ONNX model,namely 1 input,6 Conv,6 Bn,6 activation,2 elementwise-affine,and 5 max-pooling layers,and 9 YOLOv2 layers,namely 2 ReLU,2 Bn,2 Conv,1 classification,1 transform,and 1 output layer.

The layer-wise proposed model architecture is presented in Tab.1.

The proposed model is trained using selected parameters as reported in Tab.2.

It is trained on 100 epochs,because after 100 epochs,the model performance is almost stable.The number of iterations with the respective loss during training is illustrated graphically in Fig.3.

3.2 3D-Segmentation of the Leukocytes

The semantic segmentation model is proposed for the segmentation of WBCs,in which deeplabv3 is used as a bottleneck in the Xception model.The pre-trained Xception model contains 205 layers,comprising 1 input,88 2-D Conv,46 Bn,46 ReLU,3 max-pooling,12 addition,4 crop 2D,2 transpose Conv,2 depth Conv,softmax,and pixel classification layers.The segmentation model was trained from scratch on the blood smear images.The training parameters of the presented model are listed in Tab.3.

The proposed model learning with convolutional layers is plotted with activation units,as presented in Fig.4.

When the king saw how very young the prince looked, and that he was still drinking of the fountain of wonder, he said: O youth! leave aside this fancy which my daughter has conceived in the pride of her beauty

Figure 2:ONNX-YOLOv2 for multi-class detection

Table 1:The layered architecture of the proposed localization model

Table 2:Proposed localization model training parameters

Figure 3:Number of the iteration with respect to training loss

Table 3:Training parameters of the segmentation model

Figure 4:Segmentation model with activation units

3.3 Deep Features Extraction and Classification

The deep features are extracted using a pre-trained darknet53 model,which contains 184 layers,namely 1 input,53 Conv,1 global pooling,52 Bn,52 LeakyReLU,and 23 addition layers,and softmax with cross-entropy loss.The features are extracted from Conv53 layers with dimensions of 1×1000.The selection of informative features from a pool of features is difficult.Therefore,the Bhattacharyya rank-based feature selection approach is used,in which the optimum 500(50%)best features are selected out of 1000 features to improve the classification accuracy,also providing cost-effective and fast predictors.The best-selected features are further supplied to the multi-kernel SVM classifiers,such as Cubic-SVM,Quadratic SVM,O-SVM,and Gaussian SVM to classify the different types of blood cells,as depicted in Fig.5.

Figure 5:Feature extraction &selection and classification process

The SVM classifier with different kernels is trained on the best-selected feature vectors with optimum parameters,as listed in Tab.4.

Table 4:Parameters of SVM selection

Table 5:Localization results of different kinds of WBCs

Figure 6:Localization results on benchmark datasets(a)log average rate(b)average precision of different types of WBCs(c)IoU(d)average precision of blast cells

4 Experimental Setup

In this research,three publicly available benchmark datasets are used for the method evaluation.ALL-IDB1 contains 107 blood smear images,of which 33 are blasts and 74 are non-blast cells,and ALL-IDB2 contains 260 blood smear images,comprising 130 blasts and 130 non-blast cells[28–31].The LISC dataset contains blood smear images of WBCs,including eosinophils,neutrophils,monocytes,lymphocytes,and basophils.The numbers of images for each type of WBC are not equal.To balance the different types of imaging data of WBCs,data augmentation is performed by rotating the images at different angles,such as 45°,90°,180°,and 360°.After augmentation,6250 images of five types of WBCs are obtained,with each type having 1250 blood smear images[32].

4.1 Results&Discussion

The proposed work performance is validated by performing three experiments.The first experiment is performed to validate the presented localization technique by different metrics such as mean precision(mAP)and intersection over the union(IoU).The second experiment is validated to compute the segmentation model performance,while the third experiment is performed to compute the classification model performance.All experiments in this research are performed on the MATLAB 2020 Ra toolbox with 1050 K Nvidia Graphic Card.

4.2 Experiment#1:Localization of Leukocytes

Experiment 1 was performed to validate the performance of the localization approach on three benchmark datasets,LISC,ALL-IDB1,and ALL-IDB2,using IoU and mAP as metrics,as shown in Tab.5.In this experiment,six types of WBCs were localized,and the localization results are graphically depicted in Fig.6.

Figure 7:Localization results and corresponding confidence score of the proposed method on LISC dataset.Column(a)and(d)represent input images;(b)and(e)localization results;(c)and(f)confidence score

The localization outcomes in Tab.5 indicate that the method achieved the highest 0.97 IoU on blast cells.

The proposed method localizes the WBCs with confidence scores,as shown in Figs.7 and 8.

Figure 8:Localized region on LISC and ALL-IDB datasets(a)blast images(b)localization(c)confidence scores

4.3 Experiment 2:Segmentation of Leukocytes

In this experiment,the 3D segmented region is validated using different types of performance metrics,namely IoU,mean,weighted,and global accuracy,and F1-scores,as mentioned in Tab.6.The results of the proposed segmented WBCs are mapped pixel-by-pixel with ground annotated images,as illustrated in Fig.9.

Table 6:Segmentation results of the WBCs

Figure 9:3D-segmentation outcomes(a)WBCs(b)3D segmentation(c)ground annotated masks

The segmentation results in Tab.6 indicate that the proposed method achieved the highest segmentation accuracy,obtained by the pixel-by-pixel comparison of the segmented images with ground annotated images.

4.4 Experiment#3:Classification Based on the Extracted Feature

In this experiment,an optimized feature vector is fed to a multi-kernel SVM for WBC classification,and the outcomes are computed in terms of accuracy,precision,recall,and F1 scores from the LISC dataset,as displayed in Tabs.7–9.The discrimination outcomes on the LISC and ALL-IDB1&2 datasets with class labels are presented in Fig.10.

Table 7:WBCs classification results

Table 8:WBCs classification results

Table 9:WBCs classification results

A quantitative analysis is performed using an SVM with three different types of kernels,namely cubic,quadratic,and optimized.The SVM with the optimized kernel achieved a maximum overall accuracy of 98.4%.The classification results are also compared with the latest published work,as shown in Tab.10.

Tab.10 compares the classification results with the latest published existing work.The existing work achieved accuracies of 0.995,0.984,0.984,0.961,and 0.950 for lymphocytes,monocytes,basophils,eosinophils,and neutrophils,respectively.In contrast,the proposed method exhibited improved classification accuracy,with 0.996,1.00,0.987,1.00,and 0.988 for lymphocytes,monocytes,basophils,eosinophils,and neutrophils,respectively.

The classification results on the ALL-IDB1&2 datasets are presented in Tabs.11 and 12.

The classification results of blast/non-blast cells are presented in Tabs.11 and 12.An accuracy of 99.57% was achieved on the ALL-IDB1 dataset and 98.25% on the ALL-IDB2 dataset,and the results are compared with a recently published work,as provided in Tab.13.

Tab.13 presents a comparison of the numerical results,wherein the competitive results obtained from the proposed method are compared to those of the latest published work.

Figure 10:Confusion matrix(a)LISC dataset(b)ALL-IDB1 dataset

Table 10:Proposed work comparison with latest published work on LISC dataset

Table 11:WBCs classification results

Table 12:WBCs classification results

Table 13:Results comparison

5 Conclusion

In this study,deep learning approaches are proposed for the detection of WBCs.Detecting WBCs is challenging because blood smear images contain different color distributions in the cytoplasm and nucleus regions,making it difficult to segment these regions accurately.A 3-D semantic segmentation model is proposed,in which deeplabv3 is used as a bottleneck and the Xception model is used as a classification head to accurately segment the WBCs.Feature extraction/selection is another challenge for the classification of WBCs.The features are extracted from the pretrained darknet-53 model,and informative features are selected using Bhattacharyya separability criteria and passed to the SVM with different types of kernels,namely cubic,quadratic,and optimized.The proposed classification method achieved an accuracy of 99.57% on the ALL-IDB1 dataset,98.25% for the ALL-IDB2 dataset,and 98.4% for LISC datasets using the optimizable SVM kernel.The overall experimental outcomes demonstrate that the proposed technique achieved competitive outcomes by optimizing the SVM kernel.The proposed new framework based on a CNN can be used for the detection of different types of cancer,such as lung and bone cancer.It detects and classifies leukocytes at an early stage,thereby increasing the survival rate of patients.

Acknowledgement:This research was supported by Korea Institute for Advancement of Technology(KIAT).

Funding Statement:This research was supported by Korea Institute for Advancement of Technology(KIAT)grant funded by the Korea Government(MOTIE)(P0012724,The Competency Development Program for Industry Specialist)and the Soonchunhyang University Research Fund.

Conflicts of Interest:All authors declare that they have no conflicts of interest to report regarding the present study.