Intemational Journal of Computer Selanos and Information Sacurty (UCSIS), Vol. 17, No.2, February 2018 A Machine Learning Model for Hemoglobin Estimation and Anemia Classification El-Sayed M, Towfek EL-kenawy Department of Communications and Blectronies, Delta Higher Institute for Engineering &Technology (DHIET)), Mansoura, Egypt Email: drsayedkenawy @ gmail.com Keywords: CBC, machine learning, hemoglobin, regression, classification, Hematology Abstract-Hemoplobin (Hbg) is considered to have a vital role in human's body as it carries oxygen from lungs to the body's tissues and return carbon dioxide from tissues back to lungs. Estimation of Hg level is an important step for any blood analysis ‘task, Hb levels usually indicate that a person has anemia or not. In this study, we propose a Machine Learning model forthe ‘estimation of hemoglobin level and the classification of Anema based on blood test parameters. The dataset for training and ‘esting was collected from (Ahmed Shalaby lab using Mindray BC-S300) . In our proposed approach Hemoglobin Estimation and Anemia Classification (HEAC), preprocessing is done so that data i reduced and normalized before training the modes. “The results ofthis model is compared with results from diferent machine learning model, The proposed model produces a very accurate results Hemoglobin estimation and Anemia Classification, 1. Ivrropuctioy In the lat several decades, both of generating and collecting data have been increasing rapidly of imagination, so hhow to make sense of large amounts of mostly unsupervised data that is aim of data mining which is the computing process of discovering patterns in large data sets involving methods atthe intersection of machine learning, statistics, and database systems cause of the natural evolution of information technology, Data mining is being used in several applications like hospital, businesses, education, fraud detection and bioinformatics. in case of bioinformatics, helps physicians to extract useful knowledge from massive datasets gathered in biology and Patients to receive better and more affordable health services. In hematology laboratory, being & powerful tool in managing laboratory information of disease diagnosis Hematological date comment is a challenging and interesting task in medical research area so we propose model to make Compatison of Different Classification and regression Techniques using Scikit-Leata, We divide this problem of Hematology Data comments into four phases: Data Collection, data preprocessing, regression and (Classification algorithm our purpose is proceeding in the following ways: - collect hematological data from preprocessing hematological data ‘To apply hematological data in scikit-learn then find classification algorithms performance. Many related works using data mining techniques to diagnose several types of diseases have been introduced such as Automated Diagnosis of “Thalassemia Based on Data Mining Classifiers", Investigation for thalassemia depending on CBC was presented, this paper main considers MCV but we think They should need use Hemoglobin is the main feature to classified thalassemia [1] "Application of Data Mining Methods and Techniques for Diabetes Diagnosis, "In this paper a similar investigation of ‘various algorithms. This venture goes for mining the relationship in diabetes information for effective order. Be that as it may, they require proposed a model that can analyze diabetes dataset [2]. "Comparative Analysis of Data Mining Tools and Classification Techniques using WEKA in Medical Bioinformatics." [3] Considered the execution of Tree Random Forest, 148 choice tee, Naive Bayes and Lazy IBK. In this paper, they analyzed algorithms deepen on their accuracy, leaming time and error rate. They watched that there is an immediate connection between execution time in building the ‘ee model and the volume of information records, while there is Likewise a roundabout connection between execution time in building the model and the atsibute size of the informational collections, Through examination, they infer that Bayesian algorithms have better classification accuracy over and above compared algorithms [4]. 100 nps:iises.gooole.comistlesi! ISSN 1947-5500Intemational Journal of Computer Scisnos and Information Scuy (UCSIS), Vol. 17, No.2, February 2019 Al the previous related works tried to makes a model to diagnosis diesis, and most of them just try to use one data ‘mining technique, in this paper we introduce new model depend on more than one classifier to get mast significance one and diagnosis hematological data comments can be easily ‘our model summarized follow TL Dataset and Methodology In this section, we introduce the dataset that we have used in our machine learning experiments. In addition to that, the preprocessing procedures that we have made (o the data before doing the machine learning tasks will be discussed. As shown in figuee 1 Anemia Classification Module Hemoglobin Estimation Module Data Preprocessing Minaray BC-$800 Data Collection 401 nps:iistes.gooole-conisitlesi! ISSN 1947-5500Intemational Journal of Computer Selanos and Information Sacurty (UCSIS), Vol. 17, No.2, February 2018 A.CBC parameters background ‘The hematological dataset consists of the following parameters: White Blood Cell Count (WBC), Lymphocytes (LYM), Red Blood Cell Count (RBC), Mean Cellular Volume (MCV), Mean Cellular Hemoglobin (MCH), Mean Cellular Hemoglobin Concentration (MCHC), Red Blood Cell Distribution Width (RDW), Hematocrit (HCT), Platelet Count (PLT), Mean Platelet Volume (MPV) and Hemoglobin (HGB). These parameters aid in data mining tasks on the hematological data, Table [T] shows some example records of the design dataset. TABLET Sample of Hematological records wee [lym [Rac [Mev [Mi meuc [row [uct [rut [Mev [HGR 529 sis [oso faoo fais fis [sos [aor [a7 [iss saz [223 [sox [o27 [299 [aaa [isa fara [200 [79 [asa a1 [iss [sos [79 [asa |as9 [ize [soe [259 [92 [103 as [isa fas [ars [2a7 302 ise [ars [so 92 [sa B. Preprocessing The data were collected using Mindray BC-5300 Auto Hematology Analyzer which provides reliable and accurate five-part hematology results ftom just 20uL of blood. This analyzer saves more than 200 hematology parameter for each blood test In its internal records. Some of these parameters are calculated by the analyzer such as RBC, WBC, and PLT. Other parameters are inserted manually by the operator such at gender and age. We have eliminated some columns such as gender and age due to having many blank values for them. Those blank values occurred because the ‘operator dida’t save the personal information for each blood test as he was using another software to manage patient's data, The previously mentioned CBC parameters were selected catefully by a professional doctor who selected the mos {mportant parameters that affect the calculation of hemoglobin and classification of Anemia After selecting the appropriate features, we standardized the data to change the raw feature vectors into a representation that is more suitable for the different machine learning estimators. For this task, we used StandardScaler which standardize features by removing the mean and scaling to unit variance 66]. Many machine learning estimators ‘may behave badly if the individual feature does not more ot less look like standard normally distributed data. In addition to feature selection and standardization, we have divided the hematological dataset into two pars: taining data (75%) and testing data (25%). The original dataset contains 9004 records, The taining data contains 6753 records and the testing data ‘contains 2251 records. The taining data is used to train different machine learning estimators and the testing data is used to calculate the error and the accuracy of each estimator, II. Methodology and Toots ur proposed model (HEAC) bas two important stages: Regression and Classification, In the regression stage, ‘we are estimating the amount of Hemoglobin (HBG) based on the other hematological parameters. In the classification phase, we are using the estimated HBG along with other hematological parameters to classify the type of Anemia, For both stages, we are using open source software library called Scikit-Learn [5]. Scikit-leam is an efficient python tool for the tas of data mining and analysis, A. Regression (Estimation of Hemoglobin} Regression is the task of approximating a mapping function between from the input variables to the continous ‘output variable. The output in our case is hemoglobin (HBG) which is being estimated based on the input variables such 102 nps:iises.gooole.comistlesi! ISSN 1947-5500Intemational Journal of Computer Selanos and Information Sacurty (UCSIS), Vol. 17, No.2, February 2018 ‘as WBC, RBC, LYM, etc, For this task, we tested three different machine learning estimators: Random Forest, Linear Regression and Neural Networks Arifcial Neural Network Antificial Neural Network (ANN) isan information provessing paradigm inspired by biological neural networks. Nowadays, ANN is widely used in many applications in various aspects of engineering and science (6]. The neural network consists of multiple layers including input layer, output layer and hidden layer. Each layer consists of multiple neurons, Multi-Layer perceptron (MLP) is a type of feedforward neural networks that have one or more hidden layers between the input and output layers. MLP layers are fully connected which means that a neuron in one layer is connected to all neurons in the following layer. Each connection has a different weight value, MLP is trained with backpropagation algorithm. The backpropagation algorithm uses gradient descent to calculates the error contribution of each neuron after passing a batch of data to the network. Then it modifies the weights so that the network gives the expected output for a sive input, In this study, we used the hematological dataset to train a MLP using backpropagation learning algorithm, ‘The architecture of the network was designed as following: 1 input layer (10 neurons), | hidden layer (20 neurons), and 1 ‘output layer (1 neuron). The learning rate was set {0 0.001, We tested different activation functions and it seems that identity is most suitable activation function for this type of data, Practically, (wo known performance metrics was used to assess the prediction performance: the root mean square error (RMSE), and the mean absolute error (MAE). The network ‘was able to predict the hemoglobin with RMSE = 0.0954 and MAE = 0.27211 2 origina —e estimated 8 2 a 6 & 10 2 qa igus 2 Hemoglobin estinaton using Arial Neural Networks Linear Regression Linear regression is used to model the linear relationship between a dependent variable (which in our case is Hemoglobin Hgb) and one or more dependent variables (other hematological parameters). If we have only one dependent variable as an input, then itis called Simple Linear Regression, In ease we bave multiple input variables, then itis ealled ‘Multiple Linear Regression, Muliple Linear Regression is bated on Ordinary Least Squares (OLS) which wes to ft the model by minimizing the sum-of-square of the difference between the actual (observed) value and the estimated (predicted) value. The linear regression model was able to predict the hemoglobin value with RMSE = 0.0137. Figure [Fig.2] illustrates the hemoglobin estimation using linear egression model 402 nps:iises.gooole.comistlesi! ISSN 1947-5500Intemational Journal of Computer Scisnos and Information Securty (UCSIS), Vol. 7, No.2, February 2018 Random Forest In Machine Learning, combining predictions of several base estimator is called Ensemble Methods [7], The main goal ‘of Ensemble Methods is to improve generalizability and robustness over a single estimator. Random Forest is type of Ensemble methods which consists of a combination of decision tee predictors [8]. Decision Wee is a supervised learning algorithm learns from data and build a tee structure, The leaves of the tre isthe value or the class that was learned from the data, Random Forest Algorithm combines multiple decision trees to improve the generalizability and the robusiness ‘over a single decision tree. The Random Forest was able (o predict the value of hemoglobin with the lowest error in ‘comparison with other estimators (RMSE = 0.0123). Figure [Fig-3] illustrates the Hemoglobin estimation using Random Forest Algorithin =e original © estimated 1s 14 13 a2 aa 10 oo 2's 5.0 7s 100 125 180 175 igue 3 Hewogloiaeainaton wing Random Forest 104 ntpsstes.gooale comics! ISSN 1947-5500Intemational Journal of Computer Scisnos and Information Sacuty (LUCSIS), Vol 17, No.2, February 2018 ‘Table [T2] lists the results of hemoglobin estimation using Random Forest, Linear Regression and Neural Networks. As shown in the table, Random Forest is giving the lowest error compared to Linear Regression and Neural Networks. This is because Random forest can control overfiting and handle missing values, Chart [chl] illustrates the results of Hemoglobin estimation using differen estimators ‘Table Il Hemoglobin estimation results Random Forest lo.oassosesaoe lo.orzsga24ss6 Linear Regression o.07500467366 [o.01372390075 [antfcial Neural Networks loz7anissase lo.o9sado116s MAE & RMSE for different classifiers MAE mRMSE os oas o2 os < 5 oa S g © 0.08 | 5 3 ° |_| — — Random Forest Linear Regression Arial Neural Networks Tiger « Hemorlobinesiation reve B. Classification of Anemia Classification isthe task of predicting the category ofthe target variable. In this case, we are predicting the class ff Anemia based on other hematological parameters. For this task, we tested four diferent machine learning classifiers Decision Tree, Naive Bayes and Neural Network. In addition to that, we test a new model that is a combination of the best tee classifiers. The combined model bas the best accuracy and the lowest error, Decision Tree 108 ntpsstes. google comics! ISSN 1947-5500Intemational Journal of Computer Selanos and Information Sacurty (UCSIS), Vol. 17, No.2, February 2018 ‘As mentioned in section 3.1.3, Decision Tree isa supervised learning algorithm which used the training data to build a ‘use structure mode, The resulting tee leaves are the classes that we are going to predict[9]. We have tested the Decision ‘Troe classifier using Scikit-Learn and the error (RMSE) was 0.1394 Naive Bayes [Naive Bayes is a supervised learning algorithm that is based on Bayes" theorem with the assumption of independence between each pair of features, Naive Bayes calculates the posterior probability P(elx) from P(e), Pix) and Pixe)[10 Equation 2 illustrates the naive bayes algorithm: pai. ro, Peele) = Sas Equation 2) Where (chs) is the posterior probability ofa class given a predictor (xk) is the probability of the predictor given a class (6) isthe probability ofa class P(x) is the probability of a predictor Naive Bayes classifier assumes that there is no relation between the effect of the value of the predictor on a given lass and the values of other predictors. We used Scikit-Learn library to test Naive Bayes algorithm on our hematological dataset and the RMSE error was 3.0424 with score = 83.169% [Neural Nenworks ‘As mentioned in section 3.1.1, Neural Network algorithm is inspired by the biological neural networks. It can be used {or both regression and classification tasks [11], For classification of Anemia, we used SciKit-Learn to create a 3-layer neural network: Input (13 neurons), Hidden (20 neurons) and Output (1 neuron). The neural network error was 4.44 and the seore = 76.34% Hybrid Classifier For the purpose of improving generalizability and robustness ofthe prediction of the clase of Anemia, we combined the tece classifiers and calculated the class of Anemia based on the weight of each classifier. Figure [Fig-4] illustrates the proposed model structure. class of Anemia gute proposed model (combined estimators) for Anca Classifiations 108 nps:iises.gooole.comistlesi! ISSN 1947-5500Internationa Journal of Computer Selanos and Information Sacurty (UCSIS), Vol. 17, No.2, February 2018 As show in the figure, we combined the three classifiers: Decision Tree (DT), Naive Bayes (NB) and Neural [Networks (NN}, For each classifier we have assigned a specific weight. The weight are 6, 2 and 1 for DT, NB and NN in order. These weights were assigned based on the score of each classifier and using trial and error till we got the most suitable weights that make the accuracy higher than all of single classifiers and the error is lower than any error of other classifiers We used Scikit-Learn library to build our combined model and calculated the score using Weighted average probabilities. Weighted Average Probabilities or soft voting is calculated by multiplying the predicted class probabilities for each classifier with the assigned weight of the classifier and then averaged. The final class (estimated class) is then derived from the class with highest average probability Figure [Fig-S] illustrates the pseudocode for the proposed model. As show in the figure we stat by fitting the three single classifies: Decision Tree, Naive Bayes and Neural Network. Then we compute the probabilities foreach class ‘of anemia foreach classifier, After that we compute the average probability for each class of Anemia, The selected output lass is the class that has the highest average probability Fit Decision Tree to the training data Compute the probabilities for each class for Decision Tree Fit Naive Bayes to the training data Computer the probabilities for each class for Naive Bayes Fit Neural Network to the training data Computer the probabilities for each class for Neural Network Foreach class of Anemia Compute the average probability End Foreach 10. Select the output class based on highest average probability eeu aneune Figue 6Poeudcode forthe combined moved ‘The hybrid model always proved to produce a better accuracy. Diwan and Li Zhang [x5] proposed a hybrid model of Decision Tree and Naive Bayes classifiers and their model produced impressive results in the classification of real life challenging classification problems (ur proposed model (combined classifiers) has the highest score and the smallest error value compared to olber single clasifiers ag show in table [73] ‘Table ll Anemia classification results Classifier RMSE MAE Score Neural Nework 4.ac0108 0867660 0.715643 | Naive Bayes 3.042416 0.601968 0.831693 | Decision Tree 0.139463, 0.022056. 0.995928 Hybrid Classifier (HEAC ) 083814 0.015948 0.996607 IV. Conclusion 407 npssiises google. comisitlesis! ISSN 1947-5500Intemational Journal of Computer Scienos and Information Securty (LUCSIS), Vol. 17, No.2, February 2018 In this research, we did two machine learning tasks: regression and classification, For the regression, we are estimating the value of Hemoglobin using hematological parameters. We used the following three machine leaming algorithms: Random Forest, Linear Regression and Artificial Neural Networks. Random forest produced the lowest erro. After that ‘we use the hematological parameters along with the estimated hemoglobin value to classify the type of Anemia. Decision ‘Tree proved to be the winner classifier compared with Naive Bayes, Random Forest and Arlficial Neural Networks. Finally, we proposed an ensemble model (HEAC) which is a comiination of Decision Tree clasifier, Naive Bayes and ‘Random Forest. The proposed model had a belle accuracy compared to Decision Tree and other classifier. ‘As a future work, instead of using tril and error to calculate the best weights for our proposed model (HEAC), we will use Genetic Algorithm to find the best weights that make our proposed madel classify anemia with the highest accuracy. (1) Blan HL Albless. A.M. 2012), AuonmatedDiaghsis of Thalassemia Based on DuaMiniag Classis Ia the Iernatoal Conference ‘on Informatie’ and Applications ICLA2012) pp 440-45). The Society of Digillfognation and Wieess Communion 1p] Rajesh Kan V. Sangeetha. "Appiation of da mining metbods and techniques or dabses agnosis Inertia Foutal of Enginerog and “Inyovatve Tecnology (UETT Volume. 2. se 3 (2012) Ib] Bevig Satsh Kamar, Ars TM Sach, and Kha Al Rabeos, “Comparatve Anlvis of Data Mining Toole and Caesieation Techniques wing ‘WEKA se Medical Bionfrmates” Conpater Engincering ad lneliget Systems no 13 2013) 26-38 [a] Nb. 2, 2016 Machine Leung at Seale In Pro Spark Seurang (pp. 177-198). pees, Beckles, CA. [s| Sele: Machine Leasing in Pion, Pedegoe ot a, IMLR 1p. 2925-2690, 2011, [61 ale, Neral Nework and Leaming Machines, 200, Savon Pearon Beaton, ne, New eve [7] Zhe, 711, 2015 Ensembl leaming Eneslopedis of biometrics, pp 411-115 [8] Beeman. 2001. Random ores, Machine leaning, SI), 9p332 [9] Band DM, Zhang L. Rabun, CM. Hossa, MA and Strachan R. 2014, Hyd deeison ue and naive Bayes cases formulae ‘asfcaton asks. Bape stems wit Applications, 418), pp. 1937-1946. [10] Gunde K, Konvar, V Sabin, Sy Chowdhary, nd Mute. 2016. A comparative sag on tyroid diese detection wring Koearest ‘eihbor and Naive Bayes cation tehniqaee CST aneaeton on ICT, 42-4) pp 313-319 [11] Kevin P Murphy “Machine Leung” A Probable Perspective" Te MIT Pres chapter 14.3, pp 492-493 108 tps ses. google comics! ISSN 1947-5500