Examining different cost ratio frameworks for decision rule machine learning algorithms in diagnostic application

Abstract

BACKGROUND:

Artificial Intelligence (AI) plays a pivotal role in the diagnosis of health conditions ranging from general well-being to critical health issues. In the realm of health diagnostics, an often overlooked but critical aspect is the consideration of cost-sensitive learning, a facet that this study prioritizes over the non-invasive nature of the diagnostic process whereas the other standard metrics such as accuracy and sensitivity reflect weakness in error profile.

OBJECTIVE:

This research aims to investigate the total cost of misclassification (Total Cost) by decision rule Machine Learning (ML) algorithms implemented in Java platforms such as DecisionTable, JRip, OneR, and PART. An augmented dataset with conjunctiva images along candidates’ demographic and anthropometric features under supervised learning is considered with a specific emphasis on cost-sensitive classification.

METHODS:

The opted decision rule classifiers use the text features, additionally the image feature ‘a* value of CIELAB color space’ extracted from the conjunctiva digital images as input attributes. The pre-processing consists of amalgamating text and image features on a uniform scale, normalizing. Then the 10-fold cross-validation enables the classification of samples into two categories: the presence or absence of the anemia. This study utilizes the Cost Ratio ( $\rho$ ) extracted from the cost matrix to meticulously monitor the Total Cost in four different cost ratio methodologies namely Uniform (U), Uniform Inverted (UI), Non-Uniform (NU), and Non-Uniform Inverted (NUI).

RESULTS:

It has been established that the PART classifier stands out as the top performer in this binary classification task, yielding the lowest mean total cost of 629.9 compared to other selected classifiers. Moreover, it demonstrates a comparatively lower standard deviation 335.9, and lower total cost across all four different cost ratio methodologies. The ranking of algorithm performance goes as follows: PART, JRIP, DecisionTable, and OneR.

CONCLUSION:

The significance of adopting a cost-sensitive learning approach is emphasized showing the PART classifier’s consistent performance within the proposed framework for learning the anemia dataset. This emphasis on cost-sensitive learning not only enhances the recommendations in diagnosis but also holds the potential for substantial cost savings and makes it a noteworthy focal point in the advancement of AI-driven health care.

Keywords

Cost matrix cost ratio cost sensitive classifier decision rule classifier DecisionTable JRip OneR PART

1. Introduction

The advancements in technology go beyond providing medical data for diagnosis; they play a multifaceted role in enhancing various aspects of healthcare [1, 2]. In recent years, Machine Learning (ML) and Deep Learning (DL) models are more supportive to humans in most of the decision making process [3, 4, 5]. The conventional way of assessing the performance of ML or DL will be done mainly based on its classification accuracy assuming that the misclassification costs (false negative and false positive cost) are the same. However in most of the critical applications like fraud detection, cancer diagnosis, and bank loan problems the misclassification of the sample in each class will create a different impact because of a variety of reasons. For example, in loan problems false positives will cost more than false negatives [6]. In certain scenarios, such as the diagnosis of cancer, misclassifications can have significantly different consequences. For instance, a false negative (missed diagnosis) can be extremely serious as it may lead to delayed treatment and potentially cost the patient their life [7, 8]. On the other hand, a false positive (false alarm) may result in unnecessary concern and further tests, but it is less severe in comparison. In our processes, we have recognized the importance of addressing this discrepancy and have therefore reevaluated our equations and parameters for cost calculation, particularly when it comes to materials and resources involved in the classification process.

Let us consider the AI based non-invasive anemia classification problem. Anemia is a prevalent condition characterized by a decrease in the number of red blood cells or a decrease in their ability to carry oxygen effectively. It affects a substantial portion of the global population, with estimates suggesting that around a quarter of the world’s population is affected. The World Health Organization (WHO) identified anemia as a global health concern with significant prevalence and consequences [9, 10]. Diagnosis of anemia typically involves measuring hemoglobin levels through blood tests and evaluating additional parameters such as red blood cell count and morphology. Early diagnosis of anemia helps healthcare professionals to implement targeted interventions and management, mitigating the adverse effects on individuals’ health and well-being [11]. Non-invasive methods, such as conjunctival examination and digital image analysis, have emerged as promising alternatives. These techniques offer the potential for quick and cost-effective screening, making anemia diagnosis more accessible, particularly in resource-limited settings, and facilitating timely intervention and treatment. The research for developing a low cost and non-invasive diagnostic tool/device started a few decades ago. But, it is very important to consider the total cost for misclassification rather than the model development/device design cost. In non-invasive anemia screening if the test participant is misclassified as false positive, he/she will consult the health professional followed by a golden standard hemoglobin test to confirm, whereas the false negative will make either the test person to believe that he/she is normal or to check for other disease conditions with an additional test. This will make the anemic condition worse and result in additional test cost, treatment cost such as blood transfusion, long recovery period etc. The misclassification cost of False Negative is always higher than False Positives for sensitive applications like anemia diagnosis.

The proposed work evaluates the total cost involved in the misclassification of a sensitive application, non-invasive anemia diagnosis classified by rule based classifiers which was motivated by the work done by Kumaravel et al. [12]. Here, they explored the performance metric, total cost of misclassification through the cost ratio ( $\rho$ ) defined in terms of false positive and false negative frequencies in the context of IVF diagnosis using Tree cassifiers. They proposed four partitions of cost ratios namely Uniform (U), Uniform Inverted (UI), Non Uniform (NU), Non Uniform Inverted (NUI).

This article is structured as follows: Section 2 provides an overview of previous research conducted in the field of cost-sensitive classification and non-invasive anemia diagnosis. In Section 3, the materials and methods required for the proposed work are described. This is followed by the presentation of experimental results and corresponding discussions. Finally, the paper concludes with a summary of findings and key insights.

2. Related works

Cost-sensitive classification is a critical area of research in machine learning and data mining, aiming to improve the performance of classifiers by incorporating the varying costs associated with misclassification errors. Mienye et al. investigated the strength of cost-sensitive learning approaches specifically in the context of imbalanced datasets [13]. They utilized conventional machine learning methods to explore the effectiveness of cost-sensitive techniques. Their study shed light on the potential benefits of considering the varying costs associated with misclassification errors when dealing with imbalanced data. Telikani et al. delved into the domain of cost-sensitive classification using deep learning techniques [14]. They proposed a framework that involved partitioning the dataset and defining a separate cost matrix for each component. By incorporating cost-sensitive classification into the deep learning framework, they aimed to improve the overall performance of the models. While these studies primarily focused on incorporating cost sensitivity into the learning process, it is important to note that misclassification costs are not always known in many imbalanced dataset contexts. Therefore, cost-sensitive learning techniques have emerged as a valuable approach, as they take into account the misclassification costs during model construction without directly modifying the imbalanced data distribution. Assigning distinct costs to training examples has proven to be an effective approach for addressing class imbalance issues. Previous studies [15, 16] demonstrated the impact of varying the cost ratio ( $\rho$ ) uniformly on the total cost. Their findings highlighted the importance of considering cost ratios when dealing with cost-sensitive classification tasks. Additionally, Domingos [17] showcased the benefits of the meta-cost procedure in reducing costs associated with misclassification errors. Building upon previous work, our framework differs by incorporating multiple classifiers instead of relying on a single classifier, as demonstrated in the study by Thai-Nghe et al. [18].

Even though there are numerous works carried out in biomedical applications the significance of misclassification cost is very less bothered. Application of ML in the domain of ophthalmology for studying diabetic retinopathy detection was explored by Vijayan et al. (2020, 2023). The research to find the possibility of anemia screening through conjunctival examination started non-invasive anemia diagnosis a few decades ago. Previous studies [19, 20, 21, 22] assessed the effectiveness of physical examination of conjunctival pallor in anemia diagnosis. Suner et al. [23] used conjunctiva images photographed along with the 18% gray photographic standard card for non-invasive anemia screening and attained 55% sensitivity and 70% specificity. Collings et al. [24] made use of digital image acquisition devices for capturing the eye images along with a color calibration card closer to it. The impact of ambient lighting conditions on palpebral conjunctival Erythema Index (EI) was examined by a two-way analysis of variances (ANOVA) on values of pre and post standardization. This study found the Erythema Index of the palpebral conjunctiva correlated more heavily with hemoglobin concentration when compared to forniceal conjunctiva EI. Bevilacqua et al. [25] developed a head-mounted non-invasive device for capturing the conjunctiva region and to minimize the influence of ambient light. This research aims at decreasing false negatives instead of total performance. This research also revealed a* value of CIELAB color space of the manually segmented conjunctiva region, shows good correlation with both hemoglobin and Hematocrit values [26]. Muthalagu et al. [27] applied Feed Forward Neural Networks (FFNN) and Elman Neural Networks (ENN) for categorizing anemic and non-anemic cases from conjunctiva images. This study showed that ENN performs better classification than the FFNN and EI model with sensitivity of 77% and specificity of 96.11%. Dimauro et al. [28] used SMOTE and ROSE algorithms to address the class imbalance issue.

In all the aforementioned works related to non-invasive anemia diagnosis, the focus has primarily been on metrics such as accuracy and sensitivity, with insufficient consideration given to the costs associated with misclassifications [29]. In this proposed framework, the inclusion of novel metrics, namely Cost Ratio ( $\rho$ ) applied with four different methodologies (U, UI, NU, and NUI), and Total Cost is considered to reduce the loss in prediction measurements. This encourages a novel approach to the anemia classification task.

3. Materials and methods

3.1 Data description

The data utilized in this study was gathered in real-time, as stated in the study by Kasiviswanathan et al. [30]. It comprises 974 samples containing various attributes such as age, gender, height, weight, and the a* value of the L*a*b* colorspace defined International Commission on Illumination (CIELAB) derived from images of conjunctiva region [26]. The statistical information of the data, its distribution are given in Table 1 and Fig. 1 respectively.

Table 1
Statistical information of the anemia dataset

Description	Age (in years)	Height (in cm)	Weight (in kg)	a* value
Min. value	18	142	41	130.95
Max. value	73	195	105	173.99
Average	34.92	164.26	67.91	146.94
Standard deviation	13.33	10.50	12.66	7.39
Median	32	165	66	146.12
25th percentile	24	155	59.5	141.06
75th percentile	42	172	75	151.93

Figure 1.

Box plot of input features (left), distribution of male and female samples (top right) and distribution of output classes (top left).

3.2 Data preprocessing

A set of selected attributes, comprising demographic factors such as age and sex, and anthropometric features like height and weight, along with the ‘a*’ value of the CIE Lab color space, are chosen as input features due to their notable correlation with the anemic status. Previous research in the related works section highlights the significance of the a* value derived from conjunctiva images in non-invasive anemia diagnosis [25, 30, 31]. To enhance the classification performance, normalization is applied since each input attribute exists within a different range. The values are normalized using Min-max normalization, as described by Eq. (1), to rescale them to a range of (0, 1).

$\displaystyle x_{\textit{norm}}=(x-x_{\min})/(x_{\max}-x_{\min})$ (1)

where $x$ represents the original value of a given attribute before normalization, $x_{\textit{norm}}$ represents the normalized value of the attribute, $x_{\min}$ , and $x_{\max}$ are the minimum and maximum value of the attribute in the dataset. This normalization process ensures that all input attributes contribute effectively to the classification task.

3.3 Cost sensitive classifier (CSC)

A cost-sensitive classifier differs from regular machine learning algorithms in its approach to handling misclassification errors. While regular ML algorithms prioritize overall accuracy without considering the varying costs associated with different types of misclassifications, cost-sensitive classifiers explicitly incorporate these costs into the learning process. They assign different costs or weights to different types of misclassifications based on the specific domain or problem at hand. By incorporating cost information, cost-sensitive classifiers make more informed decisions that minimize the expected total cost of misclassification. This is achieved by prioritizing certain classes or minimizing misclassifications that have higher costs or consequences. Cost-sensitive classifiers often require additional inputs, such as cost matrices or misclassification cost estimates, to account for the varying costs. In summary, cost-sensitive classifiers optimize classification outcomes by explicitly considering the costs associated with misclassifications, while regular ML algorithms focus primarily on overall accuracy [12, 32]. For the proposed cost sensitive classification framework the decision rule classifiers such as Decision Table, JRip, OneR, and PART are considered [33].

3.3.1 Decision rule classifiers

Decision rule classifiers are a type of machine learning algorithm that use explicit rules to make predictions or classify data instances. These classifiers operate by defining a set of rules based on features or attributes of the data, and each rule corresponds to a specific class label or outcome. Even though the decision trees and decision rules are functionally equivalent, the rule based algorithms are considered due to their nature of linearity contributing to the readability and storage efficiency as in the inference mechanism based on knowledge base or fuzzy systems. Metrics based on confidence and support are the byproduct or automatic outputs from the decision rule algorithms. The following decision rule algorithms are trained and tested for the above discussed dataset.

a) DecisionTable Classifier

DecisionTable [34] provides a compact and expressive representation using a tabular format. They capture combinations of input conditions and corresponding actions or outcomes, enabling efficient rule-based decision-making. Decision tables are known for their ease of interpretation and implementation, making complex decision-making logic more manageable.

b) JRip Classifier

Jrip [35] is a classification algorithm implemented in the Weka machine learning toolkit. It stands for “Repeated Incremental Pruning to Produce Error Reduction (RIPPER)”, a rule-based classifier that builds a set of rules to make predictions on new instances.It is known for its efficiency and interpretability, as the resulting rule set can be easily understood and analyzed. It has been widely used in various domains, including medical diagnosis, bioinformatics, and customer churn prediction.

c) PART Classifier

The PART [36] algorithm also known as Partial C4.5 Decision Tree algorithm constructs a set of rules that cover different instances in the dataset. These rules are generated by considering various attributes and their thresholds to partition the data. The resulting rule set can be used for classification, providing human-readable rules that offer insights into the decision-making process. By utilizing the PART algorithm in Weka, you can leverage its rule-based approach to create interpretable models that provide insights and transparency in classification tasks.

d) OneR Classifier

The OneR [37] algorithm is implemented as a rule-based classifier. It aims to create a simple and interpretable classification model based on a single attribute, referred to as the “One Rule.” The OneR algorithm evaluates each attribute individually and selects the attribute that yields the best classification accuracy for prediction. This approach allows for easy understanding and explanation of the decision-making process. However, it’s important to note that the One Rule model may not capture complex relationships between attributes, and its performance might be limited compared to more sophisticated classifiers.

3.3.2 Cost sensitive learning

In the context of cost-sensitive classification, the terms “CFP” and “CFN” typically refer to the costs associated with false positives and false negatives, respectively. These costs represent the specific misclassification costs for each type of error. To incorporate these costs into a formula, we can use the following notation:

CFP: Cost of a False Positive CFN: Cost of a False Negative

Table 2 displays the confusion matrix, providing insights into the False Positive (FP) and False Negative rates associated with the anemia classification task. In this classification, the anemic condition is regarded as the positive class, while the non-anemic condition is assigned to the negative class.

Table 2
Confusion matrix

Actual class	Predicted class
	Anemic	Non-anemic
Anemic	TP	FN
Non-anemic	FP	TN

A common approach is to introduce a cost matrix, where each entry represents the cost associated with misclassifying a true instance of one class as a predicted instance of another class. In this case, the cost matrix for a binary classification problem with classes “positive” and “negative” can be defined in tabular form as shown in Table 3.

Table 3

Cost matrix

Actual class	Predicted class
	Positive	Negative
Positive	0 (C₁₁)	CFN (C₁₂)
Negative	CFP (C₂₁)	0 (C₂₂)

The notations C₁₁, C₁₂, C₂₁, and C₂₂ are the elements of the Cost Matrix. The formula to calculate the total cost using the cost matrix and the number of false positives (FP) and false negatives (FN) would be:

$\displaystyle\text{Total Cost}=(\text{CFP}*\text{FP})+(\text{CFN}*\text{FN})$ (2)

The formula presented in Eq. (2) allows you to quantify the total cost by multiplying the respective misclassification costs with the corresponding number of false positives and false negatives. It’s important to note that the specific costs (CFP and CFN) and the structure of the cost matrix may vary depending on the problem and domain. These values are typically determined based on the specific context and the relative importance or impact of each type of misclassification in the given application.

The proposed work aims to implement a cost-sensitive classification approach that incorporates four different types of Cost Ratio (U, UI, NU and NUI). The workflow of the proposed work involves a set of key steps as shown in Fig. 2. Firstly, the input data undergoes min-max normalization preprocessing technique to ensure that the data is in a suitable format and contains relevant features for classification.

Figure 2.

Workflow of the proposed cost sensitive classification approach.

The main process contains two components: Classifying and Cost Checking in each iteration. The first component is based on cost matrix-controlled cost-sensitive learning. This classifier takes into account the specific cost ratios derived from the cost matrix. This allows the classifier to assign different costs to misclassifications based on the importance and impact of each class output of an instance. Finally, the total cost of misclassification is found out.

3.4 Proposed algorithm for cost sensitive classifier by cost ratio

The main focus of the algorithm revolves around the cost ratio ( $\rho$ ), the ratio between CFP and CFN. With respect to the cost matrix the algorithm revolves around the ratio C₂₁/C₁₂ and its inverse, which can be adjusted through uniform increment or relative prime steps, resulting in four different styles. The primary objective of this process is to identify an acceptable ratio as it varies across different values in these four styles. To achieve this, a mapping $\zeta$ is constructed from the set of rational numbers (Q) to the set of real numbers (R), indicating the associated cost. Specifically, $\zeta(x)=$ c signifies a ratio of false positives (FP) to false negatives (FN), where x is a ratinal number in Q and c represents the corresponding cost. Previous studies [38, 39, 40] provide further insight into this mapping. In this study, three types of variations for x in Q are adopted, which are distinguished by four specific cases as described below.

Case 1. x takes the form 1/y where y is non negative integer. Case 2. x takes the form p/q where p, q are non-negative integers and gcd (p, q) $=$ 1. Case 3. Reciprocal of x in case 1. Case 4. Reciprocal of x in case 2.

These four cases encompass all possible values of x in Q, representing the FP: FN ratio. The selection of these cases is guided by number theory findings discussed in previous studies [15, 17, 18].

This study employs the algorithm depicted in Fig. 3 to construct cost-sensitive classifiers, which incorporate the algorithm components described earlier. The construction of these classifiers is carried out using the Weka tool [33] by adjusting the ratio $\rho$ . The interrelated four components of the main algorithm are described below where the prefixes CSC represent Cost Sensitive Classifier, and the suffixes U, UI, NU, and NUI correspond to the uniform, uniform inverse, non-uniform, and non-uniform invers components, respectively [12, 32].

a) Component for CSC-U

The cost value for CSC-U can be determined with the Cost Ratio ( $\rho$ ) which has a fixed numerator (CFP $=$ 1) and uniformly incremented denominator from 1 to 50. (CFN $=$ i, i.e. $i\in\{1,2,..,50\}$ ).

b) Component for CSC-UI

This is similar to that of CSC-UI by just inversing Components for CSC-U where CFP $=$ i, i.e. $i\in\{1,2,..,50\}$ and CFN $=$ 1.

c) Component for CSC-NU

The above algorithm is applied with the ratio $\rho$ as m: n where m & n are relatively primes and their values are within the principle range 1 to 50. The only extra complexity for a non-uniform cost algorithm is testing the relative primality condition by the greatest common divisor that is gcd (m, n) $=$ 1.

d) Component for CSC-NUI

From the ratio $\rho$ calculated by CFP/CFN, we consider its inverse and repeat the classification process to get the new results for both case uniform and non-uniform through their index in non-decreasing order. Hence we obtain the non-uniform cost algorithm namely algorithm NUI by simply inverting the loop indices in the algorithm for NU.

Figure 3 outlines the steps of the algorithm components, indicated by annotations in ‘[…]’, while the remaining steps common to all four cases are described in the rest of the figure. Dataset X comprises the underlying data instances mentioned in Section 3.1 followed by the definition of confusion matrix and cost matrix. The variable bi represents a list of decision rule classifiers, specifically chosen from the set {DecisionTable, JRip, OneR, PART}. The algorithm consists of two nested loops: the outer loop iterates over the selected decision rule classifiers, while the inner loop iterates over the index i variation, which corresponds to the cost ratio extracted from the cost matrix. Once the loop variants are fixed for the current iteration, the “TC” procedure is called to handle the training, testing, classification, and calculation of the total cost involved. The final output is determined by selecting the optimal value from the set of total costs generated by the aforementioned iterations.

Figure 3.

Pseudo code for the proposed cost ratio controlled cost sensitive classifier (CSC) framework for anemia diagnosis.

Table 4

Total cost for anemia diagnosing using CSC-U

CFP:CFN ( $\rho$ )	Total cost (CSC-U)
	DecisionTable	JRip	OneR	PART
1:1	133	96	191	33
1:2	170	133	259	51
1:3	207	170	327	69
1:4	244	207	395	87
1:5	281	244	463	105
1:6	318	281	531	123
1:7	355	318	599	141
1:8	392	355	667	159
1:9	429	392	735	177
1:10	466	429	803	195
1:11	503	466	871	213
1:12	540	503	939	231
1:13	577	540	1007	249
1:14	614	577	1075	267
1:15	651	614	1143	285
1:16	688	651	1211	303
1:17	725	688	1279	321
1:18	762	725	1347	339
1:19	799	762	1415	357
1:20	836	799	1483	375
1:21	873	836	1551	393
1:22	910	873	1619	411
1:23	947	910	1687	429
1:24	984	947	1755	447
1:25	1021	984	1823	465
1:26	1058	1021	1891	483
1:27	1095	1058	1959	501
1:28	1132	1095	2027	519
1:29	1169	1132	2095	537
1:30	1206	1169	2163	555
1:31	1243	1206	2231	573
1:32	1280	1243	2299	591
1:33	1317	1280	2367	609
1:34	1354	1317	2435	627
1:35	1391	1354	2503	645
1:36	1428	1391	2571	663
1:37	1465	1428	2639	681
1:38	1502	1465	2707	699
1:39	1539	1502	2775	717
1:40	1576	1539	2843	735
1:41	1613	1576	2911	753
1:42	1650	1613	2979	771
1:43	1687	1650	3047	789
1:44	1724	1687	3115	807
1:45	1761	1724	3183	825
1:46	1798	1761	3251	843
1:47	1835	1798	3319	861
1:48	1872	1835	3387	879
1:49	1909	1872	3455	897
1:50	1946	1909	3523	915

Table 5

Total cost for anemia diagnosing using CSC-UI

CFP:CFN ( $\rho$ )	Total cost (CSC-UI)
	DecisionTable	JRip	OneR	PART
1:1	133	96	191	33
2:1	229	155	314	48
3:1	325	214	437	63
4:1	421	273	560	78
5:1	517	332	683	93
6:1	613	391	806	108
7:1	709	450	929	123
8:1	805	509	1052	138
9:1	901	568	1175	153
10:1	997	627	1298	168
11:1	1093	686	1421	183
12:1	1189	745	1544	198
13:1	1285	804	1667	213
14:1	1381	863	1790	228
15:1	1477	922	1913	243
16:1	1573	981	2036	258
17:1	1669	1040	2159	273
18:1	1765	1099	2282	288
19:1	1861	1158	2405	303
20:1	1957	1217	2528	318
21:1	2053	1276	2651	333
22:1	2149	1335	2774	348
23:1	2245	1394	2897	363
24:1	2341	1453	3020	378
25:1	2437	1512	3143	393
26:1	2533	1571	3266	408
27:1	2629	1630	3389	423
28:1	2725	1689	3512	438
29:1	2821	1748	3635	453
30:1	2917	1807	3758	468
31:1	3013	1866	3881	483
32:1	3109	1925	4004	498
33:1	3205	1984	4127	513
34:1	3301	2043	4250	528
35:1	3397	2102	4373	543
36:1	3493	2161	4496	558
37:1	3589	2220	4619	573
38:1	3685	2279	4742	588
39:1	3781	2338	4865	603
40:1	3877	2397	4988	618
41:1	3973	2456	5111	633
42:1	4069	2515	5234	648
43:1	4165	2574	5357	663
44:1	4261	2633	5480	678
45:1	4357	2692	5603	693
46:1	4453	2751	5726	708
47:1	4549	2810	5849	723
48:1	4645	2869	5972	738
49:1	4741	2928	6095	753
50:1	4837	2987	6218	768

Figure 4.

Performance of decision rule classifiers for CSC-U.

Figure 5.

Performance of decision rule classifiers for CSC-UI.

The algorithm proposed in this study was implemented on the anemia dataset, and the obtained results were organized and presented in the form of tables and plots to facilitate a comparison of the performance of the decision rule classifiers during the 10-fold Cross Validation test based on total cost. Table 4 and Fig. 4 display the total cost generated by the decision rule classifiers for CSC-U (Cost-Sensitive Classification with Uniform cost ratio). Similarly, Table 5 and Fig. 5 depict the behavior of the total cost for the Uniform Inverse (UI) cost ratio.

During the analysis of decision rule classifiers’ performance for CSC-NU (refer to Table 6 and Fig. 6) and CSC NUI (refer to Table 7 and Fig. 7), an intriguing pattern emerged. To gain meaningful insights into these patterns, trend lines were plotted, as depicted in Figs 6 and 7. The parameters of the trend lines, such as slope (m) and intercept (c), along with the coefficient of determination, were recorded and presented in Table 8.

Table 6

Total cost for anemia diagnosing using CSC-NU

CFP:CFN ( $\rho$ )	Total cost (CSC-NU)
	DecisionTable	JRip	OneR	PART
2:3	303	229	450	84
2:5	377	303	586	120
2:7	451	377	722	156
2:11	599	525	994	228
2:13	673	599	1130	264
2:17	821	747	1402	336
2:19	895	821	1538	372
2:23	1043	969	1810	444
2:29	1265	1191	2218	552
2:31	1339	1265	2354	588
2:37	1561	1487	2762	696
2:41	1709	1635	3034	768
2:43	1783	1709	3170	804
2:47	1931	1857	3442	876
3:5	473	362	709	135
3:7	547	436	845	171
3:11	695	584	1117	243
3:13	769	658	1253	279
3:17	917	806	1525	351
3:19	991	880	1661	387
3:23	1139	1028	1933	459
3:29	1361	1250	2341	567
3:31	1435	1324	2477	603
3:37	1657	1546	2885	711
3:41	1805	1694	3157	783
3:43	1879	1768	3293	819
3:47	2027	1916	3565	891
5:7	739	554	1091	201
5:11	887	702	1363	273
5:13	961	776	1499	309
5:17	1109	924	1771	381
5:19	1183	998	1907	417
5:23	1331	1146	2179	489
5:29	1553	1368	2587	597
5:31	1627	1442	2723	633
5:37	1849	1664	3131	741
5:41	1997	1812	3403	813
5:43	2071	1886	3539	849
5:47	2219	2034	3811	921
7:11	1079	820	1609	303
7:13	1153	894	1745	339
7:17	1301	1042	2017	411
7:19	1375	1116	2153	447
7:23	1523	1264	2425	519
7:29	1745	1486	2833	627
7:31	1819	1560	2969	663
7:37	2041	1782	3377	771
7:41	2189	1930	3649	843
7:43	2263	2004	3785	879
7:47	2411	2152	4057	951
11:13	1537	1130	2237	399
11:17	1685	1278	2509	471

Table 6, continued
CFP:CFN ( $\rho$ )	Total cost (CSC-NU)
	DecisionTable	JRip	OneR	PART
11:19	1759	1352	2645	507
11:23	1907	1500	2917	579
11:29	2129	1722	3325	687
11:31	2203	1796	3461	723
11:37	2425	2018	3869	831
11:41	2573	2166	4141	903
11:43	2647	2240	4277	939
11:47	2795	2388	4549	1011
13:17	1877	1396	2755	501
13:19	1951	1470	2891	537
13:23	2099	1618	3163	609
13:29	2321	1840	3571	717
13:31	2395	1914	3707	753
13:37	2617	2136	4115	861
13:41	2765	2284	4387	933
13:43	2839	2358	4523	969
13:47	2987	2506	4795	1041
17:19	2335	1706	3383	597
17:23	2483	1854	3655	669
17:29	2705	2076	4063	777
17:31	2779	2150	4199	813
17:37	3001	2372	4607	921
17:41	3149	2520	4879	993
17:43	3223	2594	5015	1029
17:47	3371	2742	5287	1101
19:23	2675	1972	3901	699
19:29	2897	2194	4309	807
19:31	2971	2268	4445	843
19:37	3193	2490	4853	951
19:41	3341	2638	5125	1023
19:43	3415	2712	5261	1059
19:47	3563	2860	5533	1131
23:29	3281	2430	4801	867
23:31	3355	2504	4937	903
23:37	3577	2726	5345	1011
23:41	3725	2874	5617	1083
23:43	3799	2948	5753	1119
23:47	3947	3096	6025	1191
29:31	3931	2858	5675	993
29:37	4153	3080	6083	1101
29:41	4301	3228	6355	1173
29:43	4375	3302	6491	1209
29:47	4523	3450	6763	1281
31:37	4345	3198	6329	1131
31:41	4493	3346	6601	1203
31:43	4567	3420	6737	1239
31:47	4715	3568	7009	1311
37:41	5069	3700	7339	1293
37:43	5143	3774	7475	1329
37:47	5291	3922	7747	1401
41:43	5527	4010	7967	1389
41:47	5675	4158	8239	1461
43:47	5867	4276	8485	1491

Figure 6.

Performance of decision rule classifiers for CSC-NU.

Figure 7.

Performance of decision rule classifiers for CSC-NUI.

Table 7

Total cost for anemia diagnosing using CSC-NUI

CFP:CFN ( $\rho$ )	Total cost (CSC-NUI)
	DecisionTable	JRip	OneR	PART
3:2	362	251	505	81
5:2	554	369	751	111
5:3	591	406	819	129
7:2	746	487	997	141
7:3	783	524	1065	159
7:5	857	598	1201	195
11:2	1130	723	1489	201
11:3	1167	760	1557	219
11:5	1241	834	1693	255
11:7	1315	908	1829	291
13:2	1322	841	1735	231
13:3	1359	878	1803	249
13:5	1433	952	1939	285
13:7	1507	1026	2075	321
13:11	1655	1174	2347	393
17:2	1706	1077	2227	291
17:3	1743	1114	2295	309
17:5	1817	1188	2431	345
17:7	1891	1262	2567	381
17:11	2039	1410	2839	453
17:13	2113	1484	2975	489
19:2	1898	1195	2473	321
19:3	1935	1232	2541	339
19:5	2009	1306	2677	375
19:7	2083	1380	2813	411
19:11	2231	1528	3085	483
19:13	2305	1602	3221	519
19:17	2453	1750	3493	591
23:2	2282	1431	2965	381
23:3	2319	1468	3033	399
23:5	2393	1542	3169	435
23:7	2467	1616	3305	471
23:11	2615	1764	3577	543
23:13	2689	1838	3713	579
23:17	2837	1986	3985	651
23:19	2911	2060	4121	687
29:2	2858	1785	3703	471
29:3	2895	1822	3771	489
29:5	2969	1896	3907	525
29:7	3043	1970	4043	561
29:11	3191	2118	4315	633
29:13	3265	2192	4451	669
29:17	3413	2340	4723	741
29:19	3487	2414	4859	777
29:23	3635	2562	5131	849
31:2	3050	1903	3949	501
31:3	3087	1940	4017	519
31:5	3161	2014	4153	555
31:7	3235	2088	4289	591
31:11	3383	2236	4561	663
31:13	3457	2310	4697	699
31:17	3605	2458	4969	771

Table 7, continued
CFP:CFN ( $\rho$ )	Total cost (CSC-NUI)
	DecisionTable	JRip	OneR	PART
31:19	3679	2532	5105	807
31:23	3827	2680	5377	879
31:29	4049	2902	5785	987
37:2	3626	2257	4687	591
37:3	3663	2294	4755	609
37:5	3737	2368	4891	645
37:7	3811	2442	5027	681
37:11	3959	2590	5299	753
37:13	4033	2664	5435	789
37:17	4181	2812	5707	861
37:19	4255	2886	5843	897
37:23	4403	3034	6115	969
37:29	4625	3256	6523	1077
37:31	4699	3330	6659	1113
41:2	4010	2493	5179	651
41:3	4047	2530	5247	669
41:5	4121	2604	5383	705
41:7	4195	2678	5519	741
41:11	4343	2826	5791	813
41:13	4417	2900	5927	849
41:17	4565	3048	6199	921
41:19	4639	3122	6335	957
41:23	4787	3270	6607	1029
41:29	5009	3492	7015	1137
41:31	5083	3566	7151	1173
41:37	5305	3788	7559	1281
43:2	4202	2611	5425	681
43:3	4239	2648	5493	699
43:5	4313	2722	5629	735
43:7	4387	2796	5765	771
43:11	4535	2944	6037	843
43:13	4609	3018	6173	879
43:17	4757	3166	6445	951
43:19	4831	3240	6581	987
43:23	4979	3388	6853	1059
43:29	5201	3610	7261	1167
43:31	5275	3684	7397	1203
43:37	5497	3906	7805	1311
43:41	5645	4054	8077	1383
47:2	4586	2847	5917	741
47:3	4623	2884	5985	759
47:5	4697	2958	6121	795
47:7	4771	3032	6257	831
47:11	4919	3180	6529	903
47:13	4993	3254	6665	939
47:17	5141	3402	6937	1011
47:19	5215	3476	7073	1047
47:23	5363	3624	7345	1119
47:29	5585	3846	7753	1227
47:31	5659	3920	7889	1263
47:37	5881	4142	8297	1371
47:41	6029	4290	8569	1443
47:43	6103	4364	8705	1479

4. Experimental results and discussion

The comparative analysis of the mean total cost and standard deviation (SD) across different models for specific categories such as CSC-U, CSC-UI, CSC-NU, and CSC-NUI is represented in Table 8 and Fig. 8. This table aids in comparing the performance of different decision models based on their mean total cost, allowing for a comprehensive understanding of cost implications and variability across various scenarios. The standard deviation provides insights into the variability of the total cost within each model.

Table 8
Performance of decision rule classifiers based on mean of total cost across different cost ratios

Description	Mean of total cost
	DecisionTable	JRip	OneR	PART
CSC-U	1039.5	1002.5	1857	474
CSC-UI	2485	1541.5	3204.5	400.5
CSC-NU	2372.8	1899.5	3677.3	748.8
CSC-NUI	3443.8	2298.9	4675.7	694.4
Overall mean	2538.6	1832.4	3645.6	629.9
SD	1522.3	1007.1	2043.5	335.9

Table 9

Trend line information

Classifier	m	c	r²	m	c	r²	m	c	r²	m	c	r²
	CSC-U			CSC-UI			CSC-NU			CSC-NUI
DecisionTable	37	96	1	96	37	1	40.28	237.77	0.86	34.56	541.29	0.63
JRip	37	59	1	59	37	1	28.14	408.3	0.82	27.08	464.21	0.74
OneR	68	123	1	123	68	1	56.49	683.25	0.82	52.71	883.49	0.71
PART	18	15	1	15	18	1	9.19	261.79	0.68	10.41	197.29	0.88

m-slope, c-Intercept, r²-the coefficient of determination.

Figure 8.

Performance of decision tree classifiers for CSC-NUI.

Further statistical analysis for the total cost of the anemia diagnosing metric is as follows: The PART classifier has shown the lowest mean total cost of 400.5 for the CSC-UI framework and the lowest overall mean total cost of 629.9 among all the selected classifiers. It also has a relatively lower standard deviation of 335.9, indicating less variability in the total cost. Using the PART classifier can potentially lead to cost savings and more consistent results in terms of the total cost for anemia diagnosis. The OneR classifier has the highest mean total cost in all cost ratio methodologies, whereas JRIP and DecisionTable exhibit moderate performance in terms of total cost for anemia diagnosis.

The trend line information is shown in Table 9. The parameters slope (m) signifies the relationship between the input and output variables; higher values indicate a stronger relationship. The y-intercept (c) represents the value of the output variable when the input variable is zero; The R-squared (r²) values provide information about the goodness of fit of the model. Higher values indicate a better fit to the data. These facts support the some suggestions:

(a)

The JRip and OneR classifiers generally have higher slopes compared to the DT and PART classifiers, implying stronger relationships between input and output variables.

(b)

The PART classifier consistently shows the lowest slopes and y-intercepts across the four classifiers.

(c)

The DT classifier has the highest R-squared value for the third scenario (0.86), implying a relatively better fit for that specific case.

(d)

The PART classifier has the highest R-squared value for the fourth scenario (0.88), implying a relatively better fit for that specific case.

5. Conclusions

In this binary classification task, PART Classifier produces better results with minimal total cost of misclassification when compared to other decision rule classifiers for all types of cost ratio using the approaches Uniform (U), Uniform Inverted (UI), Non-Uniform (NU) and Non-Uniform Inverted (NUI) as in the proposed framework. The algorithms have been ranked based on their classification performance in terms of total cost, with the order being PART, JRIP, DecisionTable, and One R. The PART classifier has demonstrated the best performance for the specific dataset under consideration; however, it’s important to note that these rankings may vary when applied to different datasets. This highlights the need for careful consideration and evaluation of algorithm performance across diverse datasets to ensure robust and reliable results in various scenarios. Since the algorithms within the proposed framework is a generalized one, the range of cost ratio can be scaled up to cover for denser ratios. The proposed theme for analyzing the cost sensitive classifiers through the control matrix, a novel method in the direction of controlling the cost. However, it is crucial to assess the trade-offs between cost optimization and other performance metrics, such as accuracy or precision, to extend this analysis for a holistic solution.

Funding

The authors report no funding.

Data availability

This study utilized the data described in the research work by Kasiviswanathan et al. [31].

Footnotes

Acknowledgments

The authors would like to thank Dr. A. Kumaravel, Professor, Department of Information Technology and Dr. T. Vijayan, Department of Electronics and Communication Engineering, BIHER, Chennai, India for their support and guidance.

Conflict of interest

The authors declare that they have no conflict of interest.

References

Sadasivam

Thulasi Bai

. A compact diamond shaped ultra-wide band antenna system for diagnosing breast cancer. Technology and Health Care. 2023; 31(1): 57-68. Available from: doi: 10.3233/THC-220030.

Thyla

Thulasi Bai

. A very low-profile CPW based conformal antenna for wearable/implantable applications. Technology and Health Care. 2023; Pre-press: 1-13. Available from: doi: 10.3233/THC-220534.

Zhang

Sheng

Liu

Sun

Luo

. Cost supervision mining from EMR based on artificial intelligence technology. Technology and Health Care. 2023; 31(3): 1077-1091. Available from: doi: 10.3233/THC-220608.

Vijayan

Sangeetha

Kumaravel

Karthik

. Fine-tuned VGG19 convolutional neural network architecture for diabetic retinopathy diagnosis. Indian Journal of Computer Science and Engineering. 2020; 11(5): 615-622. Available from: doi: 10.21817/indjcse/2020/v11i5/201105266.

Vijayan

Sangeetha

Kumaravel

Karthik

. Feature selection for simple color histogram filter based on retinal fundus images for diabetic retinopathy recognition. IETE Journal of Research. 2023; 69(2): 987-994. Available from: doi: 10.1080/03772063.2020.1844082.

Thakkar

Desai

Ghosh

Singh

Sharma

. Clairvoyant: AdaBoost with Cost-Enabled Cost-Sensitive Classifier for Customer Churn Prediction. Computational Intelligence and Neuroscience. 2022; 2022: 1-11. Available from: doi: 10.1155/2022/9028580.

Ioannidis

JPA

Tarone

McLaughlin

. The false-positive to false-negative ratio in epidemiologic studies. Epidemiology. 2011; 22(4): 450-456. Available from: doi: 10.1097/ede.0b013e31821b506e.

Peter. The foundations of cost-sensitive learning. In: IJCAI’01: Proceedings of the 17th International Joint Conference on Artificial Intelligence. Vol. 2. 2001. pp. 973-978. Available from: doi: 10.5555/1642194.1642224.

World Health Organization (WHO). Global Nutrition Targets 2025: Anaemia policy brief. WHO/NMH/NHD/14.4.

10.

World Health Organization (WHO). Anaemia. Available from: https://www.who.int/news-room/fact-sheets/detail/anaemia.

11.

Xiong

Gao

Zhang

Song

Han

Niu

Liang

. The Effect of Dapagliflozin on Anemia in Elderly Patients with Heart Failure by Bioinformatics Analysis. Technology and Healthcare. 2023; Pre-press: 1-11. PMID: 37781829. Available from: doi: 10.3233/THC-230563.

12.

Kumaravel

Vijayan

. Comparing cost sensitive classifiers by the false-positive to false-negative ratio in diagnostic studies. Expert Systems With Applications. 2023; 227: 120303. Available from: doi: 10.1016/j.eswa.2023.120303.

13.

Mienye

Sun

. Performance analysis of cost-sensitive learning methods with application to imbalanced medical data. Informatics in Medicine Unlocked. 2021; 25: 100690. Available from: doi: 10.1016/j.imu.2021.100690.

14.

Telikani

Gandomi

Choo

KKR

Shen

. A cost-sensitive deep learning-based approach for network traffic classification. IEEE Transactions on Network and Service Management. 2022; 19(1): 661-670. Available from: doi: 10.1109/tnsm.2021.3112283.

15.

Weiss

McCarthy

Zabar

. Cost-sensitive learning vs. sampling: Which is best for handling unbalanced classes with unequal error costs? DMIN. 2007; 7: 35-41.

16.

Weiss

Elovici

Rokach

. The CASH algorithm-cost-sensitive attribute selection using histograms. Information Sciences. 2013; 222: 247-268. Available from: doi: 10.1016/j.ins.2011.01.035.

17.

Domingos

. MetaCost. In: Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1999. Available from: doi: 10.1145/312129.312220.

18.

Thai-Nghe

Gantner

Schmidt-Thieme

. Cost-sensitive learning methods for imbalanced data. The 2010 International Joint Conference on Neural Networks (IJCNN). 2010. Available from: doi: 10.1109/ijcnn.2010.5596486.

19.

Stoltzfus

Edward-Raj

Dreyfuss

Albonico

Montresor

Thapa

, et al. Clinical pallor is useful to detect severe anemia in populations where anemia is prevalent and severe. The Journal of Nutrition. 1999; 129(9): 1675-1681. Available from: doi: 10.1093/jn/129.9.1675.

20.

Sheth

Choudhry

Bowes

Detsky

. The relation of conjunctival pallor to the presence of anemia. Journal of General Internal Medicine. 1997; 12: 102-106.

21.

Glass

Batres

Selle

Garcia-Ibanez

. The value of simple conjunctival examination in field screening for anemia. Nutrition Reports International (USA). 1980; ISSN: 0029-6635.

22.

Sanchez-Carrillo

Ramirez-Sanchez

Zambrana-Castaneda

Selwyn

. Test of a noninvasive instrument for measuring hemoglobin concentration. International Journal of Technology Assessment in Health Care. 1989; 5(4): 659-667.

23.

Suner

Crawford

McMurdy

Jay

. Non-invasive determination of hemoglobin by digital photography of palpebral conjunctiva. The Journal of Emergency Medicine. 2007; 33(2): 105-111. Available from: doi: 10.1016/j.jemermed.2007.02.011.

24.

Collings

Thompson

Hirst

Goossens

George

Weinkove

. Non-invasive detection of anemia using digital photographs of the conjunctiva. PLOS ONE. 2016; 11(4). Available from: doi: 10.1371/journal.pone.0153286.

25.

Bevilacqua

Dimauro

Marino

Brunetti

Cassano

Maio

Nasca

Trotta

Girardi

Ostuni

. A novel approach to evaluate blood parameters using computer vision techniques. In: IEEE International Symposium on Medical Measurements and Applications (MeMeA). IEEE; 2016. Available from: doi: 10.1109/memea.2016.7533760.

26.

Dimauro

Caivano

Girardi

. A new method and a non-invasive device to estimate anemia based on digital images of the conjunctiva. IEEE Access. 2018; 6: 46968-46975. Available from: doi: 10.1109/access.2018.2867110.

27.

Muthalagu

Bai

Gracias

John

. Developmental screening tool: Accuracy and feasibility of non-invasive anemia estimation. Technology and Health Care. 2018; 26(4): 723-727. Available from: doi: 10.3233/THC-181291.

28.

Dimauro

Guarini

Caivano

Girardi

Pasciolla

Iacobazzi

. Detecting clinical signs of anaemia from digital images of the palpebral conjunctiva. IEEE Access. 2019; 7: 113488-113498. Available from: doi: 10.1109/ACCESS.2019.2932274.

29.

Sivachandar

Thulasi Bai

. Performance Analysis of Ensemble Model in Anaemia Detection from Unmodified Smartphone Captured Conjunctiva Images. IETE Journal of Research. 2023 (Accepted).

30.

Kasiviswanathan

Bai Vijayan

Simone

Dimauro

. Semantic segmentation of conjunctiva region for non-invasive anemia detection applications. Electronics. 2020; 9(8): 1309. Available from: doi: 10.3390/electronics9081309.

31.

Kasiviswanathan

Vijayan

John

. Ridge regression algorithm based non-invasive anaemia screening using conjunctiva images. J Ambient Intell Human Comput. 2020. Available from: doi: 10.1007/s12652-020-02618-3.

32.

Sivachandar

Thulasi Bai

. Investigating the Optimal K Value in K-Nearest Neighbors and Cost Matrix Analysis for the Global Air Quality Dataset. Knowledge Transactions on Applied Machine Learning. 2023; 1(5): 1-20. Available from: doi: 10.59567/ktAML.V1.05.01.

33.

Weka. Department of Computer Science: University of Waikato. (n.d.). Available from: http://www.cs.waikato.ac.nz.

34.

Kohavi

. The Power of Decision Tables. In: 8th European Conference on Machine Learning. Springer. 1995. pp. 174-189.

35.

Cohen

. Fast Effective Rule Induction. In: The Twelfth International Conference on Machine Learning. 1995. pp. 115-123.

36.

Frank

Witten

. Generating Accurate Rule Sets Without Global Optimization. In: Fifteenth International Conference on Machine Learning. Morgan Kaufmann. 1998. pp. 144-151.

37.

Holte

. Very simple classification rules perform well on most commonly used datasets. Machine Learning. 1993; 11: 63-91.

38.

McCrimmon

. Enumeration of the positive rationals. The American Mathematical Monthly. 1960; 67(9): 868.

39.

Sagher

. Counting the rationals. Amer. Math. Monthly. 1989; 96(9): 823.

40.

Yu-Ting

. A “Natural” enumeration of non-negative rational numbers – an informal discussion. The American Mathematical Monthly. 1980; 87(1): 25. Available from: doi: 10.2307/2320374.

Examining different cost ratio frameworks for decision rule machine learning algorithms in diagnostic application

Abstract

BACKGROUND:

OBJECTIVE:

METHODS:

RESULTS:

CONCLUSION:

Keywords

1. Introduction

2. Related works

3. Materials and methods

3.1 Data description

Table 1 Statistical information of the anemia dataset

3.3.1 Decision rule classifiers

a) DecisionTable Classifier

b) JRip Classifier

c) PART Classifier

d) OneR Classifier

3.3.2 Cost sensitive learning

Table 2 Confusion matrix

a) Component for CSC-U

b) Component for CSC-UI

c) Component for CSC-NU

d) Component for CSC-NUI

Table 8 Performance of decision rule classifiers based on mean of total cost across different cost ratios

Funding

Data availability

Footnotes

Acknowledgments

Conflict of interest

References

Table 1
Statistical information of the anemia dataset

Table 2
Confusion matrix

Table 8
Performance of decision rule classifiers based on mean of total cost across different cost ratios