A modified ID3 decision tree algorithm based on cumulative residual entropy

Abstract

In this paper, we propose a new modification of the traditional ID3 decision tree algorithm through cumulative residual entropy (CRE). We discuss the principles of decision trees, including entropy and information gain, and introduce the concept, properties, and advantages of CRE as an alternative measure of Shannon entropy in decision trees. Also, by running the proposed decision tree algorithm (named CRDT) and ID3 decision tree algorithm on ten real datasets, we evaluate and compare the accuracy and efficiency of the two decision trees algorithm using appropriate criteria which indicates that the performance of the CRDT is much more accurate and closer to reality than the ID3. Furthermore, we compare the performance of three decision tree-based algorithms, CRDT, CART, and Random Forest via MSE, RMSE, R-Square and training time criteria. The results show the superiority of new proposed model compared to alternatives.