The Use of Heuristics in Decision Tree Learning Optimization
Main Article Content
Abstract
Decision tree algorithms are among the most popular techniques for dealing with classification problems in different areas.
Decision tree learning offers tools to discover relationships, patterns and knowledge from data in databases. As the volume of data in
databases is growing up very quickly, the process of building decision trees on such databases becomes a quite complex task. The
problem with decision trees is to find the right splitting criterion in order to be more efficient and to get the highest accuracy. Different
approaches for this problem have been proposed by researchers, using heuristic search algorithms. Heuristic search algorithms can help to find optimal solutions where the search space is simply too large to be explored comprehensively. This paper is an attempt to summarize the proposed approaches for decision tree learning with emphasis on optimization of constructed trees by using heuristic search algorithms. We will focus our study on four of the most popular heuristic search algorithms, such as hill climbing, simulated annealing, tabu search and genetic algorithms.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
IJCERT Policy:
The published work presented in this paper is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. This means that the content of this paper can be shared, copied, and redistributed in any medium or format, as long as the original author is properly attributed. Additionally, any derivative works based on this paper must also be licensed under the same terms. This licensing agreement allows for broad dissemination and use of the work while maintaining the author's rights and recognition.
By submitting this paper to IJCERT, the author(s) agree to these licensing terms and confirm that the work is original and does not infringe on any third-party copyright or intellectual property rights.
References
V. Mohan, ―Decision Trees:A comparison of various algorithms for building Decision Trees‖, 2013
W.S.Alsharafat ―Steady State Genetic Algorithm in University Admission Decision‖, Contemporary Engineering Sciences, Vol. 6, 2013, no. 5, pp.245 - 254
D.Sh. Liu et al. ―A Modified Decision Tree Algorithm Based on Genetic Algorithm for Mobile User Classification Problem‖, The Scientific World Journal Volume 2014, Article ID 468324 2014
R.C.Barros et al. ―Automatic Design of Decision-Tree Algorithms with Evolutionary Algorithms‖ Massachusetts Institute of Technology 2013
M.Khanbabaei, M.Alborzi ―The Use Of Genetic Algorithm, Clustering And Feature Selection Techniques In Construction Of Decision Tree Models For Credit Scoring‖. International Journal of Managing Information Technology (IJMIT) Vol.5, No.4, 2013 DOI : 10.5121/ijmit.2013.5402
T. de la Rosa et al. ―Learning Relational Decision Trees for Guiding Heuristic Planning Association for the Advancement of Artificial Intelligence‖ (www.aaai.org). 2008
Y.Cai et al. ―A Tree-Based Tabu Search Algorithm for the Manpower Allocation Problem with TimeWindows and Job-Teaming Constraints‖, Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence 2013 pp.496-502
Y.Madadi et al. ―An Accurate Classification Algorithm with Genetic Algorithm Approach‖, International Journal of Computer & Information Technologies (IJOCIT) 2013 ISSN: 2345-3877 Vol 1, Issue 3 pp.198-210
―A Bank Customer Credit Evaluation Based on the Decision Tree and the Simulated Annealing Algorithm‖ 2007
C.J.Hinde et al. ―Evolving the input space for decision tree building‖ 2005
R.Ahmed et al ―Induction of Better Decision Trees Using Population Oriented Multi-Objective Simulated Annealing‖, 7th International Conference on Computer and Information Technology 2004
Fu Et Al. ―A Genetic Algorithm-Based Approach For Building Accurate Decision Trees Informs‖ Journal On Computing 2003 Vol. 15, No.1, ISSN:1526-5528
J.Dvoˇr´Ak , P.Savick´Y ―Softening Splits In Decision Trees Using Simulated Annealing‖ 2006
A.M.Mahmood et al. ―A New Decision Tree Induction Using Composite Splitting Criterion‖ Journal Of Applied Computer Science & Mathematics, No. 9 (4) /2010
S.H. Cha, Ch.Tappert ―A Genetic Algorithm For Constructing Compact Binary Decision Trees‖ Journal Of Pattern Recognition Research 1 (2009) pp.1-13
K.B. Bennett, J.A.Blue ―An Extreme Point Tabu Search Method For Data Mining‖ 2007
N.Mishra et al., ―Hybrid Tabu-Simulated annealing based approach to solve multi-constraint product mix decision problem‖ Expert Systems with Applications 2005 Volume 29, Issue 2, pp.446-454
Yi Jiang ―Credit Scoring Model Based on the Decision Tree and the Simulated Annealing Algorithm‖ Computer Science and Information Engineering, 2009 WRI World Congress Vol.4, pp. 18 – 22, ISBN: 978-0- 7695-3507-4
Glover, F. & Laguna, M. 1997. Tabu search. Boston: Kluwer Academic Publishers.
J. Han, Data Mining: Concepts and Techniques, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2001.
M. E. Aydin, T. C. Fogarty. ‖A Distributed Evolutionary Simulated Annealing Algorithm for Combinatorial Optimization Problems‖, in Journal of Heuristics 2004, vol. 24, no. 10, pp. 269–292.
R. Battiti. ‖Reactive search: towards self-tuning heuristics‖, in Modern heuristic search methods. Wiley&Sons, 1996, pp. 61-83.
Goldberg, D.E., 1989. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Publishing Company, Inc., Reading, MA