Review of Rule Quality Measurement: Metrics and Rule Evaluation Models
Main Article Content
Abstract
A rule-based system is a system based on the set of rules used to make inference knowledge. The system gathers knowledge into the representation of knowledge in the form of a rule. However, the knowledge in the form of the rule is inductive, meaning that the algorithm can construct the rule by studying a limited number of cases and then the induced rule of a limited number of cases and then generalize it to the general reality from time to time. This, of course, has the degree of inaccuracy in expressing reality into knowledge, or an experienced expert builds it but it is not absolute that the knowledge it possesses is 100% accurate or always consistently true from one time-space location to another time-space location. Therefore, the need for a formula that can measure the quality of the resulting rule and assess the consistency of the rule. In this study, we did a review of the ideas of people trying to measure knowledge built inductively by either the algorithm or the experts. These measurements are based on several parameters defined by them according to the underlying assumptions. This review seeks to partially present how ideas to measure the rule as knowledge representation from a varied viewpoint and how people construct evaluation models to assess the resulting regulations either from the experts or human experts as well as those resulting from the induction rule algorithm much developed.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
IJCERT Policy:
The published work presented in this paper is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. This means that the content of this paper can be shared, copied, and redistributed in any medium or format, as long as the original author is properly attributed. Additionally, any derivative works based on this paper must also be licensed under the same terms. This licensing agreement allows for broad dissemination and use of the work while maintaining the author's rights and recognition.
By submitting this paper to IJCERT, the author(s) agree to these licensing terms and confirm that the work is original and does not infringe on any third-party copyright or intellectual property rights.
References
P. Smyth and R. M. Goodman, “A n Information Theoretic Approach to Rule Induction from Databases,” vol. 4, no. 4, pp. 301–316, 1992.
R. Shinghal, “Evaluating the Interestingness of Characteristic Rules,” pp. 263–266, 1996.
S. Dreiseitl, M. Oil, C. Baumgartner, and S. Viterbo, "An evaluation of heuristics for rule ranking," Artif. Intell. Med., vol. 50, no. 3, pp. 175–180, 2010.
J. F. Roddick and M. Spiliopoulou, “A survey of temporal knowledge discovery paradigms and methods,” IEEE Trans. Knowl. Data Eng., vol. 14, no. 4, pp. 750–767, 2002.
F. Provost, “A Survey of Methods for Scaling Up Inductive Algorithms,” vol. 169, pp. 131–169, 1999.
I. Inza, P. Larrañaga, R. Etxeberria, and B. Sierra, “Feature Subset Selection by Bayesian network-based optimization,” vol. 123, pp. 157–184, 2000.
F. Provost, “Tree Induction for Probability-Based Ranking,” vol. 5, pp. 199–215, 2003.
J. Sulzmann and F. Johannes, “An Empirical Comparison of Probability Estimation Techniques for Probabilistic Rules,” no. 2003, pp. 317–331, 2009.
A. A. Freitas, “On rule interestingness measures,” KnowledgeBased Syst., vol. 12, no. March, pp. 309–315, 1999.
K. E. N. Mcgarry, “A survey of interestingness measures for knowledge discovery,” pp. 39–61, 2005.
X. Huynh, F. Guillet, J. Blanchard, and P. Kuntz, “A Graph-based Clustering Approach to Evaluate Interestingness Measures : A Tool and a Comparative Study,” vol. 50, pp. 25–50, 2007.
B. Vaillant, S. Lallich, and P. Lenca, “On the behavior of the generalizations of the intensity of implication : A data-driven comparative study,” vol. 447, pp. 421–447, 2008.
J. Hills, L. M. Davis, and A. Bagnall, “Interestingness Measures for Fixed Consequent Rules,” pp. 68–75, 2012.
P. Flach, N. Lavrac, and B. Zupan, “Rule Evaluation Measures: A Unifying View,” Proc. 9th Int. Work. Inductive Log. Program., pp. 174–185, 1999.
F. Johannes and P. A. Flach, “An Analysis of Rule Evaluation Metrics,” 2003.
D. Christensen, “David Christensen - Measuring Confirmation.pdf.” pp. 437–461, 1999.
S. Greco, R. Słowi, and I. Szcz, “Measures of rule interestingness in various perspectives of confirmation,” vol. 347, pp. 216–235, 2016.
M. Michalak, M. Sikora, and Ł. Wróbel, “Rule Quality Measures Settings in a Sequential Covering Rule Induction Algorithm - an Empirical Approach,” vol. 5, pp. 109–118, 2015.
P. F. Nada Lavrac, Bojan Cestnik, Dragan Gamberger, “Decision Support Through Subgroup Discovery : Three Case Studies and the Lessons Learned,” no. 1994, pp. 115–143, 2004.
D. M. W. Powers, “ROC-ConCert,” pp. 12–15, 2012.
P. Salgado, “Relevance as a new measure of relative importance: of sets of rules,” no. 3, pp. 3770–3777, 2000.
[22] F. Coenen and P. Leng, “An Evaluation of Approaches to Classification Rule Selection,” IEEE Int. Conf. Data Min., pp. 2– 5, 2004.
Y. Yao and B. Zhou, “Micro and Macro Evaluation of Classification Rules,” Proc. Seventh IEEE Int. Conf. Cogn. Informatics, ICCI 2008, Stanford Univ. California, USA, 2008.
D. M. W. Powers, “Evaluation: From Precision, Recall And FMeasure To Roc, Informedness, Markedness & Correlation,” vol. 2, no. 1, pp. 37–63, 2011.
H. Abe, S. Tsumoto, M. Ohsaki, and T. Yamaguchi, “Evaluating Learning Algorithms to Construct Rule Evaluation Models Based on Objective Rule Evaluation Indices,” 2007.
H. Abe, S. Tsumoto, M. Ohsaki, and T. Yamaguchi, “Evaluating Learning Algorithms to Support Human Rule Evaluation with Predicting Interestingness Based on Objective Rule Evaluation Indices,” vol. 282, no. 2008, pp. 269–282, 2008.
H. Abe and S. Tsumoto, “Comparing Accuracies of Rule Evaluation Models to Determine Human Criteria on Evaluated Rule Sets,” pp. 1–7, 2008.
H. Abe and S. Tsumoto, “Rule Evaluation Model as Behavioral Modeling,” pp. 8–15, 2009.
A. Gruca and M. Sikora, "Rule-based functional description of genes – Estimation of the multicriteria rule interestingness measure by the UTA method," Integr. Med. Res., vol. 33, no. 4, pp. 222–234, 2013.
A. Gruca and M. Sikora, “Data- and expert-driven rule induction and filtering framework for functional interpretation and description of gene sets,” pp. 1–14, 2017.
U. Stanczyk, “Weighting and Pruning of Decision Rules,” pp. 106–114, 2016.
K. K. Sethi, D. K. Mishra, and B. Mishra, “Novel Algorithm to Measure Consistency between Extracted Models from Big Dataset and Predicting Applicability of Rule Extraction,” IEEE Trans. Knowl. Data Eng., 2014.
H. Mutluri and P. Sujatha, “Challenges in Big Data using Data Mining Techniques,” Int. J. Comput. Eng. Res. Trends, vol. 2, no. 12, pp. 924–930, 2015.