Pattern Finding in Large Datasets with Big Data Analytics Mechanism
Main Article Content
Abstract
One of the most popular knowledge discovery approaches is to find frequent items from a transaction data set and derive association rules. Pattern finding is one of the most computationally expensive steps in large data sets. Patterns often referred to association rules. Association rule plays an important role in the process of mining data for sequential pattern. Association rules are used to acquire interesting rules from large collections of data which expresses an association between items or sets of items. . Apriori is a classic algorithm for learning association rules. It is designed to operate on databases containing transactions. Apriori algorithm attempts to find subsets which are common to at least a minimum number of the item sets. Apriori uses a "bottom up" approach, where frequent subsets are extended one item at a time and groups of candidates are tested against the data. The algorithm terminates when no further Successful extensions are found. In this paper we enhance Apriori algorithm to solve its complexity over large data sets. Thus in order to address the pattern finding in large datasets we used Big data analytics mechanism it is a collection of large amount of data from numerous sources and usable to be processed at much higher frequency. We first collect variety of data and then integrate both structured and unstructured data using MapReduce to find out sequential pattern from the required data sets.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
IJCERT Policy:
The published work presented in this paper is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. This means that the content of this paper can be shared, copied, and redistributed in any medium or format, as long as the original author is properly attributed. Additionally, any derivative works based on this paper must also be licensed under the same terms. This licensing agreement allows for broad dissemination and use of the work while maintaining the author's rights and recognition.
By submitting this paper to IJCERT, the author(s) agree to these licensing terms and confirm that the work is original and does not infringe on any third-party copyright or intellectual property rights.
References
Assured Research, Essay on Big Data. June 2012.
Bart Goethals. Survey on Frequent Pattern Mining.
Basel Kayyali, David Knotl, Peter Crroves and Steve Van Kuiken. The Big Data
Bid Data, ANew World of Opportunities. NESSI White Paper, December 2012.
Albert Bifet, Mining Big Data in Real Time. Informatica 37, 15-20 (2013).
Amit Ganatra, Amit Thakkar and Cletna Chand. Sequential Pattern mining: Survey and Current Research Challenges. International Journal of Soft Computing and Engineering, Volume 2, March 2012.
Aniket Mahanti and Reda Alhajj. Visual Interface for Online Watching of Frequent Itemset Generation in Apriori and Eclat, Fourth International Conference on Machine Learning and Application (2005).
Anuradha Sharma, Arvind Sehwal and Harleen Puri. An Empirical Proposal towards the Algorithmic Approach and Pattern in Web Mining for Assorted Applications. International Journal of Innovative Research in Computer and Communication Engineering, Volume 1. April 2013.
Aramburu, Juan Manuel Perez, Maria Jose, Rafael Berlanga and Torben Bach Pedersen, Integrating Data warehouse with Web Data: A Survey, IEEE transaction on knowledge and Data Engineering, Volume 20, July 2008.
Big Data Strategy- Issues Paper. March 2013.
C.I. Ezeife, R. Mabroukeh and Nizar. A Taxonomy of Sequential Pattern Mining Algorithms. ACM Computing Surveys, Volume 43, November 2010.
Challenges and Opportunities with Big Data, A Community white paper developed by leading researchers across the United States.
D. Magdalene Delighta Angeline and I. Samuel Peter James. Association Rule Generation using Apriori Mend Algorithm for Student’s Placement. Int. J. Emerging. Science. 2(1). March 2012.
D.P Agrawal, R.K. Gupta and A. Tiwari. A Survey on Frequent Pattern Mining: Current Status and Challenging Issues. Information Technology Journal 9(7), 2010.
David Loshin, Integrating Structured and Unstructured Data. TDWI Checklist Report.
DonXin, Hong Cheng, Jiawei Han and Xifeng Yan. Frequent Pattern Mining : Current Status and Future direction. Knowledge discovery Knowledge Disc, 2007.
Dr.L. DE Radt, Mining patterns in Structured Data. September 2009.
Ekta Garg and Meenakshi Bansal. A Survey on Improved Apriori Algorithm. IJERT, Volume 2, July 2013.
Huajun Chen, Jun Ma , Xiangyu Zhang, Xiaolongyu and Yang Liu. Map Reduce- Based Pattern Finding Algorithm Applied in Motif Detection for Prescription Compatibility Network, APPT 2009.
Hui Xion, Jian Pei and Yan Huang. Mining Co-Location Patterns with Rare Events from Spatial Data Sets. Geoinformatica (2006).