High Dimensional Data Clustering with Hub Based DEC

Abstract

Clustering is an important topic in various fields like machine learning and data mining. In many real applications, we often face very high dimensional data. Many dimensions are not always helpful or may even worsen the performance of the subsequent clustering algorithms. To deal with this problem one way is to employ first dimensionality reduction and then apply clustering. But if we consider the requirement of clustering in the process of dimensionality reduction and vice versus then the performance of clustering will be improved. Discriminative Embedded Clustering (DEC) is an algorithm that combines clustering and subspace learning. Hubness is the tendency of high dimensional data to have hubs. Hubs are situated near cluster centeres; therefore major hubs can be successfully used as cluster prototypes or guide during centroid based configurations. Use of hubness for clustering leads to improvement over centroid-based approaches. In this paper we propose a system for clustering high dimensional data using Discriminative Embedding Method with Hub based clustering.