Many applications of clustering require the use of normalized data, such as text or mass spectra mining. The K-Means algorithm is one of the most widely used clustering algorithm works on greedy approach. Major problems with the traditional K mean clustering is generation of empty clusters and more computations required to make the group of clusters. The proposed Entropy Based MeansClustering algorithm, is highly useful for data such as text or massive data, because it produces normalized cluster centers. Findings indicate a better performance in mining data in terms of reducing time and seed predations, avoiding the empty clusters as compared with the widely used kmeans clustering technique.