Various machine learning technology approaches have been applied to intrusion detection system (IDS). To get optimal results, it needs to take several stages for processing the traffics. Among them is the feature selection method, where irrelevant and redundant features are removed. In the previous research, the system is developed based on feature grouping that used a clustering approach as the evaluation criteria. In this research, we propose a method for improving the performance of machine learning with the feature selection approach based on feature clustering. We propose cluster based feature selection derived from the value of mutual information and Pearson correlation. The cluster hierarchy is used in forming filters that are used to create selected and reduced clusters. In developing the cluster hierarchy, single, complete, and average linkage method are used to determine the formation of the best feature clusters. The classification method with Support Vector Machine (SVM), Naïve Bayes, and J48 decision tree are applied to observe the performance of the proposed feature selection. Based on the experimental results, we find that the highest accuracy (ie, 99.842%) is obtained when a single linkage in the J48 classification is implemented in the Kyoto 2006 dataset.