Gravitational Search Algorithm (GSA) is a new heuristic algorithm which derives its inspiration from natural phenomenon. As a heuristic algorithm, GSA has a good performance in global searching. However, if a premature convergence happens, GSA will stop and lose its ability to search. Therefore, a new operator that is inspired by conservation of linear momentum is added to improve GSA’s performance. In this paper, we propose this new operator called momentum operator which aims to solve the weakness of GSA. It will be applied in a non-linear optimization and data classification. The system will be simulated in MATLAB R2009a. The result showed that this operator was successful to solve data classification and minimization problem faster and more precisely in a high dimension.