In this article, a newly hybrid nature-inspired approach (MGBPSO-GSA) is developed with a combination of Mean Gbest Particle Swarm Optimization (MGBPSO) and Gravitational Search Algorithm (GSA). The basic inspiration is to integrate the ability of exploitation in MGBPSO with the ability of exploration in GSA to synthesize the strength of both approaches. As a result, the presented approach has the automatic balance capability between local and global searching abilities. The performance of the hybrid approach is tested on a variety of classical functions, ie, unimodal, multimodal, and fixed-dimension multimodal functions. Furthermore, Iris data set, Heart data set, and economic dispatch problems are used to compare the hybrid approach with several metaheuristics. Experimental statistical solutions prove empirically that the new hybrid approach outperforms significantly a number of metaheuristics in terms of solution stability, solution quality, capability of local and global optimum, and convergence speed.