Malware detection is an essential part of any security device. Malicious code has come to be a pervasive and serious concern in today's interconnected computing world. Malware developers and detection experts see malware detection as a game of obfuscation. More than 15% of the tested antivirus software couldn't detect any malicious samples. Malware executables are identified using the proposed GWO-XGBoost system. The Gray Wolf Optimizer (GWO) is a metaheuristic set of rules with swarm intelligence that creates. . It has been widely used for a variety of optimization issues because it leads previous swarm intelligence technology in terms of capabilities. Moreover, XGBoost is a selection tree-based system that categorizes malicious executables using a gradientboosting methodology. This review contains the XGboost methodology, a GWO optimization-based method for identifying malicious executables. The suggested GWO-xgboost techniques offer 97.0% precision, 97.0% age F-degree, 0.029 Entropy value, 99.4% precision and 99.4% recall, 0.007 Flase positive cost, and 99.4% recall for the classification of malicious executables and boot code.