With the development of the web's high usage, the number of malware affecting the system are incresing. Various techniques have been used but they are incapable to identify unknown malware. To counter such threats, the proposed work makes utilization of dynamic malware investigation systems based on machine learning technique for windows based malware recognization. In this paper two methods to analyses the behaviour of the malware and feature selection of windows executables file. Cuckoo is a malicious code analysis apparatus which analyzes the malware more detail and gives the far-reaching results dependent on the arrangement of tests made by it and second, the feature selection for windows dynamic malware anaysis has been done by using Genetic Algorithm. Three classifiers have been used to compare the detection result of Windows-based malware: Support Vector Machine with detection accuracy of 81.3%, Naive Bayes classifier with accuracy of 64.7% and Random Forest classifier achieving 86.8% accurate results.