Detection of driver cognitive distraction is critical for active safety systems of road vehicles. Compared with visual distraction, cognitive distraction is more challenging for detection due to the lack of apparent exterior features. This paper presents a novel real-time detection algorithm for driver cognitive distraction by using support vector machine (SVM). Data are collected from 26 subjects, driving in typical urban and highway scenarios in a simulator. The chosen urban scenario is the stop-controlled intersection and the highway scenario is the speed-limited highway. Driver cognitive distraction while driving is induced by clock tasks which compete with the main driving tasks for visuospatial short working memory. For each subject, distracted driving instances and the equal number of non-distracted driving instances were collected (24 for urban scenario and 20 for highway scenario in total). Features concerning both driving performance and eye movement are used for training and validation. The proposed algorithm have correct rate of 93.0% and 98.5% for highway and urban scenarios respectively. Results also show that driver distraction can be recognized 6.5 s to 9.0 s after its happening, indicating good performance of the detection algorithm.