In this work, we have designed and developed a BCI racing car game in a three-dimensional (3D) virtual environment using Unity software. The 3D virtual environment consists of two racing cars, tracks, as well as surrounding terrain that includes trees, grass, buildings, mountains, and the sky. Three cameras have been set up to show a driver's view, a bird's-eye view, and a following camera's view. Kinetic parameters of the cars are chosen to simulate physical movements of the car. The two racing cars are separately controlled by two individual drivers' brainwaves. Each driver's EEG brainwaves are monitored in real time using a g.tec Nautilus 32-channels system through Matlab and Simulink. The collected brainwave signals are analyzed online in Matlab using pre-trained machine learning algorithms to decode the intended kinematics. The machine learning algorithms have been trained to classify the driver's instantaneous intention into s categories: moving forward, moving backward, turning left, turning right, and maintaining rest. A control signal is then calculated based on the decoded kinematics and sent through the TCP/IP protocol to Unity to steer the car.

We implement a hybrid decoding algorithm that combines steady state visual evoked potential (SSVEP) and imagined-body kinematics (IBK) paradigms. SSVEP can provide relatively high signal-to-noise ratio (SNR) and information transfer rate (ITR) [1] while IBK provides natural imaginary body movement [3]. A two-phase training protocol was designed to train a subject to learn to use the BCI. Signals collected during Phase 1 training are used to train the SSVEP paradigm. Canonical correlation analysis [2] is used to calculate the canonical correlation of the projected EEG and the target frequencies (7.5, 10, 12, and 15Hz) in real-time. In phase two, we train a cross-validated multiple linear regression model for decoding EEG data during IBK paradigms conditioned with two classes of resting and pushing. The overall decoding algorithm is a combination of the SSVEP and the IBK paradigms. We utilize the IBK paradigm as a gating function. If the online model detects the "pushing" state, the subject's intended direction by the SSVEP paradigm is translated into the virtual car movement. If the IBK model detects "resting" state, the virtual car remains stationary.

The platform including GUI interface, Unity 3D environment, training protocol, data acquisition processes, communication protocols, as well as online and offline decoding algorithms, has been fully implemented. The platform has been thoroughly tested to ensure it runs as expected. The current work is focused on recruiting subjects to improve accuracy and robustness of the decoding algorithms.

Conventional BCI-based biofeedback systems often fail at maintaining a user's engagement and motivation which makes it difficult to attain a satisfactory level of control. The proposed platform is designed to improve users' experience of motor imagery with a visual feedback of the users' intention on to a virtual car. It serves as a pilot for online BCI-based gaming as well as an educational tool to promote interests in BCI for the public. The setup has potential to use as a research tool for investigators in developmental psychology and behavioral science. Moreover, the developed platform opens the opportunity for us to evaluate users' immersion in virtual reality (VR) using the proposed platform in future study. The platform may suggest an effective visual feedback of vision, which can lay a foundation for BCI application.

This work was in part supported by NEURONET and a SARIF …