We present a face detection system based on a class of convolutional neural networks, namely shunting inhibitory convolutional neural networks (SICoNNets). The topology of these networks is a flexible feedforward architecture with three different connections schemes: fully-connected, toeplitz-connected and binary-connected. SICoNNets were trained, using a hybrid method based on Rprop, Quickprop and least squares, to discriminate between face and non-face patterns. All three connection schemes achieve 99% detection accuracy at 5% false alarm rate, based on a test set of 7000 face and non-face patterns. Furthermore, toeplitz-connected network was trained on a larger training set and has achieved a 99% correct classification rate with only 1% false alarm rate based on the same test set. A face detection system is built based on the trained convolutional neural networks. The system accepts an input image of arbitrary size and localizes the face patterns in the image. To localize faces of different sizes, the convolutional neural network is applied as a face detection filter at different scales. The detection scores from different scales are aggregated together to form the final decision.