Convolutional neural network (CNN) is a very effective method to recognize facial emotions. However, the preprocessing and selection of parameters of these methods heavily depend on the human experience and require a large amount of trial-and-errors. This paper presents an ensemble of convolutional neural networks method with probability-based fusion for facial expression recognition, where the architecture of CNN was adapted by using the convolutional rectified linear layer as the first layer and multiple hidden maxout layers. It was constructed by randomly varying parameters and architecture around the optimal values for CNN, where each CNN as the base classifier was trained to output a probability for each class. These probabilities were then fused through the probability-based fusion method. The conducted experiments on benchmark data sets validated our method, which had better accuracy than the compared methods. The proposed method was novel and efficient for facial expression recognition.