Deep neural networks have been shown to outperform classical machine learning algorithms in solving real-world problems. However, the most successful deep neural networks were handcrafted from scratch taking the problem domain knowledge into consideration. This approach often consumes very significant time and computational resources. In this work, we propose a novel algorithm based on particle swarm optimization (PSO), capable of fast convergence when compared with others evolutionary approaches, to automatically search for meaningful deep convolutional neural networks (CNNs) architectures for image classification tasks, named psoCNN. A novel directly encoding strategy and a velocity operator were devised allowing the optimization use of PSO with CNNs. Our experimental results show that psoCNN can quickly find good CNN architectures that achieve quality performance comparable to the state-of-the-art designs.