Census transformation and its variants have gained popularity in image classification for their simplicity and better performance. To describe a texture pattern, these approaches generally use sign information while comparing neighboring pixels. However, our observation is that sign and magnitude in a single color channel as well as in different color channels hold complementary information where sign component captures texture in an image and the saliency of that texture can be captured by the magnitude component. Considering these issues, a multi-channel complementary census transform (MCCT) is proposed in this paper by combining all of these information to capture more discriminating features. Rigorous experiments on nine different datasets which belong to six different applications such as flower, gender, aerial orthoimagery, event, leaf, indoor and outdoor scene classification demonstrate that MCCT outperforms existing state-of-the-art techniques.