This paper introduces two deep convolutional neural network training techniques that lead to more robust feature subspace separation in comparison to traditional training. Assume that dataset has M labels. The first method creates M deep convolutional neural networks called . Each of the networks is composed of a convolutional neural network () and a fully connected neural network (). In training, a set of projection matrices are created and adaptively updated as representations for feature subspaces . A rejection value is computed for each training based on its projections on feature subspaces. Each acts as a binary classifier with a cost function whose main parameter is rejection values. A threshold value is determined for network . A testing strategy utilizing is also introduced. The second method creates a single DCNN and it computes a cost function whose parameters depend on subspace separations using the geodesic distance on the Grasmannian manifold of subspaces and the sum of all remaining subspaces . The proposed methods are tested using multiple network topologies. It is shown that while the first method works better for smaller networks, the second method performs better for complex architectures.