Depth video contains depth and disparity information of a scene, which is critical for 3D video systems. In this paper, a convolutional neural network (CNN) based block upsampling method is proposed to improve the efficiency of depth video intra coding. For each largest coding tree in a depth map, it is down-sampled before sent into encoder and recovered into the original size in an intelligent way after low-resolution coding. A novel texture-assisted CNN (TACNN) is presented to handle the depth block up-sampling. The network is made up of several residual coding units and the features of texture block are extracted to assist the reconstruction of the corresponding depth block. Experimental results show that the proposed method achieves competitive rate-distortion performance compared with the state-of-the-art approaches.