With the development of deep learning, many methods on image denoising have been proposed processing images on a fixed scale or multi-scale which is usually implemented by convolution or deconvolution. However, excessive scaling may lose image detail information, and the deeper the convolutional network the easier to lose network gradient. Diamond Denoising Network (DmDN) is proposed in this paper, which mainly based on a fixed scale and meanwhile considering the multi-scale feature information by using the Diamond-Shaped (DS) module to deal with the problems above. Experimental results show that DmDN is effective in image denoising.