With the development of Earth observation programs, more and more multi-temporal synthetic aperture radar (SAR) data are available from remote sensing platforms. Therefore, it is demanding to develop unsupervised methods for SAR image change detection. Recently, deep learning-based methods have displayed promising performance for remote sensing image analysis. However, these methods can only provide excellent performance when the number of training samples is sufficiently large. In this paper, a novel simple method for SAR image change detection is proposed. The proposed method uses two singular value decomposition (SVD) analyses to learn the non-linear relations between multi-temporal images. By this means, the proposed method can generate more representative feature expressions with fewer samples. Therefore, it provides a simple yet effective way to be designed and trained easily. Firstly, deep semi-nonnegative matrix factorization (Deep Semi-NMF) is utilized to select pixels that have a high probability of being changed or unchanged as samples. Next, image patches centered at these sample pixels are generated from the input multi-temporal SAR images. Then, we build SVD networks, which are comprised of two SVD convolutional layers and one histogram feature generation layer. Finally, pixels in both multi-temporal SAR images are classified by the SVD networks, and then the final change map can be obtained. The experimental results of three SAR datasets have demonstrated the effectiveness and robustness of the proposed method.