Image alignment is an important step in various biometric authentication applications such as palmprint recognition. Most of the existing palmprint alignment methods make use of some key points between fingers or in palm boundary to establish the local coordinate system for region of interest (ROI) extraction. The ROI is consequently used for feature extraction and matching. Such alignment methods usually yield a coarse alignment of the palmprint images, while many missed and false matches are actually caused by inaccurate image alignments. To improve the palmprint verification accuracy, in this paper, we present an efficient palmprint alignment refinement method. After extracting the principal lines from the palmprint image, we apply the iterative closest point method to them to estimate the translation and rotation parameters between two images. The estimated parameters are then used to refine the alignment of palmprint feature maps for a more accurate palmprint matching. The experimental results show that the proposed method greatly improves the palmprint recognition accuracy and it works in real time.