We propose a method, called HAF, to estimate planar homography from an affine correspondence satisfying the epipolar constraint in an image pair. An affine correspondence consists of a point pair and the related local affine transformation mapping the pixels infinitely close to the point locations from the first to the second images. As a minimal solver, it estimates the homography from only one correspondence, however, it is generalized for the over-determined case as well. The required local affinities are obtained by affine-covariant feature detectors accurately. As a side-effect of the tests, the state-of-the-art affine-covariant detectors are compared to each other w.r.t. the accuracy of the estimated point-wise homographies. The proposed method is validated both on the publicly available AdelaideRMF dataset and in a synthetic testing environment.