As the video resolution increases, high computational complexity of the fractional motion estimation (FME) introduces difficulty to meet real-time constraints in a video coding. In this paper, we proposed a single-pass FME algorithm and its architecture with low hardware cost and negligible loss of the image quality. The proposed algorithm directly searches only surroundings of both the predicted fractional motion vector and the search center. To reduce the hardware cost of processing units in the proposed FME architecture, bit clipping scheme is applied to processing units reducing the hardware cost by 25%. Experimental results show that the proposed algorithm provides almost the same rate-distortion performance as the full-search algorithm. The result of hardware implementation shows that a quad full high definition video (4096×2160) can be processed in real time (24 frame/sec) using 134k gates when the operating frequency is 250MHz. Compared with the recent work supporting quad full high definition video, the proposed FME architecture has shown 70% reduction of the hardware cost.