Automatic human face detection from images in surveillance and biometric applications is a challenging task due to variations in image background, view, illumination, articulation, and facial expression. We propose a novel three-step face detection approach to addressing this problem. The approach adopts a simple-to-complex strategy. First, a linear-filtering algorithm is applied to enhance detection performance by removing most nonface-like candidates rapidly. Second, a boosting chain algorithm is adopted to combine the boosting classifiers into a hierarchical "chain" structure. By utilizing the inter-layer discriminative information, this algorithm reveals a higher efficiency than traditional approaches. Last, a postfiltering algorithm, consisting of image preprocessing; support vector machine-filter and color-filter, is applied to refine the final prediction. As only a few candidate windows remain in the final stage, this algorithm greatly improves detection accuracy with small computation cost. Compared with conventional approaches, this three-step approach is shown to be more effective and capable of handling more pose variations. Moreover, together with a two-level hierarchy in-plane pose estimator, a rapid multiview face detector is built. Experimental results demonstrate a significant performance improvement for the proposed approach over others.