Falling is an important health issue, but how can we detect falls automatically? One possibility would be a camera-based system. In this paper, we present a grayscale video processing algorithm to detect falls in video data. Grayscale video data is chosen because it has several advantages which will be presented in this paper.We start with background subtraction after which we continue to shadow removal, ellipse fitting and fall detection based on fall angle and aspect ratio. When a fall is detected, we continue to fall confirmation using vertical projection histograms.