Sign Language is an interactive system for sensory and gustatory impaired people for communicating information among them. Hand Gesture Recognition (HGR) is a system for sign language detection which uses computers for enabling communication with sensory impaired people. In this research work, an efficient and effective technique is introduced for the identification of the number of fingers opened or closed in a gesture representing an alphabet of the American Sign Language (ASL). Finger Detection is accomplished with the use of Contour Hull and Convexity Defects. Contour Hull helps to form the boundary of the hand gestures whereas the Convexity Defects helps to find tip of the fingers captured by the web camera. This proposed system applies image processing techniques to store the pre-processed data, Machine learning technique is used in supervised clustering and computer vision techniques is used to recognize the hand gestures. Hence this intelligent system does not require the hand to be perfectly aligned to the camera for detecting the signs. In addition to this, to detect the signs, this system does not need any colored markers or gloves to be wearied on the hands.