Sign Language is a language which uses visually transmitted sign patterns to convey meaning by simultaneously combining hand shapes, orientation and movement of the hands, arms or body, and facial expressions to fluently express one's thoughts/communicate with others and is commonly used by the physically impaired people who cannot speak or hear. Automatic Sign Language system requires fast and accurate techniques for identifying static signs or a sequence of produced signs to help interpret their correct meaning. Major components of a Sign Languages are Hand Gesture. In this paper, a robust approach for recognition of bare-handed static sign language is presented, using a novel combination of features. These include Local Binary Patterns (LBP) histogram features based on color and depth information, and also geometric features of the hand. Linear binary Support Vector Machine (SVM) classifiers are used for recognition, coupled with template matching in the case of multiple matches. This research aim towards working on hand gesture recognition for sign language interpretation as a Human Computer Interaction application.