Unmanned Aerial Vehicles (UAVs), which can provide an aerial view for fast responding in large-scale zones of disaster, are recently utilized for forest fire monitoring. In this paper, one general model of forest fire detection using aerial videos is investigated to prove its robustness for practical application of aerial forest fire surveillance. Fire pixels are extracted using the color and motion characteristics of fire. The fire detection performance is evaluated through a large database of various scene conditions to show the efficiency as well as deficiency of our fire detection model in previous study. Our database consists of 49 aerial videos with total of 16898 examined frames of forest fires. The accuracy rate of our forest fire detection model is 93.97 % while the false alarm rate and the miss rate are 7.08% and 6.86 %, respectively. Thick smoke which covers almost the fire is found as the main cause of miss detection in our fire detection model. To enhance the detection performance, in this study we propose one more stage of smoke detection. Smoke pixels are segmented using both color and motion characteristics of smoke. The results prove that smoke detection stage give help in detecting the fire area in case of smoke.