In this paper, creation of the Language Models (LMs) and Acoustic Models (AMs) using Kaldi speech recognition toolkit to build a robust Automatic Speech Recognition (ASR) system for Kannada language is demonstrated. The speech data is collected from the farmers of Karnataka under uncontrolled environment is used for the development of ASR models. The collected speech data needs to be translated to machine level language and hence the Indic Language Transliteration Tool (IT3 to UTF-8) is used for transcription. The dictionary for the collected speech data is created by using Indian Language Speech sound Label (ILSL12) set. The AMs are created by using Gaussian Mixture Model (GMM) and Subspace GMM (SGMM). The 80% and 20% of validated speech data is used for training and testing respectively. The accuracy and Word Error Rate (WER) of ASR models are highlighted and discussed in this work. The developed ASR models can be used in spoken query system which enables the farmers to access the on time agricultural commodity prices and weather information in Kannada language.