Internet of Things (IoT) and smart computing technologies have revolutionized every sphere of 21 ^st century humans. IoT technologies and the data driven services they offer were beyond imagination just a decade ago. Now, they surround us and influence a variety of domains such as automobile, smart home, healthcare, etc. In particular, the Agriculture and Farming industries have also embraced this technological intervention. Smart devices are widely used by a range of people from farmers to entrepreneurs. These technologies are used in a variety of ways, from finding real-time status of crops and soil moisture content to deploying drones to assist with tasks such as applying pesticide spray. However, the use of IoT and smart communication technologies introduce a vast exposure to cybersecurity threats and vulnerabilities in smart farming environments. Such cyber attacks have the potential to disrupt the economies of countries that are widely dependent on agriculture. In this paper, we present a holistic study on security and privacy in a smart farming ecosystem. The paper outlines a multi layered architecture relevant to the precision agriculture domain and discusses the security and privacy issues in this dynamic and distributed cyber physical environment. Further more, the paper elaborates on potential cyber attack scenarios and highlights open research challenges and future directions.