The popularity of unmanned aerial vehicles has exploded over the last few years, urgently demanding solutions to transfer large amounts of data from the UAV to the ground. Conversely, a control channel to the UAV is desired, in order to safely operate these vehicles remotely. This article analyzes the use of LTE for realizing this downlink data and uplink control. By means of measurements and simulations, we study the impact of interference and path loss when transmitting data to and from the UAV. Two scenarios are considered in which UAVs act as either base stations transmitting in downlink or UEs transmitting in uplink, and their impact on the respective downlink and uplink performance of an LTE ground network is analyzed. Both measurements and simulations are used to quantify such impact for a range of scenarios with varying altitude, distance from the base station, or UAV density. The measurement sets show that signal-to-interference ratio decreases up to 7 dB for UAVs at 150 m compared to ground users. Simulation results show that a UAV density of 10/km2 gives an average degradation of the signal-to-interference ratio of more than 6 dB. It is concluded that interference is going to be a major limiting factor when LTE enabled UAVs are introduced, and that strong technical solutions will have to be found.