Driven by plummeting camera prices and advances of video inference algorithms, video cameras are deployed ubiquitously and organizations usually rely on live video analytics to retrieve key information, such as the locations and identities of target objects. However, analyzing real-time video poses severe challenges to today’s network and computation systems. To balance accuracy, bandwidth usage, and latency, we present EVA, an edge-assisted real-time video analytics framework, which coordinates computationally weak cameras with more powerful edge servers to enable video analytics under the accuracy and latency requirements of applications. EVA treats the region where a target object is located as a fine-grained transmission unit and exploits the redundancies in both spatial and temporal domains to reduce the bandwidth usage. Based on the framework, we design an adaptive offloading algorithm, which coordinates the recognition process between the camera and the server. To adapt to complex environments, we then design a threshold adjustment algorithm to tune the confidence threshold dynamically. Experiments on real-world video feeds show that compared to several recent baselines on multiple video genres, EVA maintains high accuracy while reducing bandwidth usage by up to 90%.