Mobile edge computing (MEC), as a novel computing paradigm, promises dramatic reduction in latency and energy consumption by offloading computation-intensive tasks to edge clouds in close proximity to mobile users. However, compared with conventional mobile cloud computing (MCC), it has finite computation capacity, especially under the scenario of multi-user offloading. Thus, how to optimize mobile energy consumption and utilize computational resources effectively have being challenged MEC design. In this paper, we consider a multi-user MEC system design, where the multiple mobile devices ask for computation offloading to a MEC server according to the system state. We formulate the computation offloading as a stochastic dynamic programming (SDP) problem, aiming at minimizing the long-term cumulative system cost. To achieve this goal, we propose an approximate dynamic programming (ADP) approach, by employing a well-designed value function approximation (VFA) architecture and a stochastic gradient learning strategy, enabling the state value be parameterized and recursively estimated step by step. Extensive simulations are conducted to verify the effectiveness of our approach with other heuristic approaches under different parameters settings, referring to network bandwidth, task size and number of mobile devices.