The advanced Internet of Things (IoT) enables more and more interactions between people and machines in the emerging applications, which rely on real-time communication and computing. However, the limited battery capacity and low computing capacity of IoT nodes can hardly support high-performance computing applications. The integration of wireless power transmission (WPT) and mobile edge computing (MEC) is a feasible and promising solution to address the energy shortage and computing capacity limitation of IoT nodes by harvesting radio frequency signal’s energy and offloading the nodes’ computation tasks to edge computing servers (ECSs). In this work, we focus on the wireless powered MEC network with an ECS and multiple edge devices (EDs), and study the joint optimization of WPT duration, transmission time allocation of each ED and partial offloading decision to maximize the sum computation rate. First, we formulate this as a non-convex problem which is hard to solve. Second, to conquer this problem, we decompose the original offloading problem into the sub-problem of optimizing the offloading time allocation among EDs and the proportion of harvested energy allocated for offloading at each ED under a given WPT duration and the top-problem of optimizing the WPT duration. Finally, we design an online DRL-based framework where one DNN together with its exploration strategy and training strategy is adopted to learn the near-optimal WPT duration and an efficient optimal algorithm is designed to solve the sub-problem. Numerical results show that the DRL-based offloading algorithm achieves the near-maximal sum computation rate while greatly reducing the processing time by at least three orders of magnitude compared with using the solver CVX for the sub-problem and the DNN for the top-problem.