Compared with a single robot, the wireless robotic network provides more reliable and efficient services. When tasks are not independent and the movement of robots is constrained by wireless connectivity, coordination and cooperation are required to efficiently allocate tasks. Traditionally, the task allocation problem is formulated as a mixed-integer quadratically constrained quadratic programming, which is difficult to solve and the solution is not scalable. This paper studies the sequential task allocation for wireless robotic networks, where robots are subject to wireless connectivity constraints and the tasks are stochastic. The objective of this paper is to reduce the task completion time by using multiple robots. A one-dimensional motion along a straight line with applications for pipeline monitoring and tunnel exploration is considered. First, a baseline is developed for sequential task allocation using the greedy algorithm. Then, a deep reinforcement learning model with offline training is introduced, which can efficiently reduce the task completion time. To further improve the performance, the online rollout for reinforcement learning is employed. Wireless communication protocols and lower bounds of task completion time are also developed. The results show that robots can gradually learn the optimal policy and efficiently address the sequential task allocation problem.