How to tackle the unreliability in wireless channels is critical for federated learning (FL). To solve this problem, we propose a novel FL framework, namely FL with gradient recycling (FL-GR), which recycles the historical gradients of unscheduled and transmission-failure devices to improve the learning performance of FL. Based on the proposed FL-GR, we theoretically analyze how the wireless network parameters affect the convergence bound of FL-GR, revealing that scheduling devices with large staleness and increasing their transmit power in each round helps improve learning performance. Simulation results on MNIST and CIFAR-10 show that FL-GR is able to achieve higher accuracy and fast convergence speed than conventional FL algorithms without gradient recycling.