Federated learning (FL) as a promising edge-learning framework can effectively address the latency and privacy issues by featuring distributed learning at the devices and model aggregation in the central server. In order to enable efficient wireless data aggregation, over-the-air computation (AirComp) has recently attracted great attention. However, fading of wireless channels can produce aggregate distortions in an AirComp-based FL scheme. In this paper, we propose a modified federated averaging (FedAvg) algorithm by introducing the local learning rates and present the convergence analysis. To combat the distortion, the local learning rate is optimized to adapt the fading channel, which is termed as dynamic learning rate (DLR). We begin our discussion by considering multiple-input-single-output (MISO) scenario, since the underlying optimization problem is convex and has a closed-form solution. Our studies are extended to a more general multiple-input-multiple-output (MIMO) case and an iterative method is derived. We also present the asymptotic analysis and give a near-optimal and closed-form receive beamforming solution when the number of antennas approaches infinity. Extensive simulation results demonstrate the effectiveness of the proposed DLR scheme in reducing the aggregate distortion and guaranteeing the testing accuracy on the MNIST and CIFAR10 datasets. In addition, the asymptotic analysis and the close-form solution are verified through numerical simulations.