Adaptive parameter control is critical in the design and application of evolutionary algorithm (EA), so does in differential evolution. In the past decade, many adaptive evolutionary algorithms have been proposed, in which online information collected until current generation during the evolutionary search procedure is used to determine the algorithmic parameters for the next generation. Recent studies often assume that the algorithmic parameters follow some distributions, while the distributions' parameters (called hyper-parameters) are updated by the collected information. Performances of these adaptive EAs depend highly on the hyper-parameters. Notice that the experiences obtained from optimizing some related problems could provide useful guidelines on how to adaptively control the distributions' parameters. However, few existing studies sufficiently used such experiences. To fill the gap, we propose a general framework for adaptive parameter control by modeling its evolution procedure as a Markov decision process. In the framework, a neural network is employed to act as the controller. The natural evolution strategies is applied to train the neural network. The proposed framework is applied on two well-known differential evolutions (DEs), namely JADE and LSHADE. By incorporating the learned controller, two DEs, named JADE/AC and LSHADE/AC, are formed. Experimental results on the CEC 2018 benchmark suite show that in general JADE/AC and LSHADE/AC perform significantly better than their counterparts. Moreover, in comparison with some well-known EAs including three suggested best DEs in a review paper (including LSHADE, cDE and CoBiDE), the championship algorithm in the CEC 2018 competitions, a recently-developed learnable DE and recently proposed DEs, our study shows that LSHADE/AC performs the best amongst them without sacrificing much computation time.