In-network data aggregation is an inherent paradigm that extends the lifetime of resource-constrained wireless sensor networks (WSNs). By aggregating sensor data at intermediate nodes, it eliminates data redundancy, minimizes the number of transmissions and saves energy. A key component of in-network data aggregation is the design of an optimal routing structure. However, when the monitoring environment is highly dynamic, the conventional in-network aggregation routing algorithms lead to unnecessary redesign, high overhead and inferior performance, and make in-network aggregation a challenging task. This paper proposes a novel adaptive routing algorithm for in-network aggregation (RINA) in wireless sensor networks. The proposed approach employs a reinforcement learning method called Q-learning to build a routing tree based on minimal information such as residual energy, distance between nodes and link strength. In addition, it finds the aggregation points in the routing structure to maximize the number of overlapping routes in order to increase the aggregation ratio. Theoretical analysis proves the feasibility of the proposed approach. Simulation results show that the aggregation tree constructed by RINA increases the network lifetime by achieving optimum data aggregation and outperform other state-of-the-art approaches in terms of different significant features under different simulation scenarios.