Fifth generation (5G), the next generation telecommunications will be striking the markets in near future. Device‐to‐device (D2D) communication would be a part of 5G to serve communication needs for billions of connected devices to support high data rate ultrareliable low latency communications. Indoor 5G will be relying on distributed small cell solutions and D2D along with machine‐to‐machine connections. Machine learning is one of the most promising tools for providing the best set of solutions to learn the influential scenarios and certain parameters of the communication networks. This research proposes reinforcement‐learning‐based latency controlled D2D connectivity (RL‐LCDC) algorithm and its Q‐learning approach in an indoor D2D communication network for strong 5G connectivity with minimum latency. The proposed approach, RL‐LCDC efficiently discovers the neighbors, decides the D2D link, and adaptively controls the communication range for maximum network connectivity. The results show that RL‐LCDC optimizes the connectivity with lower end‐to‐end delay and better energy efficiency with efficient convergence time when compared with other conventional schemes.