Network slicing is considered a promising networking pillar of efficient resource management in beyond 5G (B5G) networks. However, the dynamic and complex characteristics of future networks pose challenges in designing novel resource allocation techniques due to the stringent quality of service (QoS) requirements and virtualized network infrastructures. To solve this issue, we propose a digital twin (DT)-enabled deep distributional Q-network (DDQN) framework that constructs a digital mirror of the physical slicing-enabled network to simulate its complex environment and predict the dynamic characteristics of the network. The DT of network slicing is expressed as a graph, and a graph neural network (GNN) is developed to learn the complicated relationships of the network slice. The graph-based network states are forwarded to the DDQN agent to learn the optimal network slicing policy. Through simulations, it is demonstrated that the proposed technique can satisfy the stringent QoS requirements and achieve near-optimal performance in a dynamic B5G network.