Size-affected dislocation-mediated plasticity is important in a wide range of materials and technologies. Here we develop a generalized size-dependent dislocation-based model that predicts strength as a function of crystal/grain size and the dislocation density. Three-dimensional (3D) discrete dislocation dynamics (DDD) simulations reveal the existence of a well-defined relationship between strength and dislocation microstructure at all length scales for both single crystals and polycrystalline materials. The results predict a transition from dislocation-source strengthening to forest-dominated strengthening at a size-dependent critical dislocation density. It is also shown that the Hall–Petch relationship can be physically interpreted by coupling with an appropriate kinetic equation of the evolution of the dislocation density in polycrystals. The model is shown to be in remarkable agreement with experiments. This work presents a micro-mechanistic framework to predict and interpret strength size-scale effects, and provides an avenue towards performing multiscale simulations without ad hoc assumptions.