In this paper, a combined viscoelasticity–viscoplasticity model, coupled with anisotropic damage and moisture effects, is developed for short fiber reinforced polymers (SFRPs) with different fiber contents and subjected to a variety of strain rates. In our model, a rate-dependent yield surface for the matrix phase is employed to identify initial yielding of the material. When an SFRP is loaded at small deformation before yielding, its viscoelastic behavior can be described using the generalized Maxwell model, while when plasticity occurs, a scalar internal state variable (ISV) is used to capture the hardening behavior caused by the polymeric constituent of the composite. The material degradation due to the moisture absorption of the composite is modeled by employing another type of ISV with different evolution equations. The complicated damage state of the SFRPs is captured by a second rank tensor, which is further decomposed to model the subscale damage mechanisms of micro-voids/cracks nucleation, growth and coalescence. It is concluded that the proposed constitutive model can be used to accurately describe complicated behaviors of SFRPs because the results predicted from the model are in good agreement with the experimental data.