Most of the research efforts in project scheduling assumed complete information about the scheduling problem to be resolved. But in the real world, there are various uncertainties during the project execution, which made the plan, become invalid. Concerned with this problem, firstly, we analyse several major random events which lead to the uncertainty of available resources in manufacturers with make-to-order production strategy and consider the stochastic resource-constrained multi-project scheduling problem (RCMPSP). Then, we establish a Markov decision processes model and relevant procedures for this stochastic RCMPSP. Moreover, in order to deal with the common problem in stochastic optimisation – the curse of dimensionality, we propose a strategy approximation method that to limit the action space and state space by utilising several existing efficient priority rules and predefined probability threshold, respectively. With that we search for the suboptimal strategy to minimise the excepted total tardiness penalty using dynamic programming. Finally, we have given the proposed approach a computational test, result showing that our solution has good practicability.