Reservoir computing is a powerful computational framework that is particularly successful in time-series prediction tasks. It utilises a brain-inspired recurrent neural network and allows biologically plausible learning without backpropagation. Reservoir computing has relied extensively on the self-organizing properties of biological spiking neural networks. We examine the ability of a rate-based network of neural oscillators to take advantage of the self-organizing properties of synaptic plasticity. We show that such models can solve complex tasks and benefit from synaptic plasticity, which increases their performance and robustness. Our results further motivate the study of self-organizing biologically inspired computational models that do not exclusively rely on end-to-end training.