We introduce a class of proper scoring rules for evaluating spatial point process forecasts based on summary statistics. These scoring rules rely on Monte-Carlo approximation of an expectation and can therefore easily be evaluated for any point process model that can be simulated. In this regard they are more flexible than the commonly used logarithmic score which cannot be evaluated for many point process models, as their density is only known up to an untractable constant. In simulation studies we demonstrate the usefulness of our scores. Furthermore we consider a scoring rule, the quantile score, that is commonly used to validate earthquake rate predictions, and show that it lacks propriety. As a consequence, several tests that are commonly applied in this context are biased and systematically favour predictive distributions that are too uniform. We suggest to remedy this issue by replacing the commonly used one-sided by two-sided tests.