Perceiving the road environment as robust and complete as possible is a fundamental requirement on the way to fully automated driving. Crucial components of perception include detection and classification of road users as well as estimating their extensions and orientation. Using data from a newly available automotive polarimetric radar, this work presents a neural network model using pre-CFAR data as input to detect road users and additionally predict their oriented bounding box. The model is trained to detect static and dynamic road users in diverse urban and rural scenarios on a large data set. A major improvement in detection performance is shown when comparing against CFAR based detection approaches. Additionally, the benefit of polarimetric information is evaluated by optimizing the model on two representations of polarimetric information and comparing it to a model on data without polarimetric information. Results show further promising performance increases when polarimetric information is available.