In this paper we give a partial answer to a 1980 question of Lazslo Babai: “Which [finite] groups admit an oriented graph as a DRR?” That is, which finite groups admit an oriented regular representation (ORR)? We show that every finite non-solvable group admits an ORR, and provide a tool that may prove useful in showing that some families of finite solvable groups admit ORRs. We also completely characterize all finite groups that can be generated by at most three elements, according to whether or not they admit ORRs.