One of the most important tools for the development of the smart grid is simulation. Therefore, analyzing, designing, modeling, and simulating the smart grid will allow to explore future scenarios and support decision making for the grid's development. In this paper, we compare two open source simulation tools for the smart grid, GridLAB-Distribution (GridLAB-D) and Renewable Alternative Power systems Simulation (RAPSim). The comparison is based on the implementation of two case studies related to a power flow problem and the integration of renewable energy resources to the grid. Results show that even for very simple case studies, specific properties such as weather simulation or load modeling are influencing the results in a way that they are not reproducible with a different simulator.