This article presents a novel open source toolbox for street network comparison based on the Sextante geoprocessing framework for the open source Geographic Information System Quantum GIS (QGIS). In the spirit of open science, the toolbox enables researchers worldwide to assess the quality of street networks such as OpenStreetMap (OSM) by calculating key performance indicators commonly used in street network comparison studies. Additionally, we suggest two new performance indicators for turn restriction and one‐way street comparisons specifically aimed at testing street network quality for routing. We demonstrate the use of this toolbox by comparing OSM and the official Austrian reference graph “Graph Integration Platform” (GIP) in the greater Vienna region.