Neighborhood physical disorder is thought to affect mental and physical health but has been difficult to measure objectively and reliably across large geographical areas or multiple locales. Virtual street audits are a novel method for assessing neighborhood characteristics. We evaluated the ecometric properties of a neighborhood physical disorder measure constructed from virtual street audit data. Eleven trained auditors assessed nine previously validated items developed to capture physical disorder (eg litter, graffiti, and abandoned buildings) on 1,826 block faces using Google Street View (Google, Inc., Mountain View, CA) imagery dating from 2007-2011 in four US cities (San Jose, California; Detroit, Michigan; New York, New York; and Philadelphia, Pennsylvania). We constructed a 2-parameter Item Response Theory scale to estimate latent levels of disorder on each block face and defined a function using kriging to estimate physical disorder levels, with confidence estimates, for any point across each city. Internal consistency reliability of the resulting scale was 0.93. The final interpolated measure of disorder was positively correlated with US Census data on unemployment and housing vacancy, and negatively correlated with US Census data on owner-occupied housing. These results suggest neighborhood physical disorder can be measured reliably and validly using virtual audits, facilitating research concerning such disorder’s health effects.