A novel metric for quantitatively measuring the content accessibility of the Web for persons with disabilities is proposed. The metric is based on the Web Content Accessibility Guidelines (WCAG) checkpoints, an internationally accepted standard, that can be automatically tested using computer programs. Problems with current accessibility evaluation and the need for a good Web accessibility metric are discussed. The proposed metric is intended to overcome the deficiencies of the current measurements used in Web accessibility studies. The proposed metric meets the requirements as a measurement for scientific research. Examples of large‐scale Web accessibility evaluations using the metric are given. The examples cover a comparison of Web accessibility of top medical journal Web sites and a longitudinal study of a Web site over time. The validity of the metric was tested using a large number of Web sites with different levels of compliance (rating categories) to the standard WCAG. The metric, which uses a predetermined simple weighting scheme, compares well to the more complex C5.0 machine learning algorithm in separating Web sites into different rating categories.