There currently exists an exciting impetus for increased diversity among medical trainees and improved equity in medical care received by patients. Yet, inclusion of disability within these efforts is often forgotten, allowing the current cultural narrative of ableism to shape medical training. National structural challenges as early as medical school admissions and ableist barriers throughout the educational pipeline have yielded 1) a concerningly low prevalence of medical students and physicians in the US who identify as disabled and 2) propagation of systemic misunderstandings on disability in our healthcare system. This perspective addresses the need for a re-evaluation of diversity in medicine which includes ability status and a commitment to anti-ableism as a critical part of the conversation. We propose reforms and important considerations that could have meaningful implications necessary for improving the culture of disability inclusion in medical education.