The development of university wide graduate attributes has become an important mechanism for setting generic learning outcomes across higher education institutions. Many universities around the world have developed sets of graduate attributes that they aim to develop through degree programs. Sustainability related attributes have been included in numerous university attribute statements, however, little research exists on methods that can be used to capture data on the application and use of these attributes by graduates. This paper evaluates an assessment tool, based on a scenario/vignette question design, which can be used to capture data on graduate attributes in context and has the potential to be used across a large graduate population which would enable comparative research of learning outcomes to be undertaken. An evaluation of the usefulness of the scenario/vignette design will be presented based on a pilot of the question design with a sample of RMIT University graduates. The results of the pilot show that scenario/vignettes address some of the limitations of existing self-assessment methods and represent a novel way to collect data on graduate sustainability attributes post-degree completion.