Fashion design is an art that reshapes the designers’ imagination into visible content which requires a significant amount of time and effort. The assistance provided by the available design tools are limited in the sense of visualizing and fitting of the generated cloth on the human body. We present, ARGAN-an Augmented Reality (AR) based Fashion Design system which is able to generate a new dress when a sketch and a theme image are provided as the input into a Controllable Generative Adversarial Network. Further, this system can visualize the generated virtual 2D apparel in realtime on a real human body using Augmented Reality. To the best of our knowledge, this work is the first attempt at utilizing Deep Generative Models (e.g. GANs) in an Augmented Reality prototype in fashion designing for generate creative fashion content in 2D and exploiting the possibility of Deep Generative Models to generate fashion designs align to a theme. Our findings show that the use of the ARGAN can support fashion designers’ during their designing process.