Rapid urbanization has brought about great challenges to our daily lives, such as traffic congestion, environmental pollution, energy consumption, public safety, and so on. Research on smart cities aims to address these issues with various technologies developed for the Internet of Things. Very recently, the research focus has shifted toward processing of massive amount of data continuously generated within a city environment, e.g., physical and participatory sensing data on traffic flow, air quality, and health care. Techniques from computational intelligence have been applied to process and analyze such data, and to extract useful knowledge that helps citizens better understand their surroundings and informs city authorities to provide better and more efficient public services. Deep learning, as a relatively new paradigm in computational intelligence, has attracted substantial attention of the research community and demonstrated greater potential over traditional techniques. This paper provides a survey of the latest research on the convergence of deep learning and smart city from two perspectives: while the technique-oriented review pays attention to the popular and extended deep learning models, the application-oriented review emphasises the representative application domains in smart cities. Our study showed that there are still many challenges ahead for this emerging area owing to the complex nature of deep learning and wide coverage of smart city applications. We pointed out a number of future directions related to deep learning efficiency, emergent deep learning paradigms, knowledge fusion and privacy preservation, and hope these would move the relevant research one step further in creating truly distributed intelligence for smart cities.