Gold nanorods (NRs) with tunable plasmon‐resonant absorption in the near‐infrared region have considerable advantages over organic fluorophores as imaging agents due to their brightness and lack of photobleaching. However, the luminescence spectral properties of NRs have not been fully characterized at the single particle level due to lack of proper analytic tools. Here, we present a spectral phasor analysis method that allows investigations of NRs' spectra at single particle level showing the spectral variance and providing spatial information during imaging. The broad phasor distribution obtained by the spectral phasor analysis indicates that spectra of NRs are different from particle to particle. NRs with different spectra can be identified in images with high spectral resolution. The spectral behaviors of NRs under different imaging conditions, for example, different excitation powers and wavelengths, were revealed by our laser‐scanning multiphoton microscope using a high‐resolution spectrograph with imaging capability. Our results prove that the spectral phasor method is an easy and efficient tool in hyper‐spectral imaging analysis to unravel subtle changes of the emission spectrum. We applied this method to study the spectral dynamics of NRs during direct optical trapping and by optothermal trapping. Interestingly, different spectral shifts were observed in both trapping phenomena. Microsc. Res. Tech. 78:283–293, 2015. © 2015 Wiley Periodicals, Inc.