We employ gigahertz frequency ultrasonic waves to non-invasively image grain microstructure in polycrystalline ceria. This technique, termed time-domain Brillouin scattering (TDBS), is a pump-probe technique that launches and monitors ultrasonic waves as they propagate into the sample. We demonstrate TDBS's capability to image microstructure by measuring mode and depth dependent acoustic velocities. The technique provides sufficient contrast to identify grains and offers means to reconstruct subsurface grain boundary's location. The results compare closely with electron backscatter diffraction measurement. This work expands the application of TDBS to 3D structural imaging and offers new route for the nondestructive characterization of grain boundary.