A good understanding of signal wave attenuation along a stairwell is necessary for successful wireless network planning. In this study, the propagation characteristics in indoor multi-floor stairwell environments are examined under the millimeter frequency band. An extensive signal measurement is conducted in two stairwells. These measurements provide information important for ensuring consistent radio transmission of data in such environment, which then enables immediate response during emergency operations. In these measurements, directional horn antennas for co-polarization and cross-polarization in transmitters and receivers are used for different millimeter frequencies (26, 28, 32, and 38 GHz). The feasibility of using the millimeter wave, which is suitable for indoor 5G wireless networks, in two different indoor stairwell environments is investigated. Four different models are employed for path loss investigation. Under single frequencies, the close-in free-space reference distance and floating-intercept path loss models are adopted. For multiple frequencies, the close-in free space reference distance with frequency dependent path loss exponent and alpha-beta-gamma models are applied. The measurements provide data on the received power at multi-floor stairwells through the stair steps, which are useful for path loss study. Accordingly, the path loss exponent values, standard deviations, and other parameters are obtained.