We describe in detail the recently proposed four-dimensional modulation format family based on 2-ary amplitude 8-ary phase-shift keying (2A8PSK), supporting spectral efficiencies of 5, 6, and 7 bits/symbol. These formats nicely fill the spectral efficiency gap between the dual-polarization (DP) quadrature PSK (QPSK) and DP 16-ary quadrature-amplitude modulation (16QAM), with excellent linear and nonlinear performance. Since these modulation formats just use different parity bit expressions in the same constellation, similar digital signal processing can be seamlessly used for different spectral efficiency. A series of nonlinear transmission simulation results shows that this modulation format family outperforms the conventional modulation formats at the corresponding spectral efficiency. We also investigate the adaptive equalizer for these modulation formats.