We investigated optical nonlinearity induced by electron tunneling through an insulating vertical gap between metals, both at terahertz frequency and at near-infrared frequency. We adopted graphene and alumina layers as gap materials to form gap widths of 3 Å and 1.5 nm, respectively. Transmission measurements show that tunneling-induced transmittance changes from strong fields at the gaps can be observed with relatively weak incident fields at terahertz frequency due to high field enhancement, whereas nonlinearity at the near-infrared frequency is restricted by laser-induced metal damages. Even when the same level of tunneling currents occurs at both frequencies, transmittance in the terahertz regime decreases much faster than that in the near-infrared regime. An equivalent circuit model regarding the tunneling as a resistance component reveals that strong terahertz nonlinearity is due to much smaller displacement currents relative to tunneling currents, also explaining small nonlinearity of the near-infrared regime with orders of magnitude larger displacement currents.