In the present study, a flame flashback phenomenon inside an ethylene-fuelled scramjet combustor equipped with a cavity flameholder is investigated under flight Mach 5.5 condition. Experimental results exhibit the quasi-periodic combustion oscillation between the fuel injectors and the leading edge of the cavity under a specified condition. As an indispensable key sub-process of combustion oscillation, the flame flashback from the boundary layer downstream of the cavity is responsible for unsteady combustion process in scramjet combustors. Numerical simulation has been carried out in the specified condition. Attributed by (i) thicker boundary layer, (ii) closer thermal disturbance and (iii) improved local mixing degree, the flame flashback phenomenon can be induced by thermal throat which is generated by interaction between separated boundary layer and intense combustion. Quantitative analysis also indicates that the flame flashing is more sensitive to temperature fluctuation downstream of the cavity. In addition, a simplified combustion opening system model has been established to analyse combustion oscillation mechanisms, which theoretically demonstrates that the three factors mentioned above can destroy the balance of heat release and dissipation, causing the system cannot self-stabilise once certain temperature fluctuation thresholds in sensitive areas are exceeded. At the same time, the auto-ignition model excludes the possibility of flame flashback generated by auto-ignition effect.