Recently, a quasi-fractional order gradient descent (QFGD) algorithm was proposed and successfully applied to solve system identification problem. The QFGD suffers from the overparameterization problem and results in estimating the redundant parameters instead of identifying only the actual parameters of the system. This study develops a novel hierarchical QFDS (HQFGD) algorithm by introducing the concepts of hierarchical identification principle and key term separation idea. The proposed HQFGD is effectively applied to solve the parameter estimation problem of input nonlinear autoregressive with exogeneous noise (INARX) system. A detailed investigation about the performance of HQFGD is conducted under different disturbance conditions considering different fractional orders and learning rate variations. The simulation results validate the better performance of the HQFGD over the standard counterpart in terms of estimation accuracy, convergence speed and robustness.