In challenging environments where human intervention is expensive, robust and persistent autonomy is a key requirement. AI Planners can efficiently construct plans to achieve this long-term autonomous behaviour. However, in plans which are expected to last over days, or even weeks, the size of the state-space becomes too large for current planners to solve as a single problem. These problems are well-suited to decomposition and abstraction planning techniques. We present a novel approach in the context of persistent autonomy in autonomous underwater vehicles, in which tasks are complex and diverse and plans cannot be precomputed. Our approach performs a decomposition into a two-level hierarchical structure, which dynamically constructs planning problems at the upper level of the hierarchy using solution plans from the lower level. Solution plans are then executed and monitored simultaneously at both levels. We evaluate the approach, showing that compared to strictly top-down hierarchical decompositions, our approach leads to more robust solution plans of higher quality.