Adaptive mutual supervision for weakly-supervised temporal action localization

C Ju, P Zhao, S Chen, Y Zhang, X Zhang… - IEEE Transactions …, 2022 - ieeexplore.ieee.org
IEEE Transactions on Multimedia, 2022ieeexplore.ieee.org
Weakly-supervised temporal action localization aims to localize actions from untrimmed long
videos with only video-level category labels. Most previous methods ignore the
incompleteness issue of Class Activation Sequences (CAS), suffering from trivial detection
results. To tackle this issue, we propose a novel Adaptive Mutual Supervision (AMS)
framework with two branches, where the base branch detects the most discriminative action
regions, while the supplementary branch localizes the less discriminative action regions …
Weakly-supervised temporal action localization aims to localize actions from untrimmed long videos with only video-level category labels. Most previous methods ignore the incompleteness issue of Class Activation Sequences (CAS), suffering from trivial detection results. To tackle this issue, we propose a novel Adaptive Mutual Supervision (AMS) framework with two branches, where the base branch detects the most discriminative action regions, while the supplementary branch localizes the less discriminative action regions through an adaptive sampler. The sampler dynamically updates the inputs for the supplementary branch using a sampling weight sequence negatively correlated with the CAS from the base branch, thus encouraging the supplementary branch to localize the action regions underestimated by the base branch. To promote mutual enhancement between two branches, we further construct mutual location supervision. Each branch adopts the location pseudo-labels generated from the other branch as the localization supervision. By alternately optimizing two branches for multiple iterations, we progressively complete action regions. Extensive experiments on THUMOS14 and ActivityNet1.2 demonstrate that the proposed AMS method significantly outperforms state-of-the-art methods.
ieeexplore.ieee.org
以上显示的是最相近的搜索结果。 查看全部搜索结果