During recent years, channel attention has attracted great interest in deep learning community. Despite significant success, it has been rarely exploited in ubiquitous human activity recognition (HAR) scenario. To decrease computational overhead, the channel attention often uses global averaging pooling (GAP) to compress each channel into a simple scalar. It is well known that GAP is equal to the lowest frequency component. Despite obvious lightweight advantage, such compression process inevitably causes severe information loss. In this paper, we propose a novel multi-frequency channel attention framework for activity recognition tasks. Considering various sensing frequencies of human activities, an intuition solution is to convert the time series from time domain to frequency domain. Instead of GAP, the discrete cosine transform (DCT) is used to compress channels. We prove that GAP can be seen as a special case of DCT, which uses the lowest frequency component only and leaves out all other frequency components unused. DCT is able to better compress channels by fully exploiting other frequency components discarded by GAP. Despite multiple frequency components used, each channel will still be represented by a scalar in order to maintain the same computational overhead. Using two frequency screening criteria, our method is able to achieve state-of-the-art results on four benchmark HAR datasets. Extensive ablation studies are conducted, which provides a better interpretability of deep model behaviors. Finally, actual inference is evaluated on an embedded platform.