LION: Linear Group RNN for 3D Object Detection in Point Clouds
The benefit of transformers in large-scale 3D point cloud perception tasks, such as 3D object
detection, is limited by their quadratic computation cost when modeling long-range …
detection, is limited by their quadratic computation cost when modeling long-range …
Revisiting Attention for Multivariate Time Series Forecasting
H Wu - arXiv preprint arXiv:2407.13806, 2024 - arxiv.org
Current Transformer methods for Multivariate Time-Series Forecasting (MTSF) are all based
on the conventional attention mechanism. They involve sequence embedding and …
on the conventional attention mechanism. They involve sequence embedding and …
Cheems: Wonderful Matrices More Efficient and More Effective Architecture
J Shi, L He, Y Wang, T He, B Wu, M Hou - arXiv preprint arXiv:2407.16958, 2024 - arxiv.org
Recent studies have shown that, relative position encoding performs well in selective state
space model scanning algorithms, and the architecture that balances SSM and Attention …
space model scanning algorithms, and the architecture that balances SSM and Attention …