A survey on time-series pre-trained models

Q Ma, Z Liu, Z Zheng, Z Huang, S Zhu… - … on Knowledge and …, 2024 - ieeexplore.ieee.org
Time-Series Mining (TSM) is an important research area since it shows great potential in
practical applications. Deep learning models that rely on massive labeled data have been …

State space model for new-generation network alternative to transformers: A survey

X Wang, S Wang, Y Ding, Y Li, W Wu, Y Rong… - arXiv preprint arXiv …, 2024 - arxiv.org
In the post-deep learning era, the Transformer architecture has demonstrated its powerful
performance across pre-trained big models and various downstream tasks. However, the …

Mambular: A sequential model for tabular deep learning

AF Thielmann, M Kumar, C Weisser, A Reuter… - arXiv preprint arXiv …, 2024 - arxiv.org
The analysis of tabular data has traditionally been dominated by gradient-boosted decision
trees (GBDTs), known for their proficiency with mixed categorical and numerical features …

Fmamba: Mamba based on fast-attention for multivariate time-series forecasting

S Ma, Y Kang, P Bai, YB Zhao - arXiv preprint arXiv:2407.14814, 2024 - arxiv.org
In multivariate time-series forecasting (MTSF), extracting the temporal correlations of the
input sequences is crucial. While popular Transformer-based predictive models can perform …

Unlocking the power of lstm for long term time series forecasting

Y Kong, Z Wang, Y Nie, T Zhou, S Zohren… - arXiv preprint arXiv …, 2024 - arxiv.org
Traditional recurrent neural network architectures, such as long short-term memory neural
networks (LSTM), have historically held a prominent role in time series forecasting (TSF) …

Structural Inference of Dynamical Systems with Conjoined State Space Models

A Wang, J Pang - The Thirty-eighth Annual Conference on Neural …, 2024 - openreview.net
This paper introduces SICSM, a novel structural inference framework that integrates
Selective State Space Models (selective SSMs) with Generative Flow Networks (GFNs) to …

TDG-Mamba: Advanced Spatiotemporal Embedding for Temporal Dynamic Graph Learning via Bidirectional Information Propagation

M Li, J Chen, B Li, Y Zhang, R Zhang… - IEEE Transactions …, 2024 - ieeexplore.ieee.org
Temporal dynamic graphs (TDGs), representing the dynamic evolution of entities and their
relationships over time with intricate temporal features, are widely used in various real-world …

Mamba4Cast: Efficient Zero-Shot Time Series Forecasting with State Space Models

SK Bhethanabhotla, O Swelam, J Siems… - arXiv preprint arXiv …, 2024 - arxiv.org
This paper introduces Mamba4Cast, a zero-shot foundation model for time series
forecasting. Based on the Mamba architecture and inspired by Prior-data Fitted Networks …

CMMamba: channel mixing Mamba for time series forecasting

Q Li, J Qin, D Cui, D Sun, D Wang - Journal of Big Data, 2024 - Springer
Transformer-based methods have achieved excellent results in the field of time series
forecasting due to their powerful ability to model sequences and capture their long-term …

Enhanced Linear and Vision Transformer-Based Architectures for Time Series Forecasting

M Alharthi, A Mahmood - Big Data and Cognitive Computing, 2024 - mdpi.com
Time series forecasting has been a challenging area in the field of Artificial Intelligence.
Various approaches such as linear neural networks, recurrent linear neural networks …