Time series forecasting / prediction

[amotl]

About

At ¹², we shared a few notes about time series anomaly detection, and forecasting/prediction. Other than using traditional statistics-based time series forecasting methods like Holt-Winters or ARIMA, and libraries like Prophet and friends, other kinds of prediction methods are emerging, based on machine learning models and outcomes from deep learning operations, like TimeGPT, or Chronos, that allow for zero-shot inference.

TimeGPT-1

• https://www.nixtla.io/timegpt
• https://docs.nixtla.io/
• https://github.com/nixtla

Note

Azul Garza, Max Mergenthaler-Canseco; Nixtla; San Francisco, CA, USA; 5 Oct 2023

In this paper, we introduce TimeGPT, the first foundation model for time series, capable of generating accurate predictions for diverse datasets not seen during training. We evaluate our pre-trained model against established statistical, machine learning, and deep learning methods, demonstrating that TimeGPT zero-shot inference excels in performance, efficiency, and simplicity.

Our study provides compelling evidence that insights from other domains of artificial intelligence can be effectively applied to time series analysis. We conclude that large-scale time series models offer an exciting opportunity to democratize access to precise predictions and reduce uncertainty by leveraging the capabilities of contemporary advancements in deep learning.

-- https://arxiv.org/pdf/2310.03589.pdf

Chronos: Learning the Language of Time Series

Chronos is a family of pretrained time series forecasting models based on language model architectures. A time series is transformed into a sequence of tokens via scaling and quantization, and a language model is trained on these tokens using the cross-entropy loss. Once trained, probabilistic forecasts are obtained by sampling multiple future trajectories given the historical context. Chronos models have been trained on a large corpus of publicly available time series data, as well as synthetic data generated using Gaussian processes.

• https://github.com/amazon-science/chronos-forecasting
• https://news.ycombinator.com/item?id=39787176

Note

Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Bohlke-Schneider, Yuyang Wang

Amazon Web Services, UC San Diego, University of Freiburg, Amazon Supply Chain Optimization Technologies; 12 Mar 2024

We introduce Chronos, a simple yet effective framework for pretrained probabilistic time series models. Chronos tokenizes time series values using scaling and quantization into a fixed vocabulary and trains existing transformer-based language model architectures on these tokenized time series via the cross-entropy loss. We pretrained Chronos models based on the T5 family (ranging from 20M to 710M parameters) on a large collection of publicly available datasets, complemented by a synthetic dataset that we generated via Gaussian processes to improve generalization.

In a comprehensive benchmark consisting of 42 datasets, and comprising both classical local models and deep learning methods, we show that Chronos models: (a) significantly outperform other methods on datasets that were part of the training corpus; and (b) have comparable and occasionally superior zero-shot performance on new datasets, relative to methods that were trained specifically on them. Our results demonstrate that Chronos models can leverage time series data from diverse domains to improve zero-shot accuracy on unseen forecasting tasks, positioning pretrained models as a viable tool to greatly simplify forecasting pipelines.

-- https://arxiv.org/pdf/2403.07815.pdf

Footnotes

1. https://kotori.readthedocs.io/en/latest/development/research/timeseries-analysis.html ↩

2. https://kotori.readthedocs.io/en/latest/development/backlog.html ↩

[amotl]

ATFNet

Adaptive Time-Frequency Ensembled Network for Long-term Time Series Forecasting.

https://github.com/YHYHYHYHYHY/ATFNet

[amotl]

EarthPT

A simple repository for training time series large observation models. This repository began its life as Andrej Karpathy's nanoGPT, and has been altered so that it is usable for time series data.

https://github.com/aspiaspace/earthPT

[amotl]

TinyTimeMixer (TTM)

About

TinyTimeMixers (TTMs) are compact pre-trained models for Multivariate Time-Series Forecasting, open-sourced by IBM Research. With less than 1 Million parameters, TTM introduces the notion of the first-ever “tiny” pre-trained models for Time-Series Forecasting.

The current open-source version supports point forecasting use-cases ranging from minutely to hourly resolutions (Ex. 10 min, 15 min, 1 hour, etc.). Note that zeroshot, fine-tuning and inference tasks using TTM can easily be executed in 1 GPU machine or in laptops too.

Details

TTM-1 currently supports 2 modes:

• Zeroshot forecasting: Directly apply the pre-trained model on your target data to get an initial forecast (with no training).
• Finetuned forecasting: Finetune the pre-trained model with a subset of your target data to further improve the forecast.

Since, TTM models are extremely small and fast, it is practically very easy to finetune the model with your available target data in few minutes to get more accurate forecasts. For more details on TTM architecture and benchmarks, refer to our paper.

HF: https://huggingface.co/ibm/TTM
Paper: https://arxiv.org/pdf/2401.03955.pdf
Repository: https://github.com/IBM/tsfm/tree/main/tsfm_public/models/tinytimemixer

[amotl]

Neural 🧠 Forecast

About

NeuralForecast offers a large collection of neural forecasting models focused on their usability, and robustness. The models range from classic networks like MLP, RNNs to novel proven contributions like NBEATS, NHITS, TFT and other architectures.

Features

• Exogenous Variables: Static, historic and future exogenous support.
• Forecast Interpretability: Plot trend, seasonality and exogenous NBEATS, NHITS, TFT, ESRNN prediction components.
• Probabilistic Forecasting: Simple model adapters for quantile losses and parametric distributions.
• Train and Evaluation Losses Scale-dependent, percentage and scale independent errors, and parametric likelihoods.
• Automatic Model Selection Parallelized automatic hyperparameter tuning, that efficiently searches best validation configuration.
• Simple Interface Unified SKLearn Interface for StatsForecast and MLForecast compatibility.
• Model Collection: Out of the box implementation of MLP, LSTM, RNN, TCN, DilatedRNN, NBEATS, NHITS, ESRNN, Informer, TFT, PatchTST, VanillaTransformer, StemGNN and HINT. See the entire collection here.

Web: https://nixtlaverse.nixtla.io/neuralforecast/
Repository: https://github.com/Nixtla/neuralforecast

[amotl]

Awesome AI for Time Series (AI4TS) Papers, Tutorials, and Surveys

A professionally curated list of papers (with available code), tutorials, and surveys on recent AI for Time Series Analysis (AI4TS), including Time Series, Spatio-Temporal Data, Event Data, Sequence Data, Temporal Point Processes, etc., at the Top AI Conferences and Journals, which is updated ASAP (the earliest time) once the accepted papers are announced in the corresponding top AI conferences/journals. Hope this list would be helpful for researchers and engineers who are interested in AI for Time Series Analysis.

Repository: https://github.com/qingsongedu/awesome-AI-for-time-series-papers

[amotl]

PatchTST (ICLR 2023)

We propose an efficient design of Transformer-based models for multivariate time series forecasting and self-supervised representation learning. It is based on two key components: (i) segmentation of time series into subseries-level patches which are served as input tokens to Transformer; (ii) channel-independence where each channel contains a single univariate time series that shares the same embedding and Transformer weights across all the series. Patching design naturally has three-fold benefit: local semantic information is retained in the embedding; computation and memory usage of the attention maps are quadratically reduced given the same look-back window; and the model can attend longer history.

Our channel-independent patch time series Transformer (PatchTST) can improve the long-term forecasting accuracy significantly when compared with that of SOTA Transformer-based models. We also apply our model to self-supervised pre-training tasks and attain excellent fine-tuning performance, which outperforms supervised training on large datasets. Transferring of masked pre-trained representation on one dataset to others also produces SOTA forecasting accuracy.

Paper: A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
Repository: https://github.com/yuqinie98/PatchTST

[amotl]

tsai

State-of-the-art Deep Learning library for Time Series and Sequences.

tsai is an open-source deep learning package built on top of Pytorch & fastai focused on state-of-the-art techniques for time series tasks like classification, regression, forecasting, imputation…

tsai is currently under active development by timeseriesAI.

Repository: https://github.com/timeseriesAI/tsai
Documentation: https://timeseriesai.github.io/tsai/

[amotl]

skforecast

Time series forecasting with scikit-learn models.

Skforecast is a Python library that eases using scikit-learn regressors as single and multi-step forecasters. It also works with any regressor compatible with the scikit-learn API (LightGBM, XGBoost, CatBoost, ...).

Homepage: https://skforecast.org/
Repository: https://github.com/JoaquinAmatRodrigo/skforecast