Skip to content

Latest commit

 

History

History
153 lines (126 loc) · 6.68 KB

README.md

File metadata and controls

153 lines (126 loc) · 6.68 KB
Raw

Usage Examples

Generating Synthetic Time Series (KernelSynth)

  • Install this package with with the training extra: 
    pip install "chronos[training] @ git+https://github.com/amazon-science/chronos-forecasting.git"
  • Run kernel-synth.py: 
    # With defaults used in the paper (1M time series and 5 max_kernels)
python kernel-synth.py

# You may optionally specify num-series and max-kernels
python kernel-synth.py \
    --num-series <num of series to generate> \
    --max-kernels <max number of kernels to use per series>
    The generated time series will be saved in a GluonTS-comptabile arrow file kernelsynth-data.arrow.

Pretraining (and fine-tuning) Chronos models

  • Install this package with with the training extra: 
    pip install "chronos[training] @ git+https://github.com/amazon-science/chronos-forecasting.git"
  • Convert your time series dataset into a GluonTS-compatible file dataset. We recommend using the arrow format. You may use the convert_to_arrow function from the following snippet for that. Optionally, you may use synthetic data from KernelSynth to follow along. 
    from pathlib import Path
from typing import List, Union

import numpy as np
from gluonts.dataset.arrow import ArrowWriter


def convert_to_arrow(
    path: Union[str, Path],
    time_series: Union[List[np.ndarray], np.ndarray],
    compression: str = "lz4",
):
    """
    Store a given set of series into Arrow format at the specified path.

    Input data can be either a list of 1D numpy arrays, or a single 2D
    numpy array of shape (num_series, time_length).
    """
    assert isinstance(time_series, list) or (
        isinstance(time_series, np.ndarray) and
        time_series.ndim == 2
    )

    # Set an arbitrary start time
    start = np.datetime64("2000-01-01 00:00", "s")

    dataset = [
        {"start": start, "target": ts} for ts in time_series
    ]

    ArrowWriter(compression=compression).write_to_file(
        dataset,
        path=path,
    )


if __name__ == "__main__":
    # Generate 20 random time series of length 1024
    time_series = [np.random.randn(1024) for i in range(20)]

    # Convert to GluonTS arrow format
    convert_to_arrow("./noise-data.arrow", time_series=time_series)
  • Modify the training configs to use your data. Let's use the KernelSynth data as an example. 
    # List of training data files
training_data_paths:
- "/path/to/kernelsynth-data.arrow"
# Mixing probability of each dataset file
probability:
- 1.0
    You may optionally change other parameters of the config file, as required. For instance, if you're interested in fine-tuning the model from a pretrained Chronos checkpoint, you should change the model_id, set random_init: false, and (optionally) change other parameters such as max_steps and learning_rate.
  • Start the training (or fine-tuning) job: 
    # On single GPU
CUDA_VISIBLE_DEVICES=0 python training/train.py --config /path/to/modified/config.yaml

# On multiple GPUs (example with 8 GPUs)
torchrun --nproc-per-node=8 training/train.py --config /path/to/modified/config.yaml

# Fine-tune `amazon/chronos-t5-small` for 1000 steps with initial learning rate of 1e-3
CUDA_VISIBLE_DEVICES=0 python training/train.py --config /path/to/modified/config.yaml \
    --model-id amazon/chronos-t5-small \
    --no-random-init \
    --max-steps 1000 \
    --learning-rate 0.001
    The output and checkpoints will be saved in output/run-{id}/.

Tip

If the initial training step is too slow, you might want to change the shuffle_buffer_length and/or set torch_compile to false.

Important

When pretraining causal models (such as GPT2), the training script does LastValueImputation for missing values by default. If you pretrain causal models, please ensure that missing values are imputed similarly before passing the context tensor to ChronosPipeline.predict() for accurate results.

  • (Optional) Once trained, you can easily push your fine-tuned model to HuggingFace🤗 Hub. Before that, do not forget to create an access token with write permissions and put it in ~/.cache/huggingface/token. Here's a snippet that will push a fine-tuned model to HuggingFace🤗 Hub at <your_hf_username>/chronos-t5-small-fine-tuned. 
    from chronos import ChronosPipeline

pipeline = ChronosPipeline.from_pretrained("/path/to/fine-tuned/model/ckpt/dir/")
pipeline.model.model.push_to_hub("chronos-t5-small-fine-tuned")

Evaluating Chronos models

Follow these steps to compute the WQL and MASE values for the in-domain and zero-shot benchmarks in our paper.

  • Install this package with with the evaluation extra: 
    pip install "chronos[evaluation] @ git+https://github.com/amazon-science/chronos-forecasting.git"
  • Run the evaluation script: 
    # In-domain evaluation
# Results will be saved in: evaluation/results/chronos-t5-small-in-domain.csv
python evaluation/evaluate.py evaluation/configs/in-domain.yaml evaluation/results/chronos-t5-small-in-domain.csv \
    --chronos-model-id "amazon/chronos-t5-small" \
    --batch-size=32 \
    --device=cuda:0 \
    --num-samples 20

# Zero-shot evaluation
# Results will be saved in: evaluation/results/chronos-t5-small-zero-shot.csv
python evaluation/evaluate.py evaluation/configs/zero-shot.yaml evaluation/results/chronos-t5-small-zero-shot.csv \
    --chronos-model-id "amazon/chronos-t5-small" \
    --batch-size=32 \
    --device=cuda:0 \
    --num-samples 20
  • Use the following snippet to compute the aggregated relative WQL and MASE scores: 
    import pandas as pd
from scipy.stats import gmean  # requires: pip install scipy


def agg_relative_score(model_df: pd.DataFrame, baseline_df: pd.DataFrame):
    relative_score = model_df.drop("model", axis="columns") / baseline_df.drop(
        "model", axis="columns"
    )
    return relative_score.agg(gmean)


result_df = pd.read_csv("evaluation/results/chronos-t5-small-in-domain.csv").set_index("dataset")
baseline_df = pd.read_csv("evaluation/results/seasonal-naive-in-domain.csv").set_index("dataset")

agg_score_df = agg_relative_score(result_df, baseline_df)