From 63fc0b03f89b8ea41a9f56b07e644a9f5a39dea6 Mon Sep 17 00:00:00 2001 From: Sam Abbott Date: Tue, 25 Jul 2023 14:16:16 +0100 Subject: [PATCH 1/2] Update README.Rmd --- README.Rmd | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.Rmd b/README.Rmd index 6741ccf..91f89ac 100644 --- a/README.Rmd +++ b/README.Rmd @@ -14,16 +14,16 @@ output: github_document # Overview The `stackr` package provides an easy way to combine predictions from individual time series or panel data models to an -ensemble. `stackr` stacks Models according to the Continuous Ranked Probability +ensemble. `stackr` stacks models according to the Continuous Ranked Probability Score (CRPS) over k-step ahead predictions. It is therefore especially -suited for timeseries and panel data. A function for +suited for time-series and panel data. A function for leave-one-out CRPS may be added in the future. Predictions need to be predictive distributions represented by predictive samples. Usually, these will be sets of posterior predictive simulation draws generated by an MCMC algorithm. # Installation -install using +Install using ``` {r eval = FALSE} devtools::install_github("epiforecasts/stackr") @@ -33,8 +33,8 @@ devtools::install_github("epiforecasts/stackr") Given some training data with true observed values as well as predictive samples generated from different models, `stackr` finds the optimal (in the sense of minimizing expected cross-validation predictive error) weights to form an -ensemble of these models. Using these weights, stackr can then provide -samples from the optimal model mixture by drawing from the predictice samples +ensemble of these models. Using these weights, `stackr` can then provide +samples from the optimal model mixture by drawing from the predictive samples of those models in the correct proportion. This gives a mixture model solely based on predictive samples and is in this regard superior to other ensembling techniques like Bayesian Model Averaging. More information @@ -78,7 +78,7 @@ score_df[, mean(crps), by = model][, setnames(.SD, "V1", "CRPS")] # References - Using Stacking to Average Bayesian Predictive Distributions, Yuling Yao, Aki Vehtari, Daniel Simpson, and Andrew Gelman, 2018, Bayesian Analysis 13, Number 3, pp. 917–1003 DOI 10.1214/17-BA1091 -- Strictly Proper Scoring Rules, Prediction,and Estimation, +- Strictly Proper Scoring Rules, Prediction, and Estimation, Tilmann Gneiting and Adrian E. Raftery, 2007, Journal of the American Statistical Association, Volume 102, 2007 - Issue 477 DOI 10.1198/016214506000001437 - Comparing Bayes Model Averaging and Stacking When Model Approximation Error Cannot be Ignored, From 3e39306cf44900f52049391e6df0f07de227466d Mon Sep 17 00:00:00 2001 From: GitHub Action Date: Tue, 25 Jul 2023 13:21:48 +0000 Subject: [PATCH 2/2] Automatic readme update --- README.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index a46b692..bed61f1 100644 --- a/README.md +++ b/README.md @@ -17,16 +17,16 @@ MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.or The `stackr` package provides an easy way to combine predictions from individual time series or panel data models to an ensemble. `stackr` -stacks Models according to the Continuous Ranked Probability Score +stacks models according to the Continuous Ranked Probability Score (CRPS) over k-step ahead predictions. It is therefore especially suited -for timeseries and panel data. A function for leave-one-out CRPS may be +for time-series and panel data. A function for leave-one-out CRPS may be added in the future. Predictions need to be predictive distributions represented by predictive samples. Usually, these will be sets of posterior predictive simulation draws generated by an MCMC algorithm. # Installation -install using +Install using ``` r devtools::install_github("epiforecasts/stackr") @@ -37,12 +37,13 @@ devtools::install_github("epiforecasts/stackr") Given some training data with true observed values as well as predictive samples generated from different models, `stackr` finds the optimal (in the sense of minimizing expected cross-validation predictive error) -weights to form an ensemble of these models. Using these weights, stackr -can then provide samples from the optimal model mixture by drawing from -the predictice samples of those models in the correct proportion. This -gives a mixture model solely based on predictive samples and is in this -regard superior to other ensembling techniques like Bayesian Model -Averaging. More information can be found in the package vignette. +weights to form an ensemble of these models. Using these weights, +`stackr` can then provide samples from the optimal model mixture by +drawing from the predictive samples of those models in the correct +proportion. This gives a mixture model solely based on predictive +samples and is in this regard superior to other ensembling techniques +like Bayesian Model Averaging. More information can be found in the +package vignette. Weights are generated using the `crps_weights` function. With these weights and predictive samples, the `mixture_from_samples` function can @@ -87,7 +88,7 @@ score_df[, mean(crps), by = model][, setnames(.SD, "V1", "CRPS")] - Using Stacking to Average Bayesian Predictive Distributions, Yuling Yao, Aki Vehtari, Daniel Simpson, and Andrew Gelman, 2018, Bayesian Analysis 13, Number 3, pp. 917–1003 DOI 10.1214/17-BA1091 -- Strictly Proper Scoring Rules, Prediction,and Estimation, Tilmann +- Strictly Proper Scoring Rules, Prediction, and Estimation, Tilmann Gneiting and Adrian E. Raftery, 2007, Journal of the American Statistical Association, Volume 102, 2007 - Issue 477 DOI 10.1198/016214506000001437