Skip to content

Commit

Permalink
update psis ref + some minor typo fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
avehtari committed Apr 12, 2024
1 parent 9f28fc4 commit ea273bb
Show file tree
Hide file tree
Showing 12 changed files with 26 additions and 40 deletions.
4 changes: 2 additions & 2 deletions R/loo_moment_matching.R
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ loo_moment_match.default <- function(x, loo, post_draws, log_lik_i,
#' @param i observation number.
#' @param x A fitted model object.
#' @param log_lik_i A function that takes `x` and `i` and returns a matrix (one
#' column per chain) or a vector (all chains stacked) of log-likeliood draws
#' column per chain) or a vector (all chains stacked) of log-likelihood draws
#' of the `i`th observation based on the model `x`. If the draws are obtained
#' using MCMC, the matrix with MCMC chains separated is preferred.
#' @param unconstrain_pars A function that takes arguments `x`, and `pars` and
Expand Down Expand Up @@ -440,7 +440,7 @@ loo_moment_match_i <- function(i,
#' `upars` and returns a matrix of log-posterior density values of the
#' unconstrained posterior draws passed via `upars`.
#' @param log_lik_i_upars A function that takes arguments `x`, `upars`,
#' and `i` and returns a vector of log-likeliood draws of the `i`th
#' and `i` and returns a vector of log-likelihood draws of the `i`th
#' observation based on the unconstrained posterior draws passed via
#' `upars`.
#' @param r_eff_i MCMC effective sample size divided by the total sample size
Expand Down
16 changes: 7 additions & 9 deletions R/loo_subsample.R
Original file line number Diff line number Diff line change
Expand Up @@ -316,7 +316,7 @@ update.psis_loo_ss <- function(object, ...,
stopifnot(is.data.frame(data) || is.matrix(data) & !is.null(draws))
}

# Compute subsample indecies
# Compute subsample indices
if (length(observations) > 1) {
idxs <- compute_idxs(observations)
} else {
Expand Down Expand Up @@ -448,7 +448,7 @@ nobs.psis_loo_ss <- function(object, ...) {
#' @details
#' The choice `psis` is returned if a `psis_loo` object
#' is converted to a `psis_loo_ss` object with `as.psis_loo_ss()`.
#' But `psis` cannot be chosen in the api of `loo_subsample()`.
#' But `psis` cannot be chosen in the API of `loo_subsample()`.
#'
#' @noRd
#' @param api The choices available in the loo API or all possible choices.
Expand Down Expand Up @@ -756,7 +756,7 @@ compute_idxs <- function(observations) {
}


#' Compare the indecies to prepare handling
#' Compare the indices to prepare handling
#'
#' @details
#' The function compares the object and sampled indices into `new`
Expand Down Expand Up @@ -1193,14 +1193,15 @@ srs_diff_est <- function(y_approx, y, y_idx) {
# eq (9) first row second `+` should be `-`
# Supplementary material eq (6) has this correct
# Here the variance is for sum, while in the paper the variance is for mean
# which explains the proporional difference of 1/n
# which explains the proportional difference of 1/N
est_list$hat_v_y <- (t_pi2_tilde + t_hat_epsilon) - # a (has been checked)
(1/N) * (t_e^2 - est_list$v_y_hat + 2 * t_pi_tilde * est_list$y_hat - t_pi_tilde^2) # b
est_list
}


#' Estimate elpd using the standard SRS estimator and SRS WOR
#' Estimate elpd using the standard simple-re-sample without
#' resampling (SRS-WOR) estimator
#' @noRd
#' @param x A `psis_loo_ss` object.
#' @return A `psis_loo_ss` object.
Expand All @@ -1220,7 +1221,7 @@ loo_subsample_estimation_srs <- function(x) {
update_psis_loo_ss_estimates(x)
}

#' Simple SRS-WOR estimation
#' Simple-re-sample without resampling (SRS-WOR) estimation
#' @noRd
#' @param y The values observed.
#' @param y_approx A vector of length N.
Expand Down Expand Up @@ -1324,6 +1325,3 @@ assert_subsampling_pointwise <- function(x) {
checkmate::assert_names(colnames(x), identical.to = c("elpd_loo", "mcse_elpd_loo", "p_loo", "looic", "influence_pareto_k", "idx", "m_i", "elpd_loo_approx"))
x
}



4 changes: 2 additions & 2 deletions man-roxygen/loo-and-compare-references.R
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
#'
#' Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
#' Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
#' accepted for publication.
#' [preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
#' 25(72):1-58.
#' [PDF](https://jmlr.org/papers/v25/19-556.html)
#'
#' Sivula, T, Magnusson, M., Matamoros A. A., and Vehtari, A. (2022).
#' Uncertainty in Bayesian leave-one-out cross-validation based model
Expand Down
6 changes: 3 additions & 3 deletions man-roxygen/loo-and-psis-references.R
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
#' [preprint arXiv:1507.04544](https://arxiv.org/abs/1507.04544)).
#'
#' Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
#' Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
#' accepted for publication.
#' [preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
#' Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
#' 25(72):1-58.
#' [PDF](https://jmlr.org/papers/v25/19-556.html)
#'
6 changes: 2 additions & 4 deletions vignettes/loo2-example.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,7 @@ encourage readers to refer to the following papers for more details:

* Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
accepted for publication.
[arXiv preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
25(72):1-58. [PDF](https://jmlr.org/papers/v25/19-556.html)


# Setup
Expand Down Expand Up @@ -302,5 +301,4 @@ Computing_. 27(5), 1413--1432. \doi:10.1007/s11222-016-9696-4.

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
accepted for publication.
[arXiv preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
25(72):1-58. [PDF](https://jmlr.org/papers/v25/19-556.html)
6 changes: 2 additions & 4 deletions vignettes/loo2-large-data.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,7 @@ Proceedings of the 23rd International Conference on Artificial Intelligence and

* Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
accepted for publication.
[arXiv preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
25(72):1-58. [PDF](https://jmlr.org/papers/v25/19-556.html)

which provide important background for understanding the methods implemented in
the package.
Expand Down Expand Up @@ -613,5 +612,4 @@ Computing_. 27(5), 1413--1432. \doi:10.1007/s11222-016-9696-4.

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
accepted for publication.
[arXiv preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
25(72):1-58. [PDF](https://jmlr.org/papers/v25/19-556.html)
3 changes: 1 addition & 2 deletions vignettes/loo2-lfo.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -642,8 +642,7 @@ Vehtari A., Gelman A., & Gabry J. (2017). Practical Bayesian model evaluation us

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
accepted for publication.
[arXiv preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
25(72):1-58. [PDF](https://jmlr.org/papers/v25/19-556.html)

<br />

Expand Down
3 changes: 1 addition & 2 deletions vignettes/loo2-mixis.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -197,8 +197,7 @@ Vehtari A., Gelman A., and Gabry J. (2017). Practical Bayesian model evaluation

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
accepted for publication.
[arXiv preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
25(72):1-58. [PDF](https://jmlr.org/papers/v25/19-556.html)



6 changes: 2 additions & 4 deletions vignettes/loo2-moment-matching.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,7 @@ papers

* Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
accepted for publication.
[arXiv preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
25(72):1-58. [PDF](https://jmlr.org/papers/v25/19-556.html)

# Example: Eradication of Roaches

Expand Down Expand Up @@ -322,5 +321,4 @@ Vehtari, A., Gelman, A., and Gabry, J. (2017). Practical Bayesian model evaluati

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
accepted for publication.
[arXiv preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
25(72):1-58. [PDF](https://jmlr.org/papers/v25/19-556.html)
3 changes: 1 addition & 2 deletions vignettes/loo2-non-factorized.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -718,5 +718,4 @@ Vehtari A., Gelman A., & Gabry J. (2017). Practical Bayesian model evaluation us

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
accepted for publication.
[arXiv preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
25(72):1-58. [PDF](https://jmlr.org/papers/v25/19-556.html)
3 changes: 1 addition & 2 deletions vignettes/loo2-weights.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -368,8 +368,7 @@ Computing_. 27(5), 1413--1432. \doi:10.1007/s11222-016-9696-4.

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
accepted for publication.
[arXiv preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
25(72):1-58. [PDF](https://jmlr.org/papers/v25/19-556.html)

Yao, Y., Vehtari, A., Simpson, D., and Gelman, A. (2018). Using
stacking to average Bayesian predictive distributions. In Bayesian
Expand Down
6 changes: 2 additions & 4 deletions vignettes/loo2-with-rstan.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,7 @@ Some sections from this vignette are excerpted from our papers

* Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
accepted for publication.
[arXiv preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
25(72):1-58. [PDF](https://jmlr.org/papers/v25/19-556.html)

which provide important background for understanding the methods implemented in
the package.
Expand Down Expand Up @@ -239,5 +238,4 @@ Computing_. 27(5), 1413--1432. \doi:10.1007/s11222-016-9696-4.

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning Research*,
accepted for publication.
[arXiv preprint arXiv:1507.02646](https://arxiv.org/abs/1507.02646)
25(72):1-58. [PDF](https://jmlr.org/papers/v25/19-556.html)

0 comments on commit ea273bb

Please sign in to comment.