Add section about Pathfinder diagnostic and using for inits #833
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bob-carpenter
requested changes
Nov 15, 2024
src/reference-manual/pathfinder.qmd
Outdated
| ## Diagnosing Pathfinder | ||
|
|
||
| Pathfinder diagnoses the accuracy of the approximation by computing the density ratio of the true posterior and | ||
| the approximation and using Pareto-$\hat{k}$ diagnostic (Vehtari et al., 2024) to assess whether these ratios can |
There was a problem hiding this comment.
We should use actual references here like "@zhang_pathfinder:2022"
src/reference-manual/pathfinder.qmd
Outdated
|
|
||
| Pathfinder diagnoses the accuracy of the approximation by computing the density ratio of the true posterior and | ||
| the approximation and using Pareto-$\hat{k}$ diagnostic (Vehtari et al., 2024) to assess whether these ratios can | ||
| be used to improve the approximation via resmapling. /, the |
There was a problem hiding this comment.
resmapling is misspelled.
I don't know what the / is doing here---should this be a different character?
src/reference-manual/pathfinder.qmd
Outdated
| be used to improve the approximation via resmapling. /, the | ||
| normalization for the posterior can be estimated reliably (Section 3, Vehtari et al., 2024), which is the | ||
| first requirement for reliable resampling. If estimated Pareto-$\hat{k}$ for the ratios is smaller than 0.7, | ||
| there is still need to further diagnose importance sampling estimates by taking into account also the expetant |
There was a problem hiding this comment.
I don't know the word "expetant"---looks like it's Latin. Can we just write this in English?
src/reference-manual/pathfinder.qmd
Outdated
| function (Section 2.2, Vehtari et al., 2024). If estimated Pareto-$\hat{k}$ is larger than 0.7, then the | ||
| estimate for the normalization is unreliable and any Mote Carlo estimate may have a big error. The resampled draws | ||
| can still contain some useful information about the location and shape of the posterior which can be used in early | ||
| parts of Bayesian workflow (Gelman et al, 2020). |
There was a problem hiding this comment.
All of these should get turned into citations.
|
I fixed references and several typos |
bob-carpenter
approved these changes
Nov 18, 2024
|
P.S. I fixed the merge conflict in the .bib file. Hope I didn't break anything in the process. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Related to #751 and recent discussion about using Pathfinder for inits in https://discourse.mc-stan.org/t/using-pathfinder-to-initalize-sampling/37221
I now just added brief sections on Pareto-k diagnostic and using Pathfinder for inits. It would be good to add much description about the Pathfinder overall.
Copyright and Licensing
Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company):
By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses: