Analysis of results
Type
$ kima-showresults -h
for help
This page assumes that you are using the pykima package.
After (and also during) a successful run,
we can analyse the results produced by the sampler.
For this, we use the kima-showresults script.
This script should have been put in your PATH when you installed pykima.
To check its usage, type
$ kima-showresults -h
By default, when you run it without any argument,
the script will read the three files generated by DNest4
(sample.txt, sample_info.txt and levels.txt),
print some information to the terminal and create actual posterior samples
(which are saved in the posterior_sample.txt file,
with extra information stored in posterior_sample_info.txt).
See this page
for a description of these files.
The information printed in the terminal
is the (log-) evidence of the current model,
the information,
which gives an idea of the number of times the prior distribution
was compressed by a factor of e to obtain the posterior distribution,
and the effective sample size,
which corresponds to the number of saved particles
with a significant posterior weight.
The posterior_sample.txt file will contain
a number of posterior samples equal to the effective sample size.
Other graphics can be generated by providing arguments
to the kima-showresults script:
$ kima-showresults rv phase planets orbital gp extra
-
rvwill plot posterior realizations of the model over the RV measurements -
phasewill also plot phase curves, assuming the maximum likelihood solution -
planetswill plot the posterior distribution for the number of planets -
orbitalwill plot posteriors for some of the orbital parameters (the semi-amplitude, eccentricity, and orbital period) -
gpwill plot the posteriors for the GP hyperparameters (if the model has a GP component) -
extrawill plot the posteriors for fiber offset, systematic velocity, and extra white noise
Note: you can get more granularity and show just one specific plot
by providing numbered arguments to kima-showresults.
It is also possible to import pykima from the Python interpreter
and call its showresults() function with any of the options above.
Again, this can be done as the code is running,
in order to get real-time diagnostics.
Calling this function from the console, as in
>>> res = pykima.showresults()will work as before and return a variable called res
which holds a number of diagnostics and methods.
For example, res.make_plot1(), etc will plot the same graphics as above,
and the numpy array res.posterior_sample contains all the posterior samples.
To get diagnostic plots from the DNest4 samples
(to check if the sampler is doing ok) call kima-showresults as
kima-showresults diagnostic
which will display three plots
-
the level of each saved particle
which should show an initial increasing trend, as new levels are added, followed by an exploration of all levels uniformly. The maximum number of levels is set by the respective parameter in theOPTIONSfile, or determined automatically by the algorithm. The quality of the sampling usually increases with more steps spent in the uniform exploration of all the levels. -
the second plot shows the compression between levels in the top, which should be close to -1 if the sampler is doing ok (see Brewer et al. 2011), and in the bottom the Metropolis acceptance fraction for the levels. A value around 20% or higher is usually a good sign.
-
in the third plot, we see the log-likelihood versus the enclosed prior mass of each sample, and the positions where new levels were created. Sometimes this plot will show the presence of phase transitions — concave-up regions in the log(L) vs log(X) curve — see Skilling (2006) for more details about this issue. The bottom panel shows the posterior weights of the samples. There should be at least one peak in this plot, but the existence of phase transitions could originate other smaller peaks.
