Purpose

Closes #341
Closes #342
Closes #343
Closes #344
Closes #345

Content

From a private repos of @KotaOK-00 - additional changes to interface

• Copy implementation of the gradient-based sampler proposals
• Implemented the calculation of ESJD
• Added esjd unit test.
• Created a new system AutodiffProtocol containing derived types GradFreeProtocol ForwardDiffProtocol and ReverseDiffProtocol. autodiff options are created with the MCMCProtocol, e.g.,

BarkerSampling() # creates BarkerSampling{ForwardDiffProtocol}()
BarkerSampling{ReverseDiffProtocol}() # creates BarkerSampling{ReverseDiffProtocol}()
pCNMHSampling() # creates pCNMHSampling{GradFreeProtocol}()

within the MCMC propose method direct calls to

ForwardDiff.gradient(x -> AdvancedMH.logdensity(model, x), current_state.params)
Symmetric(ForwardDiff.hessian(x -> AdvancedMH.logdensity(model, x), current_state.params))

Are replaced with autodiff_gradient and autodiff_hessian

autodiff_gradient(model, current_state.params, sampler)
autodiff_hessian(model, current_state.params, sampler)

that dispatch off the pre-specified autodiff protocol in the sampler. This will make adding new autodiff options easier

• Created a new type AGPJL for AbstractGPs. When building an emulator with AGPJL, As there is no optimizer of such kernels, we allow users to train e.g. with GPJL, then do the following:

opt_params_per_model = Emulators.get_params(gp_optimized)
kernel_params = [
Dict(
       "log_rbf_len" => params[1:end-2],
       "log_std_sqexp" => params[end-1],
       "log_std_noise" => params[end],
) 
for params in opt_params_per_model]
agp = GaussianProcess(AGPJL(), ...)
Emulator( agp, ... ; ..., kernel_params = kernel_params) # builds

If the user does not provide kernel_params, a useful error message appears to direct them what to do.

• Unit tests in GaussianProcesses/runtests.jl for AGP interface as above, and that GPJL vs AGP gives very similar emulator predicitions
• Unit tests in MarkovChainMonteCarlo/runtests.jl for AGP on RWM and pCN sampling - give similar posterior means to GPJL (NB it is much slower, 5-10x)
• Unit testing in MarkovChainMonteCarlo/runtests.jl for Barker update (ForwardDiff and ReverseDiff) with comparable results to RWM.

MISC

added non-Barker implementation as new issue. I have removed these algorithms as they do not perform well in unit testing and require possibly significant improvements to the tuning mechanisms to work

• I have read and checked the items on the review checklist.