Comparing finite-diff vs autodiff in cmdstanr

Interfaces
1

Can we compare the gradient of the log joint evaluated with finite diff and autodiff at the initial points using cmdstanr? This was a feature with rstan and cmdstan, but I can’t find the cmdstanr command.

The goal here is to check where a gradient doesn’t exist (if it doesn’t this can often be caught by finding a discrepancy between finite diff and autodiff).

2

Right now no, we need to expose the diagnose method of cmdstan (not to be confused with bin/diagnose): 13 Diagnosing HMC by Comparison of Gradients | CmdStan User’s Guide

There is an open issue for that: expose method 'diagnose' · Issue #156 · stan-dev/cmdstanr · GitHub

3

I wasnt aware this gets used at all so this issue was a low priority for me. Might deserve a larger one if its useful.

4

Hello,
following up on this request, as I find myself again in a situation when I would want to run this diagnostics. Thank you!

5

Hey Charles,

not sure if still relevant, but there is now a branch of cmdstanr where you can do this.

Install it using:

remotes::install_github("stan-dev/cmdstanr@diagnose_method")

Example of use:

> library(cmdstanr)

> model_path <- file.path(cmdstan_path(), "examples", "bernoulli", "bernoulli.stan")
> data_path <- file.path(cmdstan_path(), "examples", "bernoulli", "bernoulli.data.json")
> mod <- cmdstan_model(model_path)
> r <- mod$diagnose(data = data_path)
> r$gradients()
  param_idx    value    model finite_diff       error
1         0 0.118369 -3.35469    -3.35469 -3.3885e-10

The feature is missing docs, but should otherwise work as expected. In case of any issue report on the WIP PR: [WIP] Add diagnose method by rok-cesnovar · Pull Request #485 · stan-dev/cmdstanr · GitHub

Thanks!