# Sampler HMC diagnostics file

**URL:** <https://discourse.mc-stan.org/t/sampler-hmc-diagnostics-file/15386>\
**Category:** Developers\
**Created:** [May 26, 2020, 1:01am UTC](https://discourse.mc-stan.org/t/sampler-hmc-diagnostics-file/15386 "2020-05-26T01:01:01Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mitzimorris](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/mitzimorris/32/29_2.png) [@mitzimorris](https://discourse.mc-stan.org/u/mitzimorris)\
**Post date:** [May 26, 2020, 1:01am UTC](https://discourse.mc-stan.org/t/sampler-hmc-diagnostics-file/15386/1 "2020-05-26T01:01:01Z")

</div>

This is to answer the evergreen question: '“what diagnostics are available from the HMC sampler and how do I get me some?”

CmdStan can spit out different csv files -

- an output file in Stan csv format (e.g. `output_file=my_sample.csv`), which contains sampler draws on the constrained scale

- a diagnostic\_file (e.g. ` diagnostic_file=my_diag.csv`)

The diagnostic file contains the same set of initial and final comments as the output.csv file - the initial comments contain the CmdStan config, the final comments contain the timing information.

(Note - “diagnostic\_file” is confusing, and we’re planning to call this “latent\_dynamics\_file” in the CmdStanPy and CmdStanR interfaces).

The actual csv data consists of:

the sampler state variables, followed by the parameter values _on the unconstrained scale_ followed by the parameter potential energy values, followed by the parameter gradients.

e.g.: given model with 2 parameters, `mu`, `sigma`, the data columns are:

```
lp __,accept_stat__ ,stepsize __,treedepth__ ,n_leapfrog __,divergent__ ,energy__,theta,sigma,p_theta,p_sigma,g_theta,g_sigma

```

There is 1 row per saved iteration, i.e., config `save_warmup` and `thin` control how often the sampler writes to both the output and diagnostic files.

This code:

> <https://github.com/stan-dev/stan/blob/9c09195caad83a054d4dc053f347889900fff145/src/stan/mcmc/hmc/base_hmc.hpp#L57-L64>

calls this code:

> <https://github.com/stan-dev/stan/blob/9c09195caad83a054d4dc053f347889900fff145/src/stan/mcmc/hmc/hamiltonians/ps_point.hpp#L27-L45>

Hope this answers questions that @Bob_Carpenter and @s.maskell might have about gradient information for control variates, etc.

---

<div class="post-metadata">

**Author:** ![s.maskell](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/s.maskell/32/6626_2.png) [@s.maskell](https://discourse.mc-stan.org/u/s.maskell)\
**Post date:** [May 26, 2020, 3:00pm UTC](https://discourse.mc-stan.org/t/sampler-hmc-diagnostics-file/15386/2 "2020-05-26T15:00:25Z")

</div>

@mitzimorris: Thanks. This will also be relevant to others working with me, notably @PhilClemson and @LJDevlin.

---

<div class="post-metadata">

**Author:** ![PhilClemson](https://avatars.discourse-cdn.com/v4/letter/p/edb3f5/32.png) [@PhilClemson](https://discourse.mc-stan.org/u/PhilClemson)\
**Post date:** [May 26, 2020, 3:14pm UTC](https://discourse.mc-stan.org/t/sampler-hmc-diagnostics-file/15386/3 "2020-05-26T15:14:12Z")

</div>

Thanks that’s really useful - as expected I was looking in completely the wrong place!