`rstan::read_stan_csv` Fails to Read CSV Output from cmdstan When `thin=` is Specified

Developers
1

When I run cmdstan with the thin= parameter set, I get a warning when I read the resulting CSV with stan::read_stan_csv, and I suspect the data is corrupted (seeing a lot of zero fill in the matricies).

In rstan::read_stan_csv(c("/data/jeff/run20200407/chain1.csv", "/data/jeff/run20200407/chain2.csv",  :
  the number of iterations after warmup found (250) does not match iter/warmup/thin from CSV comments (50,50,50,50)

I see someone did submit a PR to fix the issue (https://github.com/stan-dev/rstan/pull/692) in the past but unclear to me if it ever made it into a release. It seems pretty straightforward to fix. If someone can confirm this a real problem I will submit a PR as soon as I can.

In case it helps, my CSV header looks like this:

# stan_version_major = 2
# stan_version_minor = 22
# stan_version_patch = 1
# model = my_model
# method = sample (Default)
#   sample
#     num_samples = 250
#     num_warmup = 3750
#     save_warmup = 0 (Default)
#     thin = 5
#     adapt
#       engaged = 1 (Default)
#       gamma = 0.050000000000000003 (Default)
#       delta = 0.90000000000000002
#       kappa = 0.75 (Default)
#       t0 = 10 (Default)
#       init_buffer = 40
#       term_buffer = 50 (Default)
#       window = 25 (Default)
#     algorithm = hmc (Default)
#       hmc
#         engine = nuts (Default)
#           nuts
#             max_depth = 10 (Default)
#         metric = diag_e (Default)
#         metric_file =  (Default)
#         stepsize = 1 (Default)
#         stepsize_jitter = 0 (Default)
# id = 2
# data
#   file = /data/data.R
# init = 2 (Default)
# random
#   seed = 12345
# output
#   file = /data/chain2.csv
#   diagnostic_file =  (Default)
#   refresh = 20
2

The Merged tag under the name indicates it was merged into the code base. That should mean it’s in any release that’s been done since the merge.

I can believe thinning for reads is a problem. Our I/O around the output is a mess we’ve been trying to fix for half a dozen years now, but there are so many conflicting opinions on how to proceed that we can’t seem to make any progress.

3

Ok copy that :)

I believe I can run the model equivalently without thinning, so I’ll probably proceed that way, thanks.