Hi Stanimals, I have problems with a rather simple Gaussian Process model on synthetic (fake) data, so I decided to use SBC to check whether I have a bug/flaw in my Stan program and if Stan is calibrated for this model (given my selected NUTS parameters and MCMC specs). I focus here on the 3 param…

I found a potential discrepancy! [image] ermeel: K1_ = cov_exp_quad(t, alpha1_, rho1_)+diag_matrix(rep_vector(1e-9,N))+diag_matrix(rep_vector(sigma_y_, N)); VS [image] ermeel: matrix[N, N] cov = cov_exp_quad(t, alpha1, rho1) + diag_matrix(rep_vector(square(sigma_y), N)); Will re-run …

Those figures definitely suggest that something is going wrong with \sigma – it could be posterior reconstruction problems or you might not be using the same observational model in the sampling and the fit. To double check the latter you can generate half-normal RNGs more simply just by applying th…

[image] betanalpha: If sigma_y_loc isn’t zero then I don’t think your implementation will work anyways, which may also be a source of problems. Thanks Michael. If I am not mistaken, then Stan implements the truncated normal in a statement sigma_y ~ normal(a,b) for real<lower=0> sigma_y (forme…

[image] ermeel: Will re-run and report here whether this solves the above. That was exactly the reason! Below the p-values and rank histograms. In my eyes this is a perfect example why SBC is so useful (the rank histogram is compatible with the discrepancy in the code)! [1] 0.6034153 0.1074…

[image] ermeel: By the way I refrained from using the new rstanarm function sbc() , as it does not provide the automatic thinning (while guaranteeing a fixed number of sample outputs). On the other hand the rstanarm implementation is parallelised, which I think is a huge advantage given the emba…

Nice. [image] ermeel: If I am not mistaken, then Stan implements the truncated normal in a statement sigma_y ~ normal(a,b) for real<lower=0> sigma_y (former is therefore equivalent to sigma_y ~ normal(a,b)T[0,] ) and not the folded normal. Thus I should use the above truncated_normal_rng instea…

Simulation Based Calibration for Gaussian Process Model

Modeling

ermeel July 14, 2019, 3:16pm 6

Here we go.

Topic		Replies	Views
Rewriting example models with SBC Developers simulation-based-calibration	8	1429	February 25, 2019
Using narrower priors for SBC General prior-choice , simulation-based-calibration	14	1701	August 6, 2021
SBC for arma model Developers simulation-based-calibration	10	1273	April 7, 2021
Simulation-based calibration case study with RStan Algorithms simulation-based-calibration	31	3385	July 30, 2019
SBC for cmdstanr General cmdstanr , simulation-based-calibration	22	1772	December 9, 2020

Simulation Based Calibration for Gaussian Process Model

Related topics