Priors for Hierarchical GP Regression

Algorithms
21

Following up on this, I tried just building a very simple heteroscedastic GP like you describe.

I think these treedepth things could just be a result of how much data is being fit. If I fit a single condition2 dataset for one person, all the treedepths are exceeded and I get posterior draws of mean and standard deviation that look like:
posterior_samples

If I artificially limit how much data I’m passing it, I don’t get those warnings. For instance, I just used 5 trajectories of the same data I got no warnings and the posterior samples of mean and standard deviation look like:
posterior_samples_5_data

This has just been bothering me cause by the pictures it seems like the model should work. Could just be as simple as you have a ton of data and so all the parameters are very tightly constrained.

22

R code is here: https://github.com/bbbales2/gp/blob/master/heteroscedastic_discourse_model.R
Stan code is here: https://github.com/bbbales2/gp/blob/master/models/heteroscedastic.stan

23

That is quite interesting. I’m not quite sure what to make of that.

I’ll take a look when I get a chance. I’ve been away for a week, so I have a bit of catching up to do before diving back into this question of hierarchical GP regression.

For what it is worth, I had tried running my model on actual data with up to 400 trials per participant (many more than in the simulation I sent) and in that case I could not get the model to sample properly even with substantial increases in max_treedepth and adapt_delta. To your point, the abundance of data may have something to do with the issues.

Thanks,

24

Too much data should make the sampling easier, though, not harder, because the posterior for the hyperparameters should be closer to multivariate normal

25

Too much data can make the sampling harder in case of non-centered parameterization.

26

Yeah, that makes sense. Honestly just looking at the posterior draws, this seems like something that an optimizer should be able to get right enough, and then we’d just sample over the hyperparameters or something.

Seems like one of the many GP libraries floating around might do this? The original model is hierarchical and all that stuff so it might be harder to write down.

@ghislaindentremont

I switched to a centered parameterization. New but different issues. Maybe it’s a warmup thing? Tons of divergences and treedepth things, and the lengthscale estimation went south.

Stan model: https://github.com/bbbales2/gp/blob/master/models/heteroscedastic_centered.stan

Maybe it’s a warmup thing, I dunno. edit: Or a bug in my model…