Raster scan likelihood surface?

Hi,

I’m trying to fit a model for a neutrino physics process to some fake data I generated using the same model. [I don’t think the details of the model are important here (yet?) but I can supply them if needed.] Stan is successfully finding a best fit to my model in the likelihood space, but the parameters are not very close to what I expect. Moreover, if I plot the parameter space Stan walked through from the output (even including the warmup draws), the region where the fake data was generated is never explored.

I’d like to try to understand the LL space Stan is seeing a bit better. Is there a way for me to ask Stan to simply raster scan through a fixed range of values in one or two parameters and report lp__ without doing any fitting so that I can plot it and see what’s going on? I’ve poked around on the forums here, done some general googling, and looked through the manual, but I haven’t found the magic incantation yet (if there is one).

I’m using CmdStan, if that’s relevant, because my model uses some external C++ code to calculate the probabilities for some components of the model.

Thanks!

Doing that is going to be a bit easier with RStan because it exposes the log_prob function that is used to evaluate the log-kernel of the density function. Basically, you “run” a chain for zero iterations to get back an empty stanfit object and then you call log_prob on the empty fit at the parameter vector you are interested in. Linking external C++ code is slightly different, see
https://cran.r-project.org/web/packages/rstan/vignettes/external.html

1 Like

Thanks for your suggestion, I’ll keep it in mind for the future. I’m completely new to R, so it wasn’t so easy for me to figure out how to set the library paths for the external code I’m linking (which has its own dependencies). I’m reasonably fluent with Makefiles so this was easier to set up in CmdStan.

All that said, I was able to sort out my problem with a few more diagnostic plots of the space that Stan has been exploring, so I’m all set for the moment.

Thanks again for your help!

If you’re already incorporating custom C++ into Stan then it should be reasonably straightforward to use the <model_name>.hpp file generated when you compile a Stan program in CmdStan. <model_name>.hpp defines a class that exposes a log_prob method and the CmdStan code demonstrates how to instantiate an instance and use the method.

Noted, I’ll keep that in mind for the next time. Thanks!