Order of parameters for log_prob

Hi,

I’m trying to use the log_prob function exposed on the fit object returned by sampling. The first argument of log_prob is a vector, but I can’t figure out how I should concatenate the parameter values to produce that vector. I thought that the flatnames field of the fit object would provide the correct order, but trying to pass a vector constructed according to flatnames produced the error

ValueError: Number of unconstrained parameters does not match that of the model (41555 vs 8278).

It seems like flatnames includes transformed parameters as well, and I conjecture based on the error that log_prob only wants actual parameters (the ones declared in the parameters block of the model). How do I construct that vector?

I noticed that the actual parameters are listed first in flatnames, and in the order that they are declared in the parameters block of the model. I tried passing the values corresponding to the 8278 first entries of flatnames to log_prob, but the value returned (-2400) is not the same as the one that stan prints during optimization (45000), regardless of whether adjust_transform is true or false. Perturbing the parameters does seem to produce logical changes in the log_prob return values, so I wonder if the parameter order might be correct, but there’s some other explanation for why the values are so different? I don’t care about the absolute values of course, but I’m just trying to verify that I’m using log_prob correctly.

The context is that I’m trying to understand why optimization produces a solution that looks a little suboptimal by comparing the log probability of the optimal parameters with the log probability of parameters that look like they should be better (based on the criteria that I’m trying to capture in the model).

Ok, it seems that I’m meant to pass a dictionary that maps parameter names to vectors (such as the one returned by stan_model.optimizing) to fit_object.unconstrain_pars, which will then return the vector that log_prob expects. I realized this when I tried to apply unconstrain_pars to my manually constructed vector and the error message sounded like it was expecting a dictionary.

1 Like

I agree that log_prob could be easier to use or better documented.

I’m planning to have a fresh look at this (see https://github.com/stan-dev/httpstan/issues/113 ) for PyStan 3. It’s not a high priority – but it is on the list of things to do soon.

2 Likes

Thanks for the reply! I didn’t find log_prob that hard to use once I understood what it expects as input and how to produce that input. (Of course it’s a bit odd and convoluted that I need to call stan_model.sampling() to access log_prob, but that’s arguably a cosmetic problem once you understand what to do.)

Clarifying the API documentation for unconstrain_pars and log_prob a little, specifically regarding what they expect as inputs, would probably go a long way. A complete example of how to evaluate log_prob for a sample model would be even more helpful, although I’m not sure where that should go to be easily discoverable.

To kryft i am facing the exact same problem. My greedy solution was to apply .sampling() with only few iterations and warmups and then in a couple second we got to access to the log_prob function and grad_log_prob as well which is extremely great for doing other stuff such as variational inference like black box stochastic VI.

Here is small example, with some settings which might help getting fit fast.

1 Like

Thanks for that! I have created a minimal example for pystan3 here.

However, some points are still obscure to me. For example,

  1. how to pass other parameters like init and seed to sampling function.
  2. How to set adjust_transform=False for grad_log_prob.
  3. Is there no equivalent one to unconstrained_param_names in pystan2? I only found constrained_param_names.