Hi,
I’m trying to understand if I can use Stan in an inference problem I’m faced with, and would appreciate any feedback. Disclaimer - I’m relatively new to Bayesian methods in general, and very new to Stan and R in particular (used python so far).
The problem domain is 2D Ising models - maximum entropy models over binary variables, constrained by means and pairwise correlations. These distributions are parametrized by the Lagrange multipliers satisfying these constraints (the “fields” and “couplings” in stat. mech terms). I have samples from a given distribution, and I’m trying to use bayesian inference to estiamate the parameters of this distribution (which are continuous variables). I’ve played around with the IsingFit library, but as far as I can tell, this library:
- Does not provide an estimate of the uncertainty in the inferred values (ideally, I’d want to see “tighter” estimates given more data).
- Ultimately, I’m interested in representing the MaxEnt model parameters as functions of other variables, and putting priors on them - I don’t think this is supported by IsingFit.
It this possible to do with Stan? Any help would be much appreciated.
Stan doesn’t work on this type of discrete model. In particular, you can’t declare any parameters to be discrete.
Maybe this package would be useful https://cran.r-project.org/web/packages/bayesImageS/index.html
In this problem, the data are discrete (binary patterns), but the parameters I’m trying to estimate are continuous variables. Once I “plug” the data to the likelihood function, doesn’t the entire problem becomes a continuous inference problem? Or is it the “plugging-in the data” part that is the problem?
For the ising model, the problem is that you can’t compute the normalising constant (aka the partition function). Stan needs and explicit expression for this.
I see, because it depends on the parameters as well? This is avoided, conveniently enough, when sampling the binary patterns themselves, since the partition function doesn’t depends on them (they’re integrated over); I hoped there might be a similar trick, here, as well.
Thanks for your help! If anyone knows other packages/refs that might be relevant, I’d appreciate it.
See my first response.
But for the most part, you have to “roll your own” for this sort of problem.
Out of curiosity, has there been any progress on this? Have you been able to implement a Ising model in stan? Thanks very much!
Hi Dominik, welcome to the community! :-)
For the reason mentioned above, I gave up on this idea. However, if you’re interested, there’s interesting literature on Bayesian approaches to fitting Exponential Random Graph Models, which are very Ising-like - see https://cran.r-project.org/web/packages/Bergm/Bergm.pdf and the references within. These approaches haven’t been implemented in Stan as far as I know (they often involve very different approaches to sampling).
Hi, @DominikDeffner : I don’t think this will be something we will ever get working because Stan doesn’t sample discrete parameters and the Ising model parameters can’t be marginalized effectively.
I’m not sure what @Adam_Haber was trying to do, but it doesn’t sound like he was trying to sample from an Ising model’s states.
My original question was about learning the parameters of an Ising-like model, which are continuous; the issue, as mentioned in the response above, is that the normalizing factor also depends on these parameters, and includes (in the general case) 2^N terms (N being the number of discrete binary variables), so explicitly calculating it is impractical for any reasonable N.
Hi @Adam_Haber & @DominikDeffner,
Julien Stoehr has previously worked on posterior sampling from doubly intractable distributions. You could have a look at his GiRaF package on CRAN for more information. Our group back in the day at University College Dublin was working on posterior sampling of Markov Random Fields; Julien and co-authors have published this work on Noisy HMC for Doubly Intractable Distributions.
Disclaimer: I am a co-author of the Bergm R package. The references within our recent publication on JSS could guide you in your search.
Thanks,
Lampros Bouranis