We are exposing the unnormalized form of distributions for the 2.25 version (see Stan 2.25 release candidate! for more info). I am writing some basic information on that but need some guidance, help.
I went looking for docs on how to do a user-specified distribution today and this was all I found which seemed rather lacking: 19.1 Examples | Stan User’s Guide
Is this doc’ed anywhere else? Separately from this I think we need more docs.
Does defining a user defined myprob_lpdf also define a myprob_lupdf?
do we need to add the _lupdf form to docs for all distributions
I think so, unfortunately lol. If people search for normal_lupdf they should end up on the normal_lpdf page. I guess it can all be boilerplate copy-paste “normal_lpdf but dropping normalization constants” kinda text.
I am updating the reduce_sum tutorial now to use them.
how many functions are gonna get changed here? wondering if we could do something for SEO and indexing w/out having to go to wholesale duplication of entries.
Functions with names ending in _lupdf and _lupmf can only be used in the model block or user-defined functions with names ending in _lpdf, _lpmf or _lp.
Which seems like a good error message.
Then I changed the reducer function name to _lpmf and it compiled and ran.
Should the recommendation for now should be for users to call their reducer functions _lpxf s so they can use _lupmf/_lupdf, or did I get an unintended behavior?
You can not use _lupdf/_lupmf it the UDF is not _lpdf/_lpmf. This will be a requirement for the foreseeable future because of the way transformed parameters are done.
This is the example that will hopefully explain how all this will work.
functions {
real foo_lpdf(real y, real x) {
return normal_lupdf(y| x, 1);
}
real goo_lpdf(real y, real x) {
return normal_lpdf(y| x, 1);
}
}
parameters {
real y;
}
model{
target += foo_lpdf(y| x); // normal_lpdf would be normalized
target += foo_lupdf(y| x); // normal_lpdf would be unnormalized
target += goo_lpdf(y| x); // normal_lpdf would be normalized
target += goo_lupdf(y| x); // normal_lpdf would be normalized
}
So you need the lupdf in the UDF + call the UDF with lupdf. I didnt push the stanc3 PR for a long time because this is not ideal, but there is just no other way.
Will separate the reduce_sum lupdf/lupmf discussion outside of this thread.
Oh I see. And this compiles and runs because reduce_sum is using the template signature without the propto, which sets propto = FALSE, which passes on to everything else (edit: I looked at the generated code – not extrapolating from what you said here). So reduce_sum doesn’t do lupmf yet effectively, though it can compile in cases. Works for meee
agreed - Functions Reference Manual needs to add _lupdf to all distributions.
I think it should be possible to write a Python munge script - more fun than doing it by hand.
And then we’ll have the implications for user-defined functions.
I think even separately from the user-defined functions Jonah wanted to talk about how we are gonna put the info out there in a way that it gets understood what’s happening.
Yeah I just wanted to make sure that when we add new things like this that there’s a way for people to understand when they should use them. For example, I think the useful offset and multiplier functionality is barely used by anyone because there’s no case study or other user friendly introductions to using it. The same thing could easily happen with this. To be clear, I’m not expecting anyone in particular to write the case studies or blaming anyone for the lack of them, I just know that people are working hard implementing this stuff so I want to make sure users actually know how and when to use the great stuff that’s getting implemented!