Hi,
I came across the ‘priorsense’ package and wanted to try it out. Unfortunately, I can’t make any of the plotting functions work. My stan object is called ‘fit’ and contains ‘lprior’ and ‘log_lik’ values calculated in the generated quantities block. Using:
powerscale_sensitivity(fit)
works just fine (even though many potential problems are detected ;)).
But trying to plot only the parameter ‘pdd1’ with:
gives:
Error in powerscale_plot_dens.powerscaled_sequence(ps, variable = variable, :
Assertion on ‘variables_per_page’ failed: Must be of type ‘number’ (or ‘NULL’), not ‘character’.
I’ve used priorsense (very cool package) but I’m not very familiar with the internals of the package or how all the arguments work. I’m tagging the maintainer @n-kall who will hopefully be able to help you out.
@buf One issue is that the keyword argument is variable not variables. So it is not actually correctly selecting pdd1 and is trying to plot all the variables in your last code block (which is why it is taking too long). Can you try with variable = "pdd1" and let me know if it works?
I think variables = "pdd1" is being partially matched and converted to variables_per_page = "pdd1", hence it is passing a character.
No problem @buf. It’s actually a bit of an oversight on my part as variables_per_page was only added recently and I didn’t think about it being partially matched by variables. I’ll try to make a fix so it would at least alert the user more informatively
I’m coming back to this after a while. I have a couple of questions regarding the plots I produced with priorsense and would be grateful for any insights. Once I use the powerscale_plot_ecdf() function (left plot), and once the powerscale_plot_quantities() function (right plot).
In the ECDF plot, one can see that power-scaling the prior with alpha = 1.25 leads to high Pareto-k values for both shown parameters ‘pdd1’ and ‘tmin_v’. When examining the quantities plot (quantity = ‘mean’), I can see how the mean of the posterior is affected by power-scaling the prior and likelihood with alphas ranging from 0.8 to 1.25. Shouldn’t we see the high Pareto-k values here for power-scaling the prior with alpha near or equal to 1.25, as we see in the ECDF plot? My understanding was that for both plots, the same things are going on in the background. And the results are just displayed in a different fashion. The quantities plot has some more calculations to do, as the interval on which alpha is changed seems rather small; the ECDF plot only uses the three fixed alpha values of 0.8, 1, and 1.25. But I would have expected to see, for alpha = 1.25, either no high Pareto-k or high Pareto-k in both plots.
My second question is regarding the power-scaling of the prior for ‘pdd1’ in the quantities plot. The prior for that parameter is a normal(45,7). When increasing the alpha from 0.8 to 1.25, I expected that the posterior mean would gradually move closer to the prior mean of 45. This happens at first, but then, approaching alpha = 1.25, the posterior mean wanders way from the prior mean.
This is surely not a ‘priorsense package plotting issue’ as the title of the thread suggests, but just my lacking understanding of the subject.
High Pareto-khats shown in ECDF plot explain also why the estimated mean can be wrong. I don’t know why high khats do not show in the mean plot (@n-kall ?) From the practical point of view, it’s not a problem that the importance sampling fails for alphas far from 1, as you can infer the sensitivity by looking at the gradient next to 1, too
@buf Thanks again for reporting the discrepancy in high pareto-k plotting. I have fixed this issue and it will be available in the next release. This discrepancy was because only the last point on the line for the quantities plot has a high k value, so the highlighting had no line segment to apply to. Now it will colour the end squares as well, so this will be clear
