I’m having some trouble making the
loo::psis function do what I think it’s supposed to do when using it to build an importance sampler. (NB: Yes. I know this is not the intended use case, but I can’t see from the code why it wouldn’t work, so I’m clearly missing something.)
My expectation for the following code is that it takes the log ratios
log_r and produces the smoothed log weights
log_w. If I ask for unnormalized weights, I expect all of these weights to be the same except for the last
attr(out,"tail_len") (which is 300 in this case) which should be the modified weights according to the fitted pareto distribution.
Hence if I fit a regression of the log-weights vs the log-ratios, I expect to get an intercept of around zero and a slope of around 1.This did not happen. Instead if I run
lm on the samples I get
Call: lm(formula = log_w ~ log_r) Coefficients: (Intercept) log_r -5.2179 0.9997
I’m clearly missing something here, but that -5.21 doesn’t make sense to me. If I ask for normalized weights I get an intercept of -9.17, which also doesn’t make sense.
Any idea what’s going on? (Specifically @avehtari but someone else may know)
Reproducible code and session info below:
library(loo) set.seed(9183) S = 10000 x = rexp(S,rate=3) log_r = dexp(x = x,rate=1,log=TRUE) - dexp(x=x,rate=3,log=TRUE) out = loo::psis(log_r,r_eff=1) log_w = weights(out,log=TRUE,normalize=FALSE) lm(log_w~log_r) mean(exp(log_r)) # 0.9603866 mean(exp(log_w)) # 0.005216566
> sessionInfo() R version 3.5.1 (2018-07-02) Platform: x86_64-apple-darwin15.6.0 (64-bit) Running under: macOS High Sierra 10.13.6 Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib locale:  en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8 attached base packages:  stats graphics grDevices utils datasets  methods base other attached packages:  loo_2.1.0 loaded via a namespace (and not attached):  compiler_3.5.1 parallel_3.5.1 tools_3.5.1  yaml_2.1.19 matrixStats_0.54.0