When encountering divergences, I often have the problem that these divergences are spread out all over the posterior distribution (or at least the part of it that was already sampled). This makes it difficult to pin down the cause of these divergences. Only in the minority of cases I encountered divergences which are thightly grouped in a certain area, thus giving a hint about what causes them.
I think that these divergences are so often spread out is because of two things:
The location where the divergence is shown is not where the energy error occurred, but the point from which the sample started moving before hitting the divergence-causing region.
Because HMC is very good at traversing the posterior, the start of the path can be somewhere completely different than where the problem occurred.
Thus, I imagine that it would be highly useful if we could get more information about the divergent transition, such as the coordinates of the leapfrog step with the largest energy error, or just simply the exact path the sampler took.
In some cases there is a region that causes the sampler to make its stepsize very small, leading to long runtimes and/or treedepth warnings, but without causing divergences. In this case diagnosing divergences doesn’t help to find the problem. Therefore it would be helpful if we could also get info about the locations and energy errors of the individual leapfrog steps for those transitions which were not divergent, to check which parts of the posterior are most difficult for the sampler.
I feel these things would greatly help me when debugging/optimizing my models.
Assuming that other people are sharing my experience, especially newcomers to stan, this could also significantly lower the entry barrier for new stan users.
I think I once saw an example of a HMC-implementation that actually had the feature of showing the sampler path of divergent transitions. Unfortunately I couldn’t find the article about it. I vaguely remember it beeing about pymc3.
That’s a fantastic thread over there! They’ve discussed a lot of aspects there already. I’ll read thought it all and then post my anwswer in that old thread.
That sounds interesting, thanks! I’ll keep it in Mind, but expect that I won’t have the time to do something useful with it.
May I suggest that you post this to the other thread which you linked above, too? Then this thread here can be closed and we have everything in one place.
Eh, keep the new thread. This thread links to it and it’s confusing to go back and try to restart these old monolithic topics. Some of that info on the code might be dated anyway.
Well, if you do feel free to report back how it goes. Plots are good :D. And if the results are confusing and not clear, that’s interesting too.
I’m interested.
Do I understand that correctly that in order to get the RNUTS Package need to install git and then create an rstudio project from a version control repository and build it from there? I’ve never built a packge, so don’t know what kinds of loops this requires jumping through. I’d rather avoid trying that (and installing additional software) if possible.
It doesn’t take that long and it’s worthwhile (and defines all the conventions). I’m usually pretty impatient with documentation but I managed :D.
ham = createHamiltonianSystem(testModelFile, list(N = 1))
I get this error:
Error in .shlib_internal(commandArgs(TRUE)) :
C++14 standard requested but CXX14 is not defined
Calls: -> .shlib_internal
Execution halted
Error in Rcpp::sourceCpp(model_code_file, env = env) :
Error 1 occurred building shared library.
Gooleing it it seems to be related to the makevars file, e.g.:
I’ve updated R, rcpp and rstan to the newest versions, but the issue persists.
The makevars file that is mentioned in that “thread” doesn’t seem to exist on my system. I’m not sure where it’s supposed to go and how to make one (or even whether it’s a good idea to do so).
I’m not sure about that path notation, can you give a more full path? I tried following the explanations I found googleing this notation, but couldn’t find this Makevars file. So either I haven’t understood this path notation correctly and searched in the wrong place, or this file doesn’t exist on my system.
The ~ refers to your home directory, so something like /home/username. If you don’t have the ~/.R directory, you can create it and put a Makevars file inside it.
I’m on Windows.
If i’m understanding this correctly the path on my System should be C:\Users\Username\.R\
At the moment there is no .R folder in this location.
Turns out my home directory was actually somewhere else …
I was able to create a makevars.win there, and R did recognize and use it.
I stil wasn’t able to get createHamiltonianSystem() working, though.
I suspect I didn’t put quite the right things in my makevars.win.
Sorry that I’m still asking about this here. I feel this is kinda not the right place.
I couldn’t find out why there was this makevars file didn’t exist in the first place on my system, nor what should be inside of it. It seems like everyone has a different one. I tried various peoples versions, but so far no luck. Is this makevars file supposed to be created with every R install? Or is it set up by certain packages (those which need to compile things?) ?
Each package may come with its Makevars, if it needs it. The system-wide Makevars you put in your home directory allows you to overrule the package-specific rules, so you it’s fine with not having it unless you need it.
I don’t understand the first part of your question, it’s easier to help by seeing exactly what you tried and how it failed.
@bbbales I got it working now and ± understand the output of oneSampleNuts() in debug mode. Am I right to assume that it’s using a diagonal metric, not doing any adaptation, and not checking or divergences? Does it use the metropolis correction?
Divergence checking and reporting of the contribution of each leapfrog step to the divergence criterion would be neccessary to test the ideas I had in mind.