Hmm, have you seen this: Issue with dual averaging ?
Sometimes when I’m running models I would like to see the lp__ in each chain, for instance. Then I could vaguely see if everything ended up in the same place.
This still goes back to the workflow paper, failing fast and all. If that chains aren’t going to around the same lp within 100-150 draws, time to kill the chains and see what is going on.
There are a few things here:
- Monitoring chains as they run
- Async cmdstan interface so it is possible to monitor stuff as it runs
- Doing analysis on killed chains – that might mean chains of different lengths at various stages of adaptation
- Restarting chains from where you killed them
I don’t mean to weigh you down with giant projects but if you’re feeling keen to write experimental R packages then yeah, we’re curious how much all these things are worth. I’ve talked to @jtimonen about this some.