Operating System:
Interface Version:
Compiler/Toolkit:
Wasn’t quite sure where to post / ask about this, but a while ago I wrote some code so I could diagnose troublesome stan models more quickly. A colleague wanted it today so I put it into a coherent function that works just like a regular call to stan, but generates simple on the fly plots with shiny. Maybe it’s handy for others to have - if so then just tell me where to put it :)
stanWplot.R (3.5 KB)
Ha, coincidentally enough I coded something similar over the past couple days. Mine is aimed at running big models on lots of cpus in parallel (I have been running several-day models on 32-core GCE machines lately). It actually re-invents the parallelism that rstan::stan will achieve automatically with its “cores” argument, but with the benefit of not locking up the R console during sampling. I also re-invented the progress reports to something a bit more sensible in my opinion. I was just about to add functions to plot diagnostics as the samples come in, so will probably be cribbing from your code!
Here’s the gist where I have the code so far:
Hey Charles, sounds cool. I’m getting some errors or warnings when I run the example though. Probably very minor but I haven’t had the time to look into it.
When it starts I get these messages:
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `Rscript.exe --slave --no-restore -e source('/var/folders/h6/14xy_35x4wd2tz542dn0qhtc0000gn/T//Rtmp6s4Q2X/stanplottemp.R') &'
then it runs the model ok but when it finishes I get
rm: /var/folders/h6/14xy_35x4wd2tz542dn0qhtc0000gn/T//Rtmp6s4Q2X/1491585202samples.csv1: No such file or directory
rm: /var/folders/h6/14xy_35x4wd2tz542dn0qhtc0000gn/T//Rtmp6s4Q2X/1491585202samples.csv2: No such file or directory
The other thing is that it doesn’t open shiny at any point when I run it.
I only tested it on windows so far, but didn’t see anything obvious that would stop it working elsewhere - which OS are you on? Shiny won’t open because it runs in the rscript command that gave a syntax error… maybe someone can see the obvious error in that line?
I’m on Mac so maybe there’s something subtle that is OS-dependent.
We’ve wanted something that does on-the-fly traceplots. Jonah Gabry’s maintaining ShinyStan, but I’m not sure that’s the right place to put this. Jonah will know what to do, but I don’t know hot ping him via this Discourse thing, so hopefull he’ll see this. If not, you could also include it as an issue for a feature request in RStan.
+1 would be neat to wrap this up for users and convince them to actually diagnose models based on the first 10-50 iterations. I recently saw a speaker who claimed he had to write his own Gibbs sampler and didn’t use Stan because he ran his model in Stan for a week (month?) and it didn’t mix well…
Hey Bob I actually already replied earlier today, but (if you’re only using
discourse through email) I guess it probably only notified you of the
initial post and not my response.
Jonah
Ok I made it a little more robust, and hopefully just removing the .exe specification gets it working on mac?
Not sure if this is of use to anyone else, but I thought I’d share that I worked out how to more efficiently check for updates in sample files from still-running chains (I think both Charles & I were simply re-reading the entire sample file). Here’s a gist with a basic demo, and here’s the principle applied to watching stan files (lines 24-35).
Nice Mike, that was definitely a bottle neck for my function, I’ll get around to patching it in at some point…