I’ve been running some stan models using Rstan on an AWS EC2 instance (using Louise Aslett’s AMI: http://www.louisaslett.com/RStudio_AMI/) to model wages. I’ve been running fairly complicated models and everything had been working fairly reasonably until I tried to substantially up the sample size above 500,000 or so. At that point I started getting an error that said “The previous R session was abnormally terminated due to an unexpected crash.”
Based on previous forum questions, I thought that this might have been due to insufficient RAM, so I moved to an AWS instance with almost a TB of ram, but this didn’t solve the problem (and based on memory usage while the code is running, it doesn’t seem that I’m even approaching using all the RAM even on instance’s with less RAM).
I cut most model features to generate something incredibly simple, and I still found this problem once the sample got much about 500,000.
I’ve put together a minimum reproducible example on github (with an EXTREMELY simplified version of the model that still generates the error):
Does anyone have any thoughts on this or advice for things to look into?
Thanks!
Alex