RStan and PyStan may fail to compile model if TMPDIR not manually set

General
1

I found recently that when attempting to compile a model on my workstation, I get the error “failed to map segment from shared object: Operation not permitted”. It looks like what is happening is that RStan and PyStan create a “.so” file somewhere under “/tmp”, but don’t have the permission to load the resulting file. My workaround, for now, is to manually set the environment variable TMPDIR to some directory other than “/tmp”. Then RStan and PyStan can create the “.so” file in a directory without that permissions issue.

I suspect that when my workstation was updated, some new policy forbade loading executables from “/tmp”, probably for security reasons. It may be a good idea to account for this issue somehow in future RStan and PyStan releases.

2

Windows?

3

Python creates .pyd files on windows to %TMPDIR%.

4

We could probably use ~/.pystan directory as we do with httpstan.

cc @ariddell

5

There’s an os-appropriate cache directory which we save cached models to
in httpstan. But this doesn’t solve the problem, I think.

If there’s some unusual workstation-specific policy about /tmp then I
think the solution is for the user to change TMPDIR to something they
control. You just do something like export TMPDIR=~/mytmp and it
will work. This came up earlier with someone working on a cluster and
they exhausted storage space in /tmp .

6

No, Linux.

Why not? Basically, what I did as a workaround amounted to making PyStan use a partly user-defined directory to cache parts of the executable model.

As far as I can tell, not allowing files in /tmp to be executable isn’t that unusual. Here’s an old method of doing it: Wayback Machine

7

I guess any directory could be made non-exec and if the TMPDIR is created under there, Stan isn’t going to work. I think we are checking that the TMPDIR is writable but I don’t know how to check whether it is permissible to set the exec bit.

8

I found that with PyStan, if TMPDIR isn’t manually set, unpickling of models also can fail. For reference, in case someone is searching for similar error messages in the future, the results that I got looked like this:

WARNING:pystan:/tmp/tmpb08wqexx/stanfit4anon_model_3ed54a397b9c2a8cb0d43309b559c7c7_3032972568303198840.cpython-36m-x86_64-linux-gnu.so: failed to map segment from shared object: Operation not permitted
WARNING:pystan:Something went wrong while unpickling the StanModel. Consider recompiling.
Traceback (most recent call last):
  File "./script_Fit_JC_model_PyStan_weak_prior_bTQ09_fA075.py", line 21, in <module>
    jc_fit = jc_model.sampling(data = my_data, seed = 12345)
  File "/home/jjramsey/.conda/envs/Bayes/lib/python3.6/site-packages/pystan/model.py", line 721, in sampling
    fit = self.fit_class(data, seed)
AttributeError: 'StanModel' object has no attribute 'fit_class'
9

This was for the model object, not the fit object?

Does this work, if you set manually the TMPDIR?

10

Yes to both questions.