NUMEXPR_MAX_THREADS with cmdstanpy

General
1

Hello all,
I am running my model with cmdstanpy in python 3.8 on ubuntu with reduce_sum.
I use 88 out of 128 cpus available (threads_per_chain=88), but I am getting the following warnings before the model starts running:

INFO:numexpr.utils:Note: detected 128 virtual cores but NumExpr set to maximum of 64, check "NUMEXPR_MAX_THREADS" environment variable.
INFO:numexpr.utils:Note: NumExpr detected 128 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
INFO:numexpr.utils:NumExpr defaulting to 8 threads.

Even if I declare os.environ["NUMEXPR_MAX_THREADS"] = "88", I still get the warnings.
However, when I hit htop, I see 88 cores working, and not 8 as stated above.
Can I just ignore these messages?

Thanks!

2

in CmdStanPy, specifying threads_per_chain=88 sets environment var STAN_NUM_THREADS=88, and evidently, this does the right thing w/r/t reduce_sum, which is why you see 88 cores working.

so yes, probably safe to ignore these messages - maybe it’s a ubuntu thing? cf https://stackoverflow.com/questions/43641247/numexpr-detecting-number-of-threads-less-than-number-of-cores - because as far as I can tell, neither CmdStanPy nor the c++ code or makefiles set or use NUMEXPR_MAX_THREADS

1 Like
3

Thanks!!