Cmdstanpy reduce_sum doesn't produce executable

mathDR · December 20, 2023, 4:53pm

Hello all! I am running the cmdstanpy example for within chain parallelization using reduce_sum.

I am doing so as a base case to show the error I am getting for a bespoke model I am building.

In short:

If I run

import cmdstanpy
import pprint

cmdstanpy.install_cmdstan()

from cmdstanpy import CmdStanModel

redcard_model = CmdStanModel(
    stan_file='redcard_reduce_sum.stan',
    # cpp_options={'STAN_THREADS': 'TRUE'},
    force_compile=True,
)
pprint.pprint(redcard_model.exe_info())

The output produced is:

{'STAN_CPP_OPTIMS': 'false',
 'STAN_MPI': 'false',
 'STAN_NO_RANGE_CHECKS': 'false',
 'STAN_OPENCL': 'false',
 'STAN_THREADS': 'false',
 'stan_version_major': '2',
 'stan_version_minor': '33',
 'stan_version_patch': '0'}

which is expected. However, when I run

redcard_model = CmdStanModel(
    stan_file='redcard_reduce_sum.stan',
    cpp_options={'STAN_THREADS': 'TRUE'},
    force_compile=True,
)
pprint.pprint(redcard_model.exe_info())

i.e. try to compile with stan_threads=true, I get:

{}

as the output.

This manifests when I try to sample because, in the former case, I run 4 chains successfully, but in the latter case, I produce:

(cmdstanpy) 11:50:04 ~/dev/eligere/eligere/models/stan main $ python compute_stan_rs.py
CmdStan install directory: /Users/daniel.marthaler/.cmdstan
CmdStan version 2.33.1 already installed
Test model compilation
11:50:08 - cmdstanpy - INFO - compiling stan file /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum.stan to exe file /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum
11:50:12 - cmdstanpy - INFO - compiled model executable: /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum
{}
11:50:13 - cmdstanpy - INFO - CmdStan start processing
chain 1 |████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 00:00 Sampling completed
chain 2 |████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 00:00 Sampling completed
chain 3 |████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 00:00 Sampling completed
chain 4 |████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 00:00 Sampling completed
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
11:50:13 - cmdstanpy - INFO - CmdStan done processing.
Traceback (most recent call last):
  File "/Users/daniel.marthaler/dev/eligere/eligere/models/stan/compute_stan_rs.py", line 34, in <module>
    redcard_fit = redcard_model.sample(data=stan_data, threads_per_chain=4)
  File "/Users/daniel.marthaler/.pyenv/versions/cmdstanpy/lib/python3.9/site-packages/cmdstanpy/model.py", line 1173, in sample
    raise RuntimeError(msg)
RuntimeError: Error during sampling:

Command and output files:
RunSet: chains=4, chain_ids=[1, 2, 3, 4], num_processes=4
 cmd (chain 1):
	['/Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum', 'id=1', 'random', 'seed=59043', 'data', 'file=/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpot9pytf4/gmhbohve.json', 'output', 'file=/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpot9pytf4/redcard_reduce_sumdaeecrj7/redcard_reduce_sum-20231220115013_1.csv', 'method=sample', 'algorithm=hmc', 'adapt', 'engaged=1']
 retcodes=[-1, -1, -1, -1]
 per-chain output files (showing chain 1 only):
 csv_file:
	/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpot9pytf4/redcard_reduce_sumdaeecrj7/redcard_reduce_sum-20231220115013_1.csv
 console_msgs (if any):
	/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpot9pytf4/redcard_reduce_sumdaeecrj7/redcard_reduce_sum-20231220115013_0-stdout.txt
Consider re-running with show_console=True if the above output is unclear!
(cmdstanpy) 11:50:13 ~/dev/eligere/eligere/models/stan main

which seems to imply that no executable was produced (and none exists in the directory).

Any thoughts?

This is an Apple Macbook Pro running an M1 Max on Sonoma 14.1.2

WardBrian · December 20, 2023, 7:12pm

Can you enable debug logging (see the documentation) and report what the output is when running this code?

mathDR · December 20, 2023, 7:23pm

Here is the whole output in all.log:

14:20:53 - cmdstanpy - INFO - compiling stan file /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum.stan to exe file /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum
14:20:53 - cmdstanpy - DEBUG - cmd: make STANCFLAGS+=--filename-in-msg=redcard_reduce_sum.stan STAN_THREADS=TRUE /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum
cwd: /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1
14:20:57 - cmdstanpy - DEBUG - Console output:

--- Translating Stan model to C++ code ---
bin/stanc --filename-in-msg=redcard_reduce_sum.stan --o=/Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum.hpp /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum.stan

--- Compiling, linking C++ code ---
clang++ -std=c++1y -Wno-unknown-warning-option -Wno-tautological-compare -Wno-sign-compare -D_REENTRANT -Wno-ignored-attributes     -DSTAN_THREADS -I stan/lib/stan_math/lib/tbb_2020.3/include    -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.78.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials    -DBOOST_DISABLE_ASSERTS          -c -include-pch stan/src/stan/model/model_header_threads_15_0.hpp.gch -x c++ -o /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum.o /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum.hpp
In file included from /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum.hpp:1:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/src/stan/model/model_header.hpp:4:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/stan/math.hpp:19:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/stan/math/rev.hpp:10:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/stan/math/rev/fun.hpp:200:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/stan/math/prim/functor.hpp:16:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/stan/math/prim/functor/integrate_ode_rk45.hpp:6:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/stan/math/prim/functor/ode_rk45.hpp:9:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/boost_1.78.0/boost/numeric/odeint.hpp:76:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/boost_1.78.0/boost/numeric/odeint/integrate/observer_collection.hpp:23:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/boost_1.78.0/boost/function.hpp:30:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/boost_1.78.0/boost/function/detail/prologue.hpp:17:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/boost_1.78.0/boost/function/function_base.hpp:21:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/boost_1.78.0/boost/type_index.hpp:29:
In file included from /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/boost_1.78.0/boost/type_index/stl_type_index.hpp:47:
/Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/boost_1.78.0/boost/container_hash/hash.hpp:132:33: warning: 'unary_function<const std::error_category *, unsigned long>' is deprecated [-Wdeprecated-declarations]
        struct hash_base : std::unary_function<T, std::size_t> {};
                                ^
/Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/boost_1.78.0/boost/container_hash/hash.hpp:692:18: note: in instantiation of template class 'boost::hash_detail::hash_base<const std::error_category *>' requested here
        : public boost::hash_detail::hash_base<T*>
                 ^
/Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/boost_1.78.0/boost/container_hash/hash.hpp:420:24: note: in instantiation of template class 'boost::hash<const std::error_category *>' requested here
        boost::hash<T> hasher;
                       ^
/Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/boost_1.78.0/boost/container_hash/hash.hpp:551:9: note: in instantiation of function template specialization 'boost::hash_combine<const std::error_category *>' requested here
        hash_combine(seed, &v.category());
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__functional/unary_function.h:23:29: note: 'unary_function<const std::error_category *, unsigned long>' has been explicitly marked deprecated here
struct _LIBCPP_TEMPLATE_VIS _LIBCPP_DEPRECATED_IN_CXX11 unary_function
                            ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__config:850:41: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX11'
#    define _LIBCPP_DEPRECATED_IN_CXX11 _LIBCPP_DEPRECATED
                                        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__config:835:49: note: expanded from macro '_LIBCPP_DEPRECATED'
#      define _LIBCPP_DEPRECATED __attribute__((deprecated))
                                                ^
1 warning generated.
clang++ -std=c++1y -Wno-unknown-warning-option -Wno-tautological-compare -Wno-sign-compare -D_REENTRANT -Wno-ignored-attributes     -DSTAN_THREADS -I stan/lib/stan_math/lib/tbb_2020.3/include    -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.78.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials    -DBOOST_DISABLE_ASSERTS               -Wl,-L,"/Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb"        /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum.o src/cmdstan/main_threads.o       -Wl,-L,"/Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb"     stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_idas.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_kinsol.a  stan/lib/stan_math/lib/tbb/libtbb.dylib stan/lib/stan_math/lib/tbb/libtbbmalloc.dylib stan/lib/stan_math/lib/tbb/libtbbmalloc_proxy.dylib -o /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum
ld: warning: duplicate -rpath '/Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb' ignored
rm -f /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum.o

14:20:57 - cmdstanpy - INFO - compiled model executable: /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum
14:20:57 - cmdstanpy - DEBUG - cmd: /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum info
cwd: None
14:20:58 - cmdstanpy - DEBUG - Command ['/Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum', 'info']
	terminated by signal 9 Unknown error: -9
14:20:58 - cmdstanpy - DEBUG - cmd: /Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum info
cwd: None
14:20:58 - cmdstanpy - DEBUG - Command: ['/Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum', 'info']
failed with error [Errno 2] No such file or directory: '/Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum'

14:20:58 - cmdstanpy - DEBUG - input tempfile: /var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/5hwl96_q.json
14:20:58 - cmdstanpy - INFO - CmdStan start processing
14:20:58 - cmdstanpy - DEBUG - idx 0
14:20:58 - cmdstanpy - DEBUG - idx 1
14:20:58 - cmdstanpy - DEBUG - running CmdStan, num_threads: 4
14:20:58 - cmdstanpy - DEBUG - idx 2
14:20:58 - cmdstanpy - DEBUG - running CmdStan, num_threads: 4
14:20:58 - cmdstanpy - DEBUG - idx 3
14:20:58 - cmdstanpy - DEBUG - CmdStan args: ['/Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum', 'id=1', 'random', 'seed=81543', 'data', 'file=/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/5hwl96_q.json', 'output', 'file=/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/redcard_reduce_sum1ltxuvpk/redcard_reduce_sum-20231220142058_1.csv', 'method=sample', 'algorithm=hmc', 'adapt', 'engaged=1']
14:20:58 - cmdstanpy - DEBUG - running CmdStan, num_threads: 4
14:20:58 - cmdstanpy - DEBUG - CmdStan args: ['/Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum', 'id=2', 'random', 'seed=81543', 'data', 'file=/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/5hwl96_q.json', 'output', 'file=/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/redcard_reduce_sum1ltxuvpk/redcard_reduce_sum-20231220142058_2.csv', 'method=sample', 'algorithm=hmc', 'adapt', 'engaged=1']
14:20:58 - cmdstanpy - DEBUG - running CmdStan, num_threads: 4
14:20:58 - cmdstanpy - DEBUG - CmdStan args: ['/Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum', 'id=3', 'random', 'seed=81543', 'data', 'file=/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/5hwl96_q.json', 'output', 'file=/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/redcard_reduce_sum1ltxuvpk/redcard_reduce_sum-20231220142058_3.csv', 'method=sample', 'algorithm=hmc', 'adapt', 'engaged=1']
14:20:58 - cmdstanpy - DEBUG - CmdStan args: ['/Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum', 'id=4', 'random', 'seed=81543', 'data', 'file=/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/5hwl96_q.json', 'output', 'file=/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/redcard_reduce_sum1ltxuvpk/redcard_reduce_sum-20231220142058_4.csv', 'method=sample', 'algorithm=hmc', 'adapt', 'engaged=1']
14:20:58 - cmdstanpy - INFO - CmdStan done processing.
14:20:58 - cmdstanpy - DEBUG - runset
RunSet: chains=4, chain_ids=[1, 2, 3, 4], num_processes=4
 cmd (chain 1):
	['/Users/daniel.marthaler/dev/eligere/eligere/models/stan/redcard_reduce_sum', 'id=1', 'random', 'seed=81543', 'data', 'file=/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/5hwl96_q.json', 'output', 'file=/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/redcard_reduce_sum1ltxuvpk/redcard_reduce_sum-20231220142058_1.csv', 'method=sample', 'algorithm=hmc', 'adapt', 'engaged=1']
 retcodes=[-1, -1, -1, -1]
 per-chain output files (showing chain 1 only):
 csv_file:
	/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/redcard_reduce_sum1ltxuvpk/redcard_reduce_sum-20231220142058_1.csv
 console_msgs (if any):
	/var/folders/rk/49593z6j4xx_hxmgmkhxh50w0000gq/T/tmpujy0cthj/redcard_reduce_sum1ltxuvpk/redcard_reduce_sum-20231220142058_0-stdout.txt
14:20:58 - cmdstanpy - DEBUG - Chain 1 console:

WardBrian · December 20, 2023, 7:34pm

This output is helpful, thanks! I think it implies that an executable was created, but it seems like running it immediately causes some failure, even in simple methods like info

Can you navigate to the folder and 1) confirm that there is a redcard_reduce_sum executable there, and 2) run the following commands and share their outputs? ldd redcard_reduce_sum and ./recard_reduce_sum help

mathDR · December 20, 2023, 7:40pm

There is not an executable produced when I compile with the stan_threads option.

One thing I noticed was the warning produced

ld: warning: duplicate -rpath '/Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb' ignored

and some googling produced this page. which implies that there may be something fishy with xcode 15? (I only use the command line tools but still…)

I am currently trying to downgrade to xcode 14 and going to rerun (since I am sure I could run this example a while ago…)

WardBrian · December 20, 2023, 7:54pm

That warning is spurious and will not exist in future versions (see Don't duplicate -rpath for TBB by WardBrian · Pull Request #2952 · stan-dev/math · GitHub)

Can you try moving the .stan file to /Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1 and then running make STAN_THREADS=1 redcard_reduce_sum?

mathDR · December 20, 2023, 8:20pm

(cmdstanpy) 15:19:40 ~/.cmdstan/cmdstan-2.33.1 $ make STAN_THREADS=1 redcard_reduce_sum      

--- Translating Stan model to C++ code ---
bin/stanc  --o=redcard_reduce_sum.hpp redcard_reduce_sum.stan

--- Compiling, linking C++ code ---
clang++ -std=c++1y -Wno-unknown-warning-option -Wno-tautological-compare -Wno-sign-compare -D_REENTRANT -Wno-ignored-attributes     -DSTAN_THREADS -I stan/lib/stan_math/lib/tbb_2020.3/include    -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.78.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials    -DBOOST_DISABLE_ASSERTS          -c -include-pch stan/src/stan/model/model_header_threads_15_0.hpp.gch -x c++ -o redcard_reduce_sum.o redcard_reduce_sum.hpp
error: PCH file built from a different branch ((clang-1500.0.40.1)) than the compiler ((clang-1500.1.0.2.5))
1 error generated.
make: *** [redcard_reduce_sum] Error 1

I will rebuild cmdstan right now and retry

Note I did just do a software update (to try to “fix” the xcode thing so that may have also updated my compiler)

mathDR · December 20, 2023, 8:27pm

Okay, running make STAN_THREADS=1 redcard_reduce_sum in the cmdstan-2.33.1 directory produced an executable.

mathDR · December 20, 2023, 8:32pm

Note ldd doesn’t exist on mac, but I ran (in the cmdstan-2.33.1 directory)

otool -L redcard_reduce_sum

and got

redcard_reduce_sum:
	@rpath/libtbb.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libtbbmalloc.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libtbbmalloc_proxy.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1600.157.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.61.1)

mathDR · December 20, 2023, 8:34pm

Now:
file redcard_reduce_sum
returns

redcard_reduce_sum: Mach-O 64-bit executable arm64

running

./redcard_reduce_sum help

returns

zsh: killed     ./redcard_reduce_sum help

and now the executable is gone! Like, doesn’t exist anymore…

WardBrian · December 20, 2023, 8:39pm

Wow, that is impressive!

Are you running the native terminal or under Rosetta?

mathDR · December 20, 2023, 8:41pm

native terminal (but using zsh and not bash)

mathDR · December 20, 2023, 8:42pm

It is like, the executable gets made from the compiler, then Stan tries to run it and it disappears!

I have never heard of such a thing!

WardBrian · December 20, 2023, 9:10pm

I found some similar-sounding issues which imply it might be a code signing issue?

github.com/golang/go

cmd/go: macOS on arm64 requires codesigning

opened 12:52AM - 18 Nov 20 UTC

closed 11:39PM - 01 Dec 20 UTC

FiloSottile

OS-Darwin NeedsInvestigation FrozenDueToAge GoCommand release-blocker arch-arm64 okay-after-beta1

On the production Apple Silicon machines, Go binaries are killed at start. https…://github.com/golang/go/issues/38485#issuecomment-729301832 It looks like all binaries need to be codesigned now, and indeed running `codesign -s -` on them lets them run correctly. This stops `go test` and `go run` from working, and requires an extra step after `go build` to get a functional binary. This also affects the bootstrapped compiler itself.

I’m pinging some people who I believe have Apple Silicon macs to see if they’ve run into this at all @andrjohns @rok_cesnovar @spinkney

andrjohns · December 20, 2023, 11:26pm

Can you rebuild the executable and then run:

lldb ./redcard_reduce_sum help
run

And post the output?

mathDR · December 21, 2023, 1:53am

(cmdstanpy) 20:52:17 ~/.cmdstan/cmdstan-2.33.1 $ lldb ./redcard_reduce_sum help

(lldb) target create "./redcard_reduce_sum"

Current executable set to '/Users/daniel.marthaler/.cmdstan/cmdstan-2.33.1/redcard_reduce_sum' (arm64).

(lldb) settings set -- target.run-args "help"

(lldb) run

**error:** process exited with status -1 (no such process.)

(lldb)

mathDR · December 21, 2023, 3:01am

This page implies there may be a linker issue with Xcode 15?

If I could figure out how to downgrade xcode back to 14 I could pinpoint this. Alas, I am out for the Holiday but will revisit next week…

andrjohns · December 21, 2023, 7:47am

Very bizarre, I can’t replicate this at all.

Are you running the terminal in a conda environment or similar?

Can you try the following dummy test to see if you can compile and execute anything?

echo "#include <iostream> \n int main() { std::cout << \"success\" << std::endl; }" > test.cpp
clang++ test.cpp -o test
./test

mathDR · December 21, 2023, 12:04pm

Oh it probably got lost in all the messages, but everything “works” (can compile and sample) if I do NOT compile with STAN THREADS.

mathDR · December 21, 2023, 2:01pm

also, i am using a fresh pyenv environment (python 3.9.10) and I

pip install cmdstanpy

also:

pip freeze

yields:

aiohttp==3.9.1
aiosignal==1.3.1
algo-dis==0.1.4
astroid==2.11.7
async-timeout==4.0.3
attrs==23.1.0
black==24.1a1
boto3==1.34.4
botocore==1.34.4
certifi==2023.11.17
charset-normalizer==3.3.2
click==8.1.7
cmdstanpy==1.2.0
croniter==2.0.1
dill==0.3.7
exceptiongroup==1.2.0
fire==0.5.0
frozenlist==1.4.1
futura==1.30.6
idna==3.6
iniconfig==2.0.0
isort==5.13.2
Jinja2==3.1.2
jmespath==1.0.1
joblib==1.3.2
jsonref==1.1.0
jsonschema==4.7.0
lazy-object-proxy==1.10.0
MarkupSafe==2.1.3
mccabe==0.7.0
multidict==6.0.4
mypy-extensions==1.0.0
numpy==1.26.2
packaging==23.2
pandas==2.1.4
pathspec==0.12.1
platformdirs==4.1.0
pluggy==1.3.0
pylint==2.13.8
pyrsistent==0.20.0
pytest==7.4.3
python-dateutil==2.8.2
pytz==2023.3.post1
PyYAML==6.0.1
requests==2.31.0
ruamel.yaml==0.18.5
ruamel.yaml.clib==0.2.8
s3transfer==0.9.0
sf-boto==1.1.0
sf-request-headers==0.1.3
sh==2.0.6
six==1.16.0
stanio==0.3.0
tabulate==0.9.0
termcolor==2.4.0
tomli==2.0.1
tqdm==4.66.1
transdag==0.0.48
typing_extensions==4.9.0
tzdata==2023.3
urllib3==1.26.18
wrapt==1.16.0
yarl==1.9.4

Topic		Replies	Views
Correct way to use MPI with cmdstanpy Modeling	9	706	August 29, 2020
CmdStanPy and multithreading Modeling	10	1252	June 27, 2024
Multiprocessing and/or multithreading problem - CmdStanPy Modeling cmdstanpy , paralellization	12	87	January 2, 2025
Parallelization CmdStan	7	88	March 20, 2025
Trouble with within-chain parallelization with cmdstan (via cmdstanr) Modeling	4	869	August 21, 2020

Cmdstanpy reduce_sum doesn't produce executable

Related topics