I’m interested in running a model on a cluster which doesn’t support ocaml. But I need a custom version of ocaml to expose the prototype embedded laplace approximation, so downloading the binaries that come with cmdstan doesn’t do the trick.
The workaround that @seantalts and I worked out was to create the C++ file on my local machine, using the custom parser, export the C++ file to the cluster and compile the latter. I did this with a homebrewed wrapper of cmdstan (based on code from @billg), but I would now like to do this with cmdstanr.
If my understanding is correct, this means creating the right mod object. Is there a way to do this with the C++ translation of the Stan file?
1 Like
Not currently, unfortunately, but I think there is cleaner way.
The cluster is running linux I suppose? If you have a linux machine you can build the OCaml binary locally (see Stanc3 as submodule of cmdstan - #6 by rok_cesnovar for instructions on how to make portable binaries) and then place the binary on a cmdstan branch in bin/linux-stanc.
Running make build on the cluster will move bin/linux-stanc to bin/stanc and that should work then as regular cmdstan.
Edit: also see Stanc3 as submodule of cmdstan - #4 by rok_cesnovar
1 Like
The cluster indeed runs on linux, and my initial plan was to build the binaries locally. However
[ccm2172@holmes laplace_approximation]$ dune
-bash: dune: command not found
meaning dune isn’t supported, so, if my understanding is correct, I can’t build the OCaml library locally.
I am not sure I understand. You cant run dune locally or on the cluster? You dont need dune on the cluster if you build the binary locally.
I can run dune locally but not on the cluster.
I must’ve misunderstood you: how do I build the binary locally on the cluster?
Ah no, I meant locally as in on your computer/laptop.
I suppose that could work – let me give it a try.
My (naive) intuition is that the binaries I generate on my machine would not work on another machine.
Make sure to checkout the thread i linked above and you should be all set.
I’m assuming the binary files are stored in _build/. Can I build the binary files on my mac, and then port these to a linux system?
Right now, I’m getting error messages with both commands (ran from my local mac):
charlesm$ dune build @install
menhir src/frontend/parsing_errors.ml
Read 455 sample input sentences and 455 error messages.
charlesm$ dune build @install --profile static
menhir src/frontend/parsing_errors.ml
Read 455 sample input sentences and 455 error messages.
ocamlopt src/stan2tfp/stan2tfp.exe (exit 2)
(cd _build/default && /Users/charlesm/.opam/4.07.0/bin/ocamlopt.opt -ccopt -static -g -o src/stan2tfp/stan2tfp.exe -I /Users/charlesm/.opam/4.07.0/lib/base -I /Users/charlesm/.opam/4.07.0/lib/base/caml -I /Users/charlesm/.opam/4.07.0/lib/base/md5 -I /Users/charlesm/.opam/4.07.0/lib/base/shadow_stdlib -I /Users/charlesm/.opam/4.07.0/lib/bin_prot -I /Users/charlesm/.opam/4.07.0/lib/bin_prot/shape -I /Users/charlesm/.opam/4.07.0/lib/core_kernel -I /Users/charlesm/.opam/4.07.0/lib/core_kernel/base_for_tests -I /Users/charlesm/.opam/4.07.0/lib/fieldslib -I /Users/charlesm/.opam/4.07.0/lib/fmt -I /Users/charlesm/.opam/4.07.0/lib/jane-street-headers -I /Users/charlesm/.opam/4.07.0/lib/menhirLib -I /Users/charlesm/.opam/4.07.0/lib/parsexp -I /Users/charlesm/.opam/4.07.0/lib/ppx_assert/runtime-lib -I /Users/charlesm/.opam/4.07.0/lib/ppx_bench/runtime-lib -I /Users/charlesm/.opam/4.07.0/lib/ppx_compare/runtime-lib -I /Users/charlesm/.opam/4.07.0/lib/ppx_deriving -I /Users/charlesm/.opam/4.07.0/lib/ppx_expect/collector -I /Users/charlesm/.opam/4.07.0/lib/ppx_expect/common -I /Users/charlesm/.opam/4.07.0/lib/ppx_expect/config -I /Users/charlesm/.opam/4.07.0/lib/ppx_hash/runtime-lib -I /Users/charlesm/.opam/4.07.0/lib/ppx_inline_test/config -I /Users/charlesm/.opam/4.07.0/lib/ppx_inline_test/runtime-lib -I /Users/charlesm/.opam/4.07.0/lib/ppx_sexp_conv/runtime-lib -I /Users/charlesm/.opam/4.07.0/lib/re -I /Users/charlesm/.opam/4.07.0/lib/result -I /Users/charlesm/.opam/4.07.0/lib/seq -I /Users/charlesm/.opam/4.07.0/lib/sexplib -I /Users/charlesm/.opam/4.07.0/lib/sexplib0 -I /Users/charlesm/.opam/4.07.0/lib/splittable_random -I /Users/charlesm/.opam/4.07.0/lib/stdio -I /Users/charlesm/.opam/4.07.0/lib/typerep -I /Users/charlesm/.opam/4.07.0/lib/uchar -I /Users/charlesm/.opam/4.07.0/lib/variantslib -I src/analysis_and_optimization -I src/common -I src/frontend -I src/middle -I src/tfp_backend /Users/charlesm/.opam/4.07.0/lib/sexplib0/sexplib0.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_sexp_conv/runtime-lib/ppx_sexp_conv_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/base/caml/caml.cmxa /Users/charlesm/.opam/4.07.0/lib/base/shadow_stdlib/shadow_stdlib.cmxa /Users/charlesm/.opam/4.07.0/lib/base/base.cmxa /Users/charlesm/.opam/4.07.0/lib/fieldslib/fieldslib.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_inline_test/config/inline_test_config.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_inline_test/runtime-lib/ppx_inline_test_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_compare/runtime-lib/ppx_compare_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_expect/common/expect_test_common.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_expect/config/expect_test_config.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_expect/collector/expect_test_collector.cmxa /Users/charlesm/.opam/4.07.0/lib/variantslib/variantslib.cmxa /Users/charlesm/.opam/4.07.0/lib/typerep/typerep_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/base/md5/md5_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/ocaml/unix.cmxa /Users/charlesm/.opam/4.07.0/lib/ocaml/bigarray.cmxa /Users/charlesm/.opam/4.07.0/lib/bin_prot/shape/bin_shape_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/bin_prot/bin_prot.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_bench/runtime-lib/ppx_bench_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_assert/runtime-lib/ppx_assert_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_hash/runtime-lib/ppx_hash_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/result/result.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_deriving/ppx_deriving_runtime.cmxa /Users/charlesm/.opam/4.07.0/lib/core_kernel/base_for_tests/base_for_tests.cmxa /Users/charlesm/.opam/4.07.0/lib/stdio/stdio.cmxa /Users/charlesm/.opam/4.07.0/lib/jane-street-headers/jane_street_headers.cmxa /Users/charlesm/.opam/4.07.0/lib/parsexp/parsexp.cmxa /Users/charlesm/.opam/4.07.0/lib/sexplib/sexplib.cmxa /Users/charlesm/.opam/4.07.0/lib/splittable_random/splittable_random.cmxa /Users/charlesm/.opam/4.07.0/lib/core_kernel/core_kernel.cmxa /Users/charlesm/.opam/4.07.0/lib/re/re.cmxa /Users/charlesm/.opam/4.07.0/lib/menhirLib/menhirLib.cmx /Users/charlesm/.opam/4.07.0/lib/fmt/fmt.cmxa /Users/charlesm/.opam/4.07.0/lib/ocaml/str.cmxa src/common/common.cmxa src/middle/middle.cmxa src/analysis_and_optimization/analysis_and_optimization.cmxa src/frontend/frontend.cmxa src/tfp_backend/tfp_backend.cmxa src/stan2tfp/.stan2tfp.eobjs/native/Stan2tfp.cmx)
ld: library not found for -lcrt0.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
File "caml_startup", line 1:
Error: Error during linking
ocamlopt src/stanc/stanc.exe (exit 2)
(cd _build/default && /Users/charlesm/.opam/4.07.0/bin/ocamlopt.opt -ccopt -static -g -o src/stanc/stanc.exe -I /Users/charlesm/.opam/4.07.0/lib/base -I /Users/charlesm/.opam/4.07.0/lib/base/caml -I /Users/charlesm/.opam/4.07.0/lib/base/md5 -I /Users/charlesm/.opam/4.07.0/lib/base/shadow_stdlib -I /Users/charlesm/.opam/4.07.0/lib/bin_prot -I /Users/charlesm/.opam/4.07.0/lib/bin_prot/shape -I /Users/charlesm/.opam/4.07.0/lib/biniou -I /Users/charlesm/.opam/4.07.0/lib/core_kernel -I /Users/charlesm/.opam/4.07.0/lib/core_kernel/base_for_tests -I /Users/charlesm/.opam/4.07.0/lib/easy-format -I /Users/charlesm/.opam/4.07.0/lib/fieldslib -I /Users/charlesm/.opam/4.07.0/lib/fmt -I /Users/charlesm/.opam/4.07.0/lib/jane-street-headers -I /Users/charlesm/.opam/4.07.0/lib/menhirLib -I /Users/charlesm/.opam/4.07.0/lib/parsexp -I /Users/charlesm/.opam/4.07.0/lib/ppx_assert/runtime-lib -I /Users/charlesm/.opam/4.07.0/lib/ppx_bench/runtime-lib -I /Users/charlesm/.opam/4.07.0/lib/ppx_compare/runtime-lib -I /Users/charlesm/.opam/4.07.0/lib/ppx_deriving -I /Users/charlesm/.opam/4.07.0/lib/ppx_expect/collector -I /Users/charlesm/.opam/4.07.0/lib/ppx_expect/common -I /Users/charlesm/.opam/4.07.0/lib/ppx_expect/config -I /Users/charlesm/.opam/4.07.0/lib/ppx_hash/runtime-lib -I /Users/charlesm/.opam/4.07.0/lib/ppx_inline_test/config -I /Users/charlesm/.opam/4.07.0/lib/ppx_inline_test/runtime-lib -I /Users/charlesm/.opam/4.07.0/lib/ppx_sexp_conv/runtime-lib -I /Users/charlesm/.opam/4.07.0/lib/re -I /Users/charlesm/.opam/4.07.0/lib/result -I /Users/charlesm/.opam/4.07.0/lib/seq -I /Users/charlesm/.opam/4.07.0/lib/sexplib -I /Users/charlesm/.opam/4.07.0/lib/sexplib0 -I /Users/charlesm/.opam/4.07.0/lib/splittable_random -I /Users/charlesm/.opam/4.07.0/lib/stdio -I /Users/charlesm/.opam/4.07.0/lib/typerep -I /Users/charlesm/.opam/4.07.0/lib/uchar -I /Users/charlesm/.opam/4.07.0/lib/variantslib -I /Users/charlesm/.opam/4.07.0/lib/yojson -I src/analysis_and_optimization -I src/common -I src/frontend -I src/middle -I src/stan_math_backend /Users/charlesm/.opam/4.07.0/lib/sexplib0/sexplib0.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_sexp_conv/runtime-lib/ppx_sexp_conv_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/base/caml/caml.cmxa /Users/charlesm/.opam/4.07.0/lib/base/shadow_stdlib/shadow_stdlib.cmxa /Users/charlesm/.opam/4.07.0/lib/base/base.cmxa /Users/charlesm/.opam/4.07.0/lib/fieldslib/fieldslib.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_inline_test/config/inline_test_config.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_inline_test/runtime-lib/ppx_inline_test_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_compare/runtime-lib/ppx_compare_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_expect/common/expect_test_common.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_expect/config/expect_test_config.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_expect/collector/expect_test_collector.cmxa /Users/charlesm/.opam/4.07.0/lib/variantslib/variantslib.cmxa /Users/charlesm/.opam/4.07.0/lib/typerep/typerep_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/base/md5/md5_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/ocaml/unix.cmxa /Users/charlesm/.opam/4.07.0/lib/ocaml/bigarray.cmxa /Users/charlesm/.opam/4.07.0/lib/bin_prot/shape/bin_shape_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/bin_prot/bin_prot.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_bench/runtime-lib/ppx_bench_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_assert/runtime-lib/ppx_assert_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_hash/runtime-lib/ppx_hash_lib.cmxa /Users/charlesm/.opam/4.07.0/lib/result/result.cmxa /Users/charlesm/.opam/4.07.0/lib/ppx_deriving/ppx_deriving_runtime.cmxa /Users/charlesm/.opam/4.07.0/lib/core_kernel/base_for_tests/base_for_tests.cmxa /Users/charlesm/.opam/4.07.0/lib/stdio/stdio.cmxa /Users/charlesm/.opam/4.07.0/lib/jane-street-headers/jane_street_headers.cmxa /Users/charlesm/.opam/4.07.0/lib/parsexp/parsexp.cmxa /Users/charlesm/.opam/4.07.0/lib/sexplib/sexplib.cmxa /Users/charlesm/.opam/4.07.0/lib/splittable_random/splittable_random.cmxa /Users/charlesm/.opam/4.07.0/lib/core_kernel/core_kernel.cmxa /Users/charlesm/.opam/4.07.0/lib/re/re.cmxa /Users/charlesm/.opam/4.07.0/lib/menhirLib/menhirLib.cmx /Users/charlesm/.opam/4.07.0/lib/fmt/fmt.cmxa /Users/charlesm/.opam/4.07.0/lib/ocaml/str.cmxa src/common/common.cmxa src/middle/middle.cmxa src/analysis_and_optimization/analysis_and_optimization.cmxa src/frontend/frontend.cmxa /Users/charlesm/.opam/4.07.0/lib/easy-format/easy_format.cmxa /Users/charlesm/.opam/4.07.0/lib/biniou/biniou.cmxa /Users/charlesm/.opam/4.07.0/lib/yojson/yojson.cmxa src/stan_math_backend/stan_math_backend.cmxa src/stanc/.stanc.eobjs/native/stanc.cmx)
ld: library not found for -lcrt0.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
File "caml_startup", line 1:
Error: Error during linking
Yes, the binary is _build/default/src/stanc.exe
Just trim the .exe on unix systems.
I am not sure if you can build on a mac and port to linux. I think the error you are seeing is related to some issue with the version of either opam, ocaml or dune. I think I saw it once before I cleaned my ocaml environment and used the stanc3 repo provided scripts as intended. But I am far from an expert of the ocaml ecosystem, so take this with a huge grain of salt.
If you would want to build a linux binary you could use Docker (I think), a VM or a free AWS instance to build on a linux machine.
Assuming the branch in question is GitHub - stan-dev/stanc3 at try-laplace_approximation
I went ahead and built a binary on my machine and pushed it to a cmdstan branch - GitHub - stan-dev/cmdstan at laplace_approximation_stanc. Just delete the branch once you move it to your branch.
1 Like
@rok_cesnovar the binary files you created work and allow me to compile my model on the cluster. So cmdstanr is now a very viable option. Mind you, it doesn’t work on mac, but I suppose this was expected. Thank you for all your help!
1 Like
Glad to help!
Yes, that is expected. To make it work on a mac, just build with dune build @install on your machine and move the generated stanc binary to the cmdstan branch to bin/mac-stanc.
I think you need to manually add it on the branch (git add bin/mac-stanc) and then you can work on all machines with this single branch which should simplify everything.
I think you need to manually add it on the branch ( git add bin/mac-stanc ) and then you can work on all machines with this single branch which should simplify everything.
Once I have the branch, it’s not too hard to build the binaries locally on my mac.
I edited the stanc3 branch to expose a new rng function. I’m trying to work out how to build binaries for linux on mac, but most of what I’m finding is building mac binaries with linux source (so opposite problem). How did you do it? I’m assuming you have a linux machine with dune on it.
You are in luck since we now have a specific job on jenkins which you can use to build all binaries.
See https://github.com/stan-dev/ci-scripts/blob/refactor-jenkins-docs/docs/jobs/stanc3-test-binaries.md
We first used this for pedantic mode testing https://github.com/stan-dev/stanc3/pull/434
Also see the pedantic mode branch to see an example on how to use this.
This might also interest @yizhang
We should have made a post about this. Its really a great piece of work by @serban-nicusor
Let me know if there is anything missing in the instructions or need additional help.
Excellent!
Looking at https://jenkins.mc-stan.org/job/stanc3-test-binaries/, it looks like the binaries I want to download are in linux-stanc (with windows and mac counterparts). Two things aren’t clear to me: (i) how to get jenkins to do the build for a specific branch, and (ii) where to store the binaries once I downloaded them (I’m guessing under cmdStan/bin/linux-stanc).
To build for a branch you need to log in and go to “build with parameters”.
The location is correct. But I would advise you to use the convenience make option I added here: https://github.com/stan-dev/cmdstan/pull/843
After building parameters, I get the following error in the log
+ git pull try-laplace_approximation
fatal: 'try-laplace_approximation' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I’m confident the repository exists. It is possible I don’t have the correct access rights?
Oh, it seems that for now this only works for open stanc3 PRs.
It should also work for branches, but that wasnt yet tested. @serban-nicusor please have a look when you have the time.
I opened a PR to test this and managed to get it working (here are the binaries: https://jenkins.mc-stan.org/job/stanc3-test-binaries/44/). I closed the PR already.
The other thing was that your branch was failing stanc3 tests and that prevents it to build.
It was missing dune runtest and dune promote. I took the libertiy to push that to your branch, feel free to reset the branch.
2 Likes
So if you are on develop cmdstan, add
STANC3_TEST_BIN_URL=https://jenkins.mc-stan.org/job/stanc3-test-binaries/44/artifact
to your make/local and you will have the laplace stanc3