Using earlier versions of Stan

I would like easily switch between different versions of Stan. E.g. use latest version of Stan most of the time (i.e. master branch) but be able to switch to earlier versions so that I can reproduce previous analysis.

I tried adapting the instructions on Getting-Started-with-CmdStan under the heading “Using a different Stan branch” but I get an error message that I don’t know how to fix.

Here are the commands I tried running.

git clone https://github.com/stan-dev/cmdstan.git --recursive
cd cmdstan
make stan-update
make build                     # successful
git checkout v2.23.0
make stan-update/v2.23.0       # recipe failed: Please specify which branch you want to merge with.
make clean-all
make build                     # error: No rule to make target 'src/cmdstan/main.o', needed by 'build'

Is there an easy way to checkout tagged commits with the appropriate submodule commits?

1 Like

Try:

make clean-all # first clean because checkout or update can fail if not cleaned previously
git checkout v2.23.0
make stan-update # just stan-update
make build

Tried your suggestion but I still get the same error. Here is the output from my terminal.

[philipm@oban cmdstan]$ make clean-all
rm -f -r doc
cd src/docs/cmdstan-guide; rm -f *.brf *.aux *.bbl *.blg *.log *.toc *.pdf *.out *.idx *.ilg *.ind *.cb *.cb2 *.upa
rm -f -r test
rm -f
rm -f
rm -f
rm -f
  removing dependency files
rm -f src/cmdstan/stanc.d
rm -f
rm -f
  cleaning sundials targets
rm -f stan/lib/stan_math/lib/sundials_5.2.0/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_5.2.0/lib/libsundials_idas.a stan/lib/stan_math/lib/sundials_5.2.0/lib/libsundials_kinsol.a stan/lib/stan_math/lib/sundials_5.2.0/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodea.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodea_io.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodes.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodes_bandpre.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodes_bbdpre.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodes_diag.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodes_direct.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodes_io.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodes_ls.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodes_nls.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodes_nls_sim.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodes_nls_stg.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodes_nls_stg1.o stan/lib/stan_math/lib/sundials_5.2.0/src/cvodes/cvodes_spils.o stan/lib/stan_math/lib/sundials_5.2.0/src/idas/idaa.o stan/lib/stan_math/lib/sundials_5.2.0/src/idas/idaa_io.o stan/lib/stan_math/lib/sundials_5.2.0/src/idas/idas.o stan/lib/stan_math/lib/sundials_5.2.0/src/idas/idas_bbdpre.o stan/lib/stan_math/lib/sundials_5.2.0/src/idas/idas_direct.o stan/lib/stan_math/lib/sundials_5.2.0/src/idas/idas_ic.o stan/lib/stan_math/lib/sundials_5.2.0/src/idas/idas_io.o stan/lib/stan_math/lib/sundials_5.2.0/src/idas/idas_ls.o stan/lib/stan_math/lib/sundials_5.2.0/src/idas/idas_nls.o stan/lib/stan_math/lib/sundials_5.2.0/src/idas/idas_nls_sim.o stan/lib/stan_math/lib/sundials_5.2.0/src/idas/idas_nls_stg.o stan/lib/stan_math/lib/sundials_5.2.0/src/idas/idas_spils.o stan/lib/stan_math/lib/sundials_5.2.0/src/kinsol/kinsol.o stan/lib/stan_math/lib/sundials_5.2.0/src/kinsol/kinsol_bbdpre.o stan/lib/stan_math/lib/sundials_5.2.0/src/kinsol/kinsol_direct.o stan/lib/stan_math/lib/sundials_5.2.0/src/kinsol/kinsol_io.o stan/lib/stan_math/lib/sundials_5.2.0/src/kinsol/kinsol_ls.o stan/lib/stan_math/lib/sundials_5.2.0/src/kinsol/kinsol_spils.o stan/lib/stan_math/lib/sundials_5.2.0/src/nvector/serial/nvector_serial.o stan/lib/stan_math/lib/sundials_5.2.0/src/sundials/sundials_band.o stan/lib/stan_math/lib/sundials_5.2.0/src/sundials/sundials_dense.o stan/lib/stan_math/lib/sundials_5.2.0/src/sundials/sundials_direct.o stan/lib/stan_math/lib/sundials_5.2.0/src/sundials/sundials_futils.o stan/lib/stan_math/lib/sundials_5.2.0/src/sundials/sundials_iterative.o stan/lib/stan_math/lib/sundials_5.2.0/src/sundials/sundials_linearsolver.o stan/lib/stan_math/lib/sundials_5.2.0/src/sundials/sundials_math.o stan/lib/stan_math/lib/sundials_5.2.0/src/sundials/sundials_matrix.o stan/lib/stan_math/lib/sundials_5.2.0/src/sundials/sundials_nonlinearsolver.o stan/lib/stan_math/lib/sundials_5.2.0/src/sundials/sundials_nvector.o stan/lib/stan_math/lib/sundials_5.2.0/src/sundials/sundials_nvector_senswrapper.o stan/lib/stan_math/lib/sundials_5.2.0/src/sundials/sundials_version.o stan/lib/stan_math/lib/sundials_5.2.0/src/sunlinsol/band/sunlinsol_band.o stan/lib/stan_math/lib/sundials_5.2.0/src/sunlinsol/dense/sunlinsol_dense.o stan/lib/stan_math/lib/sundials_5.2.0/src/sunmatrix/band/sunmatrix_band.o stan/lib/stan_math/lib/sundials_5.2.0/src/sunmatrix/dense/sunmatrix_dense.o stan/lib/stan_math/lib/sundials_5.2.0/src/sunnonlinsol/fixedpoint/sunnonlinsol_fixedpoint.o stan/lib/stan_math/lib/sundials_5.2.0/src/sunnonlinsol/newton/sunnonlinsol_newton.o
  cleaning Intel TBB targets
rm -f -rf stan/lib/stan_math/lib/tbb
rm -f bin/stanc bin/stanc2 bin/stansummary bin/print bin/diagnose
rm -f -r src/cmdstan/main.o bin/cmdstan
rm -f
rm -f examples/bernoulli/bernoulli examples/bernoulli/bernoulli.o examples/bernoulli/bernoulli.d examples/bernoulli/bernoulli.hpp
[philipm@oban cmdstan]$ git checkout v2.23.0
M       stan
HEAD is now at 5107e83... Updated version numbers
[philipm@oban cmdstan]$ make stan-update # just stan-update
git submodule update --init --recursive
[philipm@oban cmdstan]$ make build
g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare      -I stan/lib/stan_math/lib/tbb_2019_U8/include -O3 -I src -I stan/src -I lib/rapidjson_1.1.0/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.3 -I stan/lib/stan_math/lib/boost_1.72.0 -I stan/lib/stan_math/lib/sundials_5.2.0/include    -DBOOST_DISABLE_ASSERTS      -c -MT src/cmdstan/stanc.o -MM -E -MG -MP -MF src/cmdstan/stanc.d src/cmdstan/stanc.cpp
curl -L https://github.com/stan-dev/stanc3/releases/download/nightly/linux-stanc -o bin/stanc --retry 5 --retry-delay 10
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   624  100   624    0     0   1273      0 --:--:-- --:--:-- --:--:--  1270
100 16.5M  100 16.5M    0     0  2263k      0  0:00:07  0:00:07 --:--:-- 2834k
chmod +x bin/stanc
g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare      -I stan/lib/stan_math/lib/tbb_2019_U8/include -O3 -I src -I stan/src -I lib/rapidjson_1.1.0/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.3 -I stan/lib/stan_math/lib/boost_1.72.0 -I stan/lib/stan_math/lib/sundials_5.2.0/include    -DBOOST_DISABLE_ASSERTS      -c -o bin/cmdstan/stansummary.o src/cmdstan/stansummary.cpp
...
sh ../tbb_2019_U8/build/generate_tbbvars.sh
echo "INPUT (libtbb.so.2)" > libtbb.so
g++ -E -x c++ ../tbb_2019_U8/src/tbb/lin64-tbb-export.def -DDO_ITT_NOTIFY -O2 -DUSE_PTHREAD -m64 -mrtm  -fPIC  -D__TBB_BUILD=1 -Wall -Wextra -Wno-parentheses -Wno-sized-deallocation  -I../tbb_2019_U8/src -I../tbb_2019_U8/src/rml/include -I../tbb_2019_U8/include > tbb.def
g++ -fPIC -o libtbb.so.2 concurrent_hash_map.o concurrent_queue.o concurrent_vector.o dynamic_link.o itt_notify.o cache_aligned_allocator.o pipeline.o queuing_mutex.o queuing_rw_mutex.o reader_writer_lock.o spin_rw_mutex.o x86_rtm_rw_mutex.o spin_mutex.o critical_section.o mutex.o recursive_mutex.o condition_variable.o tbb_thread.o concurrent_monitor.o semaphore.o private_server.o rml_tbb.o tbb_misc.o tbb_misc_ex.o task.o task_group_context.o governor.o market.o arena.o scheduler.o observer_proxy.o tbb_statistics.o tbb_main.o concurrent_vector_v2.o concurrent_queue_v2.o spin_rw_mutex_v2.o task_v2.o   -ldl -lpthread -lrt -shared -Wl,-soname=libtbb.so.2 -m64 -Wl,-L,"/home/nag/oban/philipm/cmdstan/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/home/nag/oban/philipm/cmdstan/stan/lib/stan_math/lib/tbb" -Wl,--version-script,tbb.def
make[1]: Leaving directory '/home/nag/oban/philipm/cmdstan/stan/lib/stan_math/lib/tbb'
make: *** No rule to make target 'src/cmdstan/main.o', needed by 'build'. Stop.

In my case

make stan-update
is followed by

git submodule update --init --recursive
Submodule path ‘stan’: checked out ‘d44315be81899824ff32da81a6c74101dba995e1’
Submodule path ‘stan/lib/stan_math’: checked out ‘d4c32ed277a26b86bde275edce81ba1e35b508c5’

I am guessing in your case it cant checkout stan or math.

Try

cd stan/
git checkout v2.23.0

and if that works fine

cd lib/stan_math/
git checkout v3.2.0 (math 3.2.0 matches cmdstan 2.23.0)

1 Like

As another option, would using cmdstanr::install_cmdstan() with the release_url argument help here? To install stan and cmdstan v2.22.0, for example, you could do

cmdstanr::install_cmdstan(release_url = "https://github.com/stan-dev/cmdstan/releases/download/v2.22.0/cmdstan-2.22.0.tar.gz")

Then even if you don’t want to use CmdStanR or R at all you can use the CmdStan installation it makes for you.

1 Like

I am still getting the same error. This is really puzzling. I am pretty sure the commit ids for stan and stan_math are the same as what you posted. Here is some info about the state of my check out from git.

[philipm@oban cmdstan]$ git status
HEAD detached at v2.23.0
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)
  (commit or discard the untracked or modified content in submodules)

        modified:   stan (modified content)

no changes added to commit (use "git add" and/or "git commit -a")
[philipm@oban cmdstan]$ git diff
diff --git a/stan b/stan
--- a/stan
+++ b/stan
@@ -1 +1 @@
-Subproject commit d44315be81899824ff32da81a6c74101dba995e1
+Subproject commit d44315be81899824ff32da81a6c74101dba995e1-dirty
[philipm@oban cmdstan]$ cd stan/
[philipm@oban stan]$ git status
HEAD detached at d44315b
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)
  (commit or discard the untracked or modified content in submodules)

        modified:   lib/stan_math (untracked content)

no changes added to commit (use "git add" and/or "git commit -a")
[philipm@oban stan]$ cd lib/stan_math/
[philipm@oban stan_math]$ git status
HEAD detached at d4c32ed
Untracked files:
  (use "git add <file>..." to include in what will be committed)

        lib/boost_1.72.0/tools/build/src/engine/b2
        lib/boost_1.72.0/tools/build/src/engine/bjam

nothing added to commit but untracked files present (use "git add" to track)
[philipm@oban stan_math]$ cd ../../..
[philipm@oban cmdstan]$ cat .gitmodules
[submodule "stan"]
        path = stan
        url = https://github.com/stan-dev/stan
[philipm@oban cmdstan]$ git submodule status --recursive
 d44315be81899824ff32da81a6c74101dba995e1 stan (v2.23.0)
 d4c32ed277a26b86bde275edce81ba1e35b508c5 stan/lib/stan_math (v3.2.0)

Try checking out v2.24 again and run make clean-all.

If that does not work, remove the two files listed

lib/boost_1.72.0/tools/build/src/engine/b2
lib/boost_1.72.0/tools/build/src/engine/bjam

I have now found a solution to this. The problem was specific to v2.23.0 and to a subset of the machines I tried. Here is a minimal example.

wget https://github.com/stan-dev/cmdstan/releases/download/v2.23.0/cmdstan-2.23.0.tar.gz
tar -xzvf cmdstan-2.23.0.tar.gz
cd cmdstan-2.23.0
make build                     # error: No rule to make target 'src/cmdstan/main.o', needed by 'build'

The problem was that some implicit rules that were assumed by the Stan makefile were not defined (at least in v2.23). I guess most machines define these implicit rules for the user, but in some environments they need to be defined explicitly. Putting the following in make/local solved the problem for me.

%.o : %.c
	$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@
%.o : %.cpp
	$(CXX) $(CXXFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@
2 Likes

@philipm, I’m glad you got it sorted out. Maybe this info would help… if it doesn’t, feel free to ignore:

  • I have multiple folders containing different versions of CmdStan (and Math). I’m not creative… so I’ll have cmdstan, cmdstan2, and cmdstan3 all at different versions.
  • if you go back far enough, you’ll have to be careful how to build and follow the instructions for that version (usually in the README.md). At some point we switched from using CC as the compiler to CXX. We had started using CC well before v1.0 and it stayed around until we could make a serious overhaul of the build.
  • you might want to check you make version. There is actually quite a lot of difference in functionality between the different versions of make; we used to write workarounds in the makefiles, but we’re doing that less and encouraging upgrading to what we know works. (that might explain your implicit rule problem)
1 Like