CmdStan & Stan 2.33 release candidate

General
1

I am very happy to announce that the latest release candidates of Cmdstan and Stan are now available on Github!

This release cycle brings a new type - tuples, a new algorithm - Pathfinder, many new functions, and other improvements.

You can find the release candidate for cmdstan here. Instructions for installing are given at the bottom of this post.

Please test the release candidate with your models and report if you experience any problems. We also kindly invite you to test the new features and provide feedback. If you feel some of the new features could be improved or should be changed before the release, please do not hesitate to comment.

The Stan development team appreciates your time and help in making Stan more efficient while maintaining a high level of reliability.

If everything goes according to plan, the 2.33 version will be released in about ten days.

Below are some of the highlights of the new release.

Tuples

The Stan language gains a new type: tuples!

For any sequence of types, Stan provides a tuple data type. For example,


tuple(real, array[5] int) xi;

declares xi to be a tuple holding two values, the first of which is of type type real and the second of which a 5-dimensional array of type int. Tuples can be of arbitrary size. The values in a tuple can be of arbitrary type, but the component types must be declared along with the declaration of the tuple. Tuples can be manipulated as a whole, or their elements may be accessed and set individually.

Currently, the documentation is available here and here.

Pathfinder

A new Variational Inference algorithm is now available via CmdStan - Pathfinder!

Pathfinder is a variational method for approximately sampling from differentiable log densities. Starting from a random initialization, Pathfinder locates normal approximations to the target density along a quasi-Newton optimization path, with local covariance estimated using the inverse Hessian estimates produced by the L-BFGS optimizer. Pathfinder returns draws from the Gaussian approximation with the lowest estimated Kullback-Leibler (KL) divergence to the true posterior.

For more information see the preliminary documentation, the paper and Bob Carpenter’s tutorial.

New functions and function signatures

  • new functions qr_thin, eigendecompose_sym, eigendecompose, complex_schur_decompose, svd, and csr_extract are now available

  • eigendecompose(A) is equivalent to (eigenvectors(A), eigenvalues(A))

  • eigendecompose_sym(A) is equivalent to (eigenvectors_sym(A), eigenvalues_sym(A))

  • qr_thin(A) is equivalent to (qr_thin_Q(A), qr_thin_R(A))

  • qr(A) is equivalent to (qr_Q(A), qr_R(A))

  • svd(A) is equivalent to (svd_U(A), singular_values(A), svd_V(A))

  • csr_extract(A) is equivalent to (csr_extract_w(a), csr_extract_v(a), csr_extract_u(a))

These functions all return tuples and have a lower computational cost due to the shared work to produce the multiple results.

  • added vectorized signatures for log_sum_exp

The function now supports all arguments that are typically supported in fully vectorized functions.

Deprecations

The following deprecations have been turned into errors this version:

  • The old array syntax, e.g. int arr[5];. Use array[5] int arr; instead.

  • Distribution functions ending in _log. Use either _lpdf or _lpmf.

  • The functions binomial_coefficient_log, multiply_log, and cov_exp_quad. Use lchoose, lmultiply, and gp_exp_quad_cov respectively.

  • The if_else function. Use the ternary operator cond ? true_value : false_value

  • Use of CDFs with a , between the first and second argument. Use a |.

  • Comments beginning with #. Use //.

  • Use of <- for assignment. Use =.

  • The increment_log_prob function. Use the target += statement.

  • The get_lp() function. Use target().

  • The use of nested multi-indices on the left hand side of an assignment statement.

For this version, these can all be automatically updated with the --canonicalize=deprecations argument to the autoformatter. This is not guaranteed to work for versions following this one.

Additionally, the following identifiers are now reserved words: array, offset, multiplier, lower, and upper.

Other miscellaneous features

  • The num_chains argument for running multiple chains with a single executable is now available to all HMC variants other than the static engine

  • Improved accuracy of matrix_exp_multiply

  • using the RDump input files now shows a warning - the RDump input files will not support tuples and are unlikely to support othew new types in the future. Please switch to the JSON input.

  • Return statements now follow the same type promotion rules as assignment and function argument passing.

How to install?

Download the tar.gz file from the link above, extract it and use it the way you use any Cmdstan release. We also have an online Cmdstan guide available at CmdStan User’s Guide 1

If you are using cmdstanpy you can install the release candidate using

cmdstanpy.install_cmdstan(version='2.33.0-rc1')

With CmdStanR you can install the release candidate using

cmdstanr::install_cmdstan(version = "2.33.0-rc1", cores = 4)
13 Likes
2

Just want to give users a heads up that pathfinder is not yet available via the interfaces that wrap CmdStan, only in CmdStan. So if you install this release candidate via CmdStanR or CmdStanPy there won’t be a pathfinder method yet. Hopefully we’ll have that ready soon.

6 Likes
3

@rok_cesnovar is that a typo that qr_thin_R(A) is equivalent to (qr_thin_Q(A), qr_thin_R(A) and should just be qr_thin(A)?

3 Likes
4

Fixed, thanks. Was a typo.

1 Like
5

here’s the CmdStan documentation on Pathfinder:

Pathfinder overview:
Pathfinder_overview.pdf (42.2 KB)

Pathfinder config:
Pathfinder_config.pdf (43.3 KB)

(note: docs have been updated per PR comments)

8 Likes
6

Aki just pointed me to the cmdstanr branch that enables access to pathfinder, but in case it’s useful to anyone, here’s a by-hand way to use it:

run_pathfinder = function(data,mod){
	data_file = tempfile(fileext='.json')
	output_file = tempfile(fileext='.csv')
	cmdstanr::write_stan_json(data,file=data_file)
	processx::run(
		command = mod$exe_file()
		, args = c(
			'pathfinder'
			, 'data'
			, paste0('file=',data_file)
			, 'output'
			, paste0('file=',output_file)
		)
		, echo_cmd = T
		, stdout = ""
		, stderr = "2>&1"
	)
	
	(
		paste0("grep '^[#l]' '",output_file,"'")
		%>% system(intern=T)
		%>% strsplit('\n')
		%>% unlist()
	) -> header
	header_nlines = which(stringr::str_starts(header,'lp'))
	found_samples_col_names = unlist(strsplit(header[header_nlines],','))
	(
		data.table::fread(
			cmd = paste0(
				"tail -n+"
				, header_nlines + 1
				, " '"
				, output_file
				, "' | grep -v '^[#]' --color=never"
			)
			, data.table = FALSE
			, sep = ','
			, header = F
			, col.names = found_samples_col_names
			, colClasses = list(numeric=1:length(found_samples_col_names))
		)
		%>% as_tibble()
	) ->
		out
	return(out)
}
9 Likes
7

I think that particular cmdstanr branch is accessing an old version of CmdStan (never released) which has an earlier implementation of Pathfinder - @jonah ? but I think this code would work when running against the 2.33 RC - although maybe not the output file parsing.

8

@stevebronder would know best but my guess is that the CmdStanR branch doesn’t access any particular version of CmdStan but hasn’t been tested with the newer version.

9

I just tried pathfinder as it sounds soooo powerful. I am getting numbers for my model impressively fast… but right now the lack of support by cmdstanr is a problem for me, since the output is so different in term of the format to what my programs are written for that it is difficult to say anything. So getting cmdstanr in shape would be really nice in order to say something.

One more thing: Given that tuples land - which is awesome - is there a possibility to do something like a typedef now? I mean, I would like to define tuples for various things, but I really don’t like to retype the tuple definition over and over again…should I file a feature request to stanc/stan repo (assuming others think that this is good idea)?

I must admit that I have stopped a while ago to update Stan as 2.28 was good enough (and that’s what is on our prod system), but 2.33 is already now on my system - looking forward to see it land!

4 Likes
10

This is definitely something we want before we’d do structs, which are a natural place to go next.

I’ve also previously brought up something like C++'s auto/Java’s var keyword which lets you omit the type on the left hand side of a declare-define. This never made it to the level of design doc, but it would be nice to do still IMO

3 Likes
11

@bgoodri any news on getting RStan updated in CRAN? Otherwise, this change will cause a huge amount of problems

2 Likes
12

there’s discussion in this issue: [Stan 2.33] Pathfinder algorithm · Issue #684 · stan-dev/cmdstanpy · GitHub - your feedback welcome!

13

Until the discussion in the issue @mitzimorris linked is finished and someone has time to implement it I think @mike-lawrence’s code above should work and give you usable results in a data frame.

15

I opened a GitHub issue about this. Has there been coordination with the people working on RStan recently (@bgoodri, @andrjohns, @hsbadr) to avoid this? Last I heard a version of RStan with the new array syntax is ready to go, so we should be ok, but I don’t know if there’s a specific plan for submitting it.

16

These removals were originally scheduled for 2.32 but delayed at the request of @andrjohns because RStan 2.26 was imminent (at the time). Since then there hasn’t been much discussion

1 Like
17

People interested in trying Pathfinder through a preliminary version of cmdstanpy can use the branch from this PR:

pip install -e git+https://github.com/stan-dev/cmdstanpy@feature/pathfinder#egg=cmdstanpy

Note: when this is eventually released, it will probably be CmdStanPy 2.0 due to some breaking changes in the interfaces for optimization and ADVI (short version: the stan_variable methods had some inconsistencies which have been ironed out, but required changing some return types/behavior for those classes). So, if you heavily used those methods previously you may need to make changes

3 Likes
18

I can’t really get this to work in Windows due to some library not being statically linked. Will try to find a solution. (I guess WSL work as an alternative)

I get this error when I run the compiled model executable:

The program can’t start because libwinpthread-1.dll is missing. Try reinstalling the program to fix the problem.

which gives this result in R:

> run_pathfinder(examplemodel1_standata, examplemodel1_cmdstan_model) ->
+   pathfinder_test
Running "C:/Users/xxx/AppData/Local/Temp/RtmpCoHwpg/model_a19ad56e9a836376e1ccd44d6a405c6d.exe" pathfinder data \
  "file=C:\Users\xxx\AppData\Local\Temp\RtmpCoHwpg\file7c043b8a7ba8.json" output "file=C:\Users\xxx\AppData\Local\Temp\RtmpCoHwpg\file7c047f441aba.csv"
Opening fd 1
Error:
! ! System command 'model_a19ad56e9a836376e1ccd44d6a405c6d.exe' failed
19

I just tried the above cmdstanr branch that enables access to pathfinder with 2.33 rc and it seemed to work. Also, unless I missed something, it looked like the GitHub commits on that branch were mergeable with the latest cmdstanr. Though I think there’s some question as to what to call some of the interface options.

1 Like
20

Lots of cool stuff, looking forward to using it. I assume this discussion happened somewhere, so please point me there if relevant, but with all the backwards-incompatible changes, shouldn’t we be bumping the major verison? Could avoid some of the confusion…

21

Yes, two years ago it began in this thread before moving to a design document. The summary of the policy is these changes are allowed because a user can automatically update their model

2 Likes