PyStan 3 Beta 2 and Proposed PyStan 2 Deprecation Plans

Developers
1

PyStan 3 Beta 2 is here. It works for many people. It uses Stan 2.24 and has useful features PyStan 2 lacks.

A release candidate for PyStan 3 will wait until RStan 3 is ready since harmonizing the interfaces’ user-facing APIs is a high-priority goal.

PyStan 2 works fine but is stuck on 2.19. PyStan 2 is no longer actively maintained.

This leaves us in an uncomfortable situation. PyStan 2 is not maintained but there is no official, non-beta-version replacement for PyStan 2. And there will be no official replacement until RStan 3 is ready. This post describes one proposed way of dealing with this situation.

People who can move to PyStan 3 Beta 2 should start doing so. For those using the default sampler, PyStan 3 Beta 2 works fine. If you can draw samples using code written for PyStan 2, you can very likely draw samples using PyStan 3 Beta 2.

People who can start using PyStan 3 Beta 2 right now:

  • macOS and Linux users who have x86_64 hardware and who are drawing samples from models using the default sampler.

As for people who can’t move to or who depend on features missing from PyStan 3 Beta 2, there are some alternatives available.

People who cannot use PyStan 3 Beta 2 right now:

  • Users who use variational inference, maximization algorithms (e.g., LBFGS), or samplers other than the default sampler.
  • Users stuck on Python 2.
  • Windows users who cannot use WSL2 to emulate Linux.

For these users it makes sense to switch over to a different Stan interface (e.g., CmdStanPy, CmdStan, RStan) or use a different NUTS implementation (e.g., PyMC3). PyStan 3’s set of supported platforms is unlikely to change before 2021. People using Stan for VI or posterior maximization can use packages such as jax and pytorch. jax and pytorch support a broader range of optimization algorithms. They also have multinational corporations supporting their development.

So there’s a path available for all Python users who want to use newer versions of the Stan C++ library. No Python user is left entirely without options.

With this in mind, We (@ariddell and @ahartikainen ) would like to propose doing the following over the coming months:

  1. Announce PyStan 3 Beta 2 in a variety of places in order to attract more testers.
  2. Indicate in a variety of places that PyStan 2 is no longer being developed and that folks should start exploring their options.
2 Likes
2

does this work on Windows?

3

Depends on the definition of Windows

4

I know @ahartikainen uses Windows and has managed to get things working using WSL2. I’m not sure how easy or difficult it was to get things running.

5

WSL2 is basically same as setting up Ubuntu (I use Ubuntu 18). To make my workflow similar as in Windows, I use conda (miniconda) to keep my (conda) environments clean.

Hardest part was setting up jupyter lab with dynamical IP.

There are couple of ways depending how many ip your machine has.

One ip:

jupyter lab --ip $(hostname -I)

Multiple ip (take last):

jupyter lab --ip $(echo $(hostname -I) | rev | cut -d " " -f1 | rev)

I’m sure someone could make that last item even better.

Also, there is always CmdStanPy.

2 Likes
6

Quick update.

  • PyStan 3 Beta 3 has been released.
  • Availability of PyStan 3 announced/hinted at on the PyStan 2 documentation.
7

PyStan 3 Beta 5 is very close to being feature complete. All major bugs have been resolved on Linux and macOS.

If RStan 3 is likely more than 18 months away, it seems like it would be a good idea to revise the plan about waiting to release PyStan 3 until RStan 3 is ready.

@bgoodri @jonah Any ballpark estimates about when RStan 3 might be ready?

1 Like
8

I agree with you that we may want to reconsider the plan to wait to release PyStan until RStan is ready because I really have no clue what the timetable is for RStan 3 at this point. Because RStan has been so far behind Stan releases and so difficult to install, I’ve made it a priority in the short term to get CmdStanR off the ground, so I’ve been working on that recently instead of RStan 3. I’m not how much more progress Ben has made on RStan 3.

To be totally honest, the more I’ve worked on CmdStanR the less motivated I’ve been to work on RStan because I think CmdStanR is becoming a better interface for regular users. For R package developers RStan is necessary for getting R packages with pre-compiled Stan programs onto CRAN, so RStan is and will continue to be essential to support, but for regular users I don’t think we should be encouraging them to use RStan with an old version of Stan when they can use CmdStanR with the latest version of Stan. The biggest feature RStan has that CmdStanR is missing is exposing Stan functions to R but @rok_cesnovar even has a proof of concept for how we can do that without RStan. So going forward (once we get to CmdStanR v1.0) it seems like the only situation (or one of the only) in which RStan will be preferable is for use inside R packages with pre-compiled models.

That said, if we can figure out how to keep RStan up to date with Stan releases and make it much easier to install than it currently is then I’m totally back on the RStan bandwagon and I think RStan 3 should be the main interface we encourage people to use in R when it’s ready. But if it lags behind by many releases and continues to be hard to install then I’m much less excited about it given the success of CmdStanR.

Anyway, that’s a summary of my current thinking, but I might be overlooking something important.

2 Likes
9

Thanks for these notes. Sorry to hear about the difficulties.

Perhaps releasing PyStan 3 in January would be a good idea then. With PyStan 2 unmaintained, I feel like we can’t leave things as they are.

1 Like
10

Yeah that makes sense to me. Anyone else have any thoughts on this?