Preview of next version of PyStan

Developers
#1

There’s a PyStan 3 alpha available. It’s not ready for general use. It works well enough for me that I’m going to start using it for my day-to-day work. You can install the alpha release on most systems with python3 -m pip -pre install pystan. (No Windows support yet.)

I’d appreciate some help with the following items:

  • Try out eight schools and confirm things work on your machine.
  • Try out the new API and propose changes. The “spec” for the new API (which covers RStan and PyStan) is on the stan-dev wiki.

What’s new in PyStan 3

PyStan 3 is a rewrite. There’s a new user-facing API. Models and fits are automatically cached. Python 3.6 and higher is supported. The code has been simplified, making contributing more inviting.

User-facing changes

  • New interface.
  • Automatic caching of Stan models and fits.
  • Uses the same argument names as CmdStan (e.g., num_samples, num_warmup).
  • -DSTAN_THREADS by default.
  • Python 3.6+.

Developer-facing changes

  • Easier to maintain. 60% fewer lines of code.
  • Split into “front-end” and “back-end” packages, pystan and httpstan.
  • pystan is pure-Python, enabling faster development. (Running tests is ~80% faster.)
  • httpstan calls Stan C++ services functions and returns raw output.
  • ISC license.
7 Likes
#2

Would it be possible to call it “pystan3” so it can coexist with the current pystan?

#3

The plan is that PyStan 3.x will replace PyStan 2. PyStan 2.x will no
longer be maintained after PyStan 3 is stable.

If you have a project that needs a specific version of PyStan you can
always install a specific version using pip.

#4

I really meant to ask whether it could have a different name only during the current alpha/beta period when we may want to experiment with the new version but still use the old one for production code. Of course it’s possible to just set up virtualenv but (to me) separate names seems more straightforward.

#5

Cool! And thanks for releasing a beta—we haven’t done enough of that in the past for big changes.

I see two alternatives on naming:

  • change name of the package to PyHttpStan or something so there’s no conflict

  • leave name the same and change version number—that has to be PyStan 3 to follow semantic versioning if it doesn’t have the same interface as the current PyStan

Option 2 might get confusing as PyStan 3 will come out ahead of Stan 3 (when we remove deprecared language features and make other backward compatibility breaking changes).

#6

You need to be careful with turning on STAN_THREADS by default:

  • The RTools 4.9.3 compiler on Windows will compile Stan models, but the resulting model will just crash
  • You get ~20% performance penalty on average

(but we hopefully get all those points out of the way soon if we can settle on all details as it may have side-effects)

#7

Thanks! We’re monitoring a Windows mingw crash issue here: https://github.com/stan-dev/httpstan/pull/142 . Perhaps it’s the same thing. We do have things working with Windows clang, I believe.

We are seeing the performance hit. I hope you’ll manage to fix it. :) I’ve been following the relevant PRs/issues.

#8

Hi, is the experimental version of PyStan 3 safe to use on Linux?

When is the expected release date of the stable version?
Is it October 18th, 2019 (inferred from Tech leads Roadmap meeting update draft)?

#9

I would say that there will be a stable beta release by the end of August. It will only support one mode of sampling (the recommended NUTS one). No support for VI or MAP in the beta release.

1 Like
#10

Hi,

A few installation-related queries and issues (I’m on macOS Catalina):

  • I just tried pip install --pre pystan and it seemed to succeed, but pip created only the pystan-3.0.0.0a10.dist-info directory but not pystan itself!

  • Are there any hints or ideas for the coexistence of pystan 2 and pystan 3? I would prefer not to use virtual environments…

My idea (from above and also mentioned on GitHub) is to (temporarily) rename the package as (e.g.) pytstan3. This would enable users to have both installed at the same time without having to (e.g.) spawn separate virtual environments.

This is unrelated to version numbering: it is akin to the existence of python3 at the same time as python2==python for exactly this kind of purpose.

#11

Did import stan work?

#12

Ah, sorry if I wasn’t clear. No, it gives “module not found”.

1 Like
#13

I think that may be a broken release. I’m going to delete it.

Beta 1 is almost ready. There are a couple of pull requests which are
waiting to be merged.

#14

Thanks for the upcoming beta!

Any comments regarding the (re-)naming idea?

#15

I’m not sure renaming is a good idea right now.

If having a distinct package name is important to you, you could create a fork of pystan with a new name. It’s all open source and permissively licensed.

#16

Yep, understood that actual development is more important…

#17

Any update on this?
I’ve just installed the beta version (pystan 3.0.0b4) on Linux using pip install --pre pystan and I have the same “module not found” issue.

UPDATE
import stan instead of pystan… my bad.

#18