Problems when running pystan on windows 10

Interfaces PyStan
, ,
1

I been trying to get pystan working for quite a while on windows 10.
An package called orbit relies heavily on it and i am trying to use it without luck where the errors always appears upstream in stan.

Operating system: windows 10
Python version: 3.7.12
Pystan version: 2.19.1.1
compiler: mingw32

Code to reproduce(notice the environment variable INCLUDE, this was necessary to try to hack through the io.h file not found bug and basestd.h not found and vcruntime.h not found etc):

import pandas as pd
  import numpy as np

  pd.set_option('display.float_format', lambda x: '%.5f' % x)
  import matplotlib
  import matplotlib.pyplot as plt

  import orbit
  from orbit.models import KTRLite, KTR

  from orbit.utils.features import make_fourier_series_df, make_fourier_series
  from orbit.diagnostics.plot import plot_predicted_data, plot_predicted_components
  from orbit.diagnostics.metrics import smape
  from orbit.utils.dataset import load_iclaims, load_electricity_demand

  df = load_iclaims()

  DATE_COL = 'week'
  RESPONSE_COL = 'claims'

  print(df.shape)

  df.head()

  print(f'starts with {df[DATE_COL].min()}\nends with {df[DATE_COL].max()}\nshape: {df.shape}')

  test_size = 52

  train_df = df[:-test_size]
  test_df = df[-test_size:]

  ktr = KTR(
      date_col=DATE_COL,
      response_col=RESPONSE_COL,
      #     regressor_col=['trend.unemploy', 'trend.filling', 'trend.job'],
      regressor_col=['trend.unemploy'],
      seasonality=[52],
      seasonality_fs_order=[3],
      level_knot_scale=.1,
      level_segments=10,
      regression_segments=0,
      regression_rho=0.15,
      # pyro optimization parameters
      seed=8888,
      num_steps=1000,
      num_sample=1000,
      learning_rate=0.1,
      estimator='pyro-svi',
      n_bootstrap_draws=-1,
      ktrlite_optim_args=dict()
  )

  import os

  os.environ['CC'] = 'gcc-10'
  os.environ['CXX'] = 'g++-10'
  os.environ['INCLUDE'] = 'C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.22621.0\\ucrt;' \
                          'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\MSVC\\14.33.31629\\include;' \
                          'C:\\Program Files (x86)\\Windows Kits\\10\\Include\\10.0.22621.0\\shared'

  ktr.fit(train_df)

trace back(i cant post hte whole one due to charlimit):

WARNING:pystan:MSVC compiler is not supported
stanfit4anon_model_ce583eb84b35032d795f7056cd3c761e_9175705935281291528.cpp
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(14) : Warning Msg: Using deprecated NumPy API, disable it with #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/prim/mat/meta/seq_view.hpp(145): warning C4267: ‘return’: conversion from ‘size_t’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/prim/mat/fun/multiply_lower_tri_self_transpose.hpp(19): warning C4244: ‘initializing’: conversion from ‘Eigen::EigenBase::Index’ to ‘int’, possible loss of data
with
[
Derived=Eigen::Matrix<double,-1,-1,0,-1,-1>
]
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/prim/mat/fun/multiply_lower_tri_self_transpose.hpp(27): warning C4244: ‘initializing’: conversion from ‘Eigen::EigenBase::Index’ to ‘int’, possible loss of data
with
[
Derived=Eigen::Matrix<double,-1,-1,0,-1,-1>
]
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/prim/mat/fun/csr_u_to_z.hpp(28): warning C4267: ‘argument’: conversion from ‘size_t’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/prim/mat/fun/matrix_exp_action_handler.hpp(118): warning C4244: ‘initializing’: conversion from ‘double’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/prim/mat/fun/to_matrix.hpp(136): warning C4267: ‘initializing’: conversion from ‘size_t’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/prim/mat/fun/to_row_vector.hpp(31): warning C4267: ‘initializing’: conversion from ‘size_t’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/prim/mat/fun/to_vector.hpp(31): warning C4267: ‘initializing’: conversion from ‘size_t’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/prim/mat/fun/welford_covar_estimator.hpp(31): warning C4244: ‘return’: conversion from ‘double’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/prim/mat/fun/welford_var_estimator.hpp(31): warning C4244: ‘return’: conversion from ‘double’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\src\stan/io/dump.hpp(243): warning C4101: ‘exc’: unreferenced local variable
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\src\stan/io/dump.hpp(270): warning C4101: ‘exc’: unreferenced local variable
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\src\stan/io/dump.hpp(284): warning C4101: ‘exc’: unreferenced local variable
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/rev/mat/meta/operands_and_partials.hpp(42): warning C4267: ‘return’: conversion from ‘size_t’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/rev/mat/meta/operands_and_partials.hpp(155): warning C4267: ‘return’: conversion from ‘size_t’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/rev/scal/fun/bessel_first_kind.hpp(19): warning C4244: ‘argument’: conversion from ‘double’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/rev/scal/fun/bessel_first_kind.hpp(20): warning C4244: ‘argument’: conversion from ‘double’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/rev/scal/fun/bessel_second_kind.hpp(19): warning C4244: ‘argument’: conversion from ‘double’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/rev/scal/fun/bessel_second_kind.hpp(20): warning C4244: ‘argument’: conversion from ‘double’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/rev/scal/fun/modified_bessel_first_kind.hpp(19): warning C4244: ‘argument’: conversion from ‘double’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/rev/scal/fun/modified_bessel_first_kind.hpp(20): warning C4244: ‘argument’: conversion from ‘double’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/rev/scal/fun/modified_bessel_second_kind.hpp(19): warning C4244: ‘argument’: conversion from ‘double’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/rev/scal/fun/modified_bessel_second_kind.hpp(20): warning C4244: ‘argument’: conversion from ‘double’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/rev/scal/fun/rising_factorial.hpp(18): warning C4244: ‘argument’: conversion from ‘double’ to ‘int’, possible loss of data
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/rev/mat/fun/cholesky_decompose.hpp(82): warning C4244: ‘initializing’: conversion from ‘Eigen::EigenBase::Index’ to ‘int’, possible loss of data
with
[
Derived=Eigen::Matrixstan::math::var,-1,-1,0,-1,-1
]

C:\Users\User1\AppData\Local\Temp\pystan_xzmnrs8w\anon_model_ce583eb84b35032d795f7056cd3c761e.hpp(280): note: ‘void stan::math::assign(T_lhs &,const T_rhs &)’: expects 2 arguments - 1 provided
C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\stan\lib\stan_math\stan/math/prim/mat/fun/assign.hpp(47): note: see declaration of ‘stan::math::assign’
C:\Users\MAXIMI~1\AppData\Local\Temp\pystan_xzmnrs8w\stanfit4anon_model_ce583eb84b35032d795f7056cd3c761e_9175705935281291528.cpp(5224): warning C4267: ‘=’: conversion from ‘size_t’ to ‘int’, possible loss of data
Traceback (most recent call last):
File “C:\Users\User1\anaconda3\envs\stan_env2\lib\distutils_msvccompiler.py”, line 396, in compile
self.spawn(args)
File “C:\Users\User1\anaconda3\envs\stan_env2\lib\distutils_msvccompiler.py”, line 502, in spawn
return super().spawn(cmd)
File “C:\Users\User1\anaconda3\envs\stan_env2\lib\distutils\ccompiler.py”, line 910, in spawn
spawn(cmd, dry_run=self.dry_run)
File “C:\Users\User1\anaconda3\envs\stan_env2\lib\distutils\spawn.py”, line 38, in spawn
_spawn_nt(cmd, search_path, dry_run=dry_run)
File “C:\Users\User1\anaconda3\envs\stan_env2\lib\distutils\spawn.py”, line 81, in _spawn_nt
“command %r failed with exit status %d” % (cmd, rc))
distutils.errors.DistutilsExecError: command ‘C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\bin\HostX86\x64\cl.exe’ failed with exit status 2
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “C:\Users\User1\anaconda3\envs\stan_env2\lib\site-packages\pystan\model.py”, line 378, in init
build_extension.run()
File “C:\Users\User1\anaconda3\envs\stan_env2\lib\distutils\command\build_ext.py”, line 340, in run
self.build_extensions()
File “C:\Users\User1\anaconda3\envs\stan_env2\lib\distutils\command\build_ext.py”, line 449, in build_extensions
self._build_extensions_serial()
File “C:\Users\User1\anaconda3\envs\stan_env2\lib\distutils\command\build_ext.py”, line 474, in _build_extensions_serial
self.build_extension(ext)
File “C:\Users\User1\anaconda3\envs\stan_env2\lib\distutils\command\build_ext.py”, line 534, in build_extension
depends=ext.depends)
File “C:\Users\User1\anaconda3\envs\stan_env2\lib\distutils_msvccompiler.py”, line 398, in compile
raise CompileError(msg)
distutils.errors.CompileError: command ‘C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\bin\HostX86\x64\cl.exe’ failed with exit status 2
python-BaseException

2

It looks like you’re using the MSVC compiler, which isn’t supported:

WARNING:pystan:MSVC compiler is not supported

@ariddell it looks like the Pystan2 on Windows doc link is broken: https://pystan2.readthedocs.io/en/latest/_windows

What’s the recommended setup for Pystan2 on windows these days?

3

forgot to mention that i installed the c+±compiler following PyStan on Windows — PyStan 2.19.1.1 documentation
and checked that the distutils.cfg file declares compiler=mingw32
but the warning however seems problematic.
note the environment variable ‘C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\include;’
which is needed? since i otherwise run into a cant find vcruntime.h error.

when tracking down on the warning i found that build_extension.compiler is None here and not explicitly msvc:
module model, row 325
if build_extension.compiler in (None, ‘msvc’):
logger.warning(“MSVC compiler is not supported”)

4

Your mingw-w64 compiler is not installed correctly so Python can not find it.

Create a new conda environment and install mingw-w64 + libpython and try again.

PyStan2 is not supported officially but we can always do some debugging here.

Also, don’t try to push msvc files for the process.

1 Like
5

now i see what happened: in my distutils.cfg file i solely had [build_ext] and not [build] … this solved the error.