Knitting problem while using rstan / rethinking package

General
1

Hi all,

Firstly, for clarity. I am not using rstan directly, I am using the rethinking package which uses the function ulam to communicate with stan.

I’m having some recent trouble with R when I try to knit my document. It’s been fine for the past year but its suddenly decided it doesn’t want to work. Knitting works perfectly fine for documents that do not use stan but anything that does just doesn’t work at all. Stan (and ulam) work perfectly fine in regular R and I can fit models without any issue.

When I try and knit the document however, it always fails when it gets to the first stan model. The errors aren’t terribly helpful as it just fills the console with red text (I really do mean lines and lines of it). I’m happy to post some if that is helpful. The first error line is this:

file242868b2901.cpp:6:36: warning: ISO C++11 requires whitespace after the macro name
#define STAN__SERVICES__COMMAND_HPP#include <boost/integer/integer_log2.hpp>

Which is what makes me think the problem is with stan. The later lines mention rtools as well which I can believe as I had a lot of trouble with that recently as well which had stopped models working at all.

So far, I have tried reinstalling Rstan and Rtools (v4.00) which doesn’t seem to have helped, so any advice would be greatly appreciated.

1 Like
2

Do you encounter the issues also when not knitting?

Rtools 4.0 imply you have R 4.0, so this might be relevant

A bunch of people had similar errors, so you might want to try what worked for them:
https://discourse.mc-stan.org/search?q=ISO%20C%2B%2B11%20requires%20whitespace%20after%20the%20macro%20name

If that doesn’t help, feel free to get back here and ask some more!

3

Thanks so much for your reply.

No, I don’t have any trouble running models / code when I’m not trying to knit. Everything is working fine otherwise.

I’ve skimmed through some of the topics in that link and it seems like most people having trouble are those who can’t get stan to work at all, which isn’t the case I am in unfortunately. Stan is working, I just can’t knit a document with it. It’s pretty odd.

Is there any additional information I can give you that might help?

4

The topics might still be relevant as the knitting environment might be different than the R session you normally work with. I would definitely check PATH, CXX14FLAGS and other environment variables from within the knitting environment and see if they differ from what you get in R session. Also check the working directory of the knitting and check if it is the same as when you run from R directly.

Also is the knitting process the same version of R? (could that even happen? I don’t know :-) )

I’ve also had some charset/encoding issues with knitr, so I would check if locale is set the same…

5

That sounds like a reasonable idea. How do I go about checking those things? (sorry, not a super advanced R user).

I also don’t know, I mean one would assume so but who knows!

6

Sys.getenv, version, getwd, sessionInfo can all be called and their values displayed in the knitted document.

7

Okay, so I tried the functions you suggested and I also tried the steps in the link above and that hasn’t stopped the red text.

However, I let it run and despite pages and pages of red text it actually hasn’t stopped it from knitting the document. It takes longer to complete than it used to, but the document will now knit. It is still a little bit concerning with the warnings but maybe it doesn’t matter…

For context, from what I can tell, everything is the same output wise from those functions within a markdown doc and regular R. If you did have any other thoughts I would be nice to stop all the warnings to speed things up, but if not then it does now seem to be working.

1 Like
8

Really not sure. Could you dump as much info about your system and setup, attach the full log of all the errors (you should be able to upload files)? I’ll try to get someone with deeper knowledge of installation issues to look into that.

9

This is a warning that probably concerns the compilation of the model (which runs fine, you said), rather than the knitting. I guess there should be a different error reported further down. Could you see if you spot it?

10

I’m not sure if this helps, but here is the complete red text that I get when I knit the document after it reaches a code chunk containing a stan model. Like I say, none of this actually stops the document knitting if you leave it, but it slows it down by quite a lot.

In terms of my system I’m running R version 4.0.0, R studio version 1.2.5042, Windows 10 (64 bit). AMD Ryzen 1600 CPU, 16gb RAM.

file33502ed76c58.cpp:6:36: warning: ISO C++11 requires whitespace after the macro name
#define STAN__SERVICES__COMMAND_HPP#include <boost/integer/integer_log2.hpp>
^
In file included from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/mpl/aux_/na_assert.hpp:23,
from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/mpl/arg.hpp:25,
from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/mpl/placeholders.hpp:24,
from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/mpl/apply.hpp:24,
from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/mpl/aux_/iter_apply.hpp:17,
from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/mpl/aux_/find_if_pred.hpp:14,
from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/mpl/find_if.hpp:17,
from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/mpl/find.hpp:17,
from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/mpl/aux_/contains_impl.hpp:20,
from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/mpl/contains.hpp:20,
from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/math/policies/policy.hpp:10,
from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/math/special_functions/math_fwd.hpp:29,
from C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/math/special_functions/fpclassify.hpp:19,
from C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/stan_fit.hpp:14,
from C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/rstaninc.hpp:3,
from file33502ed76c58.cpp:7:
C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/mpl/assert.hpp:194:21: warning: unnecessary parentheses in declaration of ‘assert_arg’ [-Wparentheses]
failed ************ (Pred::************
^
C:/Users/ruben/Documents/R/win-library/4.0/BH/include/boost/mpl/assert.hpp:199:21: warning: unnecessary parentheses in declaration of ‘assert_not_arg’ [-Wparentheses]
failed ************ (boost::mpl::not_::************
^
In file included from C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/stan/math/prim/mat/prob/poisson_log_glm_log.hpp:5,
from C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/stan/math/prim/mat.hpp:336,
from C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/stan/math/rev/mat.hpp:12,
from C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/src/stan/model/log_prob_grad.hpp:4,
from C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/src/stan/model/test_gradients.hpp:7,
from C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/src/stan/services/diagnose/diagnose.hpp:10,
from C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/stan_fit.hpp:35,
from C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/rstaninc.hpp:3,
from file33502ed76c58.cpp:7:
C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/stan/math/prim/mat/prob/poisson_log_glm_lpmf.hpp: In function ‘typename stan::return_type<T_x, T_alpha, T_beta>::type stan::math::poisson_log_glm_lpmf(const T_y&, const T_x&, const T_alpha&, const T_beta&)’:
C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/stan/math/prim/mat/prob/poisson_log_glm_lpmf.hpp:64:59: warning: typedef ‘T_alpha_val’ locally defined but not used [-Wunused-local-typedefs]
typename partials_return_type<T_alpha>::type>::type T_alpha_val;
^~~~~~~~~~~
file33502ed76c58.cpp: In member function ‘void model33502566b67_48fb61b1d0aa544784d44a0562893fc7_namespace::model33502566b67_48fb61b1d0aa544784d44a0562893fc7::transform_inits(const stan::io::var_context&, std::vector&, std::vector&, std::ostream*) const’:
file33502ed76c58.cpp:134:24: warning: typedef ‘local_scalar_t__’ locally defined but not used [-Wunused-local-typedefs]
typedef double local_scalar_t__;
^~~~~~~~~~~~~~~~
In file included from C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/rstaninc.hpp:3,
from file33502ed76c58.cpp:7:
C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/stan_fit.hpp: In instantiation of ‘SEXPREC* rstan::stan_fit<Model, RNG_t>::standalone_gqs(SEXP, SEXP) [with Model = model33502566b67_48fb61b1d0aa544784d44a0562893fc7_namespace::model33502566b67_48fb61b1d0aa544784d44a0562893fc7; RNG_t = boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014, 0, 2147483563>, boost::random::linear_congruential_engine<unsigned int, 40692, 0, 2147483399> >; SEXP = SEXPREC*]’:
file33502ed76c58.cpp:500:170: required from here
C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/stan_fit.hpp:1238:9: warning: variable ‘ret’ set but not used [-Wunused-but-set-variable]
int ret = stan::services::error_codes::CONFIG;
^~~
In file included from C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/stan_fit.hpp:52,
from C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/rstaninc.hpp:3,
from file33502ed76c58.cpp:7:
C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/src/stan/services/sample/standalone_gqs.hpp: In instantiation of ‘int stan::services::standalone_generate(const Model&, const MatrixXd&, unsigned int, stan::callbacks::interrupt&, stan::callbacks::logger&, stan::callbacks::writer&) [with Model = model33502566b67_48fb61b1d0aa544784d44a0562893fc7_namespace::model33502566b67_48fb61b1d0aa544784d44a0562893fc7; Eigen::MatrixXd = Eigen::Matrix<double, -1, -1>]’:
C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/stan_fit.hpp:1239:46: required from ‘SEXPREC* rstan::stan_fit<Model, RNG_t>::standalone_gqs(SEXP, SEXP) [with Model = model33502566b67_48fb61b1d0aa544784d44a0562893fc7_namespace::model33502566b67_48fb61b1d0aa544784d44a0562893fc7; RNG_t = boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014, 0, 2147483563>, boost::random::linear_congruential_engine<unsigned int, 40692, 0, 2147483399> >; SEXP = SEXPREC*]’
file33502ed76c58.cpp:500:170: required from here
C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/src/stan/services/sample/standalone_gqs.hpp:92:22: warning: comparison of integer expressions of different signedness: ‘std::vector<std::__cxx11::basic_string >::size_type’ {aka ‘long long unsigned int’} and ‘Eigen::Index’ {aka ‘long long int’} [-Wsign-compare]
if (p_names.size() != draws.cols()) {
~^
C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/src/stan/services/sample/standalone_gqs.hpp:110:24: warning: comparison of integer expressions of different signedness: ‘size_t’ {aka ‘long long unsigned int’} and ‘Eigen::Index’ {aka ‘long long int’} [-Wsign-compare]
for (size_t i = 0; i < draws.rows(); ++i) {
^~~~~~~~~~~~
C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/src/stan/services/sample/standalone_gqs.hpp: In instantiation of ‘void stan::services::get_model_parameters(const Model&, std::vector<std::__cxx11::basic_string >&, std::vector<std::vector >&) [with Model = model33502566b67_48fb61b1d0aa544784d44a0562893fc7_namespace::model33502566b67_48fb61b1d0aa544784d44a0562893fc7]’:
C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/src/stan/services/sample/standalone_gqs.hpp:106:23: required from ‘int stan::services::standalone_generate(const Model&, const MatrixXd&, unsigned int, stan::callbacks::interrupt&, stan::callbacks::logger&, stan::callbacks::writer&) [with Model = model33502566b67_48fb61b1d0aa544784d44a0562893fc7_namespace::model33502566b67_48fb61b1d0aa544784d44a0562893fc7; Eigen::MatrixXd = Eigen::Matrix<double, -1, -1>]’
C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/stan_fit.hpp:1239:46: required from ‘SEXPREC* rstan::stan_fit<Model, RNG_t>::standalone_gqs(SEXP, SEXP) [with Model = model33502566b67_48fb61b1d0aa544784d44a0562893fc7_namespace::model33502566b67_48fb61b1d0aa544784d44a0562893fc7; RNG_t = boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014, 0, 2147483563>, boost::random::linear_congruential_engine<unsigned int, 40692, 0, 2147483399> >; SEXP = SEXPREC*]’
file33502ed76c58.cpp:500:170: required from here
C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/src/stan/services/sample/standalone_gqs.hpp:41:23: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector::size_type’ {aka ‘long long unsigned int’} [-Wsign-compare]
for (int j = 0; j < dimss[i].size(); ++j)
^~~~~~~~~~~~~~~
In file included from C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/stan/math/rev/core.hpp:45,
from C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/stan/math/rev/mat.hpp:4,
from C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/src/stan/model/log_prob_grad.hpp:4,
from C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/src/stan/model/test_gradients.hpp:7,
from C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/src/stan/services/diagnose/diagnose.hpp:10,
from C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/stan_fit.hpp:35,
from C:/Users/ruben/Documents/R/win-library/4.0/rstan/include/rstan/rstaninc.hpp:3,
from file33502ed76c58.cpp:7:
C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/stan/math/rev/core/set_zero_all_adjoints.hpp: At global scope:
C:/Users/ruben/Documents/R/win-library/4.0/StanHeaders/include/stan/math/rev/core/set_zero_all_adjoints.hpp:14:13: warning: ‘void stan::math::set_zero_all_adjoints()’ defined but not used [-Wunused-function]
static void set_zero_all_adjoints() {

1 Like
11

Let me understand: the document actually knits, but it’s slow around that block? If that’s the case then you should think of compiling and running the Stan code before knitting the whole document, and inside that block only run whatever code you need to display the results.

The alternative worth trying is to add a cache=TRUE to that knitr block, so that you will pay the price of running the model only once, so that the block will not be re-run unless you change it. I’m not sure how well caching works for Stan models, so it may not work as expected.

12

Yes, sorry to be absolutely clear. I had initially posted thinking that the knitting process didn’t work at all however after I left the knitter just running it actually did work in the end, despite all the red text. Stan models run in regular R scripts work perfectly fine and finish as quick as they ever did. Though I did also have issues with R tools after I updated to R 4.0.0 but I was able to solve them. Whenever I use knitting and it reaches a stan block, it produces a console full of red text, from what I can tell, its the same warnings every single time.

This is a new thing (in the last week) as I have been using knitr with documents containing stan code for the last few months and had no issues. Obviously markdown docs do tend to run a little slower than R scripts generally, but my main concern really is that documents that used to take ~ 5 minutes to knit can now take up to 20 because it gets slowed down so much by all the warnings. So it would be great to be able to figure out what is now causing it. Or of course if it actually does stop it running for whatever reason somewhere down the line.

1 Like
13

Not sure if you’re using rstudio, but I suspect some of the rstudio panes are causing compile slowdowns – when I build a package (with all the compile output thus going to the build pane) it takes much longer than when I compile the same models, for instance.

1 Like
14

Apparently rstudio folk have been aware of this for a long time

15

@Ruben222 I’m facing the same problem when knitting documents with {rethinking} models. Output is full or warnings/errors but code works fine. Have you found the solution?

16

Hmm, no I never did identify the solution or the problem.

Since posting this I had to replace my hard drive and reinstall everything and I don’t think the issue persisted after that. Though, I haven’t knitted any documents for quite a while.

I would also be interested to hear if you do figure out a solution or the cause.

17

I found out that it helps if you restart R and knit the document right after.

1 Like