Different output based on OS when using RScript and external C++ code

Hey all,

So after shifting from PyStan to RStan, @bfinley and I have been working on an RScript and .hpp file that would work for our use case. Unfortunately, we have ran into an issue. When using the RScript along with the .hpp file we created, I receive the following error output on my MacOS (Catalina) system:

> loadedStanModel <- stan_model(model_code = stanModel, allow_undefined = TRUE,
+                               includes = paste0('\n#include "', 
+                                                 file.path(getwd(), 'external_parabola.hpp'), '"\n'))
Error in compileCode(f, code, language = language, verbose = verbose) : 
  Compilation ERROR, function(s)/method(s) not created! In file included from <built-in>:1:
In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/Core:535:
/Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from <built-in>:1:
In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
In file included from /Library/Frameworks/R.framework/Versions/
In addition: Warning message:
In system(cmd, intern = !verbose) :
  running command '/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB file281d95ecc81.cpp 2> file281d95ecc81.cpp.err.txt' had status 1
Error in sink(type = "output") : invalid connection

However, when @bfinley runs the same RScript along with the same .hpp file, RStan produces the expected output:

Here is the RScript code:

setwd("/Users/matt/Desktop/School/Capstone_Project/RStan")

library(limma)
library(rstan)
options(mc.cores = parallel::detectCores())
rstan_options(auto_write = TRUE)

stanModel = 
  "
functions {
  real parabola_function(real A, real B, real C, real X);
#include external_parabola.hpp
}
data {
    int J; 
    int I;
    real X[J,I];
    real Y[J,I];
}
parameters {
    real A;
    real B; 
    real C;
    real<lower=0> sigma;
}
transformed parameters {
    real y_hat = parabola_function(A, B, C, X[J,I]);
}
model {
    Y[J,I] ~ normal(y_hat, sigma);
}
"

x_data = read.columns("parabola_data.csv", c("x"), sep = ",")
y_data = read.columns("parabola_data.csv", c("y"), sep = ",")

parabola_data <- list(J = 40, I = 1, X = x_data, Y = y_data)

# try(readLines(stanc(model_code = stanModel, allow_undefined = TRUE)$cppcode))

loadedStanModel <- stan_model(model_code = stanModel, allow_undefined = TRUE,
                              includes = paste0('\n#include "', 
                                                file.path(getwd(), 'external_parabola.hpp'), '"\n'))

fit <- sampling(loadedStanModel, data = parabola_data, iter=1000, chains=1, 
                sample_file='output.csv', control = list(max_treedepth = 10, adapt_delta = 0.99))

print(fit)

And here is the .hpp code:

real parabola_function(real A, real B, real C, real X)
{
    return A*X*X+B*X+C;
}

Can someone help me diagnose this issue?

Thank you,
Matt

Other people might now, but I think I’ve seen this somewhere else and the conclusion was that this is an RStudio thing? Not sure though…

Can you call stan_model with the verbose = TRUE argument and tell us the line in the output that has error: (with the colon). My guess is that it is the Catalina thing.

Interestingly enough, the error’s are traced back to the the data types of the args used in the function from the .hpp file.

In file included from file281d728030b0.cpp:85:
/Users/matt/Desktop/School/Capstone_Project/RStan/external_parabola.hpp:1:1: error: unknown type name 'real'
real parabola_function(real A, real B, real C, real X)
^
/Users/matt/Desktop/School/Capstone_Project/RStan/external_parabola.hpp:1:24: error: unknown type name 'real'
real parabola_function(real A, real B, real C, real X)
                       ^
/Users/matt/Desktop/School/Capstone_Project/RStan/external_parabola.hpp:1:32: error: unknown type name 'real'
real parabola_function(real A, real B, real C, real X)
                               ^
/Users/matt/Desktop/School/Capstone_Project/RStan/external_parabola.hpp:1:40: error: unknown type name 'real'
real parabola_function(real A, real B, real C, real X)
                                       ^
/Users/matt/Desktop/School/Capstone_Project/RStan/external_parabola.hpp:1:48: error: unknown type name 'real'
real parabola_function(real A, real B, real C, real X)
                                               ^

We originally thought that using reals in this situation would not work. However, since the RScript/.hpp worked on the @bfinley’s Windows machine, we didn’t consider it to be the issue in this case.

The real keyword is a Stan type. In the C++ file, those need to be templated. In the output of stan_model(..., verbose = TRUE), what is the first line that mentions parabola_function and the two line above that?

Is this what you are referring to?

 40: template <typename T0__, typename T1__, typename T2__, typename T3__>
 41: typename boost::math::tools::promote_args<T0__, T1__, T2__, T3__>::type
 42: parabola_function(const T0__& A,
 43:                       const T1__& B,
 44:                       const T2__& C,
 45:                       const T3__& X, std::ostream* pstream__);

Also, I guess I am slightly confused how, if the real keyword needs to be templated, the code was able to run on @bfinley’s machine?

I am completely baffled because real parabola_function(real A, real B, real C, real X) is not even valid C++ syntax. The external_parabola.hpp file should be like

template <typename T0__, typename T1__, typename T2__, typename T3__>
typename boost::math::tools::promote_args<T0__, T1__, T2__, T3__>::type
parabola_function(const T0__& A,
                  const T1__& B,
                  const T2__& C,
                  const T3__& X, std::ostream* pstream__) {

  return A*X*X+B*X+C;
}

Yep, we were equally surprised. When we initially began writing the code for the RScript, we decided to keep the parabola_function within the function{} block of the Stan Model. Within the function{} block was the code found in the external .hpp file in the initial post for this thread. After successfully loading in the Stan model using that code, we humored ourselves and tried it when attempting to implement the external .hpp file. That’s when we came across the situation of being able to run the RScript along with the .hpp file on @bfinley’s Windows machine but not my MacOS machine.


template <typename T0__, typename T1__, typename T2__, typename T3__>
typename boost::math::tools::promote_args<T0__, T1__, T2__, T3__>::type
parabola_function(const T0__& A,
                  const T1__& B,
                  const T2__& C,
                  const T3__& X, std::ostream* pstream__) {

  return A*X*X+B*X+C;
}

We attempted to use this solution prior. This is the error produced:

> loadedStanModel <- stan_model(model_code = stanModel, allow_undefined = TRUE, 
+                               includes = paste0('\n#include "', 
+                                                 file.path(getwd(), 'external_parabola.hpp'), '"\n'))
SYNTAX ERROR, MESSAGE(S) FROM PARSER:
 error in 'external_parabola.hpp' at line 1, column 0
 included from 'model4292ee44fbd_78e0a0b6a5e2ec74ec2de8949de1648e' at line 4
  -------------------------------------------------
     1: template <typename T0__, typename T1__, typename T2__, typename T3__>
       ^
     2: typename boost::math::tools::promote_args<T0__, T1__, T2__, T3__>::type
  -------------------------------------------------

PARSER EXPECTED: "}"
Error in stanc(file = file, model_code = model_code, model_name = model_name,  : 
  failed to parse Stan model '78e0a0b6a5e2ec74ec2de8949de1648e' due to the above error.

Thoughts?

Just an update in case anybody comes across this thread in the future.

Using the recommend external_parabola.hpp code that Ben suggested, along with the RScript code mentioned in the initial post of this thread, I was unable to get the code running. @bfinley and I then tried running the .hpp and RScript code on Ubuntu 18.04 LTS and it worked fine. Unfortunately, when testing this code on my MacOS partition, I forgot to install the boost library. After installing the boost library with the following homebrew command brew install boost, it worked on my MacOS partition.

Strangely enough, the code stopped working on @bfinley’s Ubuntu partition and the boost library needed to be installed (using the above homebrew command) for it to start working again. Moreover, the code stopped working on both MacOS and Ubuntu until we changed the .hpp file to include the boost library and edited the used namespaces for the file.

I will attach the edited versions of the RScript and .hpp files to this post. Unfortunately, the completed versions of parabola.r and external_parabola.hpp are being held in a private repo on Github, so I won’t be able to attach those. However, I’m hoping the edited versions that I’m attaching, along with the information in this thread, should offer additional support to those struggling with calling an external .hpp file from their RScript file when using RStan.

Best,
Matt

test.hpp (434 Bytes) test.R (926 Bytes)