Expose_stan_functions in package

Hi all,

I’m developing a package based on rstan/stan. There are some functions I’d like to pull out of .stan files and use from within the package (though, maybe not exported from the namespace; just internal functions to compute certain quantities from the posterior without using standalone gqs or inflating the model output size).

I cannot figure out how to do this properly for a package. I tried using expose_stan_functions, with cacheDir specified, and dropping the generated .cpp file(s) into src/, and running compileAttributes. However, it throws an error when building the package.

 317 In file included from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/as.h:25:0,                                                                                                                                                                                                                     
  318                  from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/RcppCommon.h:168,                                                                                                                                                                                                                   
  319                  from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/RcppEigenForward.h:26,                                                                                                                                                                                                         
  320                  from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/RcppEigen.h:25,                                                                                                                                                                                                                
  321                  from RcppExports.cpp:4:                                                                                                                                                                                                                                                                              
  322 /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/internal/Exporter.h: In instantiation of ‘Rcpp::traits::Exporter<T>::Exporter(SEXP) [with T = std::basic_ostream<char>*; SEXP = SEXPREC*]’:                                                                                                          
  323 /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/as.h:87:41:   required from ‘T Rcpp::internal::as(SEXP, Rcpp::traits::r_type_generic_tag) [with T = std::basic_ostream<char>*; SEXP = SEXPREC*]’                                                                                                     
  324 /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/as.h:152:31:   required from ‘T Rcpp::as(SEXP) [with T = std::basic_ostream<char>*; SEXP = SEXPREC*]’                                                                                                                                                
  325 /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/InputParameter.h:34:43:   required from ‘Rcpp::InputParameter<T>::operator T() [with T = std::basic_ostream<char>*]’                                                                                                                                 
  326 RcppExports.cpp:18:57:   required from here                                                                                                                                                                                                                                                                           
  327 /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/internal/Exporter.h:31:31: error: cannot convert ‘SEXP {aka SEXPREC*}’ to ‘std::basic_ostream<char>*’ in initialization                                                                                                                              
  328        Exporter( SEXP x ) : t(x){}                                                                                                                                                                                                                                                                                    
  329                                ^                                                                                                                                                                                                                                                                                      
  330 In file included from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/r/headers.h:64:0,                                                                                                                                                                                                              
  331                  from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/RcppCommon.h:29,                                                                                                                                                                                                                    
  332                  from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/RcppEigenForward.h:26,                                                                                                                                                                                                         
  333                  from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/RcppEigen.h:25,                                                                                                                                                                                                                
  334                  from RcppExports.cpp:4:                                                                                                                                                                                                                                                                              
  335 ls.h:184:16: note: class type ‘SEXPREC’ is incomplete                                                                                                                                                                                                                                                                 
  336  typedef struct SEXPREC *SEXP;                                                                                                                                                                                                                                                                                        
  337                 ^~~~~~~                                                                                                                                                                                                                                                                                               
  338 /usr/lib/R/etc/Makeconf:176: recipe for target 'RcppExports.o' failed                                                                                                                                                                                                                                                 
  339 make: *** [RcppExports.o] Error 1                                                                                                                                                                                                                                                                                     
  340 ERROR: compilation failed for package ‘omegad’                                                                                                                                                                                                                                                                        
  341 * removing ‘/home/srmart/R/x86_64-pc-linux-gnu-library/3.6/omegad’                                                                                                                                                                                                                                                    
  342 * restoring previous ‘/home/srmart/R/x86_64-pc-linux-gnu-library/3.6/omegad’                                                                                                                                                                                                                                          
  343 Error in (function (command = NULL, args = character(), error_on_status = TRUE,  :                                                                                                                                                                                                                                    
  344   System command error                                                                                         

Any ideas? Or is there an updated step-by-step for how to use defined stan functions in a package?

1 Like

Update: This is seriously puzzling.

Running expose_stan_functions in a session works just fine. No errors. Everything works as expected. I even stepped through the function with debug(), and copied the exact cpp code used to src/. Using the exact cpp code used by expose_stan_functions (and ultimately, sourceCpp), compiling the code will fail when building the package.
@bgoodri If you have any ideas, please let me know. This is driving me bonkers.

This is telling you that it can’t deal with the stream argument that gets implicitly added to all user-defined Stan functions so that the print and reject functions work. You may be able to eliminate that argument if you are not using print or reject or you can set it to 0 for a null pointer or perhaps &Rcpp::Rcout.

1 Like

Thanks @bgoodri
Just as you sent it, I was trying the removal method. That seems to work.

It’s a bit clunky though; I suppose I can just write a quick script that s/std::basic_ostream*//g (more or less).
I noticed that the .cpp files have std::ostream* pstream__ = nullptr, and that nullptr is not being parsed correctly. I assume Rcpp is throwing an error because it literally doesn’t know how to translate a pointer to std::ostream to anything R-related, so fixing it to nullptr, 0, or &Rcpp::Rcout may not work?
I’ll give it a shot.

OK! Thank you again @bgoodri

For my future self, and to any others.

  1. Use expose_stan_functions(…, cacheDir=“src”) to generate the cpp file.
  2. In src/sourceCpp* directory, copy the file*.cpp to src/.
  3. In the cpp file, there are two options:
    a. Remove all std::basic_ostream* pstream__, and all = nullptr
    b. In all exported Rcpp functions, remove std::ostream* pstream__ = nullptr from the function arguments. In the return statement, replace pstream__ with &Rcpp::Rcout

It should now compile. The basic problem is that Rcpp tries translating an std::ostream* (pointer to an output stream) argument type into an S/R equivalent, and there is not one. This fix just removes the argument from the exported function, and (if choosing b. option) sets the output stream to the one provided by Rcpp.


An option I prefer is to add some bits that just let Rcpp/R deal with these arguments by default. The curent version of rstan looks like it still has most of the C++ code that I used to set this up but this version was complete:

The current version still handles the ostream/rng arguments and the linked version also handles the accumulator functions.