Expose_stan_functions in package

Stephen_Martin · October 22, 2019, 7:53pm

Hi all,

I’m developing a package based on rstan/stan. There are some functions I’d like to pull out of .stan files and use from within the package (though, maybe not exported from the namespace; just internal functions to compute certain quantities from the posterior without using standalone gqs or inflating the model output size).

I cannot figure out how to do this properly for a package. I tried using expose_stan_functions, with cacheDir specified, and dropping the generated .cpp file(s) into src/, and running compileAttributes. However, it throws an error when building the package.

 317 In file included from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/as.h:25:0,                                                                                                                                                                                                                     
  318                  from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/RcppCommon.h:168,                                                                                                                                                                                                                   
  319                  from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/RcppEigenForward.h:26,                                                                                                                                                                                                         
  320                  from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/RcppEigen.h:25,                                                                                                                                                                                                                
  321                  from RcppExports.cpp:4:                                                                                                                                                                                                                                                                              
  322 /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/internal/Exporter.h: In instantiation of ‘Rcpp::traits::Exporter<T>::Exporter(SEXP) [with T = std::basic_ostream<char>*; SEXP = SEXPREC*]’:                                                                                                          
  323 /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/as.h:87:41:   required from ‘T Rcpp::internal::as(SEXP, Rcpp::traits::r_type_generic_tag) [with T = std::basic_ostream<char>*; SEXP = SEXPREC*]’                                                                                                     
  324 /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/as.h:152:31:   required from ‘T Rcpp::as(SEXP) [with T = std::basic_ostream<char>*; SEXP = SEXPREC*]’                                                                                                                                                
  325 /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/InputParameter.h:34:43:   required from ‘Rcpp::InputParameter<T>::operator T() [with T = std::basic_ostream<char>*]’                                                                                                                                 
  326 RcppExports.cpp:18:57:   required from here                                                                                                                                                                                                                                                                           
  327 /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/internal/Exporter.h:31:31: error: cannot convert ‘SEXP {aka SEXPREC*}’ to ‘std::basic_ostream<char>*’ in initialization                                                                                                                              
  328        Exporter( SEXP x ) : t(x){}                                                                                                                                                                                                                                                                                    
  329                                ^                                                                                                                                                                                                                                                                                      
  330 In file included from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/Rcpp/r/headers.h:64:0,                                                                                                                                                                                                              
  331                  from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include/RcppCommon.h:29,                                                                                                                                                                                                                    
  332                  from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/RcppEigenForward.h:26,                                                                                                                                                                                                         
  333                  from /home/srmart/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/RcppEigen.h:25,                                                                                                                                                                                                                
  334                  from RcppExports.cpp:4:                                                                                                                                                                                                                                                                              
  335 ls.h:184:16: note: class type ‘SEXPREC’ is incomplete                                                                                                                                                                                                                                                                 
  336  typedef struct SEXPREC *SEXP;                                                                                                                                                                                                                                                                                        
  337                 ^~~~~~~                                                                                                                                                                                                                                                                                               
  338 /usr/lib/R/etc/Makeconf:176: recipe for target 'RcppExports.o' failed                                                                                                                                                                                                                                                 
  339 make: *** [RcppExports.o] Error 1                                                                                                                                                                                                                                                                                     
  340 ERROR: compilation failed for package ‘omegad’                                                                                                                                                                                                                                                                        
  341 * removing ‘/home/srmart/R/x86_64-pc-linux-gnu-library/3.6/omegad’                                                                                                                                                                                                                                                    
  342 * restoring previous ‘/home/srmart/R/x86_64-pc-linux-gnu-library/3.6/omegad’                                                                                                                                                                                                                                          
  343 Error in (function (command = NULL, args = character(), error_on_status = TRUE,  :                                                                                                                                                                                                                                    
  344   System command error

Any ideas? Or is there an updated step-by-step for how to use defined stan functions in a package?

Stephen_Martin · October 23, 2019, 2:01am

Update: This is seriously puzzling.

Running expose_stan_functions in a session works just fine. No errors. Everything works as expected. I even stepped through the function with debug(), and copied the exact cpp code used to src/. Using the exact cpp code used by expose_stan_functions (and ultimately, sourceCpp), compiling the code will fail when building the package.
@bgoodri If you have any ideas, please let me know. This is driving me bonkers.

bgoodri · October 23, 2019, 2:35am

This is telling you that it can’t deal with the stream argument that gets implicitly added to all user-defined Stan functions so that the print and reject functions work. You may be able to eliminate that argument if you are not using print or reject or you can set it to 0 for a null pointer or perhaps &Rcpp::Rcout.

Stephen_Martin · October 23, 2019, 2:53am

Thanks @bgoodri
Just as you sent it, I was trying the removal method. That seems to work.

It’s a bit clunky though; I suppose I can just write a quick script that s/std::basic_ostream*//g (more or less).
I noticed that the .cpp files have std::ostream* pstream__ = nullptr, and that nullptr is not being parsed correctly. I assume Rcpp is throwing an error because it literally doesn’t know how to translate a pointer to std::ostream to anything R-related, so fixing it to nullptr, 0, or &Rcpp::Rcout may not work?
I’ll give it a shot.

Stephen_Martin · October 23, 2019, 3:16am

OK! Thank you again @bgoodri

For my future self, and to any others.

Use expose_stan_functions(…, cacheDir=“src”) to generate the cpp file.
In src/sourceCpp* directory, copy the file*.cpp to src/.
In the cpp file, there are two options:
a. Remove all std::basic_ostream* pstream__, and all = nullptr
b. In all exported Rcpp functions, remove std::ostream* pstream__ = nullptr from the function arguments. In the return statement, replace pstream__ with &Rcpp::Rcout

It should now compile. The basic problem is that Rcpp tries translating an std::ostream* (pointer to an output stream) argument type into an S/R equivalent, and there is not one. This fix just removes the argument from the exported function, and (if choosing b. option) sets the output stream to the one provided by Rcpp.

sakrejda · October 23, 2019, 9:12pm

An option I prefer is to add some bits that just let Rcpp/R deal with these arguments by default. The curent version of rstan looks like it still has most of the C++ code that I used to set this up but this version was complete:

github.com

stan-dev/rstan/blob/b5155ba88d0425791e2d24d8b5261c6b4f2c44b7/rstan/rstan/inst/include/exporter.h

#include <RcppCommon.h>
#include <boost/random/additive_combine.hpp>
#include <stan/math/prim/mat/fun/accumulator.hpp>
#include <iostream>

namespace Rcpp {
  SEXP wrap(boost::ecuyer1988 RNG);
  SEXP wrap(boost::ecuyer1988& RNG);
  SEXP wrap(stan::math::accumulator<double>& RNG);
  SEXP wrap(std::ostream stream);
  template <> boost::ecuyer1988 as(SEXP ptr_RNG);
  template <> boost::ecuyer1988& as(SEXP ptr_RNG);
  template <> stan::math::accumulator<double>& as(SEXP ptr_acc);
  template <> std::ostream* as(SEXP ptr_stream);
  namespace traits {
    template <> class Exporter<boost::ecuyer1988&>;
    template <> struct input_parameter<boost::ecuyer1988&>;
    template <> class Exporter<stan::math::accumulator<double>&>;
    template <> struct input_parameter<stan::math::accumulator<double>&>;
  }

This file has been truncated. show original

The current version still handles the ostream/rng arguments and the linked version also handles the accumulator functions.

Topic		Replies	Views
Rstan::expose_stan_functions inside a package? General	14	1212	January 24, 2019
Using expose_stan_functions in a package RStan	11	2227	April 9, 2024
Exposed Stan functions for generated quantities in an R package RStan	7	1221	March 18, 2018
Expose UDF from external file RStan	5	372	February 8, 2021
Building a Rstan package with custom c++ code RStan	4	455	September 26, 2019

Expose_stan_functions in package

Related topics