To generate usable all-doubles versions of functions from the Stan program we would ideally generate a separate file. That means in stan/generator.hpp we add a function called generate_function_cpp that’s like generate_cpp except that it only calls:
generate_version_comment(out);
generate_includes(out); /// is including math.hpp enough? that should be simpler.
generate_start_namespace(function_namespace, out);
generate_usings(out);
generate_typedefs(out);
generate_globals(out);
generate_functions(prog.function_decl_defs_, out);
generate_end_namespace(out);
To generate functions that could be used with Rcpp’s sourceCpp function they need to be in the global namespace so the above parts could remain as they currently are. We would need a function at the end to instantiate all the function templates in a pre-specified namespace from within (generate_template_instantiations or similar).
I think this means adding two more flags to the top level function for calling the code generator. The first flag would control whether the Stan program is called to generate a program for the whole model or just for the functions. The second flag would control what namespace the functions end up in (sourceCpp needs the global namespace but that’s generally discouraged). If the code is generated in and built into an R package I don’t think it would need to live in the global namespace either.
I can sort out the details here and get it to run/test but I want to get input on whether this is in line with your preferences (especially Bob and anybody else who deals with the code generator).
Great to see some work on this as expose_stan_functions in R is so super important to me by now and it did break in the past due to its dark background…
So what is quite annoying in R is that I cannot save the compiled code from those exposed stan functions. Instead I have to recompile the stan models to get the functions whenever I restart R. If this could taken into account when designing this thing, that would be great.
Thanks Sebastian, that’s a good point and I think straightforward to arrange on the R/Rcpp side. Much less of a hurdle than understanding the interplay between the functions in that 5k line generator.hpp file!
The generator.hpp file is a monster but I think I got option 1 figured out. We just need a generate_function_arguments function version that passes “double” for template_type_i and generate_function_body needs to get “double” in scalar_t_name when it gets called from generate_function. I think the rest is straightforward.
It’s not all of Rcpp, that deals with all sorts of namespaces and templates just fine (it is C++ after all) but nobody has bothered to figure out how to map templated types to R types so the automatic parts of Rcpp for interfacing C++ types with R types don’t work with templated types. The namespace issue is not as big and also only an issue for the automated tools (sourceCpp) that let you take a chunk of C++ code and automatically compile it and expose it to R. I think sourceCpp sticks the generated code into a namespace of its own before it compiles it so it wouldn’t take all that much more to make it deal with other namespaces. Take that with a grain of salt, it’s been a long time since I looked at sourceCpp’s internals.
I’m not sure what the rest of Rcpp does, but the relevant question
here is whether there’s a way to use Rcpp to expose to R
a C++ function written with templates and namespaces.
If Rcpp can handle the templates and namespace, then that seems like
the easier route than generating new code in a new file.
[quote]I’m not sure what the rest of Rcpp does, but the relevant question
here is whether there’s a way to use Rcpp to expose to R
a C++ function written with templates and namespaces.
If Rcpp can handle the templates and namespace, then that seems like
the easier route than generating new code in a new file.[/quote]
What the rest of Rcpp does is relevant:
1: Rcpp maps standard types (int, long, double, Eigen::Matrix, etc…) to R types. We have things like Eigen::Matrix that Rcpp does not map for us because they are not a type.
2: Rcpp also has some functionality to take a function written in C++ and expose it to R so that at the R prompt you can call your C++ function as if it were an R function. That means Rcpp figures out what the argument types should be in R, how they should be converted to the correct C++ types, what the C++ return type is, how it should be converted to an R type, and what sort of extern-marked function it needs to create as a wrapper to your C++ function.
You are conflating these two things and we need them both for calling Stan functions from R. If we do a little code generation to generate all-doubles functions or instantiate the templates with doubles to get a real type then we are making #1 possible. If you make #1 possible you get #2 for free. If we don’t do this then every time somebody wants to test a Stan function in R they have to either write boilerplate c++ (where we waste time or loose potential users) or do some hack like expose_stan_functions. The lack of namespace support is a side issue with Rcpp that could be fixed if somebody had the time to do it.
Thanks for the clarification. It explains why the answer to
my question is “no” :-(
I can see why this restriction is in place—Rcpp needs to know
the types to instantiate at compile time if it doesn’t inline
them on the fly at the point the R types are known.
I take it RcppEigen does the mapping of Eigen types.
Thanks for the clarification. It explains why the answer to
my question is “no” :-(
I can see why this restriction is in place—Rcpp needs to know
the types to instantiate at compile time if it doesn’t inline
them on the fly at the point the R types are known.
I take it RcppEigen does the mapping of Eigen types.
Bob, can you take a look at the branch for this? There are three simple tests for the regular function types (not _rng etc… yet) that compile and pass it’s at:
The diff against ‘develop’ is easy to read. Assuming this use-case works out I just want to know if the way I went about modifying the generator and compile functions is objectionable.
I haven’t actually plumbed things all the way through to R. It’s a little complicated because Rcpp requires tags in the comments to export functions and I didn’t want to plumb them into the code generator since they are R-specific.
I’m going to see how to make this work with an R package and see how it compares to doing it by hand.
Also, is the pstream function argument still used? I have some vague memory of that getting deprecated. If errors/messages come back up through exceptions that would be easier to deal with.
No, the pstreams aren’t deprecated. It’s where the output
of print() statements in user-defined functions go. So you’d
be wanting to pass in whatever the cout is for the platform,
say if you were wrapping an R function. Or if you want to suppress
the prints, you can just pass in a dummy stringstream writer.