# Cmdstanr & sourceCpp / cppfunction for exposing functions

**URL:** <https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632>\
**Category:** Interfaces\
**Tags:** cmdstanr\
**Created:** [December 4, 2020, 1:59pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632 "2020-12-04T13:59:55Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [December 4, 2020, 1:59pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/1 "2020-12-04T13:59:55Z")

</div>

Hi!

I was wondering if there has been some activity to get expose\_functions working with cmdstanr? A possible way to get this going (I think) could be via the `sourceCpp` or `cppFunction` functions from Rcpp. Has this been considered yet? Is it an option to do that (given that Rcpp has a GPL license)?

I am going to try to get some custom C++ Stan-math based function working from R. Looking at the documentation it looks feasible, but I haven’t yet done anything other than looking at the doc.

Or did someone already work this out?

Sebastian

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [December 4, 2020, 2:27pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/2 "2020-12-04T14:27:05Z")

</div>

This was my most recent attempt: [https://gist.github.com/rok-cesnovar/c91d631b66c1124ae7f221155f87516f](https://gist.github.com/rok-cesnovar/c91d631b66c1124ae7f221155f87516f)

It worked fine for me on Linux, I am almost sure it doesn’t work on Windows (yet).

It requires Cmdstan 2.25 or rather stanc3 for 2.25 (that one supports standalone functions).

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [December 4, 2020, 2:37pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/3 "2020-12-04T14:37:35Z")

</div>

It should also restore flags it sets to the previous state. The intention was just to show it can be done so there might be more user-facing holes.

I also took a bunch of tricks from Rstan so that probably makes this code GPL licensed (we possibly could get the authors of those lines to agree to give that code under a different license, but I am a newbie when it comes to licensing).

The dependency on Rcpp on its own should not make it GPL as far as I understand things.

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [December 4, 2020, 3:31pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/4 "2020-12-04T15:31:12Z")

</div>

Whow! Cool beans.

Windows will require some more care with PATH problems.

I will try this… though I am interested in a pure C++ version as I want to make a call to the Jacobian function in Stan-math and get the results.

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [December 4, 2020, 3:41pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/5 "2020-12-04T15:41:58Z")

</div>

> [@wds15](#):
>
> though I am interested in a pure C++ version as I want to make a call to the Jacobian function in Stan-math and get the results.

In that case, it should only get easier, no need for the stanc3 stuff and all the handling of the std::ostream\* stuff.

- Anything above line 20 is irrelevant
- Put your C++ code in the `code` variable in line 20
- most of the code from 28 to 110 is also irrelevant (those are just to make the the ostream and rng args work with rcpp)
- lines 116-118, 147-167 also not needed

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [December 4, 2020, 5:05pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/6 "2020-12-04T17:05:29Z")

</div>

Thanks. The code now runs and works under macOS with this small addition:

```
  tbb_dll_ext <- switch(Sys.info()["sysname"],
                        Linux = "so.2",
                        Darwin = "dylib")

```

and somehow the stream magic did not work, but defining and calling the function “manually” made things work.

What are the plans for this thing? The expose function thing is really handy for stan models. So having this in cmdstanr as a permanent feature would be very amazing. This should also open the door to having get\_log\_prob and the like…

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [December 4, 2020, 6:02pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/7 "2020-12-04T18:02:26Z")

</div>

> [@wds15](#):
>
> What are the plans for this thing?

The first step was to get this implementation working with a minimal example. You confirming it works on a Mac with the .dylib fix is a nice step forward. I am exploring some other options like cpp11 in order to have all the info before we move forward.

I think we want to support both expose\_stan\_function as well as log\_prob.

I also have a stanc3 early-stage prototype for expose\_stan\_functions that also returns gradients. But haven’t moved that much in the last month. Also not sure how much of a need is there for that.

The biggest question to me for this way of including these features is how can we use Rstan’s tricks with RCpp without being forced to GPL which is obviously a deal-breaker.

There is also another option of implementing this in cmdstan and supporting  
`model log_prob data file=data.json`  
and  
`model function name="foo" data file="data.json"`

This would have a higher overhead obviously but as far as I understand this is mostly used for debugging/exploring.

I am leaning more towards Rcpp/cpp11 but really want to make sure we avoid the rstan issues (which I think we do by using cmdstan source files but really want this to be as painless as possible).

---

<div class="post-metadata">

**Author:** ![mitzimorris](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/mitzimorris/32/29_2.png) [@mitzimorris](https://discourse.mc-stan.org/u/mitzimorris)\
**Post date:** [December 4, 2020, 6:49pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/8 "2020-12-04T18:49:13Z")

</div>

echoing what Rok says, this is going to take CmdStanR far beyond lightweight wrapper  
and it’s going to drag CmdStanR into the RCpp swamp and become a maintenance nightmare.

we desperately need an R interface to Stan that most users can install and use; we need it to be stable and well-documented. we don’t have a ton of developers. Rok is doing a heroic job adding features while putting out fires on a daily basis.

adding this to CmdStanR serves the needs of a very small set of users and will suck up time and resources. is it worth it? I say no.

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [December 4, 2020, 6:58pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/9 "2020-12-04T18:58:01Z")

</div>

Mitzi’s reply brings me to the third option I didn’t mention previously - a separate package with these kind of rcpp/cpp11 stuff: (cmd)stan\_functions or something like that.

Adding expose and log\_prob would not cause cmdstanr to be harder to install as these features would only be used by those that want it, but yeah Rcpp/handling flags is a time sinker for sure.

---

<div class="post-metadata">

**Author:** ![ssp3nc3r](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/ssp3nc3r/32/120_2.png) [@ssp3nc3r](https://discourse.mc-stan.org/u/ssp3nc3r)\
**Post date:** [December 4, 2020, 8:45pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/10 "2020-12-04T20:45:33Z")

</div>

If you’re considering a separate package, what would be advantages over using `rstan`?

I have been using `cmdstanr` for fitting (when chain threading needed or want to use newer syntax), and `rstan` to expose functions. One obvious drawback to this is that `rstan` is behind in compiling the latest stan syntax.

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [December 4, 2020, 9:32pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/11 "2020-12-04T21:32:47Z")

</div>

This would just be extending cmdstanr with a few features (if we go this route)

Some advantages are:

- new releases are available instantly
- we dont have to rely on rcppeigen, rcpparallel, etc for other dependencies. Only rcpp or cpp11 which makes this muuuuch simpler to use/install. Rcpp on its own was never really the issue.
- sources to build with are not part of another r package (stan headers)

---

<div class="post-metadata">

**Author:** ![jonah](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/jonah/32/200_2.png) [@jonah](https://discourse.mc-stan.org/u/jonah)\
**Post date:** [December 4, 2020, 11:34pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/12 "2020-12-04T23:34:37Z")

</div>

> [@rok\_cesnovar](#):
>
> The dependency on Rcpp on its own should not make it GPL as far as I understand things.

I think that’s right. Calling a function from a GPL package shouldn’t force us to be GPL, but if we included source code from a GPL package then that would. I’m not saying we should definitely depend on Rcpp, but I don’t think the license would be the reason not to do it.

> [@mitzimorris](#):
>
> adding this to CmdStanR serves the needs of a very small set of users

I think `log_prob` serves a small set of users, but `expose_stan_functions` is really essential for testing user defined functions and has a much larger set of users (and I’d argue it should have more users than it probably does). So I think it’s important to eventually offer that in some way, but I 100% agree that we need to make sure to avoid the problems that RStan has run into.

> [@ssp3nc3r](#):
>
> If you’re considering a separate package, what would be advantages over using `rstan` ?
> 
> I have been using cmdstanr for fitting (when chain threading needed or want to use newer syntax), and rstan to expose functions. One obvious drawback to this is that rstan is behind in compiling the latest stan syntax.

Yeah I think what you’re currently doing (using both packages) makes sense for now. But basically for all the reasons @rok_cesnovar said it would be good to offer this outside of RStan (unless we can somehow solve all of the RStan problems).

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [December 5, 2020, 12:41pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/13 "2020-12-05T12:41:01Z")

</div>

I woud concur that thex expose functionality is quite essential. To @mitzimorris concerns: It’s correct that the expose functionality was a long time a big headache to maintain. That was mostly due to the rather unconventional way of how it was implemented by post-processing stan model cpp files. With the parser now having an explicit mode to create cpp files intended for exporting the functions this has been made much more robust.

We absolutely want to maintain the upside of cmdstanr. To me this is ease of installation - and this is not going away when we have the expose functionality. Moreover, the expose functionality is optional and not required to get Stan models to do inference . The big thing, though, which is added in terms of dependencies here is that a working toolchain will be required in R. And this is something to decide on if that is ok. A possible option could also be to put this functionality which glues together stan and R so tightly (and requires a working toolchain) into another R package cmdstanr-tools, for example. I am not totally sure, but in any case, the expose stuff is very useful. I am about to use it to test out different CVODES tolerance settings and here this tight integration is really valuable.

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [December 12, 2020, 6:32am UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/14 "2020-12-12T06:32:40Z")

</div>

You said there is a new version of this expose thing available as prototype? Is that online already?

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [December 12, 2020, 7:18am UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/15 "2020-12-12T07:18:19Z")

</div>

Not yet, will put it on a repository (these gists are not really great for versioning) over the weekend, want to test Windows behaviour first. Its mostly just cleanup, OS support and making sure the users env variables remain as they were. And removed the requirement for rcppeigen.

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [December 13, 2020, 12:19pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/16 "2020-12-13T12:19:56Z")

</div>

I noticed that the stan-math library dependencies are hard coded into the function, which caused a hickup for me as sundials got upgraded. How about you extract the CXXFLAGS out of cmdstan with

`make print-CXXFLAGS`

?

Then things would be build with exactly the same setup as cmdstan. As alternative you can extract the library info with make print CXXFLAGS\_SUNDIALS etc.

BTW, microbenchmarking seems to work from R and that is really cool!

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [December 13, 2020, 1:06pm UTC](https://discourse.mc-stan.org/t/cmdstanr-sourcecpp-cppfunction-for-exposing-functions/19632/17 "2020-12-13T13:06:19Z")

</div>

> [@wds15](#):
>
> make print-CXXFLAGS

Oh, that is a good idea.

The harcoded paths were part of the refactor I did, I just did it with listing folders inside the lib folder and then working based on what that found. But the print-CXXFLAGS is a great idea! Thanks, should have today afternoon to publish this.

> [@wds15](#):
>
> BTW, microbenchmarking seems to work from R and that is really cool!

Used this for some OpenCL stuff as well. It beats timing in C++ any day :)
