Over the next few months, I want to focus on making it easier to develop in the Math library. I had dinner with @breckbaldwin last week and it was clear that this is something we’re lacking.
I’ll just dedicate a few hours a week doing this.
Anyone have ideas on what resources would help? I’d love to hear ideas from anyone:
Thanks, all.
Intro to C++ concepts needed to develop for Stan. I’ve written C and limited amounts of C++, however much of the Stan code is a mystery to me. I’d like to be able to understand enough of the templated code to read the code and contribute new functions.
Description of directory structure of the Stan source. I’m sure it’s quite simple once you get the hang of it, but I find myself grepping through it all to find the code I need.
Guidelines for what should be in the Stan code and what should be in user specific functions. If I am implementing, say a general inverse Gaussian distribution, should this be included in Stan math? What about more obscure and domain specific models?
Walkthrough of how how to implement, something, required documentation, pull request process etc.
I’ve read some of the documents on templating, and added to the Stan docs, but they are trivial. Some examples with references to Stan code would be great. May be something ‘typical’ of Stan, including both templates and enable_if programming, or some more complex non-trivial features.
A description of the map from Stan datatypes to C++ datatypes. (i.e. [stan -> c++] -> [vector[n] y -> Eigen::Matrix<double, Eigen::Dynamic, 1>)
There is a way to visualize the objects put on the stack (stan::math::print_stack().) A description of this feature of Stan development would be cool, we obviously want the library as efficient as possible. If there’s any other features like this we haven’t seen yet, please let us know!
More in depth description of all of the memory management features unique to the stan library (i.e. stan::math::recover_memory()).
Description of how to use/print the gradient functions in Stan, and any other diagnostic tools for making sure our code/math is accurate ( some_var_.grad(vars, grad)) and then (for (auto g : grad) { std::cout << g << std::endl;}
How does the architecture from rev-> prim work? I threw some print statements in gp_exp_quad_cov formerly (cov_exp_quad) to see how it works. rev fuctions are precompiled, and then call some of the prim functions on runtime? How does this work exactly? What kind of templating it happening to make sure rev is going to which prim? Can we have a quick visualization on a simple stan program to know what’s happening?
I haven’t done this stuff yet, but I’d like to be able to run a specific branch on my fork to test some stuff out (i.e. compare a simple generative model to GPstuff, etc). May be a description about this could be useful for beginning developers. I’ve done some derivations of the gradients as in the Autodiff paper and for some functions in the math library, but I don’t trust my math until I do some comparisons!
If there are other memory or gradient tools that we can use, please give us a description! Or, just point me to the docs!
About two weeks ago I fixed up jenkins so the doxygen site for stan math is available. May be a useful place to put some of the guidelines and concepts mentioned above
It would be nice to flesh it out more so that it looks something like Eigen or ViennaCL’s docs
https://eigen.tuxfamily.org/dox/
http://viennacl.sourceforge.net/doc/
That would be a yearlong course in C++ and numerical analysis! We just don’t have time for that. We’re going to just have to assume people know C++11 well and provide examples of how Stan fits into that.
I highly recommend Vandevoorde and Josuttis for template metaprogramming (all the expression templates, type lists, and traits we use, as well as recursive template metaprogramming, lazy function evaluation, etc. are all covered).
This is going to change a bit, but it’s at the front of our doc, including the legal dependencies. We could use some examples. The short answer is:
{prim, rev, fwd, mix} / {scal, arr, mat, meta, err }
We don’t have any other than what we think might be useful.
I think a case study on this would be great.
The top-level overview of all that’s in our reverse-mode autodiff paper on arXiv.
That would be good for a case study, but it’s literally just a loop like you laid out. We’re recommending people use our functionals for most purposes now that you have closures and binding.
I’m afraid that depends on the function.
Sean’s working on that.
Lots of stuff out there, but nothing Stan-specific.
Doc for the intended usage of metaprograms would be nice
Sure - it just wasn’t something I found in the docs. @syclik showed me this. When I’m comfortable with the library I can write a case study on this topic.
drezap: I can do this, with my first commit as an example. Is a latex document fine? Where should I send it when complete? This Stan - Case Studies send me back to this forum.
Quick question:
So I believe for rev we need implement different functions for different inputs/stan datatypes, can I just add support for a certain datatypes, with prim as the “catch all”, and come back to it when my template metaprogramming improves or I come up with a solution? I have something that’s works and I can commit, but I’m stuck at some flow control/template problem. So the autodiff will not be enabled with certain datatypes, but it’s still “functional”/“works”.
Not sure where to post this question. I just want to merge something and come back to it later.
I think you could bring a lot of people on board who could meaningfully contribute new functions to Stan math without fully understanding the details of how Stan works.
Maybe this would be enough:
Not about doc, but I feel it may help by categorizing unit tests into groups, for example by compile+run time. Before a developer sends to Jenkins, he should pass all the short tests, which takes a reasonable wall time. Current wiki requirement for finishing all the tests takes too long to be effective in practice.
Thanks for the responses. I think it’s clear that we need to separate out the docs a bit for the math library. We need a user-facing doc and dev-facing doc. Over the next couple weeks, I’ll try to come up with some organization that helps.
Yup. Heard. It’s a chicken and egg problem. We don’t strictly need all that testing if there are enough unit tests in place. The problem is explaining the combinatorics of templated programming and what that truly means for a developer to test in order to verify the code is correct. I’m going to try to write all that down too.
Don’t know if anyone else does this, but I occasionally use debug build.
No, this is good. Some other developers had used an autodebugger on another software project I worked on.
In a nutshell, what does your debug build do and how do I use it?
I put it in a PR. It just adds -g -O0 options, and allows DEBUG blocks.
For me I just need -g -O0 in order to debug with gud+gdb, as well as with valgrind.
So may be a dock with a standard way to do speed tests? I can write timing tests in-line, but this seems kind of naive and unnecessarily time consuming, as I’d have to write code for each subsection of the code I want to analyze, so I think it would be better to use some kind of profiler (i.e. perf or valgrind’s callgrind) that would do the work for me.
I’m going to open up another thread, hoping someone has suggestions on this.
EDIT: Discussion on profiling here: profiling c++ code
Your wish has been granted (for some relaxed interpretations of “your” and “wish”): https://github.com/stan-dev/performance-tests-cmdstan
Sean, this is awesome, thank you! I should look harder before asking questions like this!
Oh it’s not really published anywhere, not your fault! Probably needs more documentation too. Originally wasn’t sure how many other people would want to use it (but it runs in Jenkins daily).
This may be a bit off, and otherwise I would put this in the “people that would consider developing, but are having difficulties” category, but I think there could be a (possibly visual) map. As an example there’s the published summary structure of a BEAST 2 model, where the connection between each data, model and MCMC class/object is shown. Of course it’s very different software (and very JAVA-y), but it’s a useful conceptualization for people getting into the code and especially non-developers who are nevertheless familiar with the concept being implemented.
For instance, I know enough about MCMC to implement it and most models I need standalone in C++ (albeit in a very non-optimized, robust, or object-oriented way), but not being a developer I have a hard time going into Stan’s code and actually implementing new stuff within it.
I guess this has some overlap with what @aaronjg and @anon79882417 mentioned with the structure of the source and Stan model to Eigen/C++/etc correspondence.
I guess that would be a good connection point between the current developers and people who could potentially contribute from different angles.
Hi, I’ll attemped an answer.
The directory structure descriptions have been updated here: https://github.com/stan-dev/math/wiki, and I would take a look at the quick start wiki on that page. The datatype maps are mostly as described as above. That doc now contains a lot of what you need to start up. I’m realizing what’s inhibited my contributions most is my C++, but Stan is an extremely high quality, open-source production quality software so it’s a great way to improve C++ programming!
It’s important to be extremely specific about what you want to contribute. I would recommend starting at the the issues for the Stan math library, labeled good first issue.. Pick something simple, to develop some familiarity w/ the math lib, and then you can decide what you want to work on from there.
This is also on that wiki, but I would also recommend reading the autodiff paper: The Stan Math Library: Reverse-Mode Automatic Differentiation in C++, and there are some text book references for C++ templates, but I lost the wiki page, it’s somewhere in there.
Still a novice, though. Hope this helps!
You’re saying something like a UML diagram of a compiled Stan program? That should be pretty easy to create…