Model base class & compile time speedup PRs ready to review

The R.h file has these lines at the top

#ifndef USING_R
# define USING_R
#endif

so I believe it would be defined for anything someone could do in R using Stan.

I still am perplexed by this question. I tried to make a file called model_base.cpp that just contains

#include <stan/model/model_base.hpp>

// forward declaration for function defined in another translation unit
stan::model::model_base& new_model(stan::io::var_context& data_context,
                                   unsigned int seed,
                                   std::ostream* msg_stream);

and I can compile that. But when I do nm libStanHeaders.a | grep -F "stan" to look at what it puts into the shared library, it is only

0000000000000000 W _ZN4stan4math11stack_allocC2Em
0000000000000000 W _ZN4stan4math11stack_allocD2Ev
0000000000000000 b _ZN4stan4math12_GLOBAL__N_126global_stack_instance_initE
0000000000000000 V _ZN4stan4math22AutodiffStackSingletonINS0_4variENS0_15chainable_allocEE9instance_E
0000000000000000 W _ZN4stan4math22AutodiffStackSingletonINS0_4variENS0_15chainable_allocEED2Ev
0000000000000000 W _ZN4stan4math8internal25eight_byte_aligned_mallocEm
0000000000000010 b _ZN4stan4mathL10INV_SQRT_2E
0000000000000070 b _ZN4stan4mathL11LOG_EPSILONE
0000000000000050 b _ZN4stan4mathL11LOG_SQRT_PIE
0000000000000038 b _ZN4stan4mathL11SQRT_TWO_PIE
0000000000000040 b _ZN4stan4mathL15INV_SQRT_TWO_PIE
0000000000000028 b _ZN4stan4mathL16POISSON_MAX_RATEE
0000000000000068 b _ZN4stan4mathL19NEG_LOG_SQRT_TWO_PIE
0000000000000018 b _ZN4stan4mathL5LOG_2E
0000000000000020 b _ZN4stan4mathL6LOG_10E
0000000000000048 b _ZN4stan4mathL6LOG_PIE
0000000000000008 b _ZN4stan4mathL6SQRT_2E
0000000000000030 b _ZN4stan4mathL7SQRT_PIE
0000000000000060 b _ZN4stan4mathL8LOG_HALFE
0000000000000058 b _ZN4stan4mathL8LOG_ZEROE

as compared to CmdStan’s main.o file, which has 1538 lines that are turned up by the above grep. I feel as if more needs to be instantiated but you can’t instantiate from an abstract class.

By itself, model_base.hpp is abstract in the sense that it doesn’t provide an implementation. That’s why the new_model class is code generated to return an instance of the specific compiled model class (defined by typedef). The compiled model class is an extension of model_base_crtp, which is itself an extension of model_base. We need all three in a row to have virtual functions and backwards compatibility—it’s a complicated CRTP pattern to hack around the fact that template methods (template non-static class functions) can’t be virtual.

For reference, the code generated in the top-level namespace in 2.20 and current state of develop is:

#include <stan/model/model_header.hpp>

... model class definition ...

typedef foo_model_namespace::foo_model stan_model;

stan::model::model_base& new_model(
        stan::io::var_context& data_context,
        unsigned int seed,
        std::ostream* msg_stream) {
  stan_model* m = new stan_model(data_context, seed, msg_stream);
  return *m;
}

Great. I just created an issue to put in this simple fix. That’ll be a lot easier than managing a second file. See

I’ll submit the PR.

Was that a complicated way of saying I should #include <stan/model/model_base_crtp.hpp> instead of #include <stan/model/model_base.hpp> before the forward declaration? If so, it does not seem to have helped.

Afraid not. I meant that you need to compile in two translation units:

  • Services translation unit: This will #include <stan/model/model_base.hpp> and compile object code for the services, which can be instantiated with stan::model::model_base and the RNG we’re using. The result is a compiled services object.

  • Model translation unit: This defines a specific class extending stan::model::model_base, which itself will be defined by a new class class foo that extends stan::model::model_base_crtp<foo>). It will also need to provide some generic some way to get a model instance to the services translation unit.

In CmdStan, the services translation unit implements a main() that calls the function ::new_model(). So the services translation unit includes the header for new_model but not the definition. The model translation unit defines the model class foo and defines new_model to return an instance of foo. When they’re linked, you have a whole program with a definition of new_model returning an instance of foo and the command using that to execute.

OK, I have one line of the “services translation unit”, namely

#include <stan/model/model_base.hpp>

I am confused about the remaining lines. I think to instantiate, I would need something like

// forward declaration for function defined in another translation unit
stan::model::model_base& new_model(stan::io::var_context& data_context,
                                   unsigned int seed,
                                   std::ostream* msg_stream);
stan::model::model_base& model = new_model(?, ??, ???);

but I don’t know what to insert for ?, ??, or ??? at the time StanHeaders is compiled (i.e. before there is any actual Stan program). Can I just make ? an empty var_context, ?? be 0, and ??? be nullptr?

Using an empty var_context did not seem to work, since the compiler complains about unimplemented virtual methods

clang++-8 -Wno-unknown-pragmas -std=gnu++14 -I"/home/ben/r-devel/include" -DNDEBUG -DNO_FPRINTF_OUTPUT -I"../inst/include" -I"../inst/include/src" -include stan_sundials_printf_override.hpp -I"/home/ben/r-devel/library/RcppEigen/include" -I/usr/local/include   -fpic  -mtune=native -march=native -O3 -g0 -c model_base.cpp -o model_base.o
model_base.cpp:7:44: error: allocating an object of abstract class type 'stan::io::var_context'
stan::model::model_base& model = new_model(stan::io::var_context(), 0, nullptr);
                                           ^
../inst/include/src/stan/io/var_context.hpp:42:20: note: unimplemented pure virtual method 'contains_r' in 'var_context'
      virtual bool contains_r(const std::string& name) const = 0;
                   ^
../inst/include/src/stan/io/var_context.hpp:56:35: note: unimplemented pure virtual method 'vals_r' in 'var_context'
      virtual std::vector<double> vals_r(const std::string& name) const = 0;
                                  ^
../inst/include/src/stan/io/var_context.hpp:66:35: note: unimplemented pure virtual method 'dims_r' in 'var_context'
      virtual std::vector<size_t> dims_r(const std::string& name) const = 0;
                                  ^
../inst/include/src/stan/io/var_context.hpp:76:20: note: unimplemented pure virtual method 'contains_i' in 'var_context'
      virtual bool contains_i(const std::string& name) const = 0;
                   ^
../inst/include/src/stan/io/var_context.hpp:86:32: note: unimplemented pure virtual method 'vals_i' in 'var_context'
      virtual std::vector<int> vals_i(const std::string& name) const = 0;
                               ^
../inst/include/src/stan/io/var_context.hpp:96:35: note: unimplemented pure virtual method 'dims_i' in 'var_context'
      virtual std::vector<size_t> dims_i(const std::string& name) const = 0;
                                  ^
../inst/include/src/stan/io/var_context.hpp:104:20: note: unimplemented pure virtual method 'names_r' in 'var_context'
      virtual void names_r(std::vector<std::string>& names) const = 0;
                   ^
../inst/include/src/stan/io/var_context.hpp:112:20: note: unimplemented pure virtual method 'names_i' in 'var_context'
      virtual void names_i(std::vector<std::string>& names) const = 0;
                   ^
1 error generated.

OK, apparently we have an actual class called empty_var_context and this compiles but still does not seem to compile in the algorithms and stuff:

#include <stan/io/empty_var_context.hpp>
#include <stan/model/model_base.hpp>

// forward declaration for function defined in another translation unit
stan::model::model_base& new_model(stan::io::var_context& data_context,
                                   unsigned int seed,
                                   std::ostream* msg_stream);
stan::io::empty_var_context evc = stan::io::empty_var_context();
stan::model::model_base& model = new_model(evc, 0, nullptr);

It’s not going to be possible to build a functional model stub before you have a real model. The trick is going to be figuring out how to break everything into two translation units:

  1. everything but the model; includes model_base.hpp and all references to models should be to references to model_base.hpp.

  2. the mode; this needs to be something that implements model_base.hpp, so just what comes out of the C++ compiler (which now excludes the factory function).

What I don’t know how to do is dynamically link what you compile in (2) together with (1). What I did for CmdStan was have CmdStan invoke a new_model function in translation unit 1 using only the new_model() header. Then translation unit 2 included the definition of new_model().

How do I tell the compiler that the model_base class is already compiled when it goes to compile the thing that inherits from model_base_crtp? Do I just link to the RStan shared object? Also, why is there no gradient-related virtual methods in model_base. Am I still supposed to be calling log_prob_grad, which is still templated on the model?

The model_base_crtp has overloads defined which call the templated thing, but technically you are calling and overloaded function; so no template.

And yes, I think you just link in the already compiled class. That should do it.

I do not understand the code that I wrote, but I don’t understand what you are saying either. The C++ class in RStan that runs everything can’t be defined in terms of model_base_crtp<> because we don’t know what to put for the template parameter until the C++ code is generated for the user’s model. So, AFAICT, it has to be defined as (a pointer to) model_base. But then, I can’t call methods that are defined in the thing that inherits from model_base_crtp<>; I have to call the standalone function that is templated on the model:

That makes more sense but the generated C++ #include <stan/model/model_header.hpp> which #include <stan/model/model_base.hpp so isn’t it recompiling the model_base class from the header file before it even gets to the linker?

Hi!

I forgot a few details about model_base so let me detail a bit (after having looked at the code):

  • mode_base defines all the functions of the model which involve templates as virtual functions. Thus, all functions like log_prob are declared using overloading, but none of them is actually implemented as all are merely virtual.

  • model_base_crtp actually implements all the different overloads of these functions like log_prob using the templated model.

With this design everything you ever want to do is available in the model_base. Thus, in RStan you never have to deal with templates anymore. You only need your object factory, as discussed, to create instances of specific models (using some key to do so which maps the actual model to the type)… but after creation of the model you would only need to refer to it using a model_base pointer.

Does this make more sense now?

Is there a branch which I should look at to help?

Sebastian

The branch is “for_2.20”, which has a bunch of convoluted stuff dealing with a stan::model::model_base* that is instantiated as something that inherits from it.

But I am definitely calling the old templated log_prob_grad function on the dereferenced pointer

This compiles but I don’t know why there isn’t a stub for it in the model_base class. And I still don’t know how to get it to not recompile the model_base class each time users generate C++ code.

any instructions for me what commands I need to fire off to compile it? Thanks!

Yes, what @wds15 said. For CmdStan, I just compiled statically againt model_base (not model_base_crtp, which is only there to adapt the actual generated model class to implement model_base). Then I compiled the actual model class separately and then linked.

Then that caller has to go in the translation unit with the known model class. That’s how it’s being done with CmdStan.

It’s a pure header, so there’s nothing to compile—it’s just there to provide declarations so things can be compiled against its interface before it’s available.

There are virtual var overloads of log_prob for autodiff. Everything else is an external call that just uses the var instantiations. Such as the functionals like gradient and hessian. log_prob_grad just wraps up the call to the var overload.

It’s a utility function that’s not defined in the model class. That’s why it’s getting the model as an argument.

No, you just need the branch and then you can do R CMD build. It compiles but doesn’t work yet.

Doesn’t work is seen by doing what? Sampling any model?

Doesn’t work in the sense that I haven’t written the R code to call the C++ yet.