How does the autodiff storage arena work? I see maybe two logically separate (but probably in the same underlying chunk of raw memory?) arenas:
static std::vector<ChainableT*> var_stack_; static std::vector<ChainableT*> var_nochain_stack_; static std::vector<ChainableAllocT*> var_alloc_stack_; static stack_alloc memalloc_; // nested positions static std::vector<size_t> nested_var_stack_sizes_; static std::vector<size_t> nested_var_nochain_stack_sizes_; static std::vector<size_t> nested_var_alloc_stack_starts_;
I’d guess the first vector is used to figure out what to call
chain() on, and so the 2nd makes sense as a place not to put things to call
chain() on. What is the
var_alloc_stack_ for? And why are there nested stack sizes and starts?
IIRC this level of detail wasn’t mentioned in the autodiff paper but I might be mis-remembering - feel free to point me to some other docs.