Packing/unpacking automation

Algorithms
1

I am writing packing/unpacking code for use with ode_integrate and now map_rect and wish for a struct { int foo; real bar; } but assume that’s not going to happen anytime soon. Instead I was wondering if simply autogenerating code for packing/unpacking to/from flat types would possible/useful. E.g. given declarations like

real a;
real b[2];
matrix[3,4] c;

one could generate automatically

vector abc_pack(real a, real b[], matrix c)
{
  vector[1 + size(b) + num_elements(c)] out;
  out[1] = a;
  out[2:size(b) + 1] = to_vector(b);
  out[size(b) + 1:1 + size(b) + num_elements(c)] = to_vector(c);
  return out;
}

real abc_unpack_a(vector abc) { return abc[1]; }

...

Is this something that could/should be done (I might take a stab)? Is there any existing equivalent?

2

@mitzimorris was working on tuples so your wish will be granted … whenever that’s done… :)

3

Yes, it’s something I’ve thought about a bit. We could write a very generic serializer, with a variadic function:

vector[N] x = serialize(x1, x2, ..., xN);

where the xn are arbitrary Stan numeric types.

What I haven’t figure out how to do is deserialize easily. What we want to do is walk along the vector pulling off types. We need not just types to deserialize, but also sizes.

This is all stalled in refactoring discussions. We may just abandon Stan 2 other than for maintenance and work on Stan 3 from scratch. That’ll push back the entire roadmap.

4

Wow, I really missed things by skipping dev meetings. Is this output factor stuff wasted time now??

5

I’m just talking about the compiler (parser, AST, generator). That can be updated without disturbing the math lib or algorithms.

We need to refactor all the services so they’re not templated on the model class so that we can improve compilation time, but that’s an orthogonal issue.

Stan meetings don’t seem like a good place to discuss this kind of thing due to sporadic attendance and the majority of attendees not caring about any given engineering topic.

We had a weeklong series of onsite meetings with Maria Gorinova and Matthijs a couple months ago where we talked about plans for the language. We’re currently waiting for a spec from Matthijs along with proofs that we can do the kind of shredding Maria did in her thesis so that we can compile to the same target.

One of the things Sean and Matthijs want to do is use a different language (most likely OCaml) so that the replacement for stanc would have to be distributed as an executable per platform.

6

Watching the stancon talks now. Iif you are considering rewriting stanc anyway, may I put in a good word for Rust.

7

That’s under consideration. I’ve never programmed in any of these languages, but it’s hard to imagine anything would be worse than coding a parser in C++.

8

Coding it in R would be worse…