# Operands and partials with tuples as edge container?

**URL:** <https://discourse.mc-stan.org/t/operands-and-partials-with-tuples-as-edge-container/2902>\
**Category:** Developers\
**Created:** [January 4, 2018, 8:22pm UTC](https://discourse.mc-stan.org/t/operands-and-partials-with-tuples-as-edge-container/2902 "2018-01-04T20:22:18Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [January 4, 2018, 8:22pm UTC](https://discourse.mc-stan.org/t/operands-and-partials-with-tuples-as-edge-container/2902/1 "2018-01-04T20:22:18Z")

</div>

Hi!

… as I have dealt now a lot with this super cool beast I was wondering if a tuple based design would make sense for the edges. Right now we have 5 edges maximum fixed such that any function using less will have added overhead due to the extra edges never being used. A tuple as edge container should allow to adjust at compile time the number of available edges. Is it worthwhile to think more about this? I mean is a reduced size really relevant for performance?

Sebastian

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [January 5, 2018, 11:00pm UTC](https://discourse.mc-stan.org/t/operands-and-partials-with-tuples-as-edge-container/2902/2 "2018-01-05T23:00:53Z")

</div>

You’re right—I thought we got rid of that, but it’s still there. In the old design we had, everything was an array, so there wasn’t any wasted space. This seems like we should be specializing to the number of edges we need. Those `ops_partials_edge` structs are storing `double` values, so they are 64 bits minimum.

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [January 7, 2018, 11:50am UTC](https://discourse.mc-stan.org/t/operands-and-partials-with-tuples-as-edge-container/2902/3 "2018-01-07T11:50:03Z")

</div>

The overhead is not that bad. @seantalts introduced a nice trick with the `empty_broadcast_array` classes which cut down the overhead per unused edge to a single byte. That leaves you with 5 bytes of wasted storage for the empty configuration. That does not sound like a lot, but as this class is used a lot it could add up.

Before embarking on a large refactor here, it would be good to know if saving those bits is worthwhile. The other advantage of a tuple based design is that you are not any more limited to a fixed number of edges, but could extend just as needed.

---

<div class="post-metadata">

**Author:** ![seantalts](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/seantalts/32/32_2.png) [@seantalts](https://discourse.mc-stan.org/u/seantalts)\
**Post date:** [January 7, 2018, 4:42pm UTC](https://discourse.mc-stan.org/t/operands-and-partials-with-tuples-as-edge-container/2902/4 "2018-01-07T16:42:13Z")

</div>

I measured the speed between the old and new and found them to be the same, for whatever that’s worth.

I actually started with a tuple-based design when I first attempted this. I think ultimately it’s probably the way to go, but I encountered a lot of issues and found that others had a lot of difficulty understanding tuples and the design based on tuples. That said, now what we hammered out the non-tuple design more thoroughly, it might be a much shorter jump to tuples and save those bytes.

As far as prioritization and return-on-investment go, I would suspect this to be fairly marginal vs. other things that need work in Stan. But it might be worth measuring and seeing how much space we could save in the best-case scenario (i.e. we could assume a tuple-based design saved 4 bytes per instantiation of `operands_and_partials` for the sake of an upper-bound on memory) for some larger models and see if that buys us anything tangible in memory-land (since it didn’t give us a speed decrease to use the extra memory in the first refactor). Tuples also take longer to compile, for what it’s worth.

---

<div class="post-metadata">

**Author:** ![seantalts](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/seantalts/32/32_2.png) [@seantalts](https://discourse.mc-stan.org/u/seantalts)\
**Post date:** [January 7, 2018, 4:43pm UTC](https://discourse.mc-stan.org/t/operands-and-partials-with-tuples-as-edge-container/2902/5 "2018-01-07T16:43:56Z")

</div>

Though I also thought the current design didn’t use more memory than the old one… I forget the details though.

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [January 8, 2018, 9:49am UTC](https://discourse.mc-stan.org/t/operands-and-partials-with-tuples-as-edge-container/2902/6 "2018-01-08T09:49:25Z")

</div>

> [@seantalts](#):
>
> As far as prioritization and return-on-investment go, I would suspect this to be fairly marginal vs. other things that need work in Stan. But it might be worth measuring and seeing how much space we could save in the

I have never seen Stan programs run into memory issues. I think the extra size may lead to worse use of high-speed caches and this could be the critical. It is probably not really worth to spent resources on this at this point as it is a big effort for likely little return if I understand you right. If we haven’t seen a speed decrease with this refactor (which increased the size) then we should be good.

Do we have a facility to easily do performance tests? Currently it is a bit tedious to do as I compile two binaries with redefining the MATH variable in between. All of this could be automated a lot.

---

<div class="post-metadata">

**Author:** ![seantalts](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/seantalts/32/32_2.png) [@seantalts](https://discourse.mc-stan.org/u/seantalts)\
**Post date:** [January 8, 2018, 3:28pm UTC](https://discourse.mc-stan.org/t/operands-and-partials-with-tuples-as-edge-container/2902/7 "2018-01-08T15:28:59Z")

</div>

Not that I know of - maybe @syclik has done something like this? The only performance test I know of is in the Stan repo and just does a logistic regression 100 (or 1000?) times.

I’m not sure what the facility you’re thinking of looks like, but if there was something that made it easy to actually answer these questions empirically that would probably be pretty high ROI given how much of what we do is in the name of performance.

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [January 9, 2018, 4:35am UTC](https://discourse.mc-stan.org/t/operands-and-partials-with-tuples-as-edge-container/2902/8 "2018-01-09T04:35:45Z")

</div>

> [@seantalts](#):
>
> something that made it easy to actually answer these questions empirically that would probably be pretty high ROI given how much of what we do is in the name of performance.

Yes!

---

<div class="post-metadata">

**Author:** ![syclik](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/syclik/32/6_2.png) [@syclik](https://discourse.mc-stan.org/u/syclik)\
**Post date:** [January 9, 2018, 8:24am UTC](https://discourse.mc-stan.org/t/operands-and-partials-with-tuples-as-edge-container/2902/9 "2018-01-09T08:24:43Z")

</div>

+1!

We used to have performance tests against all the BUGS models. They were pulled out ages ago. Perhaps that was a mistake.

I’ve benchmarked specific things, but haven’t come up with a general framework for benchmarking the Stan C++ code. (Last time we thought about it, the conversation diverged.)