I think it showed up as a no-title 3 pm meeting for me. When is it?
Thursday, May 16, 10 am eastern time.
Sorry about the link. I’m still trying to work out Google’s permissions on events so I can just share the link. If that doesn’t work, I’ll have to invite everyone individually. I’ll try to figure it out over the next few days.
I want to drop this here for discussion. Can we talk about standardizing benchmarking memory and speed? Any C++ libs for timing/memory any one has a preference for, or would make it easy to compare with other software?
I did a bit with Gradient evaluations, but I’m wondering if anyone’s done memory consumption of individual lines of code as opposed to an entire Stan program.
1 Like
I’m still working out Google permissions. I’ll have the link up soon. If not, I’ll invite everyone individually.
I think I got it set up properly. I’ve also updated the original post with this information:
If you’d like an invite to the Google Calendar event, please DM me with your Google handle. This will allow you to get updates on the event as they are updated.
If instead you’d like to just have a copy of the event on your calendar, please use this link:
1 Like
I don’t know about memory per line. Memory gets allocated either in std::vector or Eigen::Matrix containers and in our underlying memory allocation for autodiff. But the actual allocation’s in big blocks, which we then divvy up. So I don’t know how useful it’d be.
@seantalts has been looking at some high-level profiling that’s the exact opposite in that it measures things like overall cache pressure, not where it’s coming from. I can’t recall the name, though.
It’s called Top-down Microarchitecture Analysis Method (TMAM) (Intel® 64 and IA-32 Architectures Optimization Reference Manual, Appendix B.1). I’m still digging into this excellent intro article but it looks super useful. If someone else has a chance to run this on some models that would be awesome, I’m heads-down on the new compiler for the next few weeks at least.
@increasechief also has a design-doc up suggesting we make a phoronix test suite to benchmark models across a variety of OS + hardware combinations, though that might not be quite what you’re looking for.
1 Like
I also threw together what I think is a minimum working example: https://github.com/ode33/standev-phoronix.
As noted in the README I haven’t run the example but the steps outlined should be representative. If I looked at it again I’d try to get it to use an existing CmdStan install as it is now the first test run downloads and compiles from source so that takes quite a while.
The actual test profile is here: https://github.com/ode33/standev-phoronix/tree/master/test-profiles/local/build-standev-cmdstan-2.19
1 Like
See you all in a few hours!
I’ll get on the call 10 minutes early; if anyone wants to troubleshoot technical issues with joining the call, feel free to get on at that time.
Hey @syclik, I didn’t get any invite despite suggesting the revival of these meetings! Can you please send me the calendar invite or whatever you’re using? Thanks.
click on the “google calendar” link in the original post.
Also:
To join the video meeting, click this link: https://meet.google.com/hhm-gnpt-jnp
Otherwise, to join by phone, dial +1 608-909-0187 and enter this PIN: 477 163 796#
Can you invite me to the event so that I get updates?
@seantalts, you’ve been invited.
If anyone else would like an invite, please ping me.
1 Like
Sorry about missing this. I completely forgot about it despite it being in my calendar. I just zombied onto the subway and went to the regular Stan meeting.
No worries. Hopefully you make it to the next meeting, June 20, 2019 at 10 am Eastern.