Whoa… hold up! Can’t compare wall time across these git hashes.
You need to be measuring against effective sample size. And since we’ve changed how random numbers are generated, you can’t do this with just with one seed. It’s one of the reasons I’m very careful when we change behavior.
I timed the
char * vs
std::string and there really was no measurable time difference. I looked at the timings with the performance test too and there wasn’t anything that was noticeable.
For this particular model, I think it would help to run it with 100 different seeds and check the wall time divided by the number of effective samples per run. And the same with the other version. (And maybe replicating that whole process to get a sense of the variability.)