-O2 vs -O3 compiler optimization level

[Bob_Carpenter]

@danluu

If it seems weird that O2 could be faster than O3, an intuition for why this is somewhat common is that O3 does a lot of optimizations that increase code side (e.g., more aggressive loop unrolling), which almost always increases performance in microbenchmarks, but can sometimes decrease performance in workloads with a larger code footprint.

Thanks—that’s a very helpful intuition.

@danluu

we’ll get better results by using PGO. Unless someone else is already doing this, I’ll probably mess with this later this week or this weekend.

We never looked into this again after Matt Hoffman told us it was a huge pain (he spent an internship at Google working on PGO, but that was at least 8 years ago).

@danluu

I’m not proposing a default change to -O2, but I figured it was worth posting preliminary results to start the discussion in case there are known cases where -O3 totally demolishes -O2 or if there are other things to consider.

If -O2 compiles faster and produces smaller output code and the time penalty is in the noise for end-to-end runs, then by all means we should switch to -O2.

@aaronjg

I’d be interested in seeing the effect of -march=native as well.

Definitely.

@aaronjg

Anecdotally, I have seen 30% speedup by using this, which enables CPU specific instructions.

I’ve always had that turned on. Didn’t realize it was that significant. Is that for code with lots of big matrix ops? Those will be the places with lots of double-based loops to unroll.