I ended up pruning the stan/ folder and the downloaded binaries macos-stanc, linux-stanc, windows-stanc, etc. in order to get the package size way down, and everything still seems to work. Thanks for the tip @ahartikainen !
I think my original questions have been largely answered so thanks everyone :) In case anyone reading this is interested, here is the work-in-progress PR we have on Prophet at the moment: [Draft] Python Wheels for PyPi by tcuongd · Pull Request #2010 · facebook/prophet · GitHub . There are still some issues with Linux we need to figure out and other optimisations we can do.