Anyone planning on getting an M1 machine to benchmark?

[laifuthegreat]

The 2020 Mac Mini Unleashed: Putting Apple Silicon M1 To The Test

One aspect we’ve never really had the opportunity to test is exactly how good Apple’s cores are in terms of memory bandwidth. Inside of the M1, the results are ground-breaking: A single Firestorm achieves memory reads up to around 58GB/s, with memory writes coming in at 33-36GB/s. Most importantly, memory copies land in at 60 to 62GB/s depending if you’re using scalar or vector instructions. The fact that a single Firestorm core can almost saturate the memory controllers is astounding and something we’ve never seen in a design before.

The 2020 Mac Mini Unleashed: Putting Apple Silicon M1 To The Test

Compared to the Intel contemporary designs, the Apple M1 is able to showcase a performance leap ahead of the best the company has to offer, with again a considerable strength in the FP score.

Considering that most of the people with scientific computing / data sciencey workloads these days are working off Macbooks, some pretty exciting stuff.

Is anyone on here planning on getting one of these fancy new ARM Macs, at least to make sure stan builds and runs correctly (though I understand there are already some problems with Big Sur even on X86)? Would also be fun to see model compiliation times and single-chain benchmarks with all these claims about the M1’s lead in single-core performance.

[Joshua_Pritikin]

@laifuthegreat

Considering that most of the people with scientific computing / data sciencey workloads these days are working off Macbooks, some pretty exciting stuff.

Meh, I’ve never been an Apple groupie due to their tendency to try to lock people into their ecosystem. I look forward to fine-tuned generic laptops with ARM CPUs that run a properly GPL’d kernel.

[jroon]

So apparently R performs very well on these machines even through Rosetta2:

Kieran Healy (kjhealy)

Thread on datascience on M1 macs on Macrumours forum also one guy says he will try Stan when he his machine is delivered:

Apple Silicon for data science

[mike-lawrence]

Anyone end up doing any benchmarks with Stan?

[rok_cesnovar]

See

Will the new M1 chip affect cmdstanR?

Hi All, These are all benchmarks of a model I’m working on: mid-2013 1.3Ghz Intel MacBook Air: ~490 seconds M1 MacBook Pro under Rosetta 2: ~240 seconds M1 MacBook Pro native Arm: ~125 seconds. It seems like going native arm64 is worth the hassle! Kind wishes, Nico This is what I did to make it work: In the Terminal.app, not the terminal window in RStudio cd ~ git clone --recursive https://github.com/stan-dev/cmdstan.git cd cmdstan/stan/lib/stan_math git remote add nicokist git@github.…

And

Link error on Apple M1

yep, I grabbed the repo and used just the CmdStan models, w/ 2.25.0, with a/c adapter plugged in. One chain with logistic0 finishes with Elapsed Time: 23.943 seconds (Warm-up) 20.146 seconds (Sampling) 44.089 seconds (Total) and logistic1 Elapsed Time: 30.971 seconds (Warm-up) 35.471 seconds (Sampling) 66.442 seconds (Total) This is with the default build options. I then added to make/local, perhaps unnecessarily`, CXX14FLAGS =…

[mike-lawrence]

Thanks! Though both of those are comparing to CPUs that came out ?8 years ago. I’d be curious how M1 compares to something more modern. I’ve preordered an asus with a 5900HS, and when I get it I might “borrow” an M1 from Apple to do a latest-to-latest comparison.

[maedoc]

@mike-lawrence

I’d be curious how M1 compares to something more modern

I didn’t do the above benchmark on it, but I had tried out a 5.3 GHz 10th gen i9 from Intel, and Scimark4 runs slower on it (score of 3300) than x86 on Rosetta2 on M1 (3600) than native (4000). Those HPC nodes I did try in the benchmark linked above score about 2700. Scimark4 is far away from Stan in terms of complexity but it’s somewhat predictive for non-tiny models.

[mike-lawrence]

Found another comparison here.

So, seems that if you don’t care about power usage, latest Ryzen series is still faster.

[maedoc]

Sure, and the HPC node I tested got faster than the m1 after 9 cores (on a 12 core). That still doesn’t bode well for x86.

[storopoli]

I’ve also have a MacBook Air 2021 M1 with R native in arm (brew binary). How can I help?

[maydin]

Hi, trying to run CmdStan via Matlab on an M1 MacBook. Ran into a problem and was suggested to have a look at this discussion.

I get two errors from the compiler:
error: half args and returns was enabled in PCH file but is currently disabled
error: PCH file was compiled for the target ‘arm64-apple-macosx11.0.0’ but the current translation unit is
being compiled for target ‘x86_64-apple-macosx11.0.0’

Matlab is x86-64, and presumably runs via rosetta2. There are no functionality issues with regards to Matlab at all.

I installed Xcode for c++, but Xcode is obviously arm64. I suspect this is what is getting me into trouble with regards to the error #2 above.

I don’t understand what the error #1 is trying to tell me.

Has anyone encountered same/similar errors? I would greatly appreciate suggestions to solve.

[bbbales2]

@maydin go ahead and make a new question for this one. I don’t know what to do but the question might get lost here.

[storopoli]

Try this inside your Makevars to compile CmdStan

CXX14FLAGS += -O3 -mtune=native -arch arm64 -ftemplate-depth-256 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

[maydin]

I was going to make a new question, but was able to solve it by building the CmdStan directly from Xcode. I suspected Matlab being an x86 app was causing the problem, but it turns out the “make build” command via the terminal presumably also runs as x86. When everything is done from Xcode, no issues whatsoever using CmdStan on M1 MacBook with BigSur.

[storopoli]

@maydin I have a LOT of problems when building stuff that happens to have the <cmath> header inside somewhere.

Here is a toy example with a file that has the #include <cmath>:

file cmath-bug.cpp

#include <cmath>
#include <iostream>

int main(){
	std::cout << "success" << "\n";
	return 0;
}

using clang++

❯ /usr/bin/clang++ cmath-bug.cpp
In file included from cmath-bug.cpp:1:
/Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/cmath:317:9: error: no member named 'signbit' in the global namespace
using ::signbit;

After much googling and tinkering I’ve found that to overcome this you need to give sudo privileges to clang (or any other CXX compiler):

using sudo clang++

❯ sudo /usr/bin/clang++ cmath-bug.cpp

No problem here, it just compiles

[maedoc]

@mike-lawrence

I’ve preordered an asus with a 5900HS, and when I get it I might “borrow” an M1 from Apple to do a latest-to-latest comparison.

Did you get that 5900HS and benchmark it? I reran the same benchmark on a i7-1185G7 and see logistic0 running in 91s, logistic1 running in 68s, and with multiple threads, runtime scales down to 19s, which isn’t far off the m1.

[mike-lawrence]

My pre-order kept getting delayed so I cancelled. Waiting to see the new MBPs expected soon (WWDC?); if they have the 8-performance core count that’s rumoured, I’ll probably go that route. I’ve lived a Windows → Mac → Linux life so far, so hopefully another transition won’t be a bad idea.

[jroon]

There is some sort of event next week - usually this timing would be about ipads n such, but sometimes they drop new computers too: Apple Events - Apple

I would deffo wait and see at least another week!