My point was that if two methods use the exact same numerical procedure to compute y_hat, then they are sampling the same posterior.
But I realize now that you are correct here so y_hat won’t be exactly the same even when using same method and same tolerances. So it is hard to do fair comparisons.
If this was the case, wouldn’t it make sense to set max_num_steps to like 1e30 or infinity.