The issue here is not a question of how good the reason for rejection is but rather what criteria for rejection are actually valid.
If the numerical ODE solver is taking too long or otherwise returning inaccurate results then you are not getting a faithful evaluation of your posterior density function and hence no algorithm will give a faithful quantification of your posterior. Similarly if the gradients of the posterior density are changing too quickly then the symplectic integrator used by Stan might become unstable, in which case the Hamiltonian Markov chains will not return faithful quantifications. In either case you can’t determine whether any empirically observed behaviors are actually supported by the posterior or are just numerical artifacts.
In order to determine whether your posterior contains undesired behavior because of an overly diffuse prior, and hence what kind of modifications you might need to make to your prior either explicitly (changing the prior model) or implicitly (rejecting based on functions of the parameters), you need an accurate quantification of your posterior distribution. Even then narrowing the prior doesn’t mean that Markov chains won’t encounter pathological behavior while trying to find the posterior typical set during warmup.
If you’re trying to discover neptune by fitting only stable orbits then the best approach is to build a model with only stable orbits, elliptical orbits. Then any numerical problems that arise are purely due to extreme stable orbits but nothing else. Of course this might have unintended side effects (limiting how close the bodies can approach, excluding resonant behavior, etc).
The second best approach would be to build a prior that concentrates around stable orbits but does but does not entirely exclude unstable orbits. In this case numerical problems might be due to extreme unstable orbital configurations in the typical set or even those found while searching for the typical set, or they might be due to extreme stable orbital configurations. The more complex the model the more ways it can break.
Finally the most ineffective approach is to let everything in and pump a ton of carbon into your numerical solver to try to resolve every stable and unstable orbit that the sampler might encounter, either as a configuration that’s consistent with the data or a configuration that is passed while finding those consistent configurations.