Could you say more about the use case? The main one we've considered is that we can compile an object file that could be linked to different models. By requiring the template parmameter, the class type indeed needs to be resolved at compile time.
Suppose we wanted to compile the Stan algorithms (L-BFGS, HMC, ADVI, diagnosis) once ahead of time. Then we'd compile a model when we got it. In order to link those, we need some interface that doesn't depend on the algorithms being instantiated with the model class they work on. I don't think a base class would be too much work or induce too much overhead. We'd just need to define it.
We may be forced to confront this. I'd very much like to work with @seantalts and @syclik on cutting down compile times. I don't see any reason in principle why we can't precompile our math and matrix library. We'd still need a header-only density library because of the crazy number of possible instantiations. Or we'd need to figure out a way to have those resolve at runtime---I don't think the dispatch would be that bad.