Sorry for taking too long to respond.
I think that - depending on how exactly the invertibility affects the rest of the model - the direct approach might be problematic. Quite possibly, the posterior would have discontinuities around say two rows being a multiple of each other (but that is just a wild guess).
My linear algebra is not good enough to be able to help you directly, so just some hints that might let you figure out a good parametrization which would enforce the constraints you care about:
- The Stan manual describes how we achieve that a correlation matrix is valid: https://mc-stan.org/docs/2_24/reference-manual/correlation-matrix-transform-section.html
- Maybe you can have a stronger constraint than invertibility that would be easier to enforce. There was some discussion of parametrizing orthogonal matrices e.g. at: How to constrain a matrix comprising several parameters to be an orthogonal matrix (my undestanding is that it unfortunately is hard).
- What you speak about seems to be similar to the ICAR model (e.g. https://mc-stan.org/users/documentation/case-studies/icar_stan.html) maybe there are some insights to be gained there?
Best of luck with your model!