That’s a good question, and I guess it depends on what you want to do. Adding a separate sigma term relaxes the assumption that all the groups share the same sigma value. Lets say you want to know the differences in Y between the groups X1, X2, etc, but you want to condition these values on Z and Phylo. Then in that case, I think you are ok doing formula = bf(Y ~ 0 + X + Z + (1|gr(phylo, cov = A)), sigma ~ 0 + X ) because you don’t really care about the individual sigma’s from Z (providing Z is a factor, not a continuous variable). Then when you calculate your differences, muX1 - muX2, and see how far they are from zero, you can also calculate the differences between sigmaX1 and sigmaX2.
Sorry, late night, I hope that made sense. If you want to learn more, check out this blog post, or look at how differences/contrasts are calculated in Ch 5 of the amazing resource Statistical Rethinking in brms.
One amazing thing about brms is knowing that the “phylo” thing is always in your back pocket. With that, it works well to ask general questions about stats in brms on this discourse, and get help from amazing people, without bringing up the phylo component, because the modeling and post-processing is always pretty much the same.