I ran into a similar issue sometimes with a similar model. I think what’s happening is the default brms priors are too loose and let the phylogenetic effect get really large and perfectly fit every observation. This lets the log-likelihood get really high and causes the Inf log-likelihoods that loo() is erroring on.
See this thread, including the functions I used to solve it near the end. I ended up using the integrated loo approach shown in section 5 of the roaches demo, though I had to evaluate the integral in R and take some additional steps to make sure it was numerically precise.