Further info for posterity.
It turns out unsurprisingly that randomly selected RBF centers don't work that well, some of them inevitably are very close to each other, and this causes numerical issues, and then other areas have basically no points, and this causes lack of control for that region. In general, the data is pretty noisy, so the RBF doesn't really remove a lot of the variation.
What I did was plot all the region centers, and then manually selected 18 or 20 locations spread around the map. This stabilized the RBF much more and now I get inference with about 50 warmups and 120 total samples that are quite reasonable in terms of traceplots etc.
In addition I set the error term in the regression to be t distributed on the parameter, so that the regional cost multiplier is RBF(x) + t_error(dof,0,scale) with unknown dof and scale. This t distribution is much lighter tailed than on the data, with something between 10 and 20 degrees of freedom, unlike the dof for the individual data, which winds up at about 2.3
With all these put into place, I get some fairly stable inferences with no warnings about divergent transitions, and this gives a nice plot of costs as a function of space using alpha for overplotting and a variously saturated red color as expenses climb into the 1.5-3 x the national average range.
Fitting a spatial function to noisy and irregularly spaced census data using full Bayes is no easy task!