In general, Bayesian Neural Networks have bias correction which may be different for each input or for each neuron. The math is in Radford Neal’s thesis A.1 - A.3. His formulation is very general and I just ignored some terms. I just added MPI stuff to make it faster.
Shards represent number of MPI enabled threads which may be executed in sequence or in parallel based on available resources. Intuitively they represent the # of groups data and parameters are split. I borrowed this name from Linear, parallell regression. Node has some number of cores. I think (I may be wrong) you want so many cores as there are shards. So sometimes you have to get few nodes. Sorry, I am not expert on this…