I discuss this here:
For Bayesian logistic regression, we want to average over our uncertainty in the regression coefficients \beta. Suppose we have data y from which we get a posterior p(\beta \mid y) over regression coefficients. Now suppose we get a new predictor \tilde{x} and we want to predict the binary outcome \tilde{y}. It’s the usual posterior predictive inference formula p(\tilde{y} \mid y) (plus predictors), which averages by integrating,
\begin{array}{rcl}
\textrm{Pr}[\tilde{y} = 1 \mid \tilde{x}, y, x]
& = & \int \textrm{Bernoulli}(\textrm{logit}^{-1}(\tilde{x} \cdot \beta)) \cdot p(\beta \mid y, x) \ \textrm{d}\beta.
\\[8pt]
& \approx & \frac{1}{M} \sum_{m=1}^M \textrm{Bernoulli}(\textrm{logit}^{-1}(\tilde{x} \cdot \beta^{(m)}))
\end{array}
The second line gives the MCMC estimate, where \beta^{(m)} is the m-th draw from the posterior p(\theta \mid y, x).