Why exactly is alpha + beta * x ~ normal(y, sigma) wrong? Is it even wrong?

This is an accident of symmetric distributions, they wind up being the same thing.
Normal distribution model:

y_i \sim \mathcal{N}(\alpha + \beta x_i, \sigma) \\ p(y_i | x_i, \alpha, \beta, \sigma) = \frac{1}{\sigma\sqrt{2\pi}} \exp(-.5\frac{(y_i - (\alpha + \beta x_i))^2}{\sigma^2}) \\

What if we switched the roles of y and \hat \mu ?

\frac{1}{\sigma\sqrt{2\pi}} \exp(-.5\frac{((\alpha + \beta x_i)-y_i)^2}{\sigma^2})

Or to make it clearer:

\frac{1}{\sigma\sqrt{2\pi}} \exp(-.5\frac{(\delta_i)^2}{\sigma^2}) \\ \frac{1}{\sigma\sqrt{2\pi}} \exp(-.5\frac{(-\delta_i)^2}{\sigma^2}) \\ \delta_i = y_i - \hat\mu_i = y_i - (\alpha + \beta x_i)

As a result, they wind up being the same, in this particular case for this particular distribution, because swapping the order of y_i and \hat\mu_i in the normal density function doesn’t matter; it evaluates to the same value.

9 Likes