First to make the terms more clear, p(y|theta) as a function of y is an observation model and p(y|theta) as a function of theta is a likelihood. waic and loo focus on (log) predictive density (in continuous case) or probability (in discrete case). Your question would make more sense if you would ask “whether WAIC (or LOOIC) can be used to compare model with different observation models ?”
- You can compare models given different discrete observation models and it’s also allowed to have different y as long as the mapping is bijective.
- You can’t mix densities and probabilities, so you can’t compare model given continuous and discrete observation models, unless you compute probabilities in intervals from the continuous model.
- You can compare models given different continuous observation models and but you have exactly the same y. If y is transformed, then the Jacobian of that transformation needs to be included.
I repeat that you need to focus on the observation model. As a bonus there is at least one case where a discrete observation model and a continuous observation model have the same continuous likelihood, so knowing the likelihood doesn’t help deciding whether the two models can be compared.