How to validate model fit with unknown truncation point?

Modeling
1

Hello. The manual has a section on estimating the unknown truncation point. I actually tried that and it got me an estimate. But how do I validate it? Would I just do a posterior predictive check by simulating from the truncated distribution? Any suggestion is appreciated.

2

It depends mainly on what you mean by “validating”; do you mean it with anything specific in mind? The term is somewhat vague since you usually don’t really validate anything, except against an expectation or alternative models.

You could indeed just perform a posterior predictive check to see if anything stands out as being inconsistent (e.g. observations beyond the estimated truncation point), but you could also use model selection criteria (WAIC, LOO, etc) to compare the model performance with a fixed truncation point, for instance.

3

Thanks for your reply. I don’t have anything specific in mind. If it is any other distribution, I would simulate from the fitted distribution to see if the actual data is typical of the simulated values (posterior predictive check). I am asking whether doing the same here suffices. But I could not directly evaluate the appropriateness of the estimated truncation point, since the actual data would be below it.

4

Again it depends on what is acceptable in the field and/or how much you can adhere to best practices of inference (which to some degree depends on the field).

If there’s a gold standard model and you are trying to improve on it (e.g. ignoring truncation, or truncating based on some heuristic or summary) it could be argued that this would be the null model, and you should perform model selection against it to see if you are actually improving on it.

If there isn’t a standard/accepted model, you could come up with a null model that allows a “fair” comparison (which is also subjective). If all you are trying to do is show that the inference doesn’t return anything crazy, i.e. it actually fits the data, a posterior predictive check may be enough. Having an estimated truncation point is in principle no different from any other estimated parameter, so it shouldn’t require any different approaches.

1 Like