Leave-future-out cross-validation for time-series models

[avehtari]

@LoicR

I am going to use the model to predict the future only for validation purpose.

Then LFO would predict what happens in that prediction better.

@LoicR

My main goal is to infer the parameters of a grey-box model to estimate thermal properties of the true physical system.

I try to be more specific. Let’s say you have time t=1,\ldots,T, a conditional model p(y_t|f_t,\phi) and time series model p(f_1,\ldots,f_T|\theta). If you are interested in \phi, LOO is probably enough. If you are interested in \theta, LOO might be enough but LFO would be better. In your model some of the parameters probably belong to the \phi category and some to the \theta category.

@LoicR

Sorry I should have clarified this point. I come to the discourse to learn about Bayesian inference in general.

No problem, I’m happy to help in general.

@LoicR

maybe I will try both and compare if the model selection with LFO and LOO leads to different recommendations.

My guess is that they give the same order for models, but when comparing models where the difference is in the timser series model part LOO shows smaller differences than LFO.