How to plot autoregressive model? And too many problematic observations LOO

Interfaces brms
1

I was trying to fit a first order auto-regressive model. There were 96 plots which has been inventoried for 10 years. I am modeling total above-ground biomass as a function of the interaction between year and diversity, to find out how diversity is affecting plot level biomass in different years. I fit the model with gamma and lognormal. In both cases I found the following warning (loo). it take days to refit the models when i put reloo is true. @avehtari

My second question is, when I plot the model with marginal effects or conditional effects, the plots look ok. However, when I try to plot it with fitted() values the confidence interval is very wide. I tried different plot names, since plot is the grouping variable in my autoregressive model. Please let me know what I am doing wrong here. @paul.buerkner

fit.cor.5<-brm(total_biomass_Mg_ha ~ year*Diversity  +(1|Block:Combination)+ar(time=year, gr=Plot, p=1, cov=TRUE),family=Gamma(link = "log"),   data=spp.biom.all.year,chains=12,cores=12,iter = 5000, control = list(adapt_delta = 0.99999, max_treedepth = 15))

fit.cor.5.loo.cor<-loo(fit.cor.5)

spp.biom.18.A.2.1<-data.frame(Diversity=c(1,2,3,4,5), year=rep(2018,5), Plot=c("x12","x1", "x14", "x17", "x11"))

plot.biomass.2018.pred.1<-fitted(fit.cor.5, newdata=spp.biom.18.A.2.1, re_formula = NA)

Warning message:

Found 766 observations with a pareto_k > 0.7 in model 'fit.cor.5'. It is recommended to set 'moment_match = TRUE' in order to perform moment matching for problematic observations. 
Computed from 30000 by 767 log-likelihood matrix

         Estimate   SE
elpd_loo  -2576.8 42.7
p_loo       657.3 17.7
looic      5153.6 85.4
------
Monte Carlo SE of elpd_loo is NA.

Pareto k diagnostic values:
                         Count Pct.    Min. n_eff
(-Inf, 0.5]   (good)       0    0.0%   <NA>      
 (0.5, 0.7]   (ok)         1    0.1%   178       
   (0.7, 1]   (bad)      725   94.5%   3         
   (1, Inf)   (very bad)  41    5.3%   0         
See help('pareto-k-diagnostic') for details.
  • Operating System: windows 10
  • brms Version: 2.16.1

EDIT: @maxbiostat edited this post for a slight reformatting.

2

This is likely because you have a parameter for each observation and when removing that obsevration the posterior for that parameter changes too much for any approximate LOO or *IC approach. See LOO package glossary — loo-glossary • loo and
Roaches cross-validation demo

reloo is doing 766 refits

If you want to do cross-validation, I recommend using k-fold-CV, e.g. with 10 so that you need only 10 refits.

3

@ahmedshovon reached out to me about the fitted() issue in a DM and I’ll answer him here. Other than the couple examples of autoregressive models I wrote about here, I have very little experience with autoregressive models in general and even less within the context of brms. More generally, though, I recommend you practice with fitting simplified versions of your model, and slowly build from there. At each stage, practice with post-processing functions like fitted() and conditional_effects() and only move forward to the next more complicated model once you fully understand the simpler ones.