Error in LOO comparison because of different data points

Modeling
1

Hi,

  I got an error message regarding LOO estimation. After testing the LOO scores and comparing the LOO scores, I get an error message: "Error: Not all models have the same number of data points". However, I can see the individual LOO scores for each model. I guess my problem with my models is due to the variables. In the first model, I used one variable. In the next model, I included two variables. In the third model, I added five or more variables. Each variable may not match with other variables. I can see the  log-likelihood matrix is different for the three models. Is that the problem? If so, how can I rectify it.

Thanks.

#.Log-likelihood matrix from the LOO score:
Model-1: Computed from 4000 by 23268 log-likelihood matrix
Model-2: Computed from 4000 by 21668 log-likelihood matrix
Model-3: Computed from 4000 by 20168 log-likelihood matrix

#.Models: I fitted the models using the BRMS package.
model1 = brm(kgd06riv ~ icowsal, family = bernoulli(link = “logit”),
data = ics,refresh = 0, chains = 4,
iter = 2000, prior = prior)

model2 = brm(kgd06riv ~ icowsal + settle, family = bernoulli(link = “logit”),
data = ics,refresh = 0, chains = 4,
iter = 2000, prior = prior)

model3 = brm(kgd06riv ~ icowsal + settle + jtdem +
atopally + cincratio,
family = bernoulli(link = “logit”),
data = ics,refresh = 0, chains = 4,
iter = 2000, prior = prior)

2

Do you have NAs in the predictor values? Those roes are silently removed.

1 Like
3

Hi,

    Thanks for the reply. Yes, I have NAs in the predictor values. Any suggestions to get rid of it or make changes in the predictor variables.

Thanks.

4

As there are only 13% rows with missing values and if they are missing at random, quick option is to drop those rows. More elaborate approach is to do imputation of the missing values, but that increases computation time. See Handle Missing Values with brms.

1 Like