Using an offset with a categorical model

Hi!

I’m fitting a foraging habitat selection model with a categorical family such as:

mod <- brm(Habitat|weights(w) ~ Sex + (1|p|Individual), data=dt, family = categorical())

Habitat: is 6 habitat categories
w: is a foraging effort weight that range from 0 to 1
Sex: “M” and “F”
Individual “id”

When modeling habitat selection, you must account for the availability of the habitats to the each animal in the environment. To do so, it was suggested to add an offset to the model of the proportion of each habitat in the environment (Kneib, Knauer, Küchenhoff - 2011 - Environmental and Ecological Statistics.pdf (822.3 KB) ). Which I did such as:

mod <- brm(Habitat|weights(w) ~ Sex + offset(Availability) + (1|p|Individual), data=dt, family = categorical())

Where Availability is a proportion ranging between 0 and 1.

Unfortunately, the results of the model with and without the offset don’t change at all, which is surprising as some habitats are in average way more available than others.

Am I specifying correctly the offset? I tried to transform the availability offset with a log and a logit link but the results are again very similar.

Any help?

Thanks!

  • Operating System: windows 10
  • brms Version: 2.13.0
1 Like

You you please show the results of make_stancode (or stancode) of your model? When I run a categorical model with offset, it correctly appears in the stan code.

Below is the model results:

 Family: categorical 
  Links: muAS = logit; muICE = logit; muSACCZ = logit; muNACCZ = logit; muNSAF = logit 
Formula: Habitat | weights(rht_w) ~ Sex + Length_cm + Length_cm:Sex + Reactiveness + Aggressiveness + Reactiveness:Aggressiveness + Reactiveness:Sex + Aggressiveness:Sex + Reactiveness:Aggressiveness:Sex + offset(Availability) + (1 | p | Id) 
   Data: dt_samp (Number of observations: 5507) 
Samples: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
         total post-warmup samples = 4000

Group-Level Effects: 
~Id (Number of levels: 70) 
                                         Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(muAS_Intercept)                           4.43      0.82     3.13     6.32 1.00     1883     2649
sd(muICE_Intercept)                          4.49      0.85     3.17     6.40 1.00     2182     2463
sd(muSACCZ_Intercept)                        3.27      0.41     2.57     4.16 1.00     1930     2647
sd(muNACCZ_Intercept)                        2.62      0.33     2.04     3.36 1.00     2873     2969
sd(muNSAF_Intercept)                         6.58      2.00     3.69    11.28 1.00     1509     1484
cor(muAS_Intercept,muICE_Intercept)          0.89      0.05     0.76     0.97 1.00     1887     2112
cor(muAS_Intercept,muSACCZ_Intercept)        0.76      0.08     0.57     0.89 1.00      883     1916
cor(muICE_Intercept,muSACCZ_Intercept)       0.78      0.08     0.58     0.90 1.00      792     1945
cor(muAS_Intercept,muNACCZ_Intercept)        0.40      0.14     0.10     0.66 1.00      852     1967
cor(muICE_Intercept,muNACCZ_Intercept)       0.44      0.14     0.15     0.68 1.00      847     2120
cor(muSACCZ_Intercept,muNACCZ_Intercept)     0.76      0.07     0.60     0.87 1.00     2820     3149
cor(muAS_Intercept,muNSAF_Intercept)        -0.04      0.22    -0.48     0.37 1.00     1395     2498
cor(muICE_Intercept,muNSAF_Intercept)        0.03      0.21    -0.39     0.44 1.00     1900     3105
cor(muSACCZ_Intercept,muNSAF_Intercept)      0.38      0.18    -0.02     0.69 1.00     3348     3637
cor(muNACCZ_Intercept,muNSAF_Intercept)      0.70      0.15     0.34     0.93 1.00     2672     3153

Population-Level Effects: 
                                         Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
muAS_Intercept                              -6.64      1.75   -10.62    -3.66 1.00     1549     1957
muICE_Intercept                             -4.68      1.44    -7.76    -2.15 1.00     1387     2167
muSACCZ_Intercept                            1.86      0.66     0.57     3.16 1.00     1983     2541
muNACCZ_Intercept                            2.08      0.55     1.04     3.15 1.00     2204     2920
muNSAF_Intercept                            -5.40      2.35   -11.01    -1.87 1.00     1987     1834
muAS_SexM                                    4.54      1.80     1.26     8.47 1.00     1458     2108
muAS_Length_cm                              -0.71      1.57    -3.86     2.39 1.00     2045     2237
muAS_Reactiveness                            0.53      1.71    -2.90     3.81 1.00     2066     2430
muAS_Aggressiveness                         -0.42      1.66    -3.83     2.84 1.00     1379     2080
muAS_SexM:Length_cm                          1.11      1.76    -2.32     4.63 1.00     1575     2137
muAS_Reactiveness:Aggressiveness             0.47      1.57    -2.49     3.87 1.00     1541     2128
muAS_SexM:Reactiveness                      -0.13      1.94    -3.90     3.76 1.00     1739     2517
muAS_SexM:Aggressiveness                    -0.73      2.29    -5.23     3.86 1.01     1228     1947
muAS_SexM:Reactiveness:Aggressiveness        0.24      2.12    -4.08     4.31 1.00     1410     1920
muICE_SexM                                   2.17      1.54    -0.72     5.42 1.00     1274     1885
muICE_Length_cm                              1.94      1.38    -0.81     4.77 1.00     2239     2490
muICE_Reactiveness                           0.58      1.33    -2.08     3.17 1.00     2079     2205
muICE_Aggressiveness                         0.06      1.56    -3.14     2.98 1.00     1287     1985
muICE_SexM:Length_cm                        -0.89      1.58    -3.96     2.27 1.00     1781     2210
muICE_Reactiveness:Aggressiveness            2.25      1.70    -0.80     5.76 1.00     1625     2037
muICE_SexM:Reactiveness                     -1.21      1.62    -4.38     2.01 1.00     1801     2156
muICE_SexM:Aggressiveness                   -1.23      2.21    -5.53     3.16 1.01     1099     1687
muICE_SexM:Reactiveness:Aggressiveness      -0.96      2.18    -5.25     3.29 1.00     1413     2149
muSACCZ_SexM                                -2.46      0.90    -4.24    -0.74 1.00     1709     2026
muSACCZ_Length_cm                            0.13      0.75    -1.30     1.64 1.00     1616     2296
muSACCZ_Reactiveness                        -0.42      0.72    -1.83     1.00 1.00     1741     2317
muSACCZ_Aggressiveness                       1.33      0.83    -0.21     3.01 1.00     1047     2122
muSACCZ_SexM:Length_cm                      -0.65      0.92    -2.53     1.15 1.00     1432     2164
muSACCZ_Reactiveness:Aggressiveness          0.22      0.72    -1.15     1.65 1.00     1435     2233
muSACCZ_SexM:Reactiveness                    0.31      0.97    -1.62     2.22 1.00     1579     2194
muSACCZ_SexM:Aggressiveness                 -2.13      1.34    -4.71     0.56 1.01      996     1948
muSACCZ_SexM:Reactiveness:Aggressiveness     0.73      1.17    -1.64     2.96 1.00     1372     2092
muNACCZ_SexM                                -3.44      0.73    -4.94    -2.00 1.00     2205     2735
muNACCZ_Length_cm                            0.16      0.63    -1.07     1.45 1.00     1721     2426
muNACCZ_Reactiveness                        -0.30      0.58    -1.42     0.80 1.00     1672     2636
muNACCZ_Aggressiveness                       1.77      0.68     0.45     3.11 1.00     1504     2418
muNACCZ_SexM:Length_cm                      -0.71      0.78    -2.28     0.81 1.00     1656     2312
muNACCZ_Reactiveness:Aggressiveness         -0.83      0.51    -1.84     0.16 1.00     1338     2306
muNACCZ_SexM:Reactiveness                   -0.09      0.79    -1.67     1.45 1.00     1875     2650
muNACCZ_SexM:Aggressiveness                 -2.95      1.09    -5.14    -0.82 1.00     1312     2770
muNACCZ_SexM:Reactiveness:Aggressiveness     1.71      0.92    -0.13     3.54 1.00     1441     2288
muNSAF_SexM                                 -7.79      3.73   -16.97    -2.17 1.00     1720     1410
muNSAF_Length_cm                            -0.90      1.92    -4.84     2.73 1.00     1990     2058
muNSAF_Reactiveness                         -1.94      1.67    -5.71     0.96 1.00     1580     1559
muNSAF_Aggressiveness                        3.75      2.29    -0.18     8.86 1.00     1603     1578
muNSAF_SexM:Length_cm                        3.99      3.21    -1.38    11.16 1.00     1604     1557
muNSAF_Reactiveness:Aggressiveness          -2.67      1.59    -6.35    -0.01 1.00     1459     1342
muNSAF_SexM:Reactiveness                    -1.40      2.78    -7.31     3.82 1.00     2112     2560
muNSAF_SexM:Aggressiveness                  -1.95      4.89   -11.72     8.30 1.00     2256     2349
muNSAF_SexM:Reactiveness:Aggressiveness      3.56      4.20    -4.26    13.06 1.00     2321     2082

Samples were drawn using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).

And here is the stancode:

// generated with brms 2.13.0
functions {
}
data {
  int<lower=1> N;  // number of observations
  int<lower=2> ncat;  // number of categories
  int Y[N];  // response variable
  vector<lower=0>[N] weights;  // model weights
  int<lower=1> K_muAS;  // number of population-level effects
  matrix[N, K_muAS] X_muAS;  // population-level design matrix
  vector[N] offsets_muAS;
  int<lower=1> K_muICE;  // number of population-level effects
  matrix[N, K_muICE] X_muICE;  // population-level design matrix
  vector[N] offsets_muICE;
  int<lower=1> K_muSACCZ;  // number of population-level effects
  matrix[N, K_muSACCZ] X_muSACCZ;  // population-level design matrix
  vector[N] offsets_muSACCZ;
  int<lower=1> K_muNACCZ;  // number of population-level effects
  matrix[N, K_muNACCZ] X_muNACCZ;  // population-level design matrix
  vector[N] offsets_muNACCZ;
  int<lower=1> K_muNSAF;  // number of population-level effects
  matrix[N, K_muNSAF] X_muNSAF;  // population-level design matrix
  vector[N] offsets_muNSAF;
  // data for group-level effects of ID 1
  int<lower=1> N_1;  // number of grouping levels
  int<lower=1> M_1;  // number of coefficients per level
  int<lower=1> J_1[N];  // grouping indicator per observation
  // group-level predictor values
  vector[N] Z_1_muAS_1;
  vector[N] Z_1_muICE_2;
  vector[N] Z_1_muSACCZ_3;
  vector[N] Z_1_muNACCZ_4;
  vector[N] Z_1_muNSAF_5;
  int<lower=1> NC_1;  // number of group-level correlations
  int prior_only;  // should the likelihood be ignored?
}
transformed data {
  int Kc_muAS = K_muAS - 1;
  matrix[N, Kc_muAS] Xc_muAS;  // centered version of X_muAS without an intercept
  vector[Kc_muAS] means_X_muAS;  // column means of X_muAS before centering
  int Kc_muICE = K_muICE - 1;
  matrix[N, Kc_muICE] Xc_muICE;  // centered version of X_muICE without an intercept
  vector[Kc_muICE] means_X_muICE;  // column means of X_muICE before centering
  int Kc_muSACCZ = K_muSACCZ - 1;
  matrix[N, Kc_muSACCZ] Xc_muSACCZ;  // centered version of X_muSACCZ without an intercept
  vector[Kc_muSACCZ] means_X_muSACCZ;  // column means of X_muSACCZ before centering
  int Kc_muNACCZ = K_muNACCZ - 1;
  matrix[N, Kc_muNACCZ] Xc_muNACCZ;  // centered version of X_muNACCZ without an intercept
  vector[Kc_muNACCZ] means_X_muNACCZ;  // column means of X_muNACCZ before centering
  int Kc_muNSAF = K_muNSAF - 1;
  matrix[N, Kc_muNSAF] Xc_muNSAF;  // centered version of X_muNSAF without an intercept
  vector[Kc_muNSAF] means_X_muNSAF;  // column means of X_muNSAF before centering
  for (i in 2:K_muAS) {
    means_X_muAS[i - 1] = mean(X_muAS[, i]);
    Xc_muAS[, i - 1] = X_muAS[, i] - means_X_muAS[i - 1];
  }
  for (i in 2:K_muICE) {
    means_X_muICE[i - 1] = mean(X_muICE[, i]);
    Xc_muICE[, i - 1] = X_muICE[, i] - means_X_muICE[i - 1];
  }
  for (i in 2:K_muSACCZ) {
    means_X_muSACCZ[i - 1] = mean(X_muSACCZ[, i]);
    Xc_muSACCZ[, i - 1] = X_muSACCZ[, i] - means_X_muSACCZ[i - 1];
  }
  for (i in 2:K_muNACCZ) {
    means_X_muNACCZ[i - 1] = mean(X_muNACCZ[, i]);
    Xc_muNACCZ[, i - 1] = X_muNACCZ[, i] - means_X_muNACCZ[i - 1];
  }
  for (i in 2:K_muNSAF) {
    means_X_muNSAF[i - 1] = mean(X_muNSAF[, i]);
    Xc_muNSAF[, i - 1] = X_muNSAF[, i] - means_X_muNSAF[i - 1];
  }
}
parameters {
  vector[Kc_muAS] b_muAS;  // population-level effects
  real Intercept_muAS;  // temporary intercept for centered predictors
  vector[Kc_muICE] b_muICE;  // population-level effects
  real Intercept_muICE;  // temporary intercept for centered predictors
  vector[Kc_muSACCZ] b_muSACCZ;  // population-level effects
  real Intercept_muSACCZ;  // temporary intercept for centered predictors
  vector[Kc_muNACCZ] b_muNACCZ;  // population-level effects
  real Intercept_muNACCZ;  // temporary intercept for centered predictors
  vector[Kc_muNSAF] b_muNSAF;  // population-level effects
  real Intercept_muNSAF;  // temporary intercept for centered predictors
  vector<lower=0>[M_1] sd_1;  // group-level standard deviations
  matrix[M_1, N_1] z_1;  // standardized group-level effects
  cholesky_factor_corr[M_1] L_1;  // cholesky factor of correlation matrix
}
transformed parameters {
  matrix[N_1, M_1] r_1;  // actual group-level effects
  // using vectors speeds up indexing in loops
  vector[N_1] r_1_muAS_1;
  vector[N_1] r_1_muICE_2;
  vector[N_1] r_1_muSACCZ_3;
  vector[N_1] r_1_muNACCZ_4;
  vector[N_1] r_1_muNSAF_5;
  // compute actual group-level effects
  r_1 = (diag_pre_multiply(sd_1, L_1) * z_1)';
  r_1_muAS_1 = r_1[, 1];
  r_1_muICE_2 = r_1[, 2];
  r_1_muSACCZ_3 = r_1[, 3];
  r_1_muNACCZ_4 = r_1[, 4];
  r_1_muNSAF_5 = r_1[, 5];
}
model {
  // initialize linear predictor term
  vector[N] muAS = Intercept_muAS + Xc_muAS * b_muAS + offsets_muAS;
  // initialize linear predictor term
  vector[N] muICE = Intercept_muICE + Xc_muICE * b_muICE + offsets_muICE;
  // initialize linear predictor term
  vector[N] muSACCZ = Intercept_muSACCZ + Xc_muSACCZ * b_muSACCZ + offsets_muSACCZ;
  // initialize linear predictor term
  vector[N] muNACCZ = Intercept_muNACCZ + Xc_muNACCZ * b_muNACCZ + offsets_muNACCZ;
  // initialize linear predictor term
  vector[N] muNSAF = Intercept_muNSAF + Xc_muNSAF * b_muNSAF + offsets_muNSAF;
  // linear predictor matrix
  vector[ncat] mu[N];
  for (n in 1:N) {
    // add more terms to the linear predictor
    muAS[n] += r_1_muAS_1[J_1[n]] * Z_1_muAS_1[n];
  }
  for (n in 1:N) {
    // add more terms to the linear predictor
    muICE[n] += r_1_muICE_2[J_1[n]] * Z_1_muICE_2[n];
  }
  for (n in 1:N) {
    // add more terms to the linear predictor
    muSACCZ[n] += r_1_muSACCZ_3[J_1[n]] * Z_1_muSACCZ_3[n];
  }
  for (n in 1:N) {
    // add more terms to the linear predictor
    muNACCZ[n] += r_1_muNACCZ_4[J_1[n]] * Z_1_muNACCZ_4[n];
  }
  for (n in 1:N) {
    // add more terms to the linear predictor
    muNSAF[n] += r_1_muNSAF_5[J_1[n]] * Z_1_muNSAF_5[n];
  }
  for (n in 1:N) {
    mu[n] = [0, muAS[n], muICE[n], muSACCZ[n], muNACCZ[n], muNSAF[n]]';
  }
  // priors including all constants
  target += student_t_lpdf(Intercept_muAS | 3, 0, 2.5);
  target += student_t_lpdf(Intercept_muICE | 3, 0, 2.5);
  target += student_t_lpdf(Intercept_muSACCZ | 3, 0, 2.5);
  target += student_t_lpdf(Intercept_muNACCZ | 3, 0, 2.5);
  target += student_t_lpdf(Intercept_muNSAF | 3, 0, 2.5);
  target += student_t_lpdf(sd_1 | 3, 0, 2.5)
    - 5 * student_t_lccdf(0 | 3, 0, 2.5);
  target += std_normal_lpdf(to_vector(z_1));
  target += lkj_corr_cholesky_lpdf(L_1 | 1);
  // likelihood including all constants
  if (!prior_only) {
    for (n in 1:N) {
      target += weights[n] * (categorical_logit_lpmf(Y[n] | mu[n]));
    }
  }
}
generated quantities {
  // actual population-level intercept
  real b_muAS_Intercept = Intercept_muAS - dot_product(means_X_muAS, b_muAS);
  // actual population-level intercept
  real b_muICE_Intercept = Intercept_muICE - dot_product(means_X_muICE, b_muICE);
  // actual population-level intercept
  real b_muSACCZ_Intercept = Intercept_muSACCZ - dot_product(means_X_muSACCZ, b_muSACCZ);
  // actual population-level intercept
  real b_muNACCZ_Intercept = Intercept_muNACCZ - dot_product(means_X_muNACCZ, b_muNACCZ);
  // actual population-level intercept
  real b_muNSAF_Intercept = Intercept_muNSAF - dot_product(means_X_muNSAF, b_muNSAF);
  // compute group-level correlations
  corr_matrix[M_1] Cor_1 = multiply_lower_tri_self_transpose(L_1);
  vector<lower=-1,upper=1>[NC_1] cor_1;
  // additionally draw samples from priors
  real prior_Intercept_muAS = student_t_rng(3,0,2.5);
  real prior_Intercept_muICE = student_t_rng(3,0,2.5);
  real prior_Intercept_muSACCZ = student_t_rng(3,0,2.5);
  real prior_Intercept_muNACCZ = student_t_rng(3,0,2.5);
  real prior_Intercept_muNSAF = student_t_rng(3,0,2.5);
  real prior_sd_1 = student_t_rng(3,0,2.5);
  real prior_cor_1 = lkj_corr_rng(M_1,1)[1, 2];
  // extract upper diagonal of correlation matrix
  for (k in 1:M_1) {
    for (j in 1:(k - 1)) {
      cor_1[choose(k - 1, 2) + j] = Cor_1[j, k];
    }
  }
  // use rejection sampling for truncated priors
  while (prior_sd_1 < 0) {
    prior_sd_1 = student_t_rng(3,0,2.5);
  }
}

The offset is indeed specified in the stancode. But I don’t see any difference between a model with offset and one without. Is it because the offset variable is a proportion (between 0 and 1)?

Thanks!

So you are using proportions as offsets on the logit scale? If the differences in the proportions are quite small, it is very plausible that they don’t matter much.

Yes I’m using a proportion to represent the availability of each habitat. The differences are not small at all and this is why I’m surprised. My guess is that I should transform the offset to another scale but I can’t really figure out what would be an appropriate transformation that would fit the model. I tried log and logit and none of them seem to produce results that make sense.
Thanks!

I restructured my data to be able to fit a model with multinomial() family. As the response variable is a table of counts (where each column is a category and each row is an individual), how can I specify an offset to each column and row of the response variable?

  bsformula <- brmsformula(counts | trials(total) ~ 
                         + Length_cm*Sex
                         + Reactiveness*Aggressiveness*Sex
                         + (1|Id))

Actually, In order to properly set the offset I need to know how the offsets_muAS (offsets_muICE, offsets_muSACCZ, etc.) in the stan code are calculated:

ex:
vector[N] muAS = Intercept_muAS + Xc_muAS * b_muAS + offsets_muAS;

Should the offset value be a probability or should I manually transform it into a log odds ratio (according to the category reference)?

Any help would be very appreciated, thanks a lot!!!

The are literally what you put into the offset function in the formula.

You can also see their values by checking out the output of make_standata.

Actually, I think I figured out what was my problem.

First, I had to compute the offsets as log odds ratios (relative to the reference category).

Odds = p / (1-p)
log Odds Ratios = log(Odds_{cat}/Odds_{catref})

Second, when visualizing the results with the conditional_effects() function, I had to set all offsets to 0 in the conditions attribute. That way predicted values take into account the availability of the habitats and return the results if like all habitats had the same availability value.

I hope this is right. Thanks so much for the help!!!