A tail ESS warning against a local level model with an ordered logistic function

Modeling
, , ,
1

Hi,
I’m encountering the following tail ESS warning. Other warnings disappeared after I had tried several changes in the codes (e.g. setting weakly informative priors), but still this warning remains. Does anyone know any solutions? I attached the R and STAN codes with accompanied data. Thank you for your cooperation.

Warning message:
Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
Running the chains for more iterations may help. See
Runtime warnings and convergence problems

STAN code

data {
  int<lower=1> N;
  int<lower=1> T;
  int<lower=1> T_pred;
  int<lower=1,upper=11> HA[N,T];
}

parameters {
  ordered[10] cutpoints;
  matrix[N,T] mu_raw; 
  matrix[N,T] phi_raw; 
  real mu_ori; 
  real<lower=0> s_mu; 
  real<lower=0> s_phi; 
  real<lower=0> s_ori; 
}

transformed parameters{
  matrix[N,T] mu; 
  matrix[N,T] phi; 
  mu[,1] = mu_ori + s_ori*mu_raw[,1] ; 
  for (t in 2:T)
      mu[,t] = mu[,t-1] + s_mu*mu_raw[,t]; 
  for (t in 1:T)
      phi[,t] = mu[,t] + s_phi*phi_raw[,t]; 
}

model {
  mu_ori ~ normal(0,1.5); 
  s_mu ~ exponential(3); 
  s_phi ~ lognormal(-0.5,0.5); 
  s_ori ~ lognormal(0,0.5); 
  for (t in 1:T){
      mu_raw[,t] ~ normal(0,1); 
      phi_raw[,t] ~ normal(0,1); 
      HA[,t] ~ ordered_logistic(phi[,t], cutpoints);
    }
}

generated quantities {
  matrix[N,T+T_pred] mu_all;
  matrix[N,T+T_pred] phi_all;
  int<lower=1,upper=11> ha_all[N,T+T_pred];
  matrix[N,T] log_lik;
  for (t in 1:T)
    for (n in 1:N){
      mu_all[n,t] = mu[n,t];
      phi_all[n,t] = phi[n,t];
      log_lik[n,t] = ordered_logistic_lpmf(HA[n,t]|phi[n,t], cutpoints); 
    }
  for (t in 1:T_pred)
    for (n in 1:N){
      mu_all[n,T+t] = normal_rng(mu_all[n,T+t-1], s_mu);
      phi_all[n,T+t] = normal_rng(mu_all[n,T+t], s_phi);
    }
  for (t in 1:T+T_pred)
    for (n in 1:N)
      ha_all[n,t] = ordered_logistic_rng(phi_all[n,t], cutpoints);
}

R code

library(rstan); library(tidyverse); library(rlist); library(ggmcmc);library(bayesplot)
library(magrittr); library(pipeR); library(Hmisc); library(readxl); library(caret);
#rstan_options(auto_write=TRUE)
#options(mc.cores=parallel::detectCores())
d <- read.csv(file='d.csv', header = TRUE)

year <- 2
data <- list(N=nrow(d),
             T=year,
             T_pred=3,
             HA=d)

stanmodel <- stan_model(file='test.stan')
fit <- sampling(
  stanmodel,
  data = data,
  seed =10,
  chains=4, iter=6000, warmup=1000, thin=5,
  init=function(){
    list(cutpoints=c(-6,-5,-4,-3,-2,-1,0,1,2,3),
         mu_raw=matrix(rnorm(nrow(d)*year, mean = 0, sd = 1), nrow = nrow(d), ncol = year),
         phi_raw=matrix(rnorm(nrow(d)*year, mean = 0, sd = 1), nrow = nrow(d), ncol = year),
         mu_ori=0,s_mu=0.1,s_phi=1,s_ori=2)
  }
)

print(fit, pars=c("s_mu", "s_phi", "s_ori"))
pairs(fit, pars=c("s_mu", "s_phi", "s_ori"))
mcmc_combo(fit, pars=c("s_mu", "s_phi", "s_ori"))
monitor(rstan::extract(fit, permuted = FALSE, inc_warmup = FALSE, 
                       pars=c("s_mu", "s_phi", "s_ori")))

test.stan (1.4 KB)
test.R (1.1 KB)
d.csv (4.8 KB)

2

For which parameter? It seems your model is highly overparameterized, and it is likely that the posterior has thick tails that are not well explored by the sampling

1 Like
3

The tail ESS for s_phi is 461. Other scale parameters, s_mu and s_ori, also have the low tail ESSs.

          Q5  Q50  Q95  Mean   SD  Rhat Bulk_ESS Tail_ESS
   s_mu  0.0  0.1  0.4   0.2  0.1     1      984     1050
   s_phi 0.2  0.4  0.7   0.4  0.2     1      572      461
   s_ori 2.8  3.0  3.2   3.0  0.1     1      920      948
4

Hi, I was on vacation. Tail_ESS > 400 is fine for the diagnostics. You can separately compute MCSE for the tail quantiles you are interested in to check whether you have sufficient accuracy for your purposes.

1 Like
5

Thank you, and sorry for my late reply.
I got a better understanding about tail ESSs for your comment.

1 Like