Feedback on bounds (included pairs plot)

Modeling
1

Hi,

I have a quick question about my output in regards to bounded parameters (I apologise for asking quite a few questions lately).

Background: I have a hierarchical choice model (see attached at the end) where individual-level parameters are a function of priors (means and sds). I have sampled with the model and it passes all computational tests, including accurately retrieving simulated data. However, I’ve noticed that the model hits the upper bound of my noise prior mean. This noise mean upper bound was informed by prior research and a simpler simulation using maximum likelihood.

Is a prior parameter’s mean hitting the upper bound a problem?

Keep in mind, my individual level parameters do not have an upper bound, just the priors as I’d like to impose some constraint on this.)

To visualise the problem I’ve attached a pairs plot for the priors.

pairs
pairs2568×1396 424 KB

As always, I appreciate your help.

Cheers,

Alex

***

Here is my model in case it’s useful

data {
  int N; // number of trials total (across participants), integer
  int nsubj;  // number of subjects,  
  int choices[N]; // the binary choice vector
  real <lower=0> x1a[N];
  real <lower=0> x2a[N];
  real <lower=0> x3a[N];
  real <lower=0> x4a[N];
  real <lower=0> x1b[N];
  real <lower=0> x2b[N];
  real <lower=0> x3b[N];
  real <lower=0> x4b[N];
  real <lower=0> p1a[N];
  real <lower=0> p2a[N];
  real <lower=0> p3a[N];
  real <lower=0> p4a[N];
  real <lower=0> p1b[N];
  real <lower=0> p2b[N];
  real <lower=0> p3b[N];
  real <lower=0> p4b[N];
  int sid[N]; 
}

parameters {
  // Group-level:
  real<lower=-2.30, upper=3.05> meannoise; // assuming mean is 0.1 to 21 - as in ca. noise estimates from Stata model with n=853
  real<lower=0, upper=1.55> sdnoise; // according sds  - calculated as sd = (b - a) / sqrt(12)
  real<lower=-2.3, upper=2.3> meanalpha; // assuming mean is 0.1 to 10 
  real<lower=0, upper=1.33> sdalpha;
  real<lower=-2.3, upper=3.76> meanM; // assuming mean is 0.1 to 43
  real<lower=0, upper=1.75> sdM;
  
  // Individual-level:
  real<lower=0.1> noise[nsubj]; // Noise, constrained it to be above [0.1, INF]
  real<lower=0.1> alpha[nsubj]; // alpha, constrained it to be above [0.1, INF]
  real<lower=0.1> M[nsubj];  // Reward Expectation, constrained it to be [0.1, INF)
  
}

model {
  real ua; // utility of the option a
  real ub; // utility of the option b
  
  // Group-level parameters:
  meannoise ~ uniform(-2.30,3.05); // assuming mean is 0.1 to 21
  sdnoise ~ uniform(0,1.55); // according sds
  meanalpha ~ uniform(-2.3,2.3); // assuming mean is 0.1 to 10
  sdalpha ~ uniform(0,1.33);
  meanM ~ uniform(-2.3, 3.76); // assuming mean is 0.1 to 43
  sdM ~ uniform(0,1.75); 
  
  // Individual-level parameters: 
  noise ~ lognormal(meannoise, sdnoise);
  alpha ~ lognormal(meanalpha, sdalpha);
  M ~ lognormal(meanM, sdM);
    
  for (i in 1:N) {

      int t = sid[i];
      
            
      ua=0.0;
      ua += p1a[i]*((x1a[i]^alpha[t])/((x1a[i]^alpha[t])+(M[t]^alpha[t])));
      ua += p2a[i]*((x2a[i]^alpha[t])/((x2a[i]^alpha[t])+(M[t]^alpha[t])));
      ua += p3a[i]*((x3a[i]^alpha[t])/((x3a[i]^alpha[t])+(M[t]^alpha[t])));
      ua += p4a[i]*((x4a[i]^alpha[t])/((x4a[i]^alpha[t])+(M[t]^alpha[t])));
      
      ub=0.0;
      ub += p1b[i]*((x1b[i]^alpha[t])/((x1b[i]^alpha[t])+(M[t]^alpha[t])));
      ub += p2b[i]*((x2b[i]^alpha[t])/((x2b[i]^alpha[t])+(M[t]^alpha[t])));
      ub += p3b[i]*((x3b[i]^alpha[t])/((x3b[i]^alpha[t])+(M[t]^alpha[t])));
      ub += p4b[i]*((x4b[i]^alpha[t])/((x4b[i]^alpha[t])+(M[t]^alpha[t])));

      choices[i] ~ bernoulli_logit(((ua-ub)*noise[t]));
      }
}
2

Hi, Alex,

Hard constraints are discouraged because they tend to be unnecessarily strict. Even if previous analyses strongly suggest a parameter is within a specific interval, this interval may not have the exact numbers you specified at first. It is better to allow your new data update the intervals.

The plots you show suggest that your posterior distribution might include values of meannoise higher than 3.05. If so, your current values might be biased.

From the prior choice recommendations:

Don’t use uniform priors, or hard constraints more generally, unless the bounds represent true constraints (such as scale parameters being restricted to be positive, or correlations restricted to being between -1 and 1). Some examples:

  • You think a parameter could be anywhere from 0 to 1, so you set the prior to uniform(0,1). Try normal(.5,.5) instead.
1 Like
3

If you actually have prior certainty that the true value isn’t larger than your supplied prior upper bound, then it’s not necessarily a problem (but see below). Otherwise, it’s a big problem. The data are telling you that they don’t think larger values for this parameter are unlikely. Without actually relaxing this prior constraint and checking, you can’t know whether the data say that the likely values are a little bit higher, or potentially wildly higher.

If you do view values larger than this upper bound as vanishingly unlikely, it would be a good idea to examine your model closely for bugs or misspecification.

1 Like
4

Thank you jsocolar and prix_tp for you comments.

I will try to run the model again with a normal distribution for the noise prior or with a much higher upper bound, such as (0.1, 200).

Given I am not necessarily after exact estimates but looking at the association between my parameter estimates and another (non-estimated) variable, the chance that there is some bias can likely be accepted as long as the bias is systematic across individuals - I’ll include sensitivity analysis across different noise specification to highlight any bias.

As for the distribution, I’ve used a uniform distribution for prior because this was suggested in prior literature when using lognormal distribution of individual level pars. I’ve had some issues with the normal before for this model/data. However, I can try a bounded normal and centre it on my expected noise term - something like:

real<lower=0.1> meannoise;
real<lower=0.1> sdnoise;

...

meannoise ~ normal(3.05,2.3); // lower bound 0.1, normal centred with mean on 21 with sd = 10 
sdnoise ~ normal(0.5,0.5); // lower bound 0.1, with mean on 1.64 with sd = 1.64
5

I’d be careful about specifying a prior with such a high bound as well. You probably know that a lot of that range is not at all plausible. Very wide priors aren’t neutral or non-informative, in particular if you don’t have lots of data. Prior predictive checks are a good way to inspect the implications of your priors, so that might be useful here.

1 Like
6

Thank you erognil, that is helpful as I did not know that about large prior bounds.