How to accelerate my model

Modeling
1

I’d like to inquire about how to accelerate the sampling process in Stan for my multi-layer hierarchical model. Let me briefly describe it: In the first layer, I draw n values from a beta distribution. In the second layer, I perform a variable transformation to convert these n variables into (alpha, beta) pairs. Then, I apply the betabinomial model (alpha_n, beta_n, times) to these transformed values. In simple terms, I want to use sampling to find the hierarchical set of n parameters. However, currently, when simulating the model, I can only use 2.5% of n (500/20000) to make it computationally feasible. I’d appreciate any advice on methods to speed up the process. Below is the code I’m using. Thank you all for your help.

The math form of my model:
\mu_i\sim Beta(\alpha_u, \beta_u)
trial_i\sim Gamma(10,10),

where pdf of gamma is
f(x|\alpha, \beta)=\frac{1}{Γ(\alpha)\beta^\alpha}x^{1-\alpha}e^{-\frac{x}{\beta}}

With parameter tranformation:

\alpha_i=\mu_i*trial_i

\beta_i=(1-\mu_i)*trial_i

The relationship between datas and parameters:

ctr_{ij}\sim Beta(\alpha_i, \beta_i)

click_{ij}\sim Bin(impression_{ij}, ctr_{ij})

Let i denote the i’th merchant, let (i,j) denote the j’th kind of goods from i’th merchant

The code is here

data{
  int<lower=0> N;
  int<lower=0> M;
  int<lower=1, upper=466> type[N];
  int<lower=0> impression[N];
  int<lower=0> click[N];
}
parameters{
  vector<lower=0.01, upper=1>[M] mu;
  vector<lower=0>[M] trial;
  real<lower=0> alpha_u;
  real<lower=0> beta_u;
}
transformed parameters{
  vector[M] alpha_i;
  vector[M] beta_i;
  alpha_i = mu.*trial;
  beta_i = (1-mu).*trial;
}
model{
  alpha_u~uniform(0.01, 200);
  beta_u~uniform(0.01, 200);
  mu~beta(alpha_u, beta_u);
  trial~gamma(10, 0.1);
  for(i in 1:N) {
    click[i]~beta_binomial(impression[i], alpha_i[type[i]], beta_i[type[i]]);
  }
}

Besides, these are the parameters.

stan_data = {'N':int(ruten_sample.shape[0]), 
             'M':int(ruten_sample.seller_number.nunique()), 
             'type': np.array(ruten_sample['seller_number']).astype(int), 
             'impression': np.array(ruten_sample['impression']).astype(int), 
             'click': np.array(ruten_sample['click']).astype(int)}
posterior = stan.build(text, data=stan_data)
fit = posterior.sample(num_chains=1, num_samples=2000)
df = fit.to_frame()
2

You don’t need to loop:

click ~ beta_binomial(impression, alpha_i[type], beta_i[type]);

works and avoids N calls.

3

Thank you. Your suggestion improves the time by 10%, but when I increase n to some big number, it still takes a lot of time.

4

So is life.

I’m not familiar with your data or model, and I’m a bit confused by your model specification. And your priors between 0.01 and 200, for example, says that all values in between are equally likely and all outside are impossible, but I don’t know your data. If it isn’t working, I would try something simpler first, e.g.,

data{
  int<lower=0> N;
  int<lower=0> M;
  int<lower=1, upper=466> type[N];
  int<lower=0> impression[N];
  int<lower=0> click[N];
}
parameters{
  vector<lower=0>[M] alpha;
  vector<lower=0>[M] beta;
}
transformed parameters{
  vector<lower=0,upper=1>[M] pi = alpha ./ (alpha + beta);
}
model{
  alpha ~ exponential(1);
  beta ~ exponential(1);
  click ~ beta_binomial(impression, alpha[type], beta[type]);
  }

If that doesn’t work, simplify more. If it does, try adding more structure back in; e.g., a hyper-prior on alpha and beta.

1 Like
5

Additionally, if you want to continue using those uniform priors, you should declare those parameters with constraints that match the priors.

1 Like
6

Thank you, again! I use uniform prior because I think it will make Stan draw samples faster. Another issue is that the lower bars of some parameters are not 0, because it seems that Stan will prohibit you to draw positive parameters from interval with zero(e.g. [0, 100]). Anyway, I will re-design my model.

7

I wonder if there are some proper distributions for drawing positive parameters. Especially the one which can speed up the algorithm :)

8

Yes, I noticed this issue earlier. It seems that Stan will prohibit you to draw positive parameters from interval with zero(e.g. [0, 100]). My plan B is to change it to 0.01.

9

I gave you one in the above example code, the exponential distribution. And yes, there are others and Stan has many built in: 20 Positive Continuous Distributions | Stan Functions Reference

10

It won’t (usually) make Stan faster :)

If you choose an exact value on some continuous interval, with infinite precision, the probability that a parameter will ever land on exactly that value in finite time is zero. The probability mass over any particular value is infinitesimal (i.e. zero). In particular, a continuous parameter has zero probability mass over an exact zero (to infinite precision). Thus there is no difference between the interval [0,100) and (0,100) for your purposes. Stan doesn’t have infinite precision, but the precision in the neighborhood of zero for a constrained parameter is close enough.

In addition to intrinsically positive distributions like the exponential, lognormal, and gamma, note that it is reasonable to think of truncated normal (or truncated t, or truncated anything) priors as proper as long as you can compute the CDF and thus the normalizing constant. Thus, combining <lower = 0> with any continuous distribution whose CDF is computable yields a proper prior.