How to run ICM methods in Stan

specification
techniques

#1

I am using R version 3.3.3 with rstan_2.15.1.

I am trying to code an Iterated Conditional Modes (ICM) method to infer parameters of a joint distribution by maximizing iteratively over conditionally independent parts.

For example, assume you have two parameters to estimate A and B, A is estimated directly from a joint posterior and B is estimated by an ICM method, where the joint distribution is decomposed into N components, an ICM sample of B is generated from each component. A final sample is done by averaging over all the ICM samples. My idea was to create B in the transformed parameters block, and make it an average of a sampled vector.

Much like the following pseudo-example:

data {
     ... // some data input
}

parameters {
  real A;        
  vector [N] ICM_B;     
}
    
transformed parameters{
  real B;
  B = mean(ICM_B);
}

model{
  for (i in 1:N)
      ICM_B[i] ~ somDist(data, A);

  A ~ somDist2(data, B);
}

I am not sure if this is the correct way to model ICM methods in Stan. Based on Stan manual version 2.14 Table 5.1; transformed parameters are updated in every leapfrog step. How does that effect ICM methods.

In R, the way to do it is analogous to the pseudo example above, since sampling is done by generating random numbers iteratively and averaging over the results.

Thanks for the help.


#2

You can try to hack this through one of the interfaces by defining appropriate joint and conditional model and then interleaving sampling with optimization calls, but we wouldn’t recommend it. That kind of algorithm is extremely fragile.