Same code (with the same seed) but different results on different platforms? Why?

General
1

I will use a repeated-measures model to demonstrate my concern. My guess is that this issue might be due to whichever default MCMC algorithm being used (however, the message shows the same “Samples were drawn using NUTS(diag_e) …”). Or, pseudo random seed is inherently different by machine.

System One: R 4.1.1; RStudio 1.4.1717; macOS 10.14.5.
System Two: R 4.0.1; RStudio 1.4.1103; Windows 7 Professional (x86_54, yep antique).

stan_code <- "
data {
  int<lower=1> N;        // number of subjects
  int<lower=2> C;        // number of conditions
  vector[C] Y[N];        // responses
}

parameters {
  vector[C] mu;         // condition means
  real<lower=0> sigma;  // standard deviation of the error
  real<lower=0> gb;     // variance of the standardized subject-specific random effects
  vector[N] b;          // subject-specific random effects
}

transformed parameters {
  real<lower=0> tau;    // sd of the subject-specific random effects
  tau = sigma * sqrt(gb);
}

model {
  // linear mixed-effects model
  for (i in 1:N) {
    Y[i] ~ normal(mu + b[i], sigma);
  }
  b ~ normal(0, tau);

  // priors
  // mu ~ implicit uniform prior     // Jeffreys prior
  target += -2* log(sigma);          // Jeffreys prior
  gb ~ scaled_inv_chi_square(1, 1);  // Rouder et al. (2012)
}"

library(rstan)
anova_stan <- stan_model(model_code = stan_code)

Y <- matrix(c(10,6,11,22,16,15,1,12,9,8,
              13,8,14,23,18,17,1,15,12,9,
              13,8,14,25,20,17,4,17,12,12), ncol = 3)
N <- nrow(Y)
C <- ncol(Y)
datlist <- list("N" = N, "C" = C, "Y" = Y)

mcmc <- sampling(anova_stan, data = datlist,
                 pars = c("mu", "sigma"), refresh = 0,
                 warmup = 200, iter = 2200, chains = 4, seed = 123)
(result <- rstan::summary(mcmc)$summary[,"mean"])

System One returns

#      mu[1]       mu[2]       mu[3]       sigma       lp__
# 11.0328955  13.0375702  14.2364598   0.8194753 -33.5446405

System Two returns

#      mu[1]       mu[2]       mu[3]       sigma       lp__
# 11.0204850  13.0253015  14.2226526   0.8188674 -33.5321822
2

See here.

1 Like
3

Thank you, Mike.

1 Like