Strange behavior in cmdstan 2.18 with matrices

Hi,

I never observed this in previous versions so it may not be something specific to cmdstan 2.18. In my stan model I have: parameters {
matrix[5, 240] beta_first;
and
transformed parameters {
vector[N] alpha = calculate_alpha(beta_first);

After fitting the model with cmdstan2.18 I run in R:
fit <- read_stan_csv(csvfile)
la <- extract(fit, permuted = TRUE)
I get
dim(la$beta_first) = c(1000, 240, 5) where 1000 is # of iterations.
i.e., beta_first is transposed.

Then in R code I expose calculate_alpha and run
alpha <- calculate_alpha(t(beta_first[1, , ])) which gives the completely different results from la$alpha[1].

I am completely confused and will appreciate any insight. The stan model is attached.
NNsigma.2.18.stan (1.5 KB)

That has always been the case. What dimensions were you expecting?

I have no problem with transpose. Since alpha is calculated both in stan and R using calculate_alpha function I am pretty sure that calculation in R is not correct. I don’t know what is wrong. In stan model I call it using:

vector[N] alpha = calculate_alpha(X, bias_first, bias_output,
beta_first, beta_output);

from R I call it using:

alpha.stan <- calculate_alpha(stan.x, la$bias_first[1, ],
la$bias_output[1], t(la$beta_first[1, , ]),
la$beta_output[1, ])

where dim(stan.x) is 12600x240, la$bias_first is 1000x5, la$bias_output is 1000, la$beta_first is 1000x240x5, la$beta_output is 1000x5. Unfortunately alpha.stan[1] is not la$alpha[1, 1]. This one confuses me.

I think you need permuted = FALSE in extract.

1 Like

You are genius. Silly me.

But syntax la$beta_first works no more. Is there a reason why permuted=TRUE returns a list while permuted=FALSE returns array?

Not that I know of

Well, I did test and matrix was completely messed up. When I printed beta_first from stan I got:

[[0.0141494,1.78998],[-0.337971,0.321494],[1.5998,-0.406764],[1.78294,0.263281],[-1.16255,-0.681286]]

However, the following statements in R:

la <- extract(fit, permuted = FALSE)
la[1,1,7:16]
produce:
beta_first[1,1] beta_first[2,1] beta_first[1,2] beta_first[2,2] beta_first[1,3]
0.0141494 -0.3379710 1.5998000 1.7829400 -1.1625500
beta_first[2,3] beta_first[1,4] beta_first[2,4] beta_first[1,5] beta_first[2,5]
1.7899800 0.3214940 -0.4067640 0.2632810 -0.6812860

All numbers are correct but in incorrect places. Matrix is transposed but headings are wrong. I am completely lost.

Ignore the headings and expect the matrix to be transposed, that’s it.

That’s what I thought. Any plans to align column names to actual content? I always used

la=extract(fit,permuted=TRUE) and la$column_name and found it very convenient but it seems I have to refrain from this.

The labels are an rstan thing so I’m not sure. My preference would be to hide this from the user entirely and mask the “transpose” but it’s likely a backwards-compatibility thing for rstan2 at this point.