Silent integer conversion in cmdstanr?

joffdd · June 6, 2024, 3:02pm

In a stack of

R
cmdstanr
CmdStan:

I noticed that when I specify model with a data variable of type int and passing a numeric type variable from R, that it gets silently converted to integers without any warning or error message. That is unexpected for me (but may be normal behavior). I couldn’t find documentation on this behavior so I thought I better post this here for confirmation.

To me, it appeared as unwanted behavior, and I personally lost some time figuring out why my model would not run as expected, and eventually understanding that I had forgotten to change my model specification from array[I] int to vector[I]. I had changed the data fed into the model in R, however, and was surprised to see that STAN had not complained about being fed non-ints to an integer data variable.

WardBrian · June 6, 2024, 3:09pm

Stan itself does complain:

Exception: int variable contained non-int values; processing stage=data initialization; variable name=y; base type=int (in 'examples/bernoulli/bernoulli.stan', line 3, column 2 to column 35)

Maybe cmdstanr is doing something odd here - do your numeric values happen to fall exactly on the integers, or do they have non-zero decimal components?

joffdd · June 6, 2024, 3:32pm

I tested it again using a model that I develop atm that has a integer data variable y_n. I can’t see an error message or warning in the output. It has non-zero decimal components.

data {
  //...
  array[N] int y_n; 
  //...
}

stan_data$y_n = stan_data$y_n+.000001

Running MCMC with 4 parallel chains...

Chain 1 Iteration:    1 / 1500 [  0%]  (Warmup) 
Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 1 Exception: gamma_lpdf: Random variable[14] is inf, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 44, column 2 to column 29)
Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 1 
Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 1 Exception: gamma_lpdf: Random variable[14] is inf, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 44, column 2 to column 29)
Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 1 
Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 1 Exception: gamma_lpdf: Random variable is 0, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 43, column 2 to column 25)
Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 1 
Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 1 Exception: gamma_lpdf: Random variable is 0, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 43, column 2 to column 25)
Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 1 
Chain 2 Iteration:    1 / 1500 [  0%]  (Warmup) 
Chain 2 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 2 Exception: gamma_lpdf: Random variable[14] is inf, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 44, column 2 to column 29)
Chain 2 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 2 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 2 
Chain 2 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 2 Exception: gamma_lpdf: Random variable[14] is inf, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 44, column 2 to column 29)
Chain 2 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 2 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 2 
Chain 2 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 2 Exception: gamma_lpdf: Random variable[1] is 0, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 44, column 2 to column 29)
Chain 2 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 2 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 2 
Chain 2 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 2 Exception: gamma_lpdf: Random variable[14] is 0, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 44, column 2 to column 29)
Chain 2 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 2 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 2 
Chain 3 Iteration:    1 / 1500 [  0%]  (Warmup) 
Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: gamma_lpdf: Random variable is 0, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 43, column 2 to column 25)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3 
Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: gamma_lpdf: Random variable is 0, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 43, column 2 to column 25)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3 
Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: gamma_lpdf: Random variable is 0, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 43, column 2 to column 25)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3 
Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: gamma_lpdf: Random variable is 0, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 43, column 2 to column 25)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3 
Chain 4 Iteration:    1 / 1500 [  0%]  (Warmup) 
Chain 4 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 4 Exception: gamma_lpdf: Random variable is 0, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 43, column 2 to column 25)
Chain 4 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 4 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 4 
Chain 4 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 4 Exception: gamma_lpdf: Random variable is 0, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 43, column 2 to column 25)
Chain 4 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 4 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 4 
Chain 4 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 4 Exception: gamma_lpdf: Random variable is 0, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 43, column 2 to column 25)
Chain 4 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 4 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 4 
Chain 4 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 4 Exception: gamma_lpdf: Random variable is 0, but must be positive finite! (in 'C:/Users/mail/AppData/Local/Temp/RtmpIZeEiW/model-7f9c78bd7865.stan', line 43, column 2 to column 25)
Chain 4 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 4 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 4 
Chain 1 Iteration:  100 / 1500 [  6%]  (Warmup) 
Chain 3 Iteration:  100 / 1500 [  6%]  (Warmup) 
Chain 2 Iteration:  100 / 1500 [  6%]  (Warmup) 
Chain 4 Iteration:  100 / 1500 [  6%]  (Warmup) 
Chain 1 Iteration:  200 / 1500 [ 13%]  (Warmup) 
Chain 3 Iteration:  200 / 1500 [ 13%]  (Warmup) 
Chain 2 Iteration:  200 / 1500 [ 13%]  (Warmup) 
Chain 4 Iteration:  200 / 1500 [ 13%]  (Warmup) 
Chain 1 Iteration:  300 / 1500 [ 20%]  (Warmup) 
Chain 3 Iteration:  300 / 1500 [ 20%]  (Warmup) 
Chain 2 Iteration:  300 / 1500 [ 20%]  (Warmup) 
Chain 4 Iteration:  300 / 1500 [ 20%]  (Warmup) 
Chain 1 Iteration:  400 / 1500 [ 26%]  (Warmup) 
Chain 3 Iteration:  400 / 1500 [ 26%]  (Warmup) 
Chain 4 Iteration:  400 / 1500 [ 26%]  (Warmup) 
Chain 2 Iteration:  400 / 1500 [ 26%]  (Warmup) 
Chain 1 Iteration:  500 / 1500 [ 33%]  (Warmup) 
Chain 3 Iteration:  500 / 1500 [ 33%]  (Warmup) 
Chain 4 Iteration:  500 / 1500 [ 33%]  (Warmup) 
Chain 2 Iteration:  500 / 1500 [ 33%]  (Warmup) 
Chain 1 Iteration:  600 / 1500 [ 40%]  (Warmup) 
Chain 3 Iteration:  600 / 1500 [ 40%]  (Warmup) 
Chain 4 Iteration:  600 / 1500 [ 40%]  (Warmup) 
Chain 1 Iteration:  700 / 1500 [ 46%]  (Warmup) 
Chain 2 Iteration:  600 / 1500 [ 40%]  (Warmup) 
Chain 3 Iteration:  700 / 1500 [ 46%]  (Warmup) 
Chain 4 Iteration:  700 / 1500 [ 46%]  (Warmup) 
Chain 1 Iteration:  800 / 1500 [ 53%]  (Warmup) 
Chain 3 Iteration:  800 / 1500 [ 53%]  (Warmup) 
Chain 2 Iteration:  700 / 1500 [ 46%]  (Warmup) 
Chain 1 Iteration:  900 / 1500 [ 60%]  (Warmup) 
Chain 4 Iteration:  800 / 1500 [ 53%]  (Warmup) 
Chain 3 Iteration:  900 / 1500 [ 60%]  (Warmup) 
Chain 2 Iteration:  800 / 1500 [ 53%]  (Warmup) 
Chain 4 Iteration:  900 / 1500 [ 60%]  (Warmup) 
Chain 3 Iteration: 1000 / 1500 [ 66%]  (Warmup) 
Chain 2 Iteration:  900 / 1500 [ 60%]  (Warmup) 
Chain 3 Iteration: 1001 / 1500 [ 66%]  (Sampling) 
Chain 1 Iteration: 1000 / 1500 [ 66%]  (Warmup) 
Chain 1 Iteration: 1001 / 1500 [ 66%]  (Sampling) 
Chain 3 Iteration: 1100 / 1500 [ 73%]  (Sampling) 
Chain 4 Iteration: 1000 / 1500 [ 66%]  (Warmup) 
Chain 4 Iteration: 1001 / 1500 [ 66%]  (Sampling) 
Chain 2 Iteration: 1000 / 1500 [ 66%]  (Warmup) 
Chain 2 Iteration: 1001 / 1500 [ 66%]  (Sampling) 
Chain 3 Iteration: 1200 / 1500 [ 80%]  (Sampling) 
Chain 1 Iteration: 1100 / 1500 [ 73%]  (Sampling) 
Chain 4 Iteration: 1100 / 1500 [ 73%]  (Sampling) 
Chain 2 Iteration: 1100 / 1500 [ 73%]  (Sampling) 
Chain 3 Iteration: 1300 / 1500 [ 86%]  (Sampling) 
Chain 4 Iteration: 1200 / 1500 [ 80%]  (Sampling) 
Chain 3 Iteration: 1400 / 1500 [ 93%]  (Sampling) 
Chain 1 Iteration: 1200 / 1500 [ 80%]  (Sampling) 
Chain 2 Iteration: 1200 / 1500 [ 80%]  (Sampling) 
Chain 3 Iteration: 1500 / 1500 [100%]  (Sampling) 
Chain 3 finished in 7.9 seconds.
Chain 4 Iteration: 1300 / 1500 [ 86%]  (Sampling) 
Chain 1 Iteration: 1300 / 1500 [ 86%]  (Sampling) 
Chain 2 Iteration: 1300 / 1500 [ 86%]  (Sampling) 
Chain 4 Iteration: 1400 / 1500 [ 93%]  (Sampling) 
Chain 2 Iteration: 1400 / 1500 [ 93%]  (Sampling) 
Chain 1 Iteration: 1400 / 1500 [ 93%]  (Sampling) 
Chain 4 Iteration: 1500 / 1500 [100%]  (Sampling) 
Chain 4 finished in 9.2 seconds.
Chain 2 Iteration: 1500 / 1500 [100%]  (Sampling) 
Chain 2 finished in 9.9 seconds.
Chain 1 Iteration: 1500 / 1500 [100%]  (Sampling) 
Chain 1 finished in 10.2 seconds.

All 4 chains finished successfully.
Mean chain execution time: 9.3 seconds.
Total execution time: 10.3 seconds.

joffdd · June 7, 2024, 10:39am

I have opened an issue on Github.

jsocolar · June 7, 2024, 9:23pm

I agree this looks problematic. I think the culprit is here cmdstanr/R/data.R at 02259ef7aa2a8b1c8de2fa3fc42a9feafd789288 · stan-dev/cmdstanr · GitHub

I’m not too troubled that it silently coerces reals to integers, but it seems really problematic to do so by rounding down, which can be very dangerous if combined with floating point operations. For example

a <- 10*(3-2.7)
mode(a) <- "integer"

> a
[1] 2

joffdd · June 7, 2024, 10:57pm

Thanks for locating the issue @jsocolar.

I’m happy to see there is a fix in the making.

Topic		Replies	Views
Error when passing factor data to Stan Interfaces cmdstanr	6	150	June 15, 2024
Vector to integer formatting Modeling rstan	7	932	July 30, 2021
Real to integer conversion? Developers	27	11819	January 15, 2024
Specifying input data as array of integers from R to Stan General rstan	6	517	February 9, 2023
Pystan converting reals to ints? Modeling pystan	3	612	November 3, 2021

Silent integer conversion in cmdstanr?

Related topics