Mysterious compilation error in mixed-effects GLM: ` cannot convert 'const stan::math::var’ to ‘double’ in return return u;`

rborder · March 31, 2018, 2:11am

I’m working on a multilevel logistic regression model with missing binary response data. Everything works fine with a single random intercept, but when I try to add a random slope, I get the below compilation error. I’m having a lot of difficulty debugging as it’s unclear to me whether it’s a compiler issue or an error in my Stan code.

The error message

 In file included from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/BH/include/boost/config.hpp:39:0,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/BH/include/boost/math/tools/config.hpp:13,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/rev/core/var.hpp:7,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/rev/core/gevv_vvv_vari.hpp:5,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/rev/core.hpp:12,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/rev/mat.hpp:4,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math.hpp:4,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/src/stan/model/model_header.hpp:4,
                 from file602c70c0cf.cpp:8:
/home/rsb/R/x86_64-pc-linux-gnu-library/3.4/BH/include/boost/config/compiler/gcc.hpp:186:0: warning: "BOOST_NO_CXX11_RVALUE_REFERENCES" redefined
 #  define BOOST_NO_CXX11_RVALUE_REFERENCES
 ^
<command-line>:0:0: note: this is the location of the previous definition
In file included from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/prim/arr/fun/promote_elements.hpp:4:0,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/prim/mat/fun/promote_elements.hpp:4,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/prim/mat/fun/promote_common.hpp:5,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/prim/mat/fun/mdivide_left.hpp:6,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/prim/mat.hpp:149,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/rev/mat.hpp:12,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math.hpp:4,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/src/stan/model/model_header.hpp:4,
                 from file602c70c0cf.cpp:8:
/home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/prim/scal/fun/promote_elements.hpp: In instantiation of ‘static T stan::math::promote_elements<T, S>::promote(const S&) [with T = double; S = stan::math::var]’:
/home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/prim/arr/fun/array_builder.hpp:37:53:   required from ‘stan::math::array_builder<T>& stan::math::array_builder<T>::add(const S&) [with S = stan::math::var; T = double]’
file602c70c0cf.cpp:473:94:   required from ‘T__ model602c410cb45f_dichotomousMissing_namespace::model602c410cb45f_dichotomousMissing::log_prob(std::vector<T2>&, std::vector<int>&, std::ostream*) const [with bool propto__ = true; bool jacobian__ = true; T__ = stan::math::var; std::ostream = std::basic_ostream<char>]’
/home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/src/stan/model/log_prob_grad.hpp:45:39:   required from ‘double stan::model::log_prob_grad(const M&, std::vector<double>&, std::vector<int>&, std::vector<double>&, std::ostream*) [with bool propto = true; bool jacobian_adjust_transform = true; M = model602c410cb45f_dichotomousMissing_namespace::model602c410cb45f_dichotomousMissing; std::ostream = std::basic_ostream<char>]’
/home/rsb/R/x86_64-pc-linux-gnu-library/3.4/rstan/include/rstan/stan_fit.hpp:1164:49:   required from ‘SEXPREC* rstan::stan_fit<Model, RNG_t>::grad_log_prob(SEXP, SEXP) [with Model = model602c410cb45f_dichotomousMissing_namespace::model602c410cb45f_dichotomousMissing; RNG_t = boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> >; SEXP = SEXPREC*]’
file602c70c0cf.cpp:848:0:   required from here
/home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/prim/scal/fun/promote_elements.hpp:24:16: error: cannot convert ‘const stan::math::var’ to ‘double’ in return
         return u;
                ^
cc1plus: warning: unrecognized command line option ‘-Wno-ignored-attributes’
cc1plus: warning: unrecognized command line option ‘-Wno-macro-redefined’
make: *** [file602c70c0cf.o] Error 1

ERROR(s) during compilation: source code errors or compiler configuration errors!

.... lots of cpp ....

Error in compileCode(f, code, language = language, verbose = verbose) : 
  Compilation ERROR, function(s)/method(s) not created! In file included from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/BH/include/boost/config.hpp:39:0,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/BH/include/boost/math/tools/config.hpp:13,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/rev/core/var.hpp:7,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/rev/core/gevv_vvv_vari.hpp:5,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/rev/core.hpp:12,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math/rev/mat.hpp:4,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/stan/math.hpp:4,
                 from /home/rsb/R/x86_64-pc-linux-gnu-library/3.4/StanHeaders/include/src/stan/model/model_header.hpp:4,
                 from file602c7309332
In addition: Warning message:
running command '/usr/lib/R/bin/R CMD SHLIB file602c73093324.cpp 2> file602c73093324.cpp.err.txt' had status 1

The Stan code that doesn’t work (with random intercept and random slope):

data {
    int<lower=1> N;                  // # of total observations of predictors
    int<lower=0> P;                  // # of covariates with fixed effects
    int<lower=1> D;                  // # of levels of sample
    int sample[N];                   // sample indicator
    int<lower=0,upper=1> y[N];       // outcome
    row_vector[P] x[N];              // covariates
    real z[N];                       // covariate with random slope
    int<lower=0,upper=1> attrited[N];// indictor of missingness of response
}
parameters {
    real a;                          // fixed intercept
    vector[P] bCov;                  // fixed effect slopes
    vector[D] alpha;                 // random intercept
    vector[D] bRandom;               // random slope
    cov_matrix[2] Sigma;             // random effect vcov mat
    // real<lower=0> varAlpha;          // random intercept variance
    // real<lower=0> varBRandom;        // random slope variance
    // real<lower=0> covAB;             // random slope / random intercept covariance
    // vector[2] mu;
}

transformed parameters {
    // matrix[2,2] Sigma = [[varAlpha,covAB],[covAB,varBRandom]];
    vector<lower=0, upper=1>[N] pi;  // probability of outcome
    for (j in 1:N) {                 // logistic function:
        pi[j] = 1/(1+exp(-(a + alpha[sample[j]] + x[j]*bCov + z[j]*bRandom[sample[j]])));
    }
}

model {
    /////// PRIORS ///////
    a ~ normal(0,5);
    // varAlpha ~ normal(0,5);
    // covAB ~ normal(0,5);
    // varBRandom ~ normal(0,5);
    // mu[1] ~ normal(0,5);
    // mu[2] ~ normal(0,5);
    for (d in 1:D) {                  // joint distribution of random effects:
        [alpha[d], bRandom[d]]' ~ multi_normal([0,0]',Sigma);
    }
    for (p in 1:P) {
        bCov[p] ~ normal(0,10);
    }
    ////// LIKELIHOOD //////

    for (j in 1:N) {
        real lp;
        if (attrited[j] == 0) {
            lp = bernoulli_lpmf( y[j] | pi[j]);
        }
        else {
            real tmp[2];
            tmp[1] = bernoulli_lpmf(1 | pi[j]);
            tmp[2] = bernoulli_lpmf(0 | pi[j]);
            lp = log_mix(pi[j], tmp[1], tmp[2]);
        }
        target += lp;
    }
}

The Stan code that does work (with random intercept only):

data {
    int<lower=1> N;                  // # total observations of predictors
    int<lower=0> P;                  // # covariates
    int<lower=1> D;                  // # levels of sample
    int sample[N];                   // sample indicator
    int<lower=0,upper=1> y[N];       // outcome
    row_vector[P] x[N];              // covariates
    int<lower=0,upper=1> attrited[N];// indictor of missingness
}
parameters {
    real a;                          // fixed intercept
    vector[P] bCov;                  // fixed effect slopes
    real alpha[D];                   // random intercept
    real<lower=0> sigma;             // random intercept variance
}

transformed parameters {
    vector<lower=0, upper=1>[N] pi; // probability of arrest
    for (j in 1:N) {  // logistic function
        pi[j] = 1/(1+exp(-(a + alpha[sample[j]] + x[j]*bCov)));
    }
}

model {
    ////// PRIORS
    a ~ normal(0,5);
    sigma ~ normal(0,5);
    for (d in 1:D) {
        alpha[d] ~ normal(0,sigma);
    }
    for (p in 1:P) {
        bCov[p] ~ normal(0,10);
    }
    ////// Likelihood

    for (j in 1:N) {
        real lp;
        if (attrited[j] == 0) {
            lp = bernoulli_lpmf( y[j] | pi[j]);
        }
        else {
            real tmp[2];
            tmp[1] = bernoulli_lpmf(1 | pi[j]);
            tmp[2] = bernoulli_lpmf(0 | pi[j]);
            lp = log_mix(pi[j], tmp[1], tmp[2]);
        }
        target += lp;
    }
}

And, for ease of readability, here are
The differences between the working code and the broken code by block:

Data - added new variable:
real z[N]; // covariate with random slope

Parameters - replaced:

real a;                          // fixed intercept
vector[P] bCov;                  // fixed effect slopes
real alpha[D];                   // random intercept
real<lower=0> sigma;             // random intercept variance

with

real a;                          // fixed intercept
vector[P] bCov;                  // fixed effect slopes
vector[D] alpha;                 // random intercept
vector[D] bRandom;               // random slope
cov_matrix[2] Sigma;             // random effect vcov mat

Transformed parameters - add random slope term to linear predictor.
From:

vector<lower=0, upper=1>[N] pi; // probability of outcome
for (j in 1:N) {  // logistic function
    pi[j] = 1/(1+exp(-(a + alpha[sample[j]] + x[j]*bCov)));
}

To:

vector<lower=0, upper=1>[N] pi;  // probability of outcome
for (j in 1:N) {                 // logistic function:
    pi[j] = 1/(1+exp(-(a + alpha[sample[j]] + x[j]*bCov + z[j]*bRandom[sample[j]])));
}

Model - model joint distribution of random intercept and random slope term as multivariate normal:
From:

a ~ normal(0,5);
sigma ~ normal(0,5);
for (d in 1:D) {
    alpha[d] ~ normal(0,sigma);
}

To:

for (d in 1:D) {                  // joint distribution of random effects:
    [alpha[d], bRandom[d]]' ~ multi_normal([0,0]',Sigma);
}

Any advice you have would be greatly appreciated! Thank you!

bgoodri · March 31, 2018, 2:23am

Compiler errors are always code generator bugs. In this case, the bug is that it does not like

for (d in 1:D) {                  // joint distribution of random effects:
    [alpha[d], bRandom[d]]' ~ multi_normal([0,0]',Sigma);
}

You could change it to

    for (d in 1:D) {                  // joint distribution of random effects:
        row_vector[2] lhs = [alpha[d], bRandom[d]];
        lhs ~ multi_normal([0,0]',Sigma);
    }

and it compiles, but a better way to do it is to declare

row_vector[2] lhs[D];

and the start of the model block and then do

for (d in 1:D) lhs[d] = [alpha[d], bRandom[d]];
lhs ~ multi_normal([0,0], Sigma);

which avoids repeated factorizations of Sigma.

rborder · March 31, 2018, 2:33am

That did the trick! Thanks for the blazing fast response–I had been banging my head against the wall trying to figure out the problem.

mitzimorris · March 31, 2018, 4:08pm

glad there’s a workaround - will investigate code generation bug further and file issue, etc.

rborder · March 31, 2018, 7:19pm

Thanks for that!

mitzimorris · April 1, 2018, 7:51pm

working against develop branch of Stan, and having edited the model that doesn’t work down to this:

data {
    int<lower=0> P;
    int<lower=1> D;
}
parameters {
    vector[P] bCov;
    vector[D] alpha;
    vector[D] bRandom;
    cov_matrix[2] Sigma;
}
model {
    for (d in 1:D) { 
        [alpha[d], bRandom[d]]' ~ multi_normal([0,0]',Sigma);
    }
}

generated code compiles. checking the generated code confirms that the compiler has properly detected that the expression:

[alpha[d], bRandom[d]]'

contains parameter vars.

are you using the latest version of Stan (RStan, PyStan, etc)?

mitzimorris · April 1, 2018, 7:56pm

reported in this issue: https://github.com/stan-dev/stan/issues/2505

rborder · April 1, 2018, 8:09pm

I’m using version:

> packageDescription("rstan")
Package: rstan
Type: Package
Title: R Interface to Stan
Version: 2.17.3
Date: 2018-01-15

and I still get an error compiling the exact code you pasted above. The full error message is attached.

P.S., would it be better to move this thread to the github issue? Thanks!

error.txt (34.3 KB)

mitzimorris · April 1, 2018, 10:31pm

you are absolutely correct - most definitely a bug in the current version.
the good news is that it will be fixed in the next release.

rborder · April 1, 2018, 10:40pm

Awesome, thanks.

Topic		Replies	Views
Compilation error when using complex transformed parameters Modeling techniques , specification	3	636	November 29, 2017
Compilation error in model (possible from Eigen?) General	4	470	October 6, 2019
Error in compileCode Modeling fitting-issues , specification	6	1279	October 3, 2017
Parse error in replicating the example from user guide section 1.9 RStan rstan	2	660	October 28, 2021
Compiling error help (beginner) Modeling rstan , fitting-issues	1	398	January 21, 2021

Mysterious compilation error in mixed-effects GLM: ` cannot convert 'const stan::math::var’ to ‘double’ in return return u;`

Related topics