Trying to compile the integrate_1d example fails

Trying to compile this example (in pystan) leads to an error:

  real normal_density(real x,          // Function argument
                    real xc,         // Complement of function argument
                                     //  on the domain (defined later)
                    real[] theta,    // parameters
                    real[] x_r,      // data (real)
                    int[] x_i) {     // data (integer)
  real mu = theta[1];
  real sigma = theta[2];

  return 1 / (sqrt(2 * pi()) * sigma) * exp(-0.5 * ((x - mu) / sigma)^2);
}
  
  
}

data {
  int N;
  real y[N];
}

transformed data {
  real x_r[0];
  int x_i[0];
}

parameters {
  real mu;
  real<lower = 0.0> sigma;
  real left_limit;
}

model {
  mu ~ normal(0, 1);
  sigma ~ normal(0, 1);
  left_limit ~ normal(0, 1);
  target += normal_lpdf(y | mu, sigma);
  target += log(integrate_1d(normal_density,
                             left_limit,
                             positive_infinity(),
                             { mu, sigma }, x_r, x_i));
}```

The error:

ValueError: Failed to parse Stan model 'anon_model_8185db55041ace7ae18bc15d93d6605e'. Error message:
SYNTAX ERROR, MESSAGE(S) FROM PARSER:
 error in 'example_int1d.stan' at line 41, column 52
  -------------------------------------------------
    39:                              left_limit,
    40:                              positive_infinity(),
    41:                              { mu, sigma }, x_r, x_i));
                                                           ^
    42: }
  -------------------------------------------------

PARSER EXPECTED: ","

Thoughts?

Did the functional format change?

Did you omit the

functions {

from the top line?

Sorry, that was a copy/paste typo. Issue holds with it included.

Just to be complete, the full model is

functions {
  real normal_density(real x,          // Function argument
                    real xc,         // Complement of function argument
                                     //  on the domain (defined later)
                    real[] theta,    // parameters
                    real[] x_r,      // data (real)
                    int[] x_i) {     // data (integer)
  real mu = theta[1];
  real sigma = theta[2];

  return 1 / (sqrt(2 * pi()) * sigma) * exp(-0.5 * ((x - mu) / sigma)^2);
}
  
  
}

data {
  int N;
  real y[N];
}

transformed data {
  real x_r[0];
  int x_i[0];
}

parameters {
  real mu;
  real<lower = 0.0> sigma;
  real left_limit;
}

model {
  mu ~ normal(0, 1);
  sigma ~ normal(0, 1);
  left_limit ~ normal(0, 1);
  target += normal_lpdf(y | mu, sigma);
  target += log(integrate_1d(normal_density,
                             left_limit,
                             positive_infinity(),
                             { mu, sigma }, x_r, x_i));
}

and in attempting to compile it, I receive the error:

ValueError: Failed to parse Stan model 'anon_model_8185db55041ace7ae18bc15d93d6605e'. Error message:
SYNTAX ERROR, MESSAGE(S) FROM PARSER:
 error in 'example_int1d.stan' at line 41, column 52
  -------------------------------------------------
    39:                              left_limit,
    40:                              positive_infinity(),
    41:                              { mu, sigma }, x_r, x_i));
                                                           ^
    42: }
  -------------------------------------------------

PARSER EXPECTED: ","

Can someone confirm they can compile this model in pystan? in cmdstan?

That is supposed to work. @bbbales2?

It compiles (I haven’t tried to run it) if you also specify a tolerance after x_i, which according to the documentation should be an optional argument. Did anything change regarding that?

what mcol said - my comments on this thread -

can fix docs once we establish what’s going on.

1 Like

Yeah it should be optional. At the Math C++ level it is so I’m guessing it’s at the Stan level. @mitzimorris do you know how to change the Stan language stuff for this? As I remember there was something weird about getting this into the language cause it has the function argument n’ such.

looks like there was a bug filed in July - https://github.com/stan-dev/stan/issues/2786

I could take a look at the current compiler and the function signatures, or the stan3 folks - @seantalts, @Matthijs et al can verify whether or not this is working for the new compiler - I’m not sure it is, as the test model https://github.com/stan-dev/stan/blob/develop/src/test/test-models/good/integrate_1d_good.stan only has tests for the sig which includes relative tolerance
.

Here’s how you can easily test with stanc3:

halair ~/tmp $ curl -L -O https://github.com/stan-dev/stanc3/releases/download/nightly/mac-stanc
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   600    0   600    0     0   3074      0 --:--:-- --:--:-- --:--:--  3076
100  9.7M  100  9.7M    0     0  5872k      0  0:00:01  0:00:01 --:--:-- 8654k
halair ~/tmp  $ chmod +x mac-stanc 
halair ~/tmp  $ pbpaste > integrate1d.stan
halair ~/tmp  $ ./mac-stanc integrate1d.stan 

Semantic error in 'integrate1d.stan', line 38, column 16 to line 41, column 53:
   -------------------------------------------------
    36:    left_limit ~ normal(0, 1);
    37:    target += normal_lpdf(y | mu, sigma);
    38:    target += log(integrate_1d(normal_density,
                         ^
    39:                               left_limit,
    40:                               positive_infinity(),
   -------------------------------------------------

A returning function was expected but an undeclared identifier 'integrate_1d' was supplied.

Looks like we don’t have integrate_1d yet - the feature was merged in March, after we stopped work on the old compiler, but no issue was made on the stanc3 repo to track it. I’ll add one now: https://github.com/stan-dev/stanc3/issues/301

The integrate_1d function exists in the Stan language but apparently you have to specify the tolerance, which is unlike the other functionals.

Added it and fixed the bug in stanc3, thanks!

One note - the Stan manual incorrectly says the integrate_1d function returns a vector. @mitzimorris which repo is the right place to file an issue?

1 Like

stan-dev/docs

What’s the progress on moving the Stan manual back into the Stan repo? Is that still on your plate?

there were a bunch of reasons we moved the docs out of the Stan repo - one of them was to cut down on CI overhead, another was because of the burden of installing the docs toolchain - pandoc, latex or equiv, etc.
another was because it gives the online docs a nice URL via GitHub pages -
https://mc-stan.org/docs/2_20/stan-users-guide/
although
https://mc-stan.org/stan/2_20/stan-users-guide/
looks OK as well, so no big.

if you think they need to be moved back, it can be done.

As a contributor to the docs repo, I find that having a separate repository makes things much easier to deal with. But I guess that if consensus is to merge with the rest, I’ll find my way through it.

Cool, that’s good to hear. I just remember some conversations around figuring out how to sync the doc versions with the code and line them up properly, but maybe @mitzimorris came up with a versioning scheme inside the docs repo instead? I’m seeing these 2_19 and 2_20 URLs on Google (so happy the manual is google-able now!) so that’s probably it. I guess this just decouples the PR process for Stan code and docs so they’re done separately. Sounds good to me, never mind :)