# Implementing prior predictive checks in stan

**URL:** <https://discourse.mc-stan.org/t/implementing-prior-predictive-checks-in-stan/23815>\
**Category:** Modeling\
**Tags:** prior-predictive\
**Created:** [August 5, 2021, 10:08am UTC](https://discourse.mc-stan.org/t/implementing-prior-predictive-checks-in-stan/23815 "2021-08-05T10:08:49Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![drstevok](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/drstevok/32/12372_2.png) [@drstevok](https://discourse.mc-stan.org/u/drstevok)\
**Post date:** [August 5, 2021, 10:08am UTC](https://discourse.mc-stan.org/t/implementing-prior-predictive-checks-in-stan/23815/1 "2021-08-05T10:08:49Z")

</div>

I am trying to perform my prior predictive checks using Stan. The [advice](https://mc-stan.org/docs/2_27/stan-users-guide/prior-predictive-checks.html#coding-prior-predictive-checks-in-stan) in the user guide suggests the following ‘complete’ program using the random number generators.

The example provided is as follows:

```stan
data {
  int<lower = 0> N;
  vector[N] x;
}
generated quantities {
  real alpha = normal_rng(0, 1);
  real beta = normal_rng(0, 1);
  real y_sim[N] = poisson_log_rng(alpha + beta * x);
}

```

When I try anything similar, I get the following error

```
/opt/homebrew/Caskroom/miniforge/base/envs/stan-base/lib/python3.8/site-packages/cmdstanpy/utils.py in scan_metric(fd, config_dict, lineno)
    714 lineno += 1
    715 if not line == '# Adaptation terminated':
--> 716 raise ValueError(
    717 'line {}: expecting metric, found:\n\t "{}"'.format(lineno, line)
    718 )

ValueError: line 1045: expecting metric, found:
	 "#"

```

My attempt at a simplified implementation that generated the error above is below. This is just a linear regression with a intercept `alpha` and slope `beta`.

```stan
// saved as STAN_GC
data {
  
  int<lower = 0> N;
  vector[N] x;
  vector[N] y;
  
  // pass in the prior specification
  real alpha_mean;
  real alpha_sd;
  real beta_mean;
  real beta_sd;
  real sigma_mean;

}

parameters {
    // Comment out as defined in the generated quantities block
    //real alpha;
    //real beta;
    //real sigma;

}

model {
    
    // Commenting out the log likelihood line so the model runs _without_ the data
    // y ~ normal(mu, sigma);
}

generated quantities {
  real alpha = normal_rng(alpha_mean, alpha_sd);
  real beta = normal_rng(beta_mean, beta_sd);
  real sigma = exponential_rng(sigma_mean);
  
  real y_sim[N] = normal_rng(alpha + beta * x, sigma);
}

```

The stan code compiles without a problem

```python
ppc_model = CmdStanModel(stan_file=STAN_GC)
ppc_model.compile(force=True)

```

but generates the `ValueError` that I pasted above when I try to _sample_ .

```python
ppc_fit = ppc_model.sample(data=ds)

```

For reference, the data that I passed to the model is as follows.

```python
df = pd.DataFrame(
    {'x': np.linspace(-1,+2,50),
     'e': np.random.normal(size=50)
    }
)

df['y_no_error'] = 0.5 + 2 * df['x'] 
df['y'] = df['y_no_error'] + df.e

df.head()

```

```

x	e	y_no_error	y
0	-1.000000	-3.320883	-1.500000	-4.820883
1	-0.938776	-0.048729	-1.377551	-1.426280
2	-0.877551	0.825229	-1.255102	-0.429873
3	-0.816327	-1.452188	-1.132653	-2.584841
4	-0.755102	-0.993722	-1.010204	-2.003926

```

which is passed in as a dictionary named `ds`…

```python
ds = {
    'N': df.shape[0],
    'x': list(df.x),
    'y': list(df.y),
    'alpha_mean': 0.5,
    'alpha_sd': 1,
    'beta_mean': 0,
    'beta_sd': 2,
    'sigma_mean': 1
}

```

I have also tried an alternative approach wherein I used the _transformed parameters_ block as per the following program, and this seemed to work without a problem.

```stan
data {
  int<lower = 0> N;
  vector[N] x;
  vector[N] y;
  // pass in the prior specification
  real alpha_mean;
  real alpha_sd;
  real beta_mean;
  real beta_sd;
  real sigma_mean;
}

parameters {
    real alpha;
    real beta;
    real sigma;

}

transformed parameters {
    vector[N] mu = alpha + beta * x;
}

model {
    alpha ~ normal(alpha_mean, alpha_sd);
    beta ~ normal(beta_mean,beta_sd);
    sigma ~ exponential(sigma_mean);
    
    // Commenting out the log likelihood line so the model runs _without_ the data
    // y ~ normal(mu, sigma);
}

```

I’d really appreciate any advice on how to go about using stan for prior predictive checks. Many thanks.

---

<div class="post-metadata">

**Author:** ![kholsinger](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/kholsinger/32/4498_2.png) [@kholsinger](https://discourse.mc-stan.org/u/kholsinger)\
**Post date:** [August 5, 2021, 6:37pm UTC](https://discourse.mc-stan.org/t/implementing-prior-predictive-checks-in-stan/23815/2 "2021-08-05T18:37:48Z")

</div>

This appears to be the problem.

> [@drstevok](#):
>
> ```no-highlight
> real y_sim[N] = poisson_log_rng(alpha + beta * x);
> 
> ```

I used Rstan rather than RPython, but when I changed that to

```no-highlight
  int y_sim[N] = poisson_log_rng(alpha + beta * x);

```

everything worked fine.

Since I don’t use Python, the error message you reported didn’t help me. The error message in RStan was a little more helpful.

```no-highlight
SYNTAX ERROR, MESSAGE(S) FROM PARSER:
Variable definition base type mismatch, variable declared as base type real[] variable definition has base type int[] error in 'model134125bd4f9ad_test' at line 8, column 51
  -------------------------------------------------
     6: real alpha = normal_rng(0, 1);
     7: real beta = normal_rng(0, 1);
     8: real y_sim[N] = poisson_log_rng(alpha + beta * x);
                                                          ^
     9: }
  -------------------------------------------------

```

---

<div class="post-metadata">

**Author:** ![drstevok](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/drstevok/32/12372_2.png) [@drstevok](https://discourse.mc-stan.org/u/drstevok)\
**Post date:** [August 5, 2021, 7:25pm UTC](https://discourse.mc-stan.org/t/implementing-prior-predictive-checks-in-stan/23815/3 "2021-08-05T19:25:46Z")

</div>

Thanks for this suggestion.  
I copied the code verbatim from the user guide but I see why the change to `int` make sense.

My problem remains unsolved, because I get the same error in the Minimal Working Example (MWE) provided where I use a linear model and that final step uses `normal_rng` where `real` would be the appropriate type.

---

<div class="post-metadata">

**Author:** ![jsocolar](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/jsocolar/32/2486_2.png) [@jsocolar](https://discourse.mc-stan.org/u/jsocolar)\
**Post date:** [August 5, 2021, 8:11pm UTC](https://discourse.mc-stan.org/t/implementing-prior-predictive-checks-in-stan/23815/4 "2021-08-05T20:11:23Z")

</div>

> [@drstevok](#):
>
> ```no-highlight
> generated quantities {
> real alpha = normal_rng(alpha_mean, alpha_sd);
> real beta = normal_rng(beta_mean, beta_sd);
> real sigma = exponential_rng(sigma_mean);
>   
> real y_sim[N] = normal_rng(alpha + beta * x, sigma);
> }
> 
> ```

If I’m not mistaken, `normal_rng(alpha + beta * x, sigma)` is generating a single real, but you are assigning it to `y_sim` which is declared as an array of `N` reals.

---

<div class="post-metadata">

**Author:** ![drstevok](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/drstevok/32/12372_2.png) [@drstevok](https://discourse.mc-stan.org/u/drstevok)\
**Post date:** [August 6, 2021, 2:57pm UTC](https://discourse.mc-stan.org/t/implementing-prior-predictive-checks-in-stan/23815/5 "2021-08-06T14:57:53Z")

</div>

Thanks for the suggestion about `normal_rng` producing just a single real.  
I rewrote the generated quantities block as below:

```stan

generated quantities {
  real alpha = normal_rng(alpha_mean, alpha_sd);
  real beta = normal_rng(beta_mean, beta_sd);
  real sigma = exponential_rng(sigma_mean);

  vector[N] y_sim;

  for (i in 1:N) {
    y_sim[i] = normal_rng(alpha + beta * x[i], sigma);
  }
}

```

But this still gives me the same error.

```
/opt/homebrew/Caskroom/miniforge/base/envs/stan-base/lib/python3.8/site-packages/cmdstanpy/utils.py in scan_metric(fd, config_dict, lineno)
    714 lineno += 1
    715 if not line == '# Adaptation terminated':
--> 716 raise ValueError(
    717 'line {}: expecting metric, found:\n\t "{}"'.format(lineno, line)
    718 )

ValueError: line 1045: expecting metric, found:
	 "#"

```

---

<div class="post-metadata">

**Author:** ![jsocolar](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/jsocolar/32/2486_2.png) [@jsocolar](https://discourse.mc-stan.org/u/jsocolar)\
**Post date:** [August 6, 2021, 2:59pm UTC](https://discourse.mc-stan.org/t/implementing-prior-predictive-checks-in-stan/23815/6 "2021-08-06T14:59:40Z")

</div>

In any case, my suggestion was wrong! I missed that `x` was a vector in your original code!

---

<div class="post-metadata">

**Author:** ![mitzimorris](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/mitzimorris/32/29_2.png) [@mitzimorris](https://discourse.mc-stan.org/u/mitzimorris)\
**Post date:** [August 6, 2021, 3:59pm UTC](https://discourse.mc-stan.org/t/implementing-prior-predictive-checks-in-stan/23815/7 "2021-08-06T15:59:35Z")

</div>

to do this in CmdStanPy, see this: [MCMC Sampling — CmdStanPy 0.9.76 documentation](https://mc-stan.org/cmdstanpy/sample.html#example-generate-data-fixed-param-true)

the error you’re hitting in CmdStanPy is due to the fact that currently, CmdStanPy requires that models with no parameters have `sample` argument `fixed_param=True`, however in latest release of CmdStan, the sampler runs `fixed_param` automatically if it detects that a model doesn’t have any parameters. ([Print errors to stderr instead of stdout; don't nag for `fixed_param` flag, just do it by mitzimorris · Pull Request #992 · stan-dev/cmdstan · GitHub](https://github.com/stan-dev/cmdstan/pull/992))

we need to update CmdStanPy to do the same. [`sample` method - proper handling of models with no parameters · Issue #425 · stan-dev/cmdstanpy · GitHub](https://github.com/stan-dev/cmdstanpy/issues/425)

---

<div class="post-metadata">

**Author:** ![drstevok](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/drstevok/32/12372_2.png) [@drstevok](https://discourse.mc-stan.org/u/drstevok)\
**Post date:** [August 12, 2021, 8:37pm UTC](https://discourse.mc-stan.org/t/implementing-prior-predictive-checks-in-stan/23815/8 "2021-08-12T20:37:15Z")

</div>

Just following up here to say the amazing CmdStanPy team have helped out, and there’s an update that handles models with no parameters and cases like this. Thanks!  
See [`sample` method - proper handling of models with no parameters · Issue #425 · stan-dev/cmdstanpy · GitHub](https://github.com/stan-dev/cmdstanpy/issues/425) and [Bugfix/425 fixed params by mitzimorris · Pull Request #427 · stan-dev/cmdstanpy · GitHub](https://github.com/stan-dev/cmdstanpy/pull/427)