# Log target vs. non linear model

**URL:** <https://discourse.mc-stan.org/t/log-target-vs-non-linear-model/20664>\
**Category:** brms\
**Tags:** brms\
**Created:** [February 7, 2021, 11:17am UTC](https://discourse.mc-stan.org/t/log-target-vs-non-linear-model/20664 "2021-02-07T11:17:01Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Duncan\_Stoddard](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/duncan_stoddard/32/6421_2.png) [@Duncan\_Stoddard](https://discourse.mc-stan.org/u/Duncan_Stoddard)\
**Post date:** [February 7, 2021, 11:17am UTC](https://discourse.mc-stan.org/t/log-target-vs-non-linear-model/20664/1 "2021-02-07T11:17:01Z")

</div>

Key question: should I use log(y) ~ … OR y ~ exp(…) OR y ~ … with lognormal family…or something else…

I’ve inherited a model that looks like this:

```
m1 <- brm(
  bf(y / off_set ~ exp(b0 + b1 * f1 + b2 * f2),
     b0 ~ 1,
     b1 ~ 1,
     b2 ~ 1, 
     nl=TRUE),
  prior = c(
    prior(normal(0, 5), nlpar = "b0"),
    prior(normal(0, 5), nlpar = "b1"),
    prior(normal(0, 5), nlpar = "b2")
  ),
  data = df,
  family = gaussian()
)

```

It gives reasonable predictions, but, given y \> 0, off\_set \> 0, the ppc\_check doesn’t look ideal to me:

 ![Screenshot 2021-02-07 at 11.07.20](https://canada1.discourse-cdn.com/flex030/uploads/mc_stan/original/2X/b/bdf97006204acaf11690d6dcab0c44e820dd1365.png)

I"d thought a better and simpler way to specify the model would be

```
m2 <- brm(log(y / off_set) ~ f1 + f2,
           data = df)

```

Which gives a nice looking ppc\_check plot:

 ![Screenshot 2021-02-07 at 11.07.10](https://canada1.discourse-cdn.com/flex030/uploads/mc_stan/original/2X/0/0a65209210f9b39ef2897dae660d5af7c8536264.jpeg)

But on a crude measure of fit, the first seems to perform better - that is, comparing y to exp(pred) \* off\_set vs. pred \* offset for the first model…

Any help / guidance much appreciated!

---

<div class="post-metadata">

**Author:** ![bbbales2](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bbbales2/32/77_2.png) [@bbbales2](https://discourse.mc-stan.org/u/bbbales2)\
**Post date:** [February 8, 2021, 4:52pm UTC](https://discourse.mc-stan.org/t/log-target-vs-non-linear-model/20664/2 "2021-02-08T16:52:42Z")

</div>

> [@Duncan\_Stoddard](#):
>
> But on a crude measure of fit

I would question that measure of fit then. The PPCs for the second thing look way better – maybe it’s something to do with the scales here? One is on the log scale and the other is not.

From that second plot too, just go ahead and use a lognormal family if your data is constrained positive. Use whichever scale makes more sense for your problem. If it’s the log thing that you’re concerned with, just do `log(y) ~ normal(...)`. If the application makes more sense on the non-log scale, then do `y ~ lognormal(...)`.

---

<div class="post-metadata">

**Author:** ![Duncan\_Stoddard](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/duncan_stoddard/32/6421_2.png) [@Duncan\_Stoddard](https://discourse.mc-stan.org/u/Duncan_Stoddard)\
**Post date:** [February 8, 2021, 5:24pm UTC](https://discourse.mc-stan.org/t/log-target-vs-non-linear-model/20664/3 "2021-02-08T17:24:55Z")

</div>

Super, thanks for your reply, appreciate it!

---

<div class="post-metadata">

**Author:** ![Duncan\_Stoddard](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/duncan_stoddard/32/6421_2.png) [@Duncan\_Stoddard](https://discourse.mc-stan.org/u/Duncan_Stoddard)\
**Post date:** [February 14, 2021, 5:14pm UTC](https://discourse.mc-stan.org/t/log-target-vs-non-linear-model/20664/4 "2021-02-14T17:14:25Z")

</div>

One follow up question…

The distribution i’m most interested in is the sum of the predictive posteriors. I.e. once i’ve got e.g. 500 samples for N data points, i want to sum over N and take summary stats from that distribution.

The problem is with `x ~ lognormal(...)` i get way high values. e.g. a median ~ 6x larger than the actual sum of x in the training data.

Is the solution to truncate the distribution? If so, how do I get samples for truncated lognormal with below generated quantities?

Thanks in advance

```
generated quantities{
      real y_pred[N];

      for(n in 1:N){
           y_pred[n] = lognormal_rng(mu[n], sigma); 
      }
 }
```

---

<div class="post-metadata">

**Author:** ![bbbales2](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bbbales2/32/77_2.png) [@bbbales2](https://discourse.mc-stan.org/u/bbbales2)\
**Post date:** [February 15, 2021, 4:05pm UTC](https://discourse.mc-stan.org/t/log-target-vs-non-linear-model/20664/5 "2021-02-15T16:05:34Z")

</div>

> [@Duncan\_Stoddard](#):
>
> Is the solution to truncate the distribution?

I’m not sure what the solution is but the diagnostic is similar to the previous one.

When we make predictions with this new thing, even though the distribution of `y` and `y_rep` looks similar (at least I’m assuming on a log scale they look like the second plot here: [Log target vs. non linear model](https://discourse.mc-stan.org/t/log-target-vs-non-linear-model/20664)), `sum(y_rep)` produces values far away from `sum(y)` (make a plot of this probly). Probably make a plot of this to see what is happening.

I’m not sure about a solution though. Maybe it is that you are able to fit individual outputs but the sum seems off? It could be the individuals aren’t fit as well as we think, or the outputs weren’t IID or something – I’m not sure. It is definitely fair game for a new post (that way you might attract more new eyes to the problem and it is different than original question here).

---

<div class="post-metadata">

**Author:** ![BrittaL](https://avatars.discourse-cdn.com/v4/letter/b/5fc32e/32.png) [@BrittaL](https://discourse.mc-stan.org/u/BrittaL)\
**Post date:** [February 16, 2021, 9:39am UTC](https://discourse.mc-stan.org/t/log-target-vs-non-linear-model/20664/6 "2021-02-16T09:39:11Z")

</div>

I might have similarly structured data and would be happy to share my experience, but I am not sure how similar it is. Can you talk more about what y and off\_set mean?  
How variable is off\_set? Why are you interested in the sum over N datapoints?

---

<div class="post-metadata">

**Author:** ![Duncan\_Stoddard](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/duncan_stoddard/32/6421_2.png) [@Duncan\_Stoddard](https://discourse.mc-stan.org/u/Duncan_Stoddard)\
**Post date:** [February 16, 2021, 10:12am UTC](https://discourse.mc-stan.org/t/log-target-vs-non-linear-model/20664/7 "2021-02-16T10:12:49Z")

</div>

Sure! y and offset are sales and population by geographic region, respectively. Both right skewed and the ratio between them has been seen as an appropriate target variable (though ultimately sales predictions are what we’re after).

And the aggregation we want is by group - i.e. the prediction distribution for a number of categories (e.g. shop or city etc).

Thnx