Systematic error treatment

Interfaces
1

I hope this question is not too trivial, however since I was not able to find a definitive answer I would like to ask it here.
I found that systematical errors are supposed to be treated as a nuisance prior in a fully bayesian approach, however I am unsure how to implement such a nuisance prior in a model.

In case I am terrible at explaining the question, I hope this clarifies the issue.
I am wondering how to implement a respective nuisance prior in a model similar to the following:

model = """
data {
    int<lower=0> N;                          // number of datapoints
    vector[N] x;                                 // measured x-value
    vector[N] y;                                 // measured y-value
    real<lower=0> sigma_stat[N];   // statistical y-error
    real<lower=0> sigma_syst[N];   // systematical y-error
}
parameters {
    real alpha;             // a in the linear model ax+b
    real beta;              // b in the linear model ax+b
}
model {
    // priors
    alpha ~ normal(0, 1);          
    beta ~ normal(0, 1);  
    
    // likelihood
    y ~ normal(alpha * x + b, sigma_stat);
}
"""
1 Like
2

There are some error measurement model examples here:

2 Likes
3

Thank you for this answer.
I did read the Guide, however I am under the impression that the chapter 6.1 deals with statistical errors and not systematical errors, unless the last passage can be generalized to also be a way to deal with systematical errors.

2 Likes
4

Not sure if that’s still relevant or if you manage to resolve this. However, I don’t think I understand what you mean by “nuisance priors” or how those could resolve systematic errors in data. Some sort of measurement error model can be helpful sometimes (as @Ara_Winter mentioned), but in general systematic errors are hard to deal with statistically… Do you have a reference for this claim?

5

It is definitely still of interest.
What I meant by nuisance prior was not exactly precise, given that I came across papers in which it was reference that systematical errors could be treated by a nuisance parameter (since all the knowledge about the estimated deviation of the systematical error has to go into the prior I was calling it a nuisance prior), however I was unable to find anything further, which is why asked this question in the first place.
I found an interesting article on this (https://link.springer.com/article/10.1140/epjc/s10052-019-6644-4) and would currently say that a systematical error should propably be modeled as an offset to the relevant variable with a prior that incorporates the estimations on the systematical error, although this is still vague.
In the case that I am dealing with that might be helpful, given that the model does not include another constant offset that is fit to the observable.
I would be grateful for anything else you could add to this.

1 Like
6

So just to be clear, when speaking about “systematic error”, I mean something like the wiki page, in particular it is an error that is the same (or almost the same) for all observations of the variable. Is that the meaning you are using as well?

I quickly skimmed the reference you provided and my impression is that they actually do not expect a systematic bias, but rather a (specific form of) statistical uncertainty, which looks actually quite similar to the measurement error model.

This could definitely be done. I think a potential problem here is that such model could be IMHO very sensitive to model misspecification - the posterior for such a systematic error term will be (crudely speaking) informed by the deviation of the data from a perfect fit. But if the perfect fit is wrong, this could be misleading.

The reference also notes:

First nuisance parameters are introduced into the model to parameterize the systematic uncertainty, and then these parameters are constrained by means of control measurements

the second part seems crucial to me: if you have control measurements to constrain the systematic errors, the dependence on the details of the rest of the model could likely be greatly reduced.

I hope I didn’t misunderstand something and that this musing is at least a bit helpful :-)

1 Like
7

Yes, that is the kind of error I am refering to.

This is definitely true, although I would consider the problems resulting from deviations, compared to a perfect fit, to be unavoidable. In the end the result of this parameter is just an estimate with an estimated deviation.
Furthermore I would say that one should be able to judge the potential problems resulting from such deviations based on the results of a reasonable cross-validation of the fit.

No, this is definitely helpful.
Thank you very much. :)