Interpretation of CIs of acc statistic

Can anyone provide any information about how the acc statistic and the corresponding CIs are calculated during the variable selection procedure with projpred?

Many thanks

Hi @T_nick,

acc is the classification accuracy, i.e., the proportion of correctly classified observations. For this, the response category (“class”) with highest probability (the probabilities are model-based) is taken as the prediction (“classification”) for an observation. I think we need to explain this in the documentation, thanks for asking.

1 Like

Thanks for the prompt response and the nice explanation! Can you please also provide some clarity about the calculation of CIs of acc values?

Oh yes, I forgot that. For the acc, the CIs are based on a normal approximation. The relevant lines are and where you can see that essentially, the SE of the “correct classification proportion” (i.e., the SE of the mean of the observation-wise zero-one values indicating correct classification) is calculated and after that, this SE is used to build a normal-approximation CI.

Perfect this is exactly what I needed! Thanks again

1 Like