Can anyone provide any information about how the acc statistic and the corresponding CIs are calculated during the variable selection procedure with projpred?

Many thanks

Can anyone provide any information about how the acc statistic and the corresponding CIs are calculated during the variable selection procedure with projpred?

Many thanks

Hi @T_nick,

`acc`

is the classification accuracy, i.e., the proportion of correctly classified observations. For this, the response category (â€śclassâ€ť) with highest probability (the probabilities are model-based) is taken as the prediction (â€śclassificationâ€ť) for an observation. I think we need to explain this in the documentation, thanks for asking.

1 Like

Thanks for the prompt response and the nice explanation! Can you please also provide some clarity about the calculation of CIs of acc values?

Oh yes, I forgot that. For the `acc`

, the CIs are based on a normal approximation. The relevant lines are https://github.com/stan-dev/projpred/blob/560616ba87420beea21cfbee6620dbacbe63f961/R/summary_funs.R#L434-L445 and https://github.com/stan-dev/projpred/blob/560616ba87420beea21cfbee6620dbacbe63f961/R/summary_funs.R#L471-L474 where you can see that essentially, the SE of the â€ścorrect classification proportionâ€ť (i.e., the SE of the mean of the observation-wise zero-one values indicating correct classification) is calculated and after that, this SE is used to build a normal-approximation CI.

Perfect this is exactly what I needed! Thanks again

1 Like