Application of type M and S errors to meta analysis

General
1

Hi everyone,

I am looking to apply the concept of type M and S errors to a network meta-analysis I’m working on, hoping to use the following function developed by Andrew

retrodesign <- function(A, s, alpha=.05, df=Inf, n.sims=10000){
z <- qt(1-alpha/2, df) 
p.hi <- 1 - pt(z-A/s, df) 
p.lo <- pt(-z-A/s, df)  
power <- p.hi + p.lo
typeS <- p.lo/power
estimate <- A + s*rt(n.sims,df)
significant <- abs(estimate) > s*z
exaggeration <- mean(abs(estimate)[significant])/A
return(list(power=power, typeS=typeS, exaggeration=exaggeration))
}

Where A is your hypothesized true effect and s is the standard error of your estimate. The code for NMA calculates basic parameters which in my case is the mean difference between each active treatment and placebo. These are then used to calculate all the remaining pairwise comparisons. My question is an embarassingly simple one, and is really just meant to make sure I am not making a huge error:

Am I right to calculate the standard error for a comparison based on the SD of the mcmc results for that parameter (e.g. sd/sqrt(n))? Or do I just use the SD of the estimate in lieu of the standard error?

2

I have no idea about Andrew’s function, but given a parameter,

  • its MCMC standard deviation is the variation of the parameter in the posterior, whereas

  • its MCMC standard error is the (estimated) error in estimating the posterior mean, which is defined as the MCMC standard deviation divided by the square root of the effective sample size (n_eff in the interfaces).

So they measure completely different things. With more posterior draws, the MCMC standard deviation doesn’t change, but the MCMC standard error goes down as you get a larger sample with which to estimate a mean.