# Data for Bayesian hierarchical weighting adjustment and survey inference

I am going to use your hierarchical weighting method from your 2017 paper for an online survey project to do post-stratified inference. While the paper is pretty clear, the example code shown in the appendix isn’t as useful as it could be because the data referenced in the code isn’t available from the paper.

Do y’all have that data? It’s apparently from the ACS. I could dig up ACS data myself but I wanted to use the same dataset y’all used in the paper.

Also if you have your simulation code available, that would be really helpful too.

I’m posting on here as I assume others might be interested in being able to run the full example code. Also there is a function in the code, `mrp_structured`, that is referenced in rstanarm but that I can’t find in rstanarm on Github or CRAN.

As a reference, code cut and pasted from the appendix is below:

``````model_based_cell_weights <- function(object, cell_table) {
stopifnot(
is.data.frame(cell_table),
colnames(cell_table) == c("N", "n")
)
draws <- as.matrix(object)
Sigma <- draws[, grep("^Sigma\\[", colnames(draws)), drop = FALSE]
sigma_theta_sq <- rowSums(Sigma)
sigma_y_sq <- draws[, "sigma"]^2
Ns <- cell_table[["N"]] # population cell counts
ns <- cell_table[["n"]] # sample cell counts
J <- nrow(cell_table)
N <- sum(Ns)
n <- sum(ns)
# implementing equation 7 in the paper (although i did some algebra first to
# simplify the expression a bit)
Nsy2 <- N * sigma_y_sq
ww <- matrix(NA, nrow = nrow(draws), ncol = J)
for (j in 1:J) {
ww[, j] <-
(Nsy2 + n * Ns[j] * sigma_theta_sq) / (Nsy2 + N * ns[j] * sigma_theta_sq)
}
return(ww)
}
# prepare population data: acs_ad has age, eth, edu and inc
mutate(
cell_id = paste0(age, eth, edu, inc)
acs_design <- svydesign(id = ~1, weights = ~perwt, data = acs_ad)
agg_pop <-
svytable( ~ age + eth + edu + inc, acs_design) %>%
as.data.frame() %>%
rename(N = Freq) %>%
mutate(
cell_id = paste0(age, eth, edu, inc)
) %>%
# prepare data to pass to rstanarm
# SURVEYdata has 4 weighting variables: age, eth, edu and inc; and outcome Y
dat_rstanarm <-
SURVEYdata %>%
mutate(
cell_id = paste0(age, eth, edu, inc)
)%>%
group_by(age, eth, edu, inc) %>%
summarise(
sd_cell = sd(Y),
n = n(),
Y = mean(Y),
cell_id = first(cell_id)
) %>%
mutate(sd_cell = if_else(is.na(sd_cell), 0, sd_cell)) %>%
left_join(agg_pop[, c("cell_id", "N")], by = "cell_id")
# Stan fitting under structured prior in rstanarm
fit <-
stan_glmer(
formula =
Y ~ 1 + (1 | age) + (1 | eth) + (1 | edu) + (1 | inc) +
(1 | age:eth) + (1 | age:edu) + (1 | age:inc) +
(1 | eth:edu) + (1 | eth:inc) +
(1 | age:eth:edu) + (1 | age:eth:inc),
data = dat_rstanarm, iter = 1000, chains = 4, cores = 4,
prior_covariance =
rstanarm::mrp_structured(
cell_size = dat_rstanarm\$n,
cell_sd = dat_rstanarm\$sd_cell,
group_level_scale = 1,
group_level_df = 1
),
seed = 123,
prior_aux = cauchy(0, 5),
prior_intercept = normal(0, 100, autoscale = FALSE),
)
# model-based weighting
cell_table <- fit\$data[,c("N","n")]
weights <- model_based_cell_weights(fit, cell_table)
weights <- data.frame(w_unit = colMeans(weights),
cell_id = fit\$data[["cell_id"]],
Y = fit\$data[["Y"]],
n = fit\$data[["n"]]) %>%
mutate(
w = w_unit / sum(n / sum(n) * w_unit), # model-based weights
Y_w = Y * w
)
with(weights, sum(n * Y_w / sum(n)))# mean estimate
``````

Thanks much for writing the method and for your help!

Bob

2 Likes

I have nothing to offer regarding the code, but you might want to check this thread for information on mrp_structured(): https://discourse.mc-stan.org/t/fitting-a-large-data-hierarchical-model-can-i-trust-variational-inference-algorithm/1780.

Specifically, @bgoodri said (in '17) that it’s available in one of rstanarm’s branches and that it can be installed via:

``````devtools::install_github("stan-dev/rstanarm", ref = "structured_prior_merge",
args = "--preclean", build_vignettes = FALSE)
``````

I do have the data but unfortunately I don’t think I’m allowed to share it at the moment. The ACS stuff is public but most of the data required to run our code is from the New York City Longitudinal Study of Wellbeing and I don’t think they have made that data public yet (they definitely will at some point). @lauren do you by any chance have any idea what the timeline is for that?

And yeah it looks like the arxiv version of the paper needs to be updated to mention that the rstanarm version required is just on a branch. If we publish the paper then we’ll likely try to get it implemented in an rstanarm release at some point.

@Jonah can you link the paper so I can check what years you used so I can check?

Hello, have there been any developments on this thread? I am also interested in reproducing the analysis from this paper (if possible)…
Thanks!

1 Like

What do we want? Structured multilevel regression & post-stratification.

Where do we want it? It’d be nice to have it out in the `rstanarm` repo.

1 Like

Hey @lauren, I’m pretty sure this is the paper:

1 Like

Hi, I am trying to install this branch using code

``````devtools::install_github("stan-dev/rstanarm", ref = "structured_prior_merge",
args = "--preclean", build_vignettes = FALSE)
``````

Unfortunately, it failed

``````if (!require(devtools)) {
install.packages("devtools")
library(devtools)
}
install_github("stan-dev/rstanarm", ref = "structured_prior_merge", args = "--preclean", build_vignettes = FALSE)
#>
#>          checking for file 'C:\Users\liliz\AppData\Local\Temp\RtmpUN5WGe\remotes80448383394\stan-dev-rstanarm-b46c14f/DESCRIPTION' ...  v  checking for file 'C:\Users\liliz\AppData\Local\Temp\RtmpUN5WGe\remotes80448383394\stan-dev-rstanarm-b46c14f/DESCRIPTION'
#>       -  preparing 'rstanarm':
#>    checking DESCRIPTION meta-information ...     checking DESCRIPTION meta-information ...   v  checking DESCRIPTION meta-information
#> -  cleaning src
#>       -  running 'cleanup.win'
#>       -  checking for LF line-endings in source and make files and shell scripts (2.7s)
#>       -  checking for empty or unneeded directories
#>       -  building 'rstanarm_2.15.4.tar.gz'
#>   Warning:     Warning: file 'rstanarm/cleanup' did not have execute permissions: corrected
#>
#>
#> Installing package into 'C:/Users/liliz/OneDrive/Documents/R/win-library/4.0'
#> (as 'lib' is unspecified)
#> Warning in i.p(...): installation of package 'C:/Users/liliz/AppData/Local/Temp/
#> RtmpUN5WGe/file8041ce74daa/rstanarm_2.15.4.tar.gz' had non-zero exit status
#> Warning: 1 components of `...` were not used.
#>
#> We detected these problematic arguments:
#> * `args`
#>
#> Did you misspecify an argument?
``````

Do you have any suggestions? Thanks!

I think the developers of the `install_github` function have changed its arguments and it no longer has an `args` argument. It looks like the `...` argument to `install_github` is passed to `install.packages` so maybe instead of `args = "--preclean"` you could try `INSTALL_opts = "--preclean"` and see if that works, but I’m not entirely sure.

Also just a heads up: in theory this rstanarm branch should still work, but it’s quite old at this point, was always “experimental”, and hasn’t been maintained for a while so no guarantees unfortunately.

1 Like

Thank you for your quick response. I tried again, but still failed.

``````devtools::install_github("stan-dev/rstanarm", ref = "structured_prior_merge", INSTALL_opts = "--preclean", build_vignettes = FALSE)
#>
#>          checking for file 'C:\Users\liliz\AppData\Local\Temp\Rtmp8GTV3T\remotes62084cab602e\stan-dev-rstanarm-b46c14f/DESCRIPTION' ...  v  checking for file 'C:\Users\liliz\AppData\Local\Temp\Rtmp8GTV3T\remotes62084cab602e\stan-dev-rstanarm-b46c14f/DESCRIPTION'
#>       -  preparing 'rstanarm':
#>    checking DESCRIPTION meta-information ...     checking DESCRIPTION meta-information ...   v  checking DESCRIPTION meta-information
#> -  cleaning src
#>       -  running 'cleanup.win'
#>       -  checking for LF line-endings in source and make files and shell scripts (2.7s)
#>       -  checking for empty or unneeded directories
#>       -  building 'rstanarm_2.15.4.tar.gz'
#>   Warning:     Warning: file 'rstanarm/cleanup' did not have execute permissions: corrected
#>
#>
#> Installing package into 'C:/Users/liliz/OneDrive/Documents/R/win-library/4.0'
#> (as 'lib' is unspecified)
#> Warning in i.p(...): installation of package 'C:/Users/liliz/AppData/Local/Temp/
#> Rtmp8GTV3T/file62084a7643ab/rstanarm_2.15.4.tar.gz' had non-zero exit status
``````

Does this imply that there is a problem with the rstanarm branch?

Hmm, I’m not sure. Can you try this and see if it changes anything?

``````Sys.setenv("R_REMOTES_NO_ERRORS_FROM_WARNINGS" = "true")

remotes::install_github("stan-dev/rstanarm", ref = "structured_prior_merge", INSTALL_opts = "--preclean", build_vignettes = FALSE)
``````

Still same. There is ERROR: compilation failed for package ‘rstanarm’.

``````Sys.setenv("R_REMOTES_NO_ERRORS_FROM_WARNINGS" = "true")

remotes::install_github("stan-dev/rstanarm", ref = "structured_prior_merge", INSTALL_opts = "--preclean", build_vignettes = FALSE)
#>
#>          checking for file 'C:\Users\liliz\AppData\Local\Temp\Rtmpig5MV0\remotes2d604db4909\stan-dev-rstanarm-b46c14f/DESCRIPTION' ...  v  checking for file 'C:\Users\liliz\AppData\Local\Temp\Rtmpig5MV0\remotes2d604db4909\stan-dev-rstanarm-b46c14f/DESCRIPTION'
#>       -  preparing 'rstanarm':
#>    checking DESCRIPTION meta-information ...     checking DESCRIPTION meta-information ...   v  checking DESCRIPTION meta-information
#> -  cleaning src
#>       -  running 'cleanup.win'
#>       -  checking for LF line-endings in source and make files and shell scripts (2.5s)
#>       -  checking for empty or unneeded directories
#>       -  building 'rstanarm_2.15.4.tar.gz'
#>   Warning:     Warning: file 'rstanarm/cleanup' did not have execute permissions: corrected
#>
#>
#> Installing package into 'C:/Users/liliz/OneDrive/Documents/R/win-library/4.0'
#> (as 'lib' is unspecified)
#> Warning in i.p(...): installation of package 'C:/Users/liliz/AppData/Local/Temp/
#> Rtmpig5MV0/file2d6053991a97/rstanarm_2.15.4.tar.gz' had non-zero exit status
``````
``````make: *** [C:/PROGRA~1/R/R-40~1.5/etc/x64/Makeconf:229: Modules.o] Error 1
ERROR: compilation failed for package 'rstanarm'
* removing 'C:/Users/liliz/OneDrive/Documents/R/win-library/4.0/rstanarm'
Warning message:
In i.p(...) :
installation of package ‘C:/Users/liliz/AppData/Local/Temp/RtmpQZ7rjs/file48784f883b9a/rstanarm_2.15.4.tar.gz’ had non-zero exit status
``````

Is there anyway you can share the R function of

``````rstanarm::mrp_structured
``````

as I am really interested in this structured prior distributions. Thanks in advance!!

1 Like