The Monster Model | Hierarchical ODE models

Modeling
, ,
16

I was trying to express the difficulty of communicating the importance of time_step, which some dynamic modelers around me regard as nuts and bolts and should not asymptotically impact the outcome. So I wanted to ask how you choose your time_step. From your answer, it seems to be adaptively chosen, but for now, the simulation in dynamic modeling software (called Vensim) does not support changing time_step. Experts have heuristic of halving the time_step until state time-series matrix doesn’t change much.

In Stan community, \frac{t_{est}}{t_{gen}} > 1, but in dynamic simulation community, \frac{t_{est}}{t_{gen}} << 1 where t_{gen}, t_{est} are time spent in designing generator (parameter scaler values to state time-series matrix) and estimator (reverse of generator). From trustable model perspective, \frac{t_{est}}{t_{gen}} \sim 1 and symmetric generator and estimator are ideal. However, I discovered some model components that were asymmetrically focused in two communities. Topic of adaptable time_step, identifiablity, diagnostics are much valued in estimator-focused community where as process noise, aggregation get greater attention in generator-focused community. The difficulty of SBC lies in matching these asymmetry which were not surfaced when only one side of the joint model (joint of generator and estimator which forms joint distribution of parameter and observation) was the focus.

Tagging @maxbiostat as we had similar conversation (difficulty of communicating the importance of nuts and bolts). For instance, S, M, N, P, Q, R from the mapping below affects SBC result greatly (what the new SBC paper partly addresses.

Reversible mapping between \tilde{\theta}_{P, S}, Y_{Q, N}, \theta_{P, SM}

command in stanify in Bayes-inference community (software: Stan) in dynamic simulation community (software: Vensim) example
stanify command
draws2data(model, S) generate: map from parameter \theta region to observation Y region run
data2draws(model, M) estimate: map from observation Y region to parameter \theta region MCMC
draws2data2draws(model, S, M, N) composition of draws2data and data2draws x
model.set_prior() estimated_parameter and its prior distribution .voc has names of estimated_parameter and range model.set_prior(“prey_birth_frac”, “normal”, 0.8, 0.08, lower = 0), model.set_prior(“pred_birth_frac”, “normal”, 0.05, 0.005, lower = 0)
model.set_numeric() assign numeric vector to driving data
model.update_numeric() assign numeric scalar to assumed_parameter, assign numeric vector to driving data express difference between generator and estimator
model assumption
vensim mdl filepath . vensim_files/prey_predator.mdl
setting names of estimated_parameter, target_simulated_stock, driving_data binding parameter, target, driving { ‘estimated_parameter’:(‘prey_birth_frac’, ‘pred_birth_frac’,), ‘driving_vector_name’: ‘process_noise_uniform’, ‘target_simulated’: (‘prey’, ‘predator’)}
numeric prior information for estimated parameters {‘prey_birth_frac’: (0.8, 0.08, ‘normal’),‘predator_birth’: (0.05, 0.005, ‘normal’)}
classification settings
estimated_parameter parameters in .voc prey_birth_frac, pred_birth_frac
assumed_parameter all parameters in .mdl except estimated_parameter prey_death_frac: .05, pred_death_frac: .8
numeric settings
S # of draws from prior # of synthetic datasets (sensitivity check run) 1**
M # of draws from posterior (# of chains * # of draws from each chain) # of effective MCMC samples 4 * 100
N length of driving_data (final_time - initial_time)/time_step 200
P # of estimated_parameter # of parameters in .voc file 2 (prey_birth_frac, pred_birth_frac)
Q # of target_simulated_stock # of time series vectors to be matched 2 (prey, predator)
R*** # of subgroups for hierarchical Bayes elmcount(subscript) 2 region: R1, R2
noise settings
p_noise process noise
m_noise measurement noise, additive** **
1 Like