Thanks for all the inputs! In short, AIC/WAIC seems it could fulfill what I am after, will test and let you know about the results. Some more thoughts/rambling follow.
I am definitely open to the possibility that I am completely wrong :-D Also I don’t think I should ignore prediction, just that optimizing for prediction seems to lead me to more complex models than I think is justifiable by the data+context.
The main problem IMHO is that you generally want to extrapolate from a single/a few experiment(s) to what is actually happening in the cells. But the experiments have many degrees of freedom that cannot be randomized. Just choosing a medium the cells grow on is at least 5 continuous parameters with highly non-linear effects on outcome. From my understanding (I am not a biologist), the experimenter basically tweaks the setup until they get reproducible results (in practice this can reduce to something like “similarly looking result at least twice in a row” or even “the cells don’t die most of the time”). This is not to bash the practices of the field, it just happens to be really hard to do most experiments. Sometimes people spend years before they are able to get the experiment running.
For this reason, measures akin to cross validation are IMHO not that relevant, because you know for sure that your data is not representative and squeezing it too hard is just chasing noise. On the other hand, we have a reasonably good understanding on how gene regulation happens at the level of molecules. You have to simplify a lot to be able to do inference over the model, but it lets us believe that extrapolating from the few observed cases is not completely doomed to failure.
Most often you just want to rank the possible regulations in the order of the level of trust you have in the results. The most direct further decision is then choosing some individual regulations for an in vitro or even in vivo experiment that can (in most cases) determine if the regulation takes place with high certainty. Those experiments are expensive, so you can do them on ~1% of the candidates.
Yes, basically automating PPC. Had some mixed results with this before: Automated posterior predictive checks
Turns out it is not really applicable here, so nevermind :-)