Seeking feedback on an educational paper on a Bayesian inference workflow in pharmacometrics using Pumas

Hi everyone,

This is my first post here but I have been a silent follower of this forum for a while now and I learnt a lot from many of the folks here. Since this is my first post, let me briefly introduce myself first. I am Mohamed Tarek, a scientist at PumasAI Inc., a company which develops a proprietary statistical software for pharmacometrics called Pumas that is free for academics. Before joining PumasAI, I was a developer on the Turing.jl team and a few of you may know me from there. My background is in mechanical engineering and computer science and I am a self-taught Bayesian who came into the Bayesian world from the open source software development side. I learnt almost everything I know about Bayesian inference from books, videos, documentation and papers produced by people on this forum and I have had my fair share of struggles getting NUTS to work well for some models that I worked with.

Together with my co-authors, I recently completed an educational manuscript titled “A Practitioner’s Guide to Bayesian Inference in Pharmacometrics using Pumas” and I am seeking some feedback and constructive criticism from both the experts and non-experts on this forum. In this paper, I attempted to give a somewhat comprehensive guide to doing Bayesian inference in pharmacometrics using Pumas the software. There is a tutorial-like section with code snippets and there is a more theoretical section in which I try to explain all of the main concepts that a user may need to know about to effectively use Bayesian inference in pharmacometrics. This is an arguably bold move given the many great resources out there already. But I wanted to write something more concise, accessible and comprehensive than anything I could find out there and I wanted it to be customised to pharmacometrics to have a common language with my target audience. In this theoretical section, I tried to focus on the intuition first and the mathematical details second, if at all. The target audience is people in the pharmacometrics space with no Bayesian background and who would like to get their feet wet with using MCMC in their work but they don’t understand how to use it or what it does, but more importantly they are not interested in understanding all of the details. The paper was largely inspired from the fantastic Bayesian workflow paper by Gelman et al and the Torsten paper by Margossian et al. so I would like to sincerely thank the authors of both these papers.

I hope this post is not an inappropriate way to seek feedback on my manuscript. In the paper, I also have a brief section on “Limitations of Other Software” in which I hope I didn’t step on too many toes but I am open to correcting any claims made there if anything I said was inaccurate. Anyways, I will leave you to it now and please let me know if you have any comments or questions. You can post comments here, or you can personally message me on Discourse or by email. My email is in the paper. I intend to iterate on this paper for the next few months until I am happy with it so whenever you get to it, feel free to contact me with your feedback.

Best regards,
Mohamed

5 Likes

bumping to get more views

Hi, @mohamed82008 and @storopoli (both are authors of this paper): It’s fine to ask whatever you’d like on our forums. The reason you’re not getting any responses is that (a) the post isn’t Stan-related, and (b) the software is proprietary.

I don’t want to look too closely because you’re behind a proprietary license. I did have a look at your section 2.6 on limitations. I agree that reduce_sum can be challenging for people who aren’t used to writing low-level code. I didn’t understand this criticism:

However, the APIs of both Stan and Torsten do not provide the best user experience for PMx users because they are based on C++, an arguably difficult-to-use, non-interactive programming language.

I’m not sure which API you’re talking about. Did you mean the Stan language (which itself feels a bit like C, but not C++) or the R/Python/Julia APIs?

I also didn’t understand this:

The second limitation of other software is that they do not support other non-Bayesian workflows that are common in PMx which means that users with a PMx background need to translate their models from other software to the PPL to compare the Bayesian and non-Bayesian results.

Are you talking about NONMEM and Monolix and the use of max marginal likelihood? Does your new package just accept NONMEM model specifications so users don’t have to rewrite anything? Stan only supports MCMC, penalized maximum likelihood, and variational inference.

As for background and related projects, you might also want to mention GNU MCSim - GNU Project - Free Software Foundation. It’s still being used.

2 Likes

That’s understandable. Thanks for taking some time yourself to look at it. A lot of the paper is actually somewhat generic, e.g. explaining how NUTS works or explaining diagnostics, but I understand that many people here may not like the idea of a paper based on proprietary software.

The Stan language itself. Fair point though, we should say C not C++.

For the most part yes along with some standard diagnostics and plots commonly done in Monolix or R.

You can define hierarchical models in it and it supports the NONMEM and Monolix algorithms for maximum marginal likelihood, together with the standard workflow, e.g. simulation, diagnostics and plots commonly used in the field.

đź‘Ť