I’m really happy to share the first official CRAN release of the Bayesian Measurement Modeling (bmm) R package, developed together with @g.frischkorn 🥳 🥳 🥳

**tldr;**

🥇 Hierarhical Bayesian estimation of complex measurment models in nearly any design

🥈 Uses

brmsas a translation engine for Stan and integrates into the existing R infrastructure🥉 Provides a general Bayesian development framework for domain-specific models via highly modular code base, well-documented developer tools and useful templates

❓

Is this post for me if I am not a behavioral scientist?

✔️ If interested in implementing complex domain models, you might find the 2nd half relevant

👀

We are looking for collaborators, feedback, ideas…

🙏 Please get in touch if interested or share with colleagues/students who might be

## The what

What does **bmm** do? I briefly mentioned it in a comment to @jsocolar’s thread a few months ago: **bmm** translates complex psychological measurement models into **brms** syntax, allowing psychologists to estimate such models in a hierarhical bayesian framework, with minimal coding effort, for nearly any experimental design.

With the exception of drift diffusion models^{[1]}, this was not previously feasible without project-specific custom Stan or Jags scripting, usually accompanied by a lot of cursing, debugging and general despair (… just me?) **bmm** aims to solve this problem by doing for cognitive modeling what **brms** did for generalized multi-level regression. Because it uses **brms** under the hood, **bmm** returns an object that inherits from **brmsfit**; because of that, **bmm** integrates directly in the existing Bayesian R infrastructure. All the excellent tools built by this community for postprocessing, plotting, inference, model comparison, etc, can be used out of the box with bmm outputs.

In this blog post we give an overview of the package: the problems it solves, how to use it, and what its design principles are. There we focused more on the *user side*. But for the community here I thought it would be more valuable to highlight the potential role **bmm** could play as a general model development framework (stil very much a work in progress - we welcome all feeback, ideas & collaboration).

## The how

Our key insight was that despite their relatively higher complexity, many substantive scientific models^{[2]} can be expressed as *distributional regression problems*. Although 95% of **brms** users only care about how the conditional mean of the response changes with explanatory variables^{[3]}, **brms** truly begins to shine when used for distributional regression. Unlike its frequentist uncle **lme4**, **brms** lets you specify a predictor formula for any free parameter of supported distributions. In other words, there is nothing theoretically special about the mean/location parameter^{[4]} - you can just as easily estimate how the variance, scale, shift or any other parameter changes with any combination of explanatory variables.

In the end, if you can write down a theoretical model’s likelihood, you can treat it like any other distribution and estimate it via *brms*, taking advantage of it powerful and flexible formula syntax. Sometimes that’s easy to do with custom families, other times you might have to “abuse” some mathematical tricks and the non-linear formula syntax beyond their original design… but hey, if it works, it works.

In essence, just as **Stan** is a higher-level interface for **C++**, and **brms** is in turn a higher-level interface for Stan, **bmm** serves as a yet another level of abstraction over **brms** (“It’s turtles all the way down!”^{[5]} ). We are not the first to think of this, and I recently made a short list/review of all packages I could find that do something similar. These packages all appeared in the last year or so, and they share in common the idea of “hacking” **brms** to estimate various domain models. This is truly a testament to what an incredibly useful tool @paul.buerkner has built^{[6]}.

## The why

So if **brms** already offers custom_families and can be “hacked” to do this, why a separate package? We never planned to make a package - when we discovered how to implement some of our cognitive models 2 years ago, we initially wrote a tutorial about how to do precisely that. But at some point it occured to us that building a general framework that abstracts away has many benefits - drasticly increased code efficiency, no need to look-up the “formula tricks” for different projects, standardized and well documented models with known properties available to the community, lowering the technical barrier to colleagues not trained in math and modeling. Also, since the models we implement are grounded in particular subject-matter theories, **bmm** can provide more opinionated, empirically or theoretically grounded defaults for priors, comprehensive data checks, plausible initial values to improve sampling, etc.

## The future

Even though our current focus is on cognitive/psychological models, we have been consciously developing the package to be very modular, such that eventually it could be used as a general purpose framework for developing and testing domain-specific models from any discipline. Even currently, If a model’s likelihood function can be defined in Stan, and the parameters’ can be expressed as a generalized (non)-linear function of predictors (i.e. anything implementable in a brmsformula), **bmm** can probably implement it, regardless of the domain.

The core of the code base is built such that the model translation, data processing, prior configuration, etc, are all done via generic S3 methods. Thus, all it takes to implement a new model is to define it as a new S3 class, and provide corresponding S3 methods for each step in the pipeline.

To make this process easier, we have written a detailed developer’s guide and created a use_model_template() function, which will generate all necessary boilerplate code to define a new model class and the corresponding methods. If you want to add a new model, you only need to call this function, and fill out only the parts that are unique for the new model.

This makes it really easy to add new models *if you fork the github bmm package repo and want work with the developmental package*. But this makes the workflow a bit tedious, unless you absolutely plan to contribute the model to the package. I am now imagining a much better API for a future release, in which any user who loads the package via the good old `library(bmm)`

call, would be able to use constructor functions to define new models that will work immediately with bmm.

The same API could be used to develop new packages for models specific fields. That is, rather than making **bmm** a one-stop-shot with hundreds of models added over time, I imagine a core package that provides the interface and generics, which anyone can use with models defined in other packages. @jsocolar suggested something along those lines a few months ago, and I like this idea more and more.

This, at least for me would be incredibly useful when developing novel computational models. When we do that, it usually requires many different iterations over possible functional forms and parametrizations to find a version of the explanatory model that can account for many differrent target phenomena. So rather than my previous workflow of keeping dozens of variations of stan files, which I need to adjust the moment I want to test the models on a different set of experiments, we could just define all the different model versions as `bmmodel`

objects, take advantage of the modularized configuration functions that calibrate the models with experimental variables, and just do model estimation with bmm, getting a bmmfit result that then can be systematically compared across the family of candidate models.

These final thoughts are very much just at the ideation stage, but anything that let’s me spend less time organizing and rewriting code and more time thinking about theory is a win for me. I’m hoping that the right set of code design can make **bmm** not only useful for end users who just want to apply an established measurement model to their data for inferential purposes, but also to theoreticians who develop such models.

thanks to the native

**brms**and**Stan**support for the Wiener distribution ↩︎by “substantive” I mean models with ontological commitments whose parameters have a clear theoretical interpretation. In such models parameters are thought to represent measurable attributes of real entities and processes; as opposed to the data-agnostic off-the-shelf statistical models in which parameters are summary statistics. The line is sometimes blury, of course. ↩︎

I’m not trying to be…

*mean*. Those are Paul’s own words :) ↩︎It is only trivially special due to historical reasons and traditional statistical notation in R. The formula syntax

`y ~ x1`

does not mean that “the response y is distributed as x1”, as it might be in Stan. A typical call such as`lm(y ~ x, family = gaussian())`

or`brm(y ~ x, family = gaussian())`

means that “y follows a normal distribution whose \mu parameter is a linear function of predictor x1”, or :y \sim N(\mu, \sigma) \\ \mu = \beta_0+\beta_1xIn contrast to other regression functions in R,

**brms**let you specify a group of formulas, one for each distributional parameter. However, it has kept the`resp ~ predictor`

convention to denote the formula for the location parameter. Relevant discussions here and here. For me, an ideal API would make it transparent that y follows a certain distribution, and would separate that from the regression formulas of the parameters. The current mixed convention of`bf(y ~ x1, sigma ~ x1)`

is just confusing. Which is why we eventually decided to build a wrapper around`brmsformula()`

called`bmmformula()`

, which makes all parameters explicit and specifies the response in the model object instead. ↩︎…and they all speak math ↩︎

Paul has also been incredibly open to changes in brms that made the integration easier ↩︎