I’ve developed some code that does periodic Bayesian updating: fitting a beta distribution to [0,1]-bounded time series data. I implemented “forgetting” at each update by expanding the posterior’s (phi and lambda’s) SDs as the posterior becomes the prior for the subsequent iteration. The goal is to adapt to any drift, but also to minimize potential overfitting to fluctuations in the data. I think it works great!
My post isn’t really “publicity”; rather, it’s a request for pointers to any similar work.
- Any pointers to work on Bayesian updating with “forgetting” by manipulating distributions?
- Any other approaches that accomplish similar results? Perhaps somehow “de-weighting” older data’s contributions to the posterior distributions? There’s a ton of time series work for de-weighting older observations for computing means. (E.g, exponential smoothing.) But are there ways to “de-weight” older data when computing a full posterior distribution?
- Perhaps there are particular application areas (presumably that deal with time series data) where I might focus my literature search?
Many thanks in advance for any pointers!
If you’re interested, a brief summary of my approach – and a few plots – follow. The steps:
- Generate fake [0,1]-bounded time series data drawn from a time-varying beta distribution.
- Divide the data up into sequential “chunks.”
- Fit a beta distribution to the first chunk (using Stan). Extract the posterior mean and SD of the two parameters, phi and lambda.
- For update #1 (with the 2nd chunk of data), use the previous posterior (as summarized by phi and lambda’s means and SDs) as the new prior, but with phi and lambda’s SDs multiplied by, e.g., 1.4. (Which diffuses the new prior, relative to the previous posterior.)
- Repeat steps 3 and 4 for each subsequent update on each subsequent chunk of data…
(Many thanks to @LucC for suggesting I use the phi/lambda parameterization; big help!)
And some plots…
In each plot I show three time series: the raw data, a sliding window approach, and “post”: the posterior using my “forgetting” approach. The top frame in each plot just shows the series’ means, to provide a rough idea of how the three approaches compare. Below are separate frames for the three individual series which show the distributions’ 80% Percentile Intervals – to give at least a general feel for the posteriors’ distributions at each update. (For the Percentile Intervals for the data, I just used a non-Bayesian estimation for each data “chunk.”)
In the first plot I’ve implemented “forgetting” via phi and lambda SD multipliers of 1.4. (You can see the results in blue.) There’s a nice smoothness to the “forgetting” (and thereby in adapting to the new data). But also a bit of resistance to the “perturbation” at iterations 16-18. (By comparison, the sliding window approach, in green, of course “forgets” rather abruptly at iteration #8.)
Increasing the SD multipliers to 1.6 increases the degree of “forgetting.”
As an aside, here’s one phi and lambda posterior plot. The left figure is a posterior directly from Stan. The middle figure is 4000 points sampled from Stan’s marginal summary statistics of that posterior. Not exactly the same, but “close.” (There appears to be a slight parameter correlation in the “true” posterior on the left.) And the right figure is 4000 points sampled from an expanded posterior.
For comparison, here’s a plot with no forgetting, where new data has less and less impact on the posterior, due to the decreasing relative contributions of each subsequent “chunk” of data to each update’s posterior. (“No forgetting” in my code via just multiplying phi and lambda’s SDs by 1.0.)
A final aside is that the above plot is close to – but not identical to – just computing the posteriors at each iteration with the cumulative data to that point. (So there’s a little bit of loss from summarizing the successive full posteriors by only four marginal parameter values each…)