Hi,
sorry for not getting to you earlier, your question is relevant and well written. If I understand you
Unfortunately, you cannot use splines as “random” or “varying” effects, because the penalized splines from mgcv which brms relies on are already implemented as a varying effect (see e.g. Random effects and penalized splines are the same thing - Higher Order Functions for background).
So if you want a smooth term that is at the same time a “random” effect, you need to work directly with non-penalized splines. That’s IMHO usually not a big deal, since you rarely want to allow the per-individual spline to be very complex (you can’t learn them reliably anyway), so a low-degree-of-freedom spline should work just well. So something like:
y ~ Treatment + s(t, by = Treatment) + (1 + bs(t, df = 2)|individual)
Could be sensible. (this requires the splines package. You may also want to use ns but I would expect the results to be quite comparable)
I’ll tag @paul.buerkner if he knows about other solutions.
EDIT: I realized probably forgot to attend to the core of your question - if you have s(t, by = Treatment), it is not a problem that you get estimates separately for both groups. Intuitively: you fit a separate splines for both groups which is well-defined and the splines from s() do not contain intercept which is the only term that could interact with the the other terms already in the model.
Note that in GAM models you do not remove some of the terms “just because”. You remove them, because they are shared and implicitly already in the model. So e.g. if you have two binary predictors A and B and have y ~ A * B you are actually fitting 2x2 separate means. The four groups are called “Intercept”, “Intercept + ATrue”, “Intercept + BTrue” and “Intercept + ATrue + BTrue + ATrue:BTrue”. So it is not that you would treat A as having just one level. What happens is that one of the levels is already present in the model thanks to the intercept, so you do not add it once more. The per-group splines are however coded so that the terms are not shared (akin to what you get with y ~ 0 + A). Hope the previous paragraph did make at least some sense :-)
Best of luck with your model!