Hi! I’m fairly new to Stan. I have a large Supply-Use Table balancing problem with ~700k parameters. For small data, Stan works fine, but when I scale to 5–10% of real data, sampling becomes extremely slow (8+ hours for 500 warmup) and the chains appear stuck. I tried reduce_sum, raising adapt_delt…

Did you start small and build up the model from an extremely basic one? That’s the user guide advice, because we are building software products when we compile Stan models. I’ve always stuck strictly to that, and it highlights where the bottlenecks are as one adds complications one at a time. I susp…

[image] simschul: csr_matrix_times_vector(N_pr, N_use, w_pr_use, v_pr_use, u_pr_use, x_use) N_pr, N_use, w_pr_use, v_pr_use, and u_pr_use are all data. Consequently, a lot of the work that csr_matrix_times_vector is doing just repeats the same thing each iteration, reconstructing the sparse m…

Thanks for your suggestions. [image] simonbrauer: e. I’m not familiar with the efficiency of csr_matrix_times_vector but I suspect you could speed this up a lot by pre-computing the matrix in the transformed data block. The reason I use csr_matrix_times_vector is that the matrices (which map…

Thanks for your comment and the encouragement (I put myself into that difficult position :) ). Yes, indeed I started with a very small, simple toy data set, for which the model works fine (also for very high prior imbalances it finds a solution). The problem really seems to be when increasing the si…

[image] simschul: The goal of my model is to make sure that: The sum of all inputs into an activity equals the sum of all outputs (activity balance) The sum of all products supplied equals the sum of all products used (product balance) (… other constraints that don’t matter for now) Is it t…

thanks @Bob_Carpenter for your thoughts! really appreciate. [image] Bob_Carpenter: Is it the goal of the model to ensure that or to evaluate whether it holds? The goal is indeed to ensure it. Our “unbalanced” data come from many sources and show discrepancies (e.g., country A reports 100 t o…

[image] Bob_Carpenter: You can parameterize directly with x_sup and x_use by declaring Oh yes, this looks cleaner, I will try.

[image] simschul: Using your example, my problem is that I have cake pieces of different height and color. I need that the pieces have to add up to a cake when sorting them by height and color. So it’s more complicated than a single simplex. If anyone has examples or references on enforcing mult…

[image] simschul: If anyone has examples or references on enforcing multi‐dimensional constraints in Stan, I’d really appreciate them! Do you mean you have a matrix where the rows have constrained sums and the columns have constrained sums? As @jsocolar noted, @spinkney is about to roll out …

@Bob_Carpenter I thought a doubly stochastic matrix is one where all rows and columns are simplexes? Doubly stochastic matrix - Wikipedia But now I’m not totally sure which one @spinkney is working on–a sum-to-zero or the wikipedia version.

Stan model with ~700K parameters: Troubleshooting slow, stuck chains

Modeling

jsocolar March 31, 2025, 7:11pm 9

@spinkney has a parameterization of doubly stochastic matrices in the works, per this post

Topic		Replies	Views
Inequality constraints on linear combinations of parameters Modeling specification , priors	21	775	March 23, 2025
Stuck at Warmup iteration with no error : CmdStanR CmdStan techniques , fitting-issues	48	3925	April 21, 2020
Benchmarking with Inequality constraints on the true value in Stan Modeling	7	244	February 4, 2025
Single Valued Chains Modeling	9	1305	May 8, 2017
Using identities in model formulation Modeling	3	805	January 17, 2018

Stan model with ~700K parameters: Troubleshooting slow, stuck chains

Related topics