To help solve the stray csv files problem, the workflow I imagine for stanflow has a helper bash script, stan, that writes the csvs to a model dependent output directory. read_stan reads only csvs from this output directory.
this API requires the user to specify a name for the output csv files - no defaults. is this too unpythonic to contemplate?
there’s a corresponding branch in the cmdstanpy repo that has the wrappers to compile a model and run the sampler implemented. wrapping the cmdstan utilities stansummary and diagnose should be fairly trivial. it’s the last step - creating a PosteriorSample object in a way thats efficient for downstream processing that’s the concern.
Is the idea that rather than specifying a bunch of .csv files, you’d just specify one for CmdStan?
Does anything need to be done other than concatenation assuming we can ignore all the rest of the comments?
What really needs to happen is that the whole CSV parser needs to be refactored into a comment parser and CSV parser. But then we’re going to take the structured stuff and write it out with real structure, so probably no point in doing this [rewriting csv parser].