Hi, I’m working on a JSON output format for MCMC and other Monte Carlo samplers. It has a basic python tool to convert back and forth to TSV format. I’ve opened issues to highlight future work. GitHub - bredelings/MCON I saw that there was previously a proposal for JSON output format for STAN, …

Apologies, I thought you had linked to https://github.com/stan-dev/design-docs/blob/master/designs/0032-stan-output-formats.md , which is a design-doc which was accepted and does include JSON (but not for samples)

The issues with Inf/-Inf/NaN came up in the Boost.JSON project as well. The discussion here is pretty informative: https://github.com/boostorg/json/issues/397 One additional approach that is mentioned there is to write 1e9999 for Inf and -1e99999 for -Inf. Those are both valid JSON, and most pa…

I believe there are often two complaints about CSV as an output format: Being text based, you have to choose between accuracy and file size It is annoying to have to flatten objects with rich structure like matrices in order to output them as one row The pros are that it is human readable (ish), …

Hell yeah, always good taste around here

I’m looking at Parquet – very interesting. It looks like it would be possible to represent vectors of variable size using the “REPEATED” repetition type. If Parquet output is added to Stan, would the Stan team work on getting parquet input added to packages like coda?

One that I’m going to have to update soon to keep using it 🤦

Pardon my ignorance, but would I be correct to surmise that both Arrow and Parquet can handle variable-shape records (e.g. records that contain a variable-length vector)?

I’m a bit concerned about a situation where every probabilistic programming software package defines their own output format, and that output can only be analyzed by a separate tool released along with the MCMC sampler. I see for example that pymcmc can “save” samples in an HDF5 format. 6. Saving …

Is it currently the case that diagnostics tools exist (besides the provided stansummary) that directly analyze the Stan CSV files? My understanding is that most of these rely on you getting the data in memory yourself and then providing it for analysis (this is certainly the case in Python, where a…

JSON Output for STAN

Developers

mike-lawrence September 14, 2023, 2:51pm 12

Btw, see here for a branch of cmdstan that uses Arrow

Topic		Replies	Views
Notes on Stan Output Serialization Options (YAML, Protobuf, Avro, CBOR) Developers	13	3454	July 14, 2021
Proof of concept: Binary output format for cmdstan Developers	74	1969	April 8, 2026
Status of the IO re-factor? Developers	16	1541	September 18, 2020
Speed difference between rstan and cmdstan for a simple model CmdStan rstan , techniques	25	4104	November 7, 2021
Alternative .csv reader Developers maintenance , rstan	53	4614	September 11, 2018

JSON Output for STAN

Related topics