general questions: for all CmdStanX wrappers, should part of the contract be “minimal library dependencies”? If so, how do we decide when using a library is OK? context: discussion from Getting error 'Error: Can't find columns `lp__`, `treedepth__`, `divergent__` in `.data`.' · Issue #243 · stan…

This is the first problematic thing that popped up, its seems to be a problem only for R 3.6 and I think it mostly comes down to not propperly set minimum version of the packages used. I am not sure the comparison is completely fair though. cmdstanpy only relies on numpy and pandas, but they are bo…

A minor non-R point: I find the major appeal of working with CmdStan is simply that the C++ toolchain aspects are all debuggable in pure shell, Make, g++ etc. What libraries are involved in the wrapper itself is mostly irrelevant in my opinion, if they have any positive value. If it were me, I wou…

[image] jonah: And some package for dealing with JSON too, right? And os right? Plus all the dependencies of Numpy, Pandas (like how vroom also has its own dependencies). Good question! Just checked on everything thats being used in CmdStanPy - aside from Python standard libraries , CmdStanP…

[image] maedoc: support vroom with fallback to whatever standard parser is available +1

[image] rok_cesnovar: I personally think fast IO is important and the addition of vroom also enables reading only selected columns, which is also very useful imo and will also take some time to do in cmdstan (the issue for this in cmdstan is ~3 years old). We can always revert, would not oppose …

[image] maedoc: I would support vroom I hadn’t heard of vroom before. It looks like a persistence framework for C++: http://ftp.lyx.org/lip6/softs/vroom/vroom.ps We’d have to be able to control the serialization and not just use defaults, because our low level C++ changes a lot.

@maedoc is referring to the vroom R package we are using in cmdstanr to read the CSV files with samples as opposed to utils::read.csv rstan is using in read_stan_csv. We use it because its faster in general and because it allows reading in only selected columns of the CSV to not waste memory (and r…

thanks for the detailed explanation. I didn’t mean to be criticizing the use of vroom in this thread - yes, I was concerned, but now I’m not. I think that @maedoc ’s suggestion of how to handle this is a fine solution and a great way to move forward. [image] rok_cesnovar: Alternative packages …

I hope you don’t mind a suggestion from a user. Consider making vroom optional – put it in the suggests section of the DESCRIPTION file and do a check for its presence in the code, with a fallback if it’s not installed. I’m still reinstalling packages after updating R to 4.0.0 3 months ago – many o…

Lightweight interfaces - keeping it light

Developers

stevebronder July 31, 2020, 10:53pm 12

From the benchmarks here (vroom’s own benchmarks for all numeric data) it looks like data.table is a lot faster right? + data.table has zero depends

Topic		Replies	Views
"ServerStan" implementation language poll Developers	44	2514	August 21, 2020
Speed difference between rstan and cmdstan for a simple model CmdStan rstan , techniques	25	3925	November 7, 2021
Alternative .csv reader Developers maintenance , rstan	53	4319	September 11, 2018
CmdStanPy - ready for beta testing! Developers pystan	23	2451	August 6, 2019
Prototype: Simplified CLI to Stan General cmdstan	28	1923	September 17, 2020

Lightweight interfaces - keeping it light

Related topics