I’m very confused about how people are supposed to cite Stan and the interfaces. I think there are two issues that lead to this confusion:
(1) Currently the “How to Cite Stan” section at mc-stan.org/citations does not correspond to the suggested citations in our R packages and maybe elsewhere as well. For example, the citations for all the interfaces say “Stan Development Team” but by default R will suggest citing packages using the names of the authors in the package metadata (the names of the humans who wrote the code). Do we want to override R’s default and have it suggest that every one of our R packages (that’s rstan, rstanarm, rstantools, shinystan, bayesplot, loo) be cited with the author as “Stan Development Team”?
(2) The Stan system paper (JSS) was submitted so long ago that the authors listed on the paper do not correspond to the active developers. But this is the first citation listed under “How to Cite Stan” and I’m starting to see this being used as the sole citation for Stan in papers. Many people aren’t going to cite Stan and also cite the interface to Stan that they used, just one or the other. I saw someone use rstanarm but just cite it using the Stan system paper. Strangely, that means that Peter Li was cited but I was not in a paper that relied on an R package I co-wrote and Peter Li wrote none of. I can only assume this was because the Stan system paper is the JSS paper and highlighted on our website as the first thing to cite. This also means that people will cite Peter Li when they use ADVI instead of Alp and Dustin, etc. I don’t mean to pick on Peter Li at all, it just seems very strange.
I will certainly admit that (2) is at least partially a selfish concern (not being cited for my work), but I’d also be fine with “Stan Development Team” used for everything. Now I’m also seeing that Ben is changing the citations for StanHeaders and RStan to be the JSS paper because that’s what JSS said to do, but why is JSS dictating how we tell people to cite our software? Especially when that means the authors in the citations don’t correspond to the authors of the software package?
Another option would be to explicitly suggest/request that people cite both Stan (with author = Stan Development Team) and the particular interface (with author = interface authors). For our R packages this can be done by providing both citations in inst/CITATION so that when citation("rstan") is called from R (or citation("shinystan"), citation("rstanarm"), etc.) it prints both.
I just removed that top JSS citation and left it in the “papers about
Stan” section.
I tried to go with “Stan Development Team” throughout specifically
to not piss people off in this way.
We can go further and remove all the names everywhere. I don’t care.
Or someone can try to add them in everywhere as long as that someone
agrees to maintain the lists going forward.
I agree JSS shouldn’t dictate what goes where on our distributions.
Let’s just say I haven’t been happy with this whole 3-year long process
and have no idea when or if it’ll ever end. As I’ve said before, my
preference would be to just retract the paper and be done with it.
I’d be fine with either “Stan Development Team” as the author on everything everything or with the individual author names. It just seemed very strange to do neither of those things and have only the JSS paper authors on everything.
I’m not an author on the paper so I shouldn’t have a say in whether you actually retract it or not, but if I were you I would retract it immediately and never think about JSS again. Seems like an enormous waste of time/energy/effort.
I wouldn’t mind writing those papers, but I don’t think we should have to write those papers just to get cited for the software. That’s the thing I don’t understand. The software itself is citable and is a separate thing from a paper about the software. JSS and perhaps other journals are conflating the two.
Off-topic: About JSS, just for information for those who had not previously heard: years ago we did retract GPstuff paper from JSS, because the process was taking years and the requests on every revision round were ridiculous (we were not as patient as Bob has been). The shorter version of the GPstuff paper was then published quickly in JMLR. I want to warn everyone reading this that if you submit to JSS, be prepared for a long ridiculous process (I’m not a co-author in Stan-JSS paper, so it’s not my decision, but I would have retracted the paper already).
On-topic: I agree that citing instruction on the web page could be better. Whatever citation is recommended, I think it would be good to recommend to report also the version of Stan or interface used for the computations (of course sometimes papers cite Stan without paper having any results made with Stan). I’ve seen quite many referring to Stan reference manual with version number included. Citable code would be good idea, too.
One is that academics often only want to cite peer-reviewed
paper and only use peer-reviewed papers for tenure and promotion
and hiring. So Andrew wanted a “real” paper for Stan.
Second, JSS seems to want to promote itself. This is more like
a marketing arrangement than an academic journal. And their
process has been ridiculously long. These reviews should take
weeks, not years!
I think it’s more specific than ‘academics’: in ecology, evolutionary biology, biostatistics, and at least my corner of infectious disease/epi I see people would be happy to cite Stan in whatever way we suggest (maybe minus a few especially stogy journals). Don’t know about promotion/hiring. Maybe by ‘academics’ you mean ‘economists’(?)
Well, it started with the Wikipedia. They wanted a "real"
publication, not a blog, not a manual, but something more
legitimate. They rejected all our initial attempts to create
a Stan page because of lack of citations that met their criteria.
But I’m also thinking about tenure and promotion committees. I
don’t know what it’s like at UMass, but in my experience, they’re
all bean counters. And things that aren’t peer reviewed aren’t
beans they’ll count.
I believe if you’re a biologist, the beans are all and only
what gets published in PubMed.
For grant proposals, they wouldn’t take manuals, etc., on our
lists of publications. Only in our “synergistic activities” or
whatever they call that part of the biosketch.
I don’t know about journals since I try to avoid them. If they
let you cite arXiv papers, more power to them. I’ll always try
to cite the arXiv version of a paper rather than a paywalled journal
version.
I’m sure it’s the same at UMass but I’m just a post-doc here so I’m blissfully and willfully ignorant of that topic. The time will come. Did you guys ever look at PLOS Computational Biology? Looks like they ask for (simple) pre-submission inquiries when it comes to software papers so it might give a hint on whether they’d be straightforward to work with. Here’s the link to their rules: http://journals.plos.org/ploscompbiol/s/submission-guidelines#loc-software-submissions
What does that even mean? I don’t see it on their site. Don’t they let you just point them at Stan 2.13.1? It makes sense for crap one-off R packages meant to get a paper published but otherwise…
Here the library eats much of that cost (last I heard) so it’s not so bad but, yeah, no solution to that one.
No, they insist on their own custom distribution with all the
examples in the paper in a runnable state. And it looks like
PLOS requires the same thing.
For us, that’s a 30MB pile of CmdStan. That I know how to use
locally, but I don’t even know how to install RStan and PyStan
locally from source.
They used to insist on GPL, so I had to get everyone’s permission
to release under a separate license, but they have since
relaxed that to GPL-compatible licenses.