Hello! The Stan Governing Board is looking into redesigning the Stan website (mc-stan.org), and we’d like to get feedback from the Stan community in term of what works and doesn’t work as a user/developer experience. If you have any suggestions on how the experience can be improved on the site it would be helpful if you could reply to this post with a brief statement about the issue(s) you have faced and what you think the experience should be (e.g. what you think the user journey should be). Please keep in mind that Stan has multiple interfaces and the site should cater to new/returning users and developers coming from those interfaces.
Here’s a non-exhaustive outline of some of the site features we are considering,
Facilitating the journey of a new/returning user/developer
Quick start guides for new users
Information about all the interfaces with links to documentation
Stan documentation
Links to the Stan Twitter, GitHub Org page, and Discourse
Information about the code of conduct
About us for SGB/Devs
SGB weekly/monthly updates
Should be easily editable by developers
You can take a look at the following websites of existing open source projects to get an idea of what has been done by others.
I’ll start. I like the suggestion from @doug_johnsonhere about using Google Colab notebooks. This would help new users (temporarily) skip the installation phase and allow them to quickly use the Stan language and play around with simple models. Rust has a similar playground area on their site.
The Stan docs are in a weird order and their names aren’t meaningful (Stan Language Reference Manual vs Stan Language Functions Reference). Users don’t need to know about the Stan math library.
This should all be clarified and reordered with insight from people who aren’t devs. I had to tell my students “you basically always want the third one” a lot.
There should be clean links to the Stan version in each major interface (almost no one is using 2.23 but they’re the only easily available docs).
The case studies etc don’t make sense on that page. You have to either know they’re there or happen to scroll quite far down.
I think it would help to make things a lot more attractive for complete beginners if you put case studies up front (e.g. as one of the ribbons?). Having 1 or 2 flagship case studies with a lot of visualizations & easy to follow introductory text up top would also be very useful imho. Right now, the top two (i.e. the latest) case studies are about parallelization and Notebooks in cloud which might not be relevant to everyone. To show the power & flexibility of Stan, the models in the case studies could start simple & after that they could be expanded (e.g. heterogeneous variance, random effects) to fit the data better. There’s a couple examples like that in BDA3.
I completely agree, as a student I found Stan docs very confusing. I had never heard of Stan until 2 months ago and finding “Stan User Guide” “Stan Functions-Reference” and " Stan Reference Manual" made the journey as a beginner a bit difficult.
The new user journey from the main Stan page to Rstan isn’t intuitive to follow.
Suggestions :
Introduction to Stan which introduces the entire pipeline of key concepts from defining the program blocks to Convergence Diagnostics example codes and plots.
Having links to installation and download instructions for each platform.
1 Combined Stan manual that is structured keeping beginners in mind with no prior experience with Stan
One of the websites I referred when starting out as a complete beginner which gave me a direction to proceed and what to search for was when modelling my dataset was the following https://ourcodingclub.github.io/tutorials/stan-intro/
PS: I am not sure if this thread is only for main governing body and Stan developers. Feel free to let me know if I should not be posting here
We should def sort these out and have a repository somewhere with older versions. Maybe have rstan and pystan links for whatever version both of them are using
The problem with the website has always been that it’s trying to accommodate too many audiences and failing to accommodate any one of them particularly well. Only by very carefully deciding who the website is for will one be able to motivate a better design.
Typical audiences include people interested in the project, people interested in development, people interested in using Stan, and experienced Stan users looking for general documentation or additional pedagogical material. The targeting is complicated by the fact the Stan ecosystem covers multiple components – an autodiff library, algorithm library, probabilistic programming language, and interfaces – which each potential audiences will interact with differently. Further complicating matters is the fact that using Stan well requires a reasonable statistical background and consequently the best way to guide new users will vary depending on their particular statistical background. Every introduction to Stan needs to be a little different, and a website trying to cater to new people will never be able to capture all of those introductions.
The original version of the current design tried to compromise with a landing page that directed a few of the audiences (namely potential devs and users) to the best place for them to start. People then added more directions to accommodate more audiences and quickly the whole thing became overburdened and confusing (the same thing is happening to the manual, but that’s another story).
At some point @ariddell and I discussed an iterative navigation scheme that would guide visitors to targeted starting pages, but the design quickly got out of hand before anything came out of it.
Anyways, site design is hard but it’s exceptionally hard for Stan given the breadth of the project and the diversity of the community. Before any design is considered I strongly recommend identifying a few audiences to prioritize and where they need to start.
I’ve been lobbying for years to formulate our RStan install instructions this way, but I seem to recall @bgoodri had objections and preferred it the way it is.
If we do that, can we also keep the standard dev-friendly terminology we’re currently using? It’s very standard for doc to be divided into a language reference (our reference manual), library reference (that’s the function’s reference), and user’s guide with programming tips (our user’s guide). We could put it back together.
That’s exactly how the user’s guide is structured. One of the problems we have is that we have too much documentation!
I agree.
An alternative I proposed is to knock down our monolithic web site and instead make web sites focused from the interface level. So CmdStan, PyStan, and RStan would all get their own web sites. That would make the scope of these web sites much more like all the comparisons everyone’s throwing around.
I think the solution for the web site may be not having a single big page with everyting on it. The difference between Stan and all these other packages is that we have multiple different languages and multiple higher-level interfaces like RStan and then even higher-level interfaces like brms.
I think it’s going to be hard to have something easy for new users that tries to cover all of Stan.
We have lots of intros to this written. The problem we’re finding is that eveyrone has a slightly different idea of the ideal path.
The other problem we face is that a lot of our doc is reference material, not case studies or tutorial guides. That’s necessary for completeness and advanced users, but always confusing for users. It’s the unix man-page problem—you can only understand a unix man page if you already understand the function.
We invite everyone to comment in our public forums. We’d lock them down if we didn’t want others commenting!
I’d call that an introductory tutorial more than a manual. We have something like that for our different interfaces (e.g., getting started with RStan). But it’s not at all ocordinated.
A reference manual has the definitive definitions of everything and is hard to write for beginners.
Those are there, but not so easy to find if you don’t know where to look. That’s why we need a redesign!
By platform, do you mean OS plus hardware? We also need to have it for every Stan interface: RStan, PyStan, brms, CmdStan, rstanarm, bayesplot, loo, etc. The combinatorics would lead to well over 100 pages of instructions, which still might be the right way to go. But I think it’d work better one interface at a time rather than trying to put them all together.
You mean broken apart rather than combined as in the RStan docs?
Exactly. And I’m not convinced one site design for every Stan project together can achieve this.
At least from their high-level sketch, that’s what we have now for pretty much every one of our 20 or so interfaces. Just separating dev-facing from user-facing APIs would be a good start.
I added a couple case studies this year, and I have another little one in the pipeline. These are handy for copy-paste code examples. They’re different than the manual cause we can include interface code there.
I think the Case Studies page should be more compact to make it easier to browse the page itself (https://mc-stan.org/users/documentation/case-studies.html). It works fine to just link them manually, but right now I don’t think that page is great for browsing or anything.