Discourse - issue/question triage and need for a FAQ

hijacked conversation about warning messages - starting new topic -

starting point was @martinmodrak’s comment here:

This came up because Discourse is where people come to ask questions, but lots of those questions have already been answered (many times). The problem is that Discourse’s “Your question is similar to” doesn’t do a real good job of finding similariaties (IMHO), so this feature is only sometimes useful (at least for me).

4 Likes

I’ve been thinking about the best form of a FAQ quite a lot over the past months. I currently believe that the best way forward would be to move Discourse more towards being a one-stop shop whenever you need anything Stan, containing both discussions/help center and documentation/pointers to relevant, more detailed documentation.

Having some basic “howto” topics here on Discourse (e.g. “Introduction to divergences”, “Prior choice”, “Model selection”, “Using Stan Discourse effectively”) that would at least chart the landscape of other resources the user can use would have the advantages that

  • Those topics could pop up in the “Your question is similar to”
  • The users can immediately ask clarifications in the “howto” topics, so we get more feedback whether the instruction is useful and what is missing
  • Once users studied documentation and failed to resolve their problem we want them to ask a question on Discourse. As they are already on reading docs on Discourse, this should lower the barrier.
  • Making the post a wiki, all TL1 and above users on Discourse can directly contribute, in a format that they are already familiar with (this assumes everybody who has rights to edit GitHub wikis/push to main Stan site/… is on Discourse, which I think is true, while the converse is obviously not true)
  • Generally, lower barrier for users outside the core contributors to make completely new “howto” topics themselves and have them easily discovered by the community.
  • We don’t really have a single place to point people looking for “information to become better at Stan”. The forums are the least “movable” part of our information ecosystem - e.g. user-oriented GitHub wikis or directories of case studies on the main site can move to Discourse. Moving Discourse discussion to GitHub/main site is not possible.
  • It is reasonably easy to get going and learn as we build it.

By the way we can customize the intro texts shown when people start composing their questions, so if we had good starting points for common problems (e.g. for divergences) we can link to them from there.

Storing docs along with discussions on Discourse is what http://meta.discourse.org does and it works quite good for them. The main disadvantage I see is that Discourse topics can become disorganized and disconnected in a way that a good old-school docs isn’t. When you know exactly what you need, it is sometimes hard to find it on Discourse Meta, even though it is there. On the other, accidentally discovering a piece of good info that you wouldn’t look for in the first place is - in my experience there - not uncommon. I don’t think we have the resources to really build and maintain another tome of knowledge on the scale of the Stan user’s manual, so less friction to create and discover the docs seems a good tradeoff for a bit lower usability for the user.

With that said, I am slightly skeptical a FAQ would, by itself, have a large impact on the number of inquiries on Discourse. In my view many questions that look almost identical to an experienced user, seem disconnected to a novice user. In other words, many users lack a lot of background knowledge that is necessary to make all the connections between their particular problem and the (often excellent) content already on Discourse. The bigger (and harder) task is IMHO to create resources to let users acquire this background knowledge more easily. Mike’s recent case studies are IMHO a very good step in this direction, although they are IMHO not suitable for all audiences. Our FAQs should definitely make it easier for users to discover those resources.

A good FAQ will help the forums and we should build one. If we do a good job, the FAQ could also be a good step towards resources that help users get the robust background knowledge they need.

5 Likes

I attempted to draft how such a documentation wiki topic might look like: Divergent transitions - a primer happy to receive feedback.

1 Like

perhaps I’m misinterpreting what you mean by “one-stop shop”. this isn’t really possible because different users prefer different venues therefore we need both Discourse and good readily available documentation, online (html) and offline (pdf) versions. some people take their questions to the reference librarian; others head directly to the stacks.

1 Like

Yeah, sorry, I could have been clearer. The current official documentation is great and should be kept and maintained. What I meant is that Discourse could (and I currently believe also should) eventually subsume the “less official” parts of the Docs - the field guides, and the user-oriented GitHub wikis (the dev-oriented part is probably better at GitHub as all devs are on GitHub, while much less users are on GitHub), possibly also the index of tutorials/case studies. Additionally the user’s guide should IMHO become easier to discover from Discourse via the proposed FAQ/howto/overview topics.

I see the manual (reference, user’s guide) as the home of more stable and permanent documentation that is relatively thoroughly vetted, while Discourse should try to centralize the more fluid best practices and recommendations and take advantage of fast feedback and involvement by the community.

So when a user searches say “HMM” on the Discourse (or even Google “HMM Stan”), one of to hits would hopefully be something like “Overview of HMMs” which will link to the User’s guide chapter on HMMs but also various case studies/papers/blogs/packages/Discourse topics that use HMM in Stan. The best content from all of those other sources could also serve as candidates on what to include in the User’s guide.

EDIT: Should really take more care in writing thoughts down. There are two parts to what I think as Discourse being “one-stop shop”: 1) make sure (almost) every piece of relevant documentation/guidelines/… is easily discoverable when searching Discourse. I guess this is not very controversial. 2) actually maintain and store a part of the documentation, especially best practices, how tos, etc. on Discourse instead of GitHub wikis and Stan site. I guess this is more controversial and would be happy to hear arguments against it.

Does that make sense?

@martinmodrak I like this experiment with wiki-style posts. Do you have an idea for how to group these together and make them easy for people to find? (I guess other than the howto tag)

Glad you like it. If we end up having a lot of those, they might earn their own category, but I think the primary mode of discovery should be search/suggestions when starting a topic and links between the wiki posts (notably the “Welcome to Stan Forums” post that is pinned for new users). I don’t think a lot of users want to just brows a list of how-tos, so I wouldn’t worry about grouping too much.

1 Like

this is fundamentally an SEO issue - what we want, above all, is that when someone does a Google search on “Stan non-informative priors” or “RStan on Windows” or whatever, something useful on Discourse is among the top 3-5 suggestions.

the problem is that the same thing goes by so many names in so many different domains, also problematic is the name “Stan” which seems to be used ever more often in the Eminem (sp) sense.

an example of what we need, qua Rosetta stone and perhaps also for SEO tagging is: All the names for hierarchical and multilevel modeling | Statistical Modeling, Causal Inference, and Social Science

does this make sense?

2 Likes