Jupyter notebook/codemirror support


#1

As one of the few python users in last week’s Learn Bayes workshop, it was frustrating to see lack of syntax highlighting in jupyter notebooks. I noticed that stan was not supported in codemirror and hence the jupyter editor was unable to recognize stan code. I got through with a custom js file I wrote. I would request the team to consider the following (given that a lot of work has already been done in R)

  • officially support stan mode in the list of codemirror language list. (I just templated c++ mode file and copied STAN functions). It shouldn’t take very long. (https://github.com/codemirror/codemirror)
  • Jupyter project periodically upgrades codemirror.js file and STAN will automatically have syntax highlighting and hinting support in its code cell
  • Add a %%stan magic by including a jupyter extension to activate stan specific features for a code cell (again a small javascript/python file)
  • If possible, add a pygments file for nbconvert

Kindly ignore this if it is already a part of your roadmap. Also let me know if you need any help.

Thanks


#2

This is something I would like to be implemented in the future. %% magic sounds great (stan code is normally written inside str, so will it work otherways).

Some ideas:

The %% method should output to either variable -o model_code or a file -f modelx.stan

So should we build custom extension first to use %% plus highlighting?


#3

All that is required for syntax highlighting is the following javascript code that needs to be injected during the load phase of notebook (via an extension or a modifying a custom js file during pystan install)…I manually inserted in custom.js in my jupyter directory

require([‘notebook/js/codecell’], function(codecell) {
codecell.CodeCell.options_default.highlight_modes[‘magic_text/x-stan’] = {‘reg’:[/^%%stan/]} ;
Jupyter.notebook.events.one(‘kernel_ready.Kernel’, function(){
Jupyter.notebook.get_cells().map(function(cell){
if (cell.cell_type == ‘code’){ cell.auto_highlight(); } }) ;
});
});

This is assuming codemirror.js file supports stan (x-stan) (I have a locally hacked version of this)

It would be great if the codemirror part is handled first (support x-stan) as it would ensure support for jupyter and host of other editors once they pick up their latest copy of codemirror mode files. the effort is simple and will give great mileage to support stan in host of editors (the following file needs to be customized as stan is similar to c++ - https://github.com/codemirror/CodeMirror/blob/master/mode/clike/clike.js)

A custom extension would be great for other functionality you mentioned.


#4

Ah, if we only had a roadmap. That’s not anyone’s to-do list as far as I know, so if you want it done, you may have to contribute it yourself.

There are lots of Stan syntax highlighter modes around. Jeff Arnold wrote on for pygments if that’ll help—it’s in their releases. But I don’t even know what nbconvert is, so that’s just a wild guess.


#5

Thanks Bob. Understand. I have uploaded the codemirror file I am using at https://github.com/Arvinds-ds/Stan-Code/tree/master/codemirror/mode/stan. I can submit a PR to codemirror to add stan mode but I feel language specific features which gets updated with every release should be a part of the main project and my effort is kind of a hack. Given that codemirror is the base for many code editors - jupyter notebook/atom/brackets, I felt this effort by the team could be worth it.

I did base my pygments file for nbconvert off of jeff arnolds pygments (a couple of versions older) file and updated it by scraping functions from 2.16 manual. nbconvert is used to convert jupyter notebooks to html/pdf and can use pygments for syntax highlighting


#6

To be a little more blunt than @Bob_Carpenter, we’re barely keeping up with our own internal documentation (see how often the manual has to be updated due to code that has become deprecated in style or performance) and don’t have the person-power to internally support a feature like this. Unfortunately this is the reality of trying to scale up an open source project like Stan!

The highlighting in pygments (and Emacs, and Atom) have all been supported by Jeff Arnold independently.

We very much welcome external contributions, both in terms of setting up and maintaining new syntax highlighters!


#7

+1 to that!


#8

OK. Thanks