Why do we promote myst-nb instead of nbsphinx ?

[12rambau]

We decided to promote myst-nb (I assume because they are used by Jupyter books) but is ther any other rational behind it ?I think it is not the prefered notebook interpreter in the sphinx community and I don't see why we should decide.
This descision is currently leading to many issues related to nbsphinx compatibility like #1338 that we should see from our side when testing the documentation.

Note as well that our documentation is currently blocked with sphinx 5.3 (current version is 7) because they have hard pinned it in their deps. Sphinx is changing a lot lately and being bound to 5.3 prevents us to see the future deprecation in our docs build. On the other hand nbsphinx is fully compatible.

here are some numbers to illustrate my thoughts:

	nbsphinx	myst-nb
users	34.2k	4.2k
stars	423	175
supported sphinx version	any	<6

The other point that I would like to raise is what these libs are made for ?
by reading the following thread spatialaudio/nbsphinx#420, I have the feeling that nbsphinx and myst-nb are following 2 different approaches:

• myst-nb is extending the myst parser tools inside the notebooks cells, making it possible to create admonitions and all types of directives. It's extremely useful when you need to build a Jupyter Book because you can simply write down codes in a notebook and publish it directly using the Sphinx extention with richer interfaces. These directives will downgrade disgraccefully as code block in a normal Jupyter notebook as the rich directives are not part of the current markdown. in short myst-nb is publication driven.
• nbsphinx, don't add anything to markdown, it simply interprete the cells. They decided to go down that route because the main purpose is to render Jupyter notebook that you can then execute as they are: jupyterlab, google colab, visual studiocode, you name it. It's extremely useful in documentations when you build reproducable examples. nbsphinx is execution driven.

What are we ?

To me pydata-sphinx-theme only make sense to document big libs with many document pages and huge API. It overcomes the problem of having all the toctree in the left bar which would be overwelming for numpy, pandas, geopandas, xarray etc ....

So to my knowledge a documentation based on pydata-sphinx-theme will require these execution libs mainly for examples like:

• opensar toolkit: https://opensartoolkit.readthedocs.io/en/latest/example/latest_S1_scene.html
• or from mkdoc: https://geemap.org/notebooks/00_geemap_key_features/#use-drawing-tools

So (again according to me) promoting nbsphinx seems like the natural choice for a documentation theme instead of myst-nb (which on the other hand is the right choice for jupyterbooks).

Happy to read your thoughts !

[drammock]

I never use either nbsphinx nor myst-nb in my other projects, so I don't think I can help much here. I'll only repeat what I said in #1475 (comment):

we are very short on maintainer time so any simplifications that save us time seem good to me

[gabalafou]

Did you get the last row in your table backwards? I don't have time to investigate right now on my own, but the paragraph above your table reads to me like a contradiction of the last row.

[12rambau]

@gabalafou yep, let me update my message

[trallard]

I am the wrong user on this case as I rarely build docs from notebooks these days but I know many people do_. I will tag @choldgraf here as I am sure he will have more context on the matter or how this came to be.

From reading through both projects and their rationale, it would seem from a top-view that myst-nb is more closely targetted/developed with the scientific (computing) community in mind, which seems to be also well aligned with our user base in PST.

I also assume it might have to do with other projects in the ExecutableBooks ecosystem building off this theme (as in PST is a dependency). Or this might be completely unrelated, and I am trying to find connections where they are not 🤷🏽‍♀️

[trallard]

Follow-up - While I appreciate the metrics table, I would like to get more insight into how is the number of users calculated.

I am always careful when also using stars as a reliable metric because I also know this is a metric that can be easily inflated or hacked (not saying this is the case). Still, both projects seem also actively maintained (PRs are moving, issues are dealt with and the repos have committed in the last 2-3 weeks) and it looks myst-nb is waiting on a new release to support Sphinx >6 (likely 7 a per executablebooks/MyST-NB#543).

I am trying to be thorough and cover all the bases before jumping onto a decision of swapping libraries or approaches.

[12rambau]

From reading through both projects and their rationale, it would seem from a top-view that myst-nb is more closely targetted/developed with the scientific (computing) community in mind, which seems to be also well aligned with our user base in PST.

I agree on your conclusion on the differences between myst-nb and nbsphinx but I disagree on the purpose of PST. To me PST is primarily designed for Python libs and large documentations (I discovered the theme through pandas and numpy). They are not doing scientific publication per-se. Also the theme parameters are I think overwelming for someone that just want to share science. I feel that the use case you describe is more the one of JupyterBooks. and based on its success, it makes a very good job !

on that note, #1338 was raised and solved in the context of pandas documentation, they should not do tricks like that, it's the cradle of this theme.

[trallard]

I believe I was not clear or maybe went off definitions in my head. The scientific computing community/ecosystem for me includes all those tools/packages in the pydata ecosystem and beyond (such as numpy, pandas, etc.)
Which intersects with groups of folks using notebooks for purposes like tutorials (I e. Numpy's tutorials website is entirely made up with notebooks) and dissemination.

Then there is an entirely adjacent portion of the community that uses notebooks entirely for research dissemination (maybe research community is a good term?).
Anyway mystnb serves both cases.

PST is for sure better suited for the former group as it adds a lot of features that other themes are missing for large library docs.
But I also do not think we have a complete view of who/how is using PST beyond big/substantiative libraries. Because if there is something that I have learned through making tools is that your user will always find ways to use your tool beyond the one/many ways you originally intended or anticipated.
I am not sure how many folks would be impacted in a way or another if we drop one for the other in a haste.

[drammock]

Just so I'm clear: I think the proposal here is to change our docs so that we don't demo MystNB (this is what is meant by "promoting it" I think)? And the motivation/result is that our docs can be built with something newer than sphinx 5?

That seems fine to me as long as we create a new CI job that builds a simple site using our theme and MystNB, so that we can still know that it works (if you use a suitably old version of Sphinx). then our docs can still say "MystNB is supported" (with relevant caveats), even if we don't include a demo MystNB page in our public docs.

Does that seem like a route forward that everyone can endorse?

[choldgraf]

A quick note from me:

I don't understand the distinction between being "execution driven" and "publication driven". MyST-NB executes notebooks in pretty much the same way as nbsphinx (though with the added functionality that myst provides, like inserting outputs elsewhere in the document). I feel like a better distinction is that MyST-NB more directly integrates with Sphinx functionality (via the MyST Markdown parser), while nbsphinx does to via a translation layer of rST -> markdown.

Also note that there is a jupyterlab-myst extension to provide myst functionality in a notebook UI: https://github.com/executablebooks/jupyterlab-myst

That said, I don't have strong opinions if others think that there's clear value in demoing nbsphinx over myst-nb. I don't see this as a critical question for the theme's strategy, and @drammock's suggestion seems sensible to me but I'm happy to go with whatever.

[drammock]

That seems fine to me as long as we create a new CI job that builds a simple site using our theme and MystNB, so that we can still know that it works

@12rambau are you OK with this route? If so can you update #1482 accordingly?

[12rambau]

I'm ok with that route but I need to solve the pandoc issue first.

[maximlt]

FWIW I also don't understand the distinction made between nbsphinx and myst-nb. In the HoloViz ecosystem we use myst-nb to build pretty large websites, executing many notebooks.

While I appreciate the metrics table, I would like to get more insight into how is the number of users calculated.

I'm also interested in that! I'd love to how many users our OSS packages have 🙃 Another useful stats is the number of downloads on PyPI, with for the last month: 462K for nbsphinx and 335K for myst-nb.