Strategy for optional MyST syntax

[hukkin]

The current state is that mdformat-myst only supports syntax that is enabled by default in myst-parser. User @ichard26 has voiced the need for optional syntax support (colon fence in particular). The purpose of this issue is to plan how support for optional syntax should be implemented.

I like that there is way to enable default syntax only so we shouldn't simply add optional syntax support the same way we do for non-optionals, as that would prevent installing "myst core" only. So that leaves us with a few options:

1. Add optional syntax support to this plugin but disable it by default. Add a CLI argument for enabling the optional syntax.
2. Make a mdformat-myst-extensions (or similar name) plugin that depends on mdformat-myst but adds optionals on top.
3. Very similar to option 2 but different naming. Make a mdformat-myst-core and move this plugin in its current state there. Then make this plugin require it and add optional syntax support.
4. Keep this plugin as "myst core" only and do not create a "mdformat full" plugin at all. Require separate plugin for each optional syntax that a user wishes to enable. We then end up having mdformat-colon-fence, mdformat-amsmath etc.

I think my favorite at least currently is option number 4, having the following strong points:

• keeps mdformat-myst simple and stable
• learning from the past: it's been useful how e.g. tables syntax is in its own plugin that can be collaboratively enabled by mdformat-gfm or mdformat-myst (whichever happens to run first)
• it's not yet clear if there's demand for any other optionals besides colon-fence
• transition from option 4 to any of the others is easy, if desired later

[hukkin]

@ichard26 sorry for a horribly long post, but in a nutshell, I propose that for colon-fence support we simply create mdformat-colon-fence plugin (or similar name), and for MyST+colon_fence support one would need to

pip install mdformat-myst mdformat-colon-fence

I don't have the need or motivation to create that plugin right now myself, but will help and support you or anyone else willing to do so. Here's a plugin template to get started btw.

[ichard26]

Sounds good, I'll look into it, totally new to this but it looks fun too!

[hukkin]

Some implementation pointers:

The colon fence extension for the Markdown parser lies in mdit_py_plugins PyPI package.

To enable that in the mdformat plugin one needs to do

from markdown_it import MarkdownIt
from mdit_py_plugins.colon_fence import colon_fence_plugin

def update_mdit(mdit: MarkdownIt) -> None:
    mdit.use(colon_fence_plugin)

Then what is needed is the renderer function for "colon_fence" syntax. Probably should run the (already existing) renderer function of "fence" syntax, and use a regex or something to replace the leading and trailing ` characters with : chars.

[chrisjsewell]

yep for executablebooks/rst-to-myst#18, will need colon_fence, deflist, and substitution (all from mdit_py_plugins)

[ichard26]

Soooooo, I totally forgot about this since psf/black ended up not using colon_fence, but I am still willing to implement colon_fence, although if you want it in a quick timeframe, don't count on me 😅

[chrisjsewell]

do you want to transfer this repo to executablebooks @hukkin

[hukkin]

Makes sense to me. Transferred!

EDIT: Seems I lost admin rights in the process. @chrisjsewell would you be able to fix that?

[chrisjsewell]

yep you should have maintenance rights now 👍

[dylan-thomas832]

I would really like the ability to turn on/off MyST syntax, so I can easily use mdformat-myst on some libraries. Primarily, the colon fence feature is of interest to me. I know in #9 there is the idea of adding a --myst-extensions CLI argument, and I quite like that option, but I'm not sure what is needed to finish this and then start work on adding similar functionality for colon fences.

I just want to open the discussion back up and offer my help. What is the current block to progressing this forward?

Thanks for your work, mdformat is a great tool!

[LecrisUT]

I've made a comment of another common strategy to control mdformat plugins.