Support CommonMark via commonmarker markup provider

[noraj]

The issue Render: 2 vs 4 white spaces in markdown lists #1155 was the fault of redcarpet not of yard or github_markup as you can read in github/markup#1142.

For a ton of reason readcarpet is not doing the thing the right way, and that's why now GitHub and their GFM use commonmark standard for their GFM specs. Ref. here.

As you can see there are lot of markdown issues specifically due to readcarpet like issue #1032 or #1155.

That's why a lot of big and serious project (like github-markup used by github.com and a lot more) now use only commonmarker (repo) that respect CommonMark markdown. Any other way will lead to hazardous random due to unclear / ambiguous specs.

That's why I ask for yard to use github-markup / commonmarker by default instead of github-markup / redcarpet and a least to add commonmarker in the markup providers to allow using the --markup-provider switch.

Even before commonmarker was heavily used a lot of projects deprecated the use of RDiscount, Redcarpet, or RedCloth for kramdown.

Steps to reproduce

For example see #1155 for bad rendering with redcarpet.

Actual Output

--markup-provider commonmarker with yard doc is giving:

[error]: Invalid markup type 'markdown' or markup provider (commonmarker) is not registered.

because commonmarker is not in the markup providers.

I have read the Contributing Guide.

[noraj]

PS : http://commonmark.org/

Why is CommonMark needed?

John Gruber’s canonical description of Markdown’s syntax does not specify the syntax unambiguously.

In the absence of a spec, early implementers consulted the original Markdown.pl code to resolve these ambiguities. But Markdown.pl was quite buggy, and gave manifestly bad results in many cases, so it was not a satisfactory replacement for a spec. Markdown.pl was last updated December 17th, 2004.

Because there is no unambiguous spec, implementations have diverged considerably over the last 10 years. As a result, users are often surprised to find that a document that renders one way on one system (say, a GitHub wiki) renders differently on another (say, converting to docbook using Pandoc). To make matters worse, because nothing in Markdown counts as a “syntax error,” the divergence often isn’t discovered right away.

There’s no standard test suite for Markdown; MDTest is the closest thing we have. The only way to resolve Markdown ambiguities and inconsistencies is Babelmark, which compares the output of 20+ implementations of Markdown against each other to see if a consensus emerges.

We propose a standard, unambiguous syntax specification for Markdown, along with a suite of comprehensive tests to validate Markdown implementations against this specification. We believe this is necessary, even essential, for the future of Markdown.

That’s what we call CommonMark.

[lsegal]

A PR to add commonmarker to the markup providers list would be accepted if you want to tackle that. I think we would have to wait a little longer and see how the lib performs if we were to want to change the default, though.

[noraj]

@lsegal I would be proud to help but I'm not a experienced dev, in fact I'm not a dev at all. I always make a PR when I'm able to; but here I don't know how to add the support for commonmarker.

Update: sorry I didn't read / remember this before https://github.com/lsegal/yard/blob/master/CONTRIBUTING.md#asking-for-a-feature. Do I need to join the mailing list to ask everyone how to properly implement this?

[lsegal]

Sorry for the late reply:

You're free to ask on the mailing list if you need more info. You can also look at https://github.com/lsegal/yard/blob/master/lib/yard/templates/helpers/markup_helper.rb where markup libs are defined and play around with those definitions.

[mvz]

I'm interested in providing a pull request to refactor the markup provider loading code to make it easier to:

• Pick a particular provider.
• Use a custom provider.