Related: Tomorrow [29-SEP-2016] afternoon at 15:00-15:40 CET, Jon Corbet of LWN, is giving a talk[1] at Linux Recipes conference in Paris on this subject.
The main problem is not the format of the documentation, but lack thereof. Many important functions and high level implementation tasks are not documented or the docs are inadequate.
I (being the kernel documentation maintainer) have already seen an increase in activity around the docs as a result of the change - it makes things far more accessible.
It's also not really correct to say there's not much documentation. We have quite a bit if you look at it, especially if you count the 55,000 kerneldoc comments in the source itself. The quality of some of it and the organization of all of it is another matter...but we're working on that.
It is almost certainly correct that this is a contributing factor. It is unlikely to be the only factor or even the primary factor.
The very human element, that most programmers simply don't like writing documentation is certainly a larger factor. If this holds true for the Kernel no new format can fix the problem by itself. At a minimum it will require a encouragement from the top to improve.
Definitely the case in my office. A couple of opinionated and (relatively) senior devs that think the code is the documentation. No, I can't read this 40 yo fortran code and use it as the basis for a new program on a different architecture. It uses gotos and variables like "yxtgn" that mean nothing to the reader (turned out y meant it was an integer, weird). I spent a month on that because there was no documentation, which took up a page once I was done writing it after my reverse engineering effort.
And then documentation, where it exists, is often poorly CM'd word documents, and poorly OCR'd PDFs (or doc->pdf that somehow lost the ability to be searchable and selectable text, what option did they select?!?).
The offending systems are only maintainable because a few people haven't retired (< 1 year left for several of them) and sheer force of will for the rest of us.
They missed a format that's ubiquitous, has a lot of formatting options, and has search facilities. Manpages. When someone's looking for a piece of documentation, they are usually working on a system as they do it, and man -k <thing> is a lot easier to remember than some other search tool that requires its own set of packages, etc.
Manpages are not a format to write the documentation, but to display it.
You can `make mandocs` in the linux tree, then `make installmandocs` to make them easily accessible with the man command, i.e. `man struct_sk_buff` will display the definition of an sk_buff.
The problem with that approach is that you are limiting yourself to fundamentally primitive documentation (and by primitive I mean non-interactive). Granted, for a kernel it might be less important, but it's great when there is a good architecture diagram/image instead of 7 paragraphs.
Manpages can have a rather large amount of interactivity. Documents can be formatted as text, as HTML, with links, etc. The problem is that manpages are an afterthought in too much of the Linux ecosystem, so people have low expectations and low beliefs in what they can actually do. They can have tables, they can have equations, they can have graphics.
Out of genuine curiosity, could you link me to an existing guide from the Linux community that is based on manpages and features images and/or other interactive content?
Have you actually tried to use this for something real? Sphinx source doesn't
seem suited for man pages at all, and formatting seems the least important bit
that doesn't fit.
ReST can be better extended without creating a variant of the markup language [1]. Extension support is first class [2], and it is by that means that a good number of the core features are implemented.
[1] For example, github flavour, stackoverflow flavour, reddit flavour, ...
One large difference is that Markdown exclusively targets HTML as an output format where ReST tries to be format agnostic. If you only ever target HTML this might ease some things but not if you want html, man pages, pdfs, plain text, …
ReST is a whole multi-document system, as opposed to MD which is just the formatting (at least before you mix in the dozens of mutually incompatible extensions). ReST is a superset.
>Embed HTML (more or less preferred, depending on your situation)
But then you can't embed markdown inside the HTML, like code blocks inside a table. Sure you can use <pre><code> tags, but then you lose syntax highlighting.
Actually I've always wondered why no MD renderer supports something like
Not only are you correct, you allowed me to find an equivalent feature in CommonMark! If there is an empty line between some content and its wrapping HTML tags then that content will be interpreted as markdown. So
> But then you can't embed markdown inside the HTML, like code blocks inside a table. Sure you can use <pre><code> tags, but then you lose syntax highlighting.
Pandoc supports this with the `markdown_in_html_blocks` option.
Not like you have to do anything to get it to work OOB, but your mileage may vary. One of the benefits of Markdown is that it's easily understood by people new to the doc space. RST is slightly more complex and has more rules added in.
I find it surprising that Michael Kerrisk's Linux man-pages project isn't mentioned here [1]. He and other contributors done a major job documenting the Linux man pages. It's usually the first place I go to when I need to read up on the Kernel APIs or C libraries.
URL[2] to live streaming.
[1] https://kernel-recipes.org/en/2016/talks/kernel-documentatio...
[2] https://air.mozilla.org/kernel-recipes-2016-09-29-PM-Session...