Documentation ignores the \b characters in docstrings

[gtristan]

As is documented in click here, the docstrings for click commands are expected to contain \b characters before a paragraph which should not be made to wrap.

This allows better formatting of docstrings in the help output, without this it is impossible to make any kind of formatting of the help such as point form lists which describe the meaning of each click.Choice() option.

This works well for click's help output, but is also broken for the click-man generated man pages, and for sphinx-click generated html (although the man pages are slightly less broken than the html).

See this example of the breakage and compare with the corresponding help output below:

Usage: bst show [OPTIONS] TARGET

  Show elements in the pipeline

  By default this will only show the specified element, use the --deps
  option to show an entire pipeline.

  FORMAT
  ~~~~~~
  The --format option controls what should be printed for each element,
  the following symbols can be used in the format string:

      %{name}     The element name
      %{key}      The abbreviated cache key (if all sources are consistent)
      %{full-key} The full cache key (if all sources are consistent)
      %{state}    cached, buildable, waiting or inconsistent
      %{config}   The element configuration
      %{vars}     Variable configuration
      %{env}      Environment settings
      %{public}   Public domain data

  The value of the %{symbol} without the leading '%' character is understood
  as a pythonic formatting string, so python formatting features apply,
  examle:

      build-stream show target.bst --format \
          'Name: %{name: ^20} Key: %{key: ^8} State: %{state}'

  If you want to use a newline in a format string in bash, use the '$'
  modifier:

      build-stream show target.bst --format \
          $'---------- %{name} ----------\n%{vars}'

Options:
  -d, --deps [all|build|run]  Optionally specify a dependency scope to show
  --order [stage|alpha]       Staging or alphabetic ordering of dependencies
  -f, --format FORMAT         Format string for each element
  -a, --arch TEXT             The target architecture (default: x86_64)
  --variant TEXT              A variant of the specified target
  --help                      Show this message and exit.

[stephenfin]

Good point. Currently we render the help text as is and let Sphinx/docutils process that as is. If we were to support this, we'd need to modify that workflow to at least check for \b characters.

I'll try to get to this eventually but it's not something I presently use so it's rather low priority for me. If someone want to take this upon themselves, I'd be more than happy to review any/all patches.

[gtristan]

@stephenfin, thanks for your reply and being receptive to patches.

I've looked at this briefly but sphinx and docbook are admittedly not my strong suit, as I understand (or not), it looks like ClickDirective._generate_nodes() needs some work to handle \b, maybe in the description or summary part, and the result we are generating is basically the reStructuredText that sphinx will format.

Judging from the comment around "# Description", it looks like there may be a workaround which would be to include restructured text in the help output itself (which may turn out to look not too bad in plain text as well).

I will try to look further into this but dont know how much time I can focus on this...

[philipstarkey]

I've just attempted to fix this (see here). Works for my use case. I'll try and do a bit more testing and make a pull request soon.

[gtristan]

@philipstarkey, I tried your branch and it is an improvement, thanks.

This further exposes an oversight I had made.

Ideally it would be nice if the rendered HTML appeared like the paragraphs which appear in the doc comments and help text, however this falls flat on it's face without monospace fonts, and tolerance of leading whitespace on lines.

It seems unwieldy to try to support this in a sphinx-click contrib package, and wrong to limit the whole document to monospace just because someone might have used \b. This leads me to wonder if it would be nicer to support markdown in sphinx proper; and have the --help output itself use some ansi escape codes to render titles and bullets and such on a best effort basis (and of course this would leave the door open to generating even nicer documentation)...

[stephenfin]

@philipstarkey Looking forward to it. That would make for a good 1.1 release

@gtristan I'm not sure I understand you. What exactly are you looking for?

[gtristan]

@stephenfin I guess I just went on a tangent on what might be the ideal solution...

So, basically I have three outputs of my doc strings:

• application --help output
• man pages generated with the click_man thing
• documentation generated with sphinx-click

With the application help output, I can do some alignments and tables and stuff, like:

    \b
    Values for the --frobtype argument
        foo        Frob it with foo
        bar        Bar it from any frobbing
        baz        Frob it baz

So I can create aligned output in the additional text I write in the documentation body and am not limited to the alignment provided by argument options; but in html this alignment is lost because of lack of monospace fonts.

I think it makes sense with the current approach to force monospace fonts in the rendered docs, otherwise there is no way to use any alignment in your help text.

Looking at the reverse approach

With man pages we have more freedom of expression than with help output, and with HTML we have even more freedom.

The current approach with click is to write a very, very simple format for help output (the format is just text with a special meaning of the \b character); and then use that same format to generate man pages and html documentation.

Things would make more sense the other way around:

• Click command docstrings should support markdown
• Click itself should honor the markdown on a best effort basis when printing --help output
• The man pages can do a better job of honoring the markdown
• The html can do an even better job of honoring the markdown

Just saying, this would be a much nicer experience :)

[jjmaestro]

+1 to this, I was just looking into click + docstring + --help / sphinx etc and found this. Would be awesome to have a better experience around docstrings / documentation :D

[KnownNexus]

@gtristan I've recreated philipstarkey's patch to be compatible with the latest version of master.
Do you want to try and get this merged, as it does solves buildstreams current issue, or would you rather wait until you've come up with an alternative solution?

https://github.com/KnownNexus/sphinx-click/tree/Slash_b_ignored_in_docstrings

[stephenfin]

Fixed in the above commit.