Sphinx extensions

Sphinx is the best way to document your Python CLI. Click Extra provides several utilities to improve the quality of life of maintainers.

Important

For these helpers to work, you need to install click_extra’s additional dependencies from the sphinx extra group:

$ pip install click_extra[sphinx]

Setup

Once Click Extra is installed, you can enable its extensions in your Sphinx’s conf.py:

extensions = ["click_extra.sphinx", ...]

Click directives

Click Extra adds two new directives:

• .. click:example:: to display any Click-based Python code blocks in Sphinx (and renders like .. code-block:: python)

• .. click:run:: to invoke the CLI defined above, and display the results as if was executed in a terminmal (within a .. code-block:: ansi-shell-session)

Thanks to these, you can directly demonstrate the usage of your CLI in your documentation. You no longer have to maintain screenshots of you CLIs. Or copy and paste their outputs to keep them in sync with the latest revision. Click Extra will do that job for you.

Hint

These directives are based on the official Pallets-Sphinx-Themes from Click’s authors, but augmented with support for ANSI coloring. That way you can show off your user-friendly CLI in all its glory. 🌈

See also

Click Extra’s own documentation extensively use .. click:example:: and .. click:run:: directives. Look around in its Markdown source files for advanced examples and inspiration.

Usage

Here is how to define a simple Click-based CLI with the .. click:example:: directive:

Markdown (MyST)

Attention

As you can see in the example below, these Click directives are not recognized as-is by the MyST parser, so you need to wrap them in {eval-rst} blocks.

```{eval-rst}
.. click:example::
    from click_extra import echo, extra_command, option, style

    @extra_command
    @option("--name", prompt="Your name", help="The person to greet.")
    def hello_world(name):
        """Simple program that greets NAME."""
        echo(f"Hello, {style(name, fg='red')}!")
```

Thanks to the .. click:run:: directive, we can invoke this CLI with its --help option:

```{eval-rst}
.. click:run::
    invoke(hello_world, args=["--help"])
```

Warning

CLI states and references are lost as soon as an {eval-rst} block ends. If you need to run a .. click:example:: definition multiple times, all its .. click:run:: calls must happens within the same rST block.

A symptom of that issue is the execution failing with tracebacks such as:

Exception occurred:
  File "<docs>", line 1, in <module>
NameError: name 'hello_world' is not defined

reStructuredText

.. click:example::
    from click_extra import echo, extra_command, option, style

    @extra_command
    @option("--name", prompt="Your name", help="The person to greet.")
    def hello_world(name):
        """Simple program that greets NAME."""
        echo(f"Hello, {style(name, fg='red')}!")

Thanks to the .. click:run:: directive, we can invoke this CLI with its --help option:

.. click:run::
    invoke(hello_world, args=["--help"])

Placed in your Sphinx documentation, the two blocks above renders to:

from click_extra import echo, extra_command, option, style

@extra_command
@option("--name", prompt="Your name", help="The person to greet.")
def hello_world(name):
    """Simple program that greets NAME."""
    echo(f"Hello, {style(name, fg='red')}!")

$ hello-world --help
Usage: hello-world [OPTIONS]

  Simple program that greets NAME.

Options:
  --name TEXT               The person to greet.
  --time / --no-time        Measure and print elapsed execution time.  [default:
                            no-time]
  --color, --ansi / --no-color, --no-ansi
                            Strip out all colors and all ANSI codes from output.
                            [default: color]
  -C, --config CONFIG_PATH  Location of the configuration file. Supports glob
                            pattern of local path and remote URL.  [default:
                            ~/.config/hello-
                            world/*.{toml,yaml,yml,json,ini,xml}]
  --show-params             Show all CLI parameters, their provenance, defaults
                            and value, then exit.
  --verbosity LEVEL         Either CRITICAL, ERROR, WARNING, INFO, DEBUG.
                            [default: WARNING]
  -v, --verbose             Increase the default WARNING verbosity by one level
                            for each additional repetition of the option.
                            [default: 0]
  --version                 Show the version and exit.
  -h, --help                Show this message and exit.

This is perfect for documentation, as it shows both the source code of the CLI and its results.

See for instance how the CLI code is properly rendered as a Python code block with syntax highlighting. And how the invocation of that CLI renders into a terminal session with ANSI coloring of output.

You can then invoke that CLI again with its --name option:

.. click:run::
    invoke(hello_world, args=["--name", "Joe"])

Which renders in Sphinx like it was executed in a terminal block:

$ hello-world --name Joe
Hello, Joe!

Tip

.. click:example:: and .. click:run:: directives works well with standard vanilla click-based CLIs.

In the example above, we choose to import our CLI primitives from the click-extra module instead, to demonstrate the coloring of terminal session outputs, as click-extra provides fancy coloring of help screens by default.

Inline tests

The .. click:run:: directive can also be used to embed tests in your documentation.

These blocks are Python code executed at build time, so you can use them to validate the behavior of your CLI. This allow you to catch regressions, outdated documentation or changes in terminal output.

For example, here is a simple CLI:

from click_extra import echo, extra_command, option, style

@extra_command
@option("--name", prompt="Your name", help="The person to greet.")
def hello_world(name):
    """Simple program that greets NAME."""
    echo(f"Hello, {style(name, fg='red')}!")

If we put this CLI code in a .. click:example:: directive, we can associate it with the following .. click:run:: block:

.. click:run::
    result = invoke(hello_world, args=["--help"])

    assert result.exit_code == 0, "CLI execution failed"
    assert not result.stderr, "error message in <stderr>"
    assert "--show-params" in result.stdout, "--show-params not found in help screen"

See how we collect the result of the invoke command, and inspect the exit_code, stderr and stdout of the CLI with assert statements.

If for any reason our CLI changes and its help screen is no longer what we expect, the test will fail and the documentation build will break with a message similar to:

Exception occurred:
File "<docs>", line 5, in <module>
AssertionError: --show-params not found in help screen

Having your build fails when something unexpected happens is a great signal to catch regressions early.

On the other hand, if the build succeed, the .. click:run:: block will render as usual with the result of the invocation:

$ hello-world --help
Usage: hello-world [OPTIONS]

  Simple program that greets NAME.

Options:
  --name TEXT               The person to greet.
  --time / --no-time        Measure and print elapsed execution time.  [default:
                            no-time]
  --color, --ansi / --no-color, --no-ansi
                            Strip out all colors and all ANSI codes from output.
                            [default: color]
  -C, --config CONFIG_PATH  Location of the configuration file. Supports glob
                            pattern of local path and remote URL.  [default:
                            ~/.config/hello-
                            world/*.{toml,yaml,yml,json,ini,xml}]
  --show-params             Show all CLI parameters, their provenance, defaults
                            and value, then exit.
  --verbosity LEVEL         Either CRITICAL, ERROR, WARNING, INFO, DEBUG.
                            [default: WARNING]
  -v, --verbose             Increase the default WARNING verbosity by one level
                            for each additional repetition of the option.
                            [default: 0]
  --version                 Show the version and exit.
  -h, --help                Show this message and exit.

Tip

In a way, you can consider this kind of inline tests as like doctests, but for Click CLIs.

Look around in the sources of Click Extra’s documentation for more examples of inline tests.

Hint

The CLI runner used by .. click:run:: is a custom version derived from the original click.testing.CliRunner.

It is called ExtraCliRunner and is patched so you can refine your tests by inspecting both <stdout> and <stderr> independently. It also provides an additional <output> stream which simulates what the user sees in its terminal.

ANSI shell sessions

Sphinx extensions from Click Extra automaticcaly integrates the new ANSI-capable lexers for Pygments.

This allows you to render colored shell sessions in code blocks by referring to the ansi- prefixed lexers:

Markdown (MyST)

```ansi-shell-session
$ # Print ANSI foreground colors.
$ for i in {0..255}; do \
>     printf '\e[38;5;%dm%3d ' $i $i \
>     (((i+3) % 18)) || printf '\e[0m\n' \
> done
  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15 
 16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33 
 34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51 
 52  53  54  55  56  57  58  59  60  61  62  63  64  65  66  67  68  69 
 70  71  72  73  74  75  76  77  78  79  80  81  82  83  84  85  86  87 
 88  89  90  91  92  93  94  95  96  97  98  99 100 101 102 103 104 105 
106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 
124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 
142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 
160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 
178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 
196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 
214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 
232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 
250 251 252 253 254 255
```

reStructuredText

.. code-block:: ansi-shell-session

    $ # Print ANSI foreground colors.
    $ for i in {0..255}; do \
    >     printf '\e[38;5;%dm%3d ' $i $i \
    >     (((i+3) % 18)) || printf '\e[0m\n' \
    > done
      0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15 
     16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33 
     34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51 
     52  53  54  55  56  57  58  59  60  61  62  63  64  65  66  67  68  69 
     70  71  72  73  74  75  76  77  78  79  80  81  82  83  84  85  86  87 
     88  89  90  91  92  93  94  95  96  97  98  99 100 101 102 103 104 105 
    106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 
    124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 
    142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 
    160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 
    178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 
    196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 
    214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 
    232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 
    250 251 252 253 254 255

In Sphinx, the snippet above renders to:

$ # Print ANSI foreground colors.
$ for i in {0..255}; do \
>     printf '\e[38;5;%dm%3d ' $i $i \
>     (((i+3) % 18)) || printf '\e[0m\n' \
> done
  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15 
 16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33 
 34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51 
 52  53  54  55  56  57  58  59  60  61  62  63  64  65  66  67  68  69 
 70  71  72  73  74  75  76  77  78  79  80  81  82  83  84  85  86  87 
 88  89  90  91  92  93  94  95  96  97  98  99 100 101 102 103 104 105 
106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 
124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 
142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 
160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 
178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 
196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 
214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 
232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 
250 251 252 253 254 255

click_extra.sphinx API

        classDiagram
  Directive <|-- DeclareExampleDirective
  Directive <|-- RunExampleDirective
  Domain <|-- ClickDomain
  EchoingStdin <|-- EofEchoingStdin
  ExtraCliRunner <|-- ExampleRunner
    

Helpers and utilities for Sphinx rendering of CLI based on Click Extra.

Danger

This module is quite janky but does the job. Still, it would benefits from a total clean rewrite. This would require a better understanding of Sphinx, Click and MyST internals. And as a side effect will eliminate the dependency on pallets_sphinx_themes.

If you’re up to the task, you can try to refactor it. I’ll probably start by moving the whole pallets_sphinx_themes.themes.click.domain code here, merge it with the local collection of monkey-patches below, then clean the whole code to make it more readable and maintainable. And finally, address all the todo-list below.

Todo

Add support for plain MyST directives to remove the need of wrapping rST into an {eval-rst} block. Ideally, this would allow for the following simpler syntax in MyST:

```{click-example}
from click_extra import echo, extra_command, option, style

@extra_command
@option("--name", prompt="Your name", help="The person to greet.")
def hello_world(name):
    "Simple program that greets NAME."
    echo(f"Hello, {style(name, fg='red')}!")
```

```{click-run}
invoke(hello_world, args=["--help"])
```

Todo

Fix the need to have both .. click:example:: and .. click:run:: directives in the same {eval-rst} block in MyST. This is required to have both directives shares states and context.

See also

This is based on Pallets’ Sphinx Themes, released under a BSD-3-Clause license.

Compared to the latter, it: - Forces the rendering of CLI results into ANSI shell sessions, via the

.. code-block:: ansi-shell-session directive.

class click_extra.sphinx.EofEchoingStdin(input, output)

Bases: EchoingStdin

Like click.testing.EchoingStdin but adds a visible ^D in place of the EOT character ().

ExampleRunner.invoke() adds  when terminate_input=True.

click_extra.sphinx.patch_modules()

Patch modules to work better with ExampleRunner.invoke().

subprocess.call` output is redirected to ``click.echo so it shows up in the example output.

class click_extra.sphinx.ExampleRunner

Bases: ExtraCliRunner

click.testing.CliRunner with additional features.

This class inherits from click_extra.testing.ExtraCliRunner to have full control of contextual color settings by the way of the color parameter. It also produce unfiltered ANSI codes so that the Directive sub-classes below can render colors in the HTML output.

force_color: bool = True

Force color rendering in invoke calls.

isolation(*args, **kwargs)

Copy of click.testing.CliRunner.isolation() with extra features.

• An additional output stream is returned, which is a mix of <stdout> and <stderr> streams if mix_stderr=True.

• Always returns the <stderr> stream.

Caution

This is a hard-copy of the modified isolation() method from click#2523 PR which has not been merged upstream yet.

Todo

Reduce the code duplication here by using clever monkeypatching?

invoke(cli, args=None, prog_name=None, input=None, terminate_input=False, env=None, _output_lines=None, **extra)

Like CliRunner.invoke() but displays what the user would enter in the terminal for env vars, command args, and prompts.

Parameters:

• terminate_input – Whether to display “^D” after a list of input.

• _output_lines – A list used internally to collect lines to be displayed.

declare_example(source)

Execute the given code, adding it to the runner’s namespace.

run_example(source)

Run commands by executing the given code, returning the lines of input and output. The code should be a series of the following functions:

• invoke(): Invoke a command, adding env vars, input, and output to the output.

• println(text=""): Add a line of text to the output.

• isolated_filesystem(): A context manager that changes to a temporary directory while executing the block.

close()

Clean up the runner once the document has been read.

env: t.Mapping[str, t.Optional[str]]

click_extra.sphinx.get_example_runner(document)

Get or create the ExampleRunner instance associated with a document.

class click_extra.sphinx.DeclareExampleDirective(name, arguments, options, content, lineno, content_offset, block_text, state, state_machine)

Bases: Directive

Add the source contained in the directive’s content to the document’s ExampleRunner, to be run using RunExampleDirective.

See ExampleRunner.declare_example().

has_content = True

May the directive have content?

required_arguments = 0

Number of required directive arguments.

optional_arguments = 0

Number of optional arguments after the required arguments.

final_argument_whitespace = False

May the final argument contain whitespace?

run()

class click_extra.sphinx.RunExampleDirective(name, arguments, options, content, lineno, content_offset, block_text, state, state_machine)

Bases: Directive

Run commands from DeclareExampleDirective and display the input and output.

See ExampleRunner.run_example().

has_content = True

May the directive have content?

required_arguments = 0

Number of required directive arguments.

optional_arguments = 0

Number of optional arguments after the required arguments.

final_argument_whitespace = False

May the final argument contain whitespace?

run()

class click_extra.sphinx.ClickDomain(env)

Bases: Domain

Declares new directives: - .. click:example:: - .. click:run::

name = 'click'

domain name: should be short, but unique

label = 'Click'

domain label: longer, more descriptive (used in messages)

directives: dict[str, type[Directive]] = {'example': <class 'click_extra.sphinx.DeclareExampleDirective'>, 'run': <class 'click_extra.sphinx.RunExampleDirective'>}

directive name -> directive class

merge_domaindata(docnames, otherdata)

Merge in data regarding docnames from a different domaindata inventory (coming from a subprocess in parallel builds).

data: dict[str, Any]

data value

env: BuildEnvironment

click_extra.sphinx.delete_example_runner_state(app, doctree)

Close and remove the ExampleRunner instance once the document has been read.

click_extra.sphinx.setup(app)

Register new directives, augmented with ANSI coloring. :rtype: None

Danger

This function activates some monkey-patches:

• sphinx.highlighting.PygmentsBridge is updated to set its default HTML formatter to an ANSI capable one for the whole Sphinx app.