CLI testing

Todo

Write example and tutorial.

click_extra.testing API

        classDiagram
  AssertionError <|-- RegexLineMismatch
  BytesIO <|-- BytesIOCopy
  CliRunner <|-- ExtraCliRunner
    

CLI testing and simulation of their execution.

click_extra.testing.PROMPT = '$ '

Prompt used to simulate the CLI execution.

Hint

Use ASCII characters to avoid issues with Windows terminals.

click_extra.testing.INDENT = '  '

Constants for rendering of CLI execution.

click_extra.testing.args_cleanup(*args)

Flatten recursive iterables, remove all None, and cast each element to strings.

Helps serialize pathlib.Path and other objects.

It also allows for nested iterables and None values as CLI arguments for convenience. We just need to flatten and filters them out.

Return type:

tuple[str, ...]

click_extra.testing.format_cli_prompt(cmd_args, extra_env=None)

Simulate the console prompt used to invoke the CLI.

Return type:

str

click_extra.testing.render_cli_run(args, result, env=None)

Generates the full simulation of CLI execution, including output.

Mostly used to print debug traces to user or in test results.

Return type:

str

click_extra.testing.print_cli_run(args, result, env=None)

Prints the full simulation of CLI execution, including output.

Return type:

None

click_extra.testing.INVOKE_ARGS = {'args', 'catch_exceptions', 'cli', 'color', 'env', 'input', 'self'}

Parameter IDs of click.testing.CliRunner.invoke().

We need to collect them to help us identify which extra parameters passed to invoke() collides with its original signature.

Warning

This has been reported upstream to Click project but has been rejected and not considered an issue worth fixing.

class click_extra.testing.BytesIOCopy(copy_to)

Bases: BytesIO

Patch io.BytesIO to let the written stream be copied to another.

Caution

This has been proposed upstream to Click project but has not been merged yet.

flush()

Does nothing.

Return type:

None

write(b)

Write bytes to file.

Return the number of bytes written.

Return type:

int

class click_extra.testing.ExtraCliRunner(charset='utf-8', env=None, echo_stdin=False, catch_exceptions=True)

Bases: CliRunner

Augment click.testing.CliRunner with extra features and bug fixes.

force_color: bool = False

Global class attribute to override the color parameter in invoke.

Note

This was initially developed to force the initialization of the runner during the setup of Sphinx new directives. This was the only way we found, as to patch some code we had to operate at the class level.

invoke(cli, *args, input=None, env=None, catch_exceptions=True, color=None, **extra)

Same as click.testing.CliRunner.invoke() with extra features.

• The first positional parameter is the CLI to invoke. The remaining positional parameters of the function are the CLI arguments. All other parameters are required to be named.

• The CLI arguments can be nested iterables of arbitrary depth. This is useful for argument composition of test cases with @pytest.mark.parametrize.

• Allow forcing of the color property at the class-level via force_color attribute.

• Adds a special case in the form of color="forced" parameter, which allows colored output to be kept, while forcing the initialization of Context.color = True. This is not allowed in current implementation of click.testing.CliRunner.invoke() because of colliding parameters.

• Strips all ANSI codes from results if color was explicirely set to False.

• Always prints a simulation of the CLI execution as the user would see it in its terminal. Including colors.

• Pretty-prints a formatted exception traceback if the command fails.

Parameters:

• cli (Command) – CLI to invoke.

• *args –

  can be nested iterables composed of str, pathlib.Path objects and None values. The nested structure will be flattened and None values will be filtered out. Then all elements will be casted to str. See args_cleanup() for details.

• input (str | bytes | IO | None) – same as click.testing.CliRunner.invoke().

• env (Mapping[str, str | None] | None) – same as click.testing.CliRunner.invoke().

• catch_exceptions (bool) – same as click.testing.CliRunner.invoke().

• color (Union[bool, Literal['forced'], None]) – If a boolean, the parameter will be passed as-is to click.testing.CliRunner.isolation(). If "forced", the parameter will be passed as True to click.testing.CliRunner.isolation() and an extra color=True parameter will be passed to the invoked CLI.

• **extra –

  same as click.testing.CliRunner.invoke(), but colliding parameters are allowed and properly passed on to the invoked CLI.

Return type:

Result

env: cabc.Mapping[str, str | None]

click_extra.testing.unescape_regex(text)

De-obfuscate a regex for better readability.

This is like the reverse of re.escape().

Return type:

str

exception click_extra.testing.RegexLineMismatch(regex_line, content_line, line_number)

Bases: AssertionError

Raised when a regex line does not match the corresponding content line.

click_extra.testing.regex_fullmatch_line_by_line(regex, content)

Check that the content matches the given regex.

If the regex does not fully match the content, raise an AssertionError, with a message showing the first mismatching line.

This is useful when comparing large walls of text, such as CLI output.

Return type:

None