CLICK-EXTRA-TEST-SUITE(1) CLICK-EXTRA-TEST-SUITE(1)

click-extra test-suite - Run declarative CLI test cases against a command or binary.

click-extra test-suite [OPTIONS]

Run declarative CLI test cases against a command or binary.

Resolves the suite by precedence: --suite-file or --suite-envvar, then the [tool.click-extra.test-suite] config (cases, then file), then a built-in default. Each case invokes the target with its parameters and checks the exit code and output.

Cases run in parallel by default (see --jobs): each is an independent process invocation, so they overlap well. Pass --jobs max to use every logical core, or --jobs 1 for sequential execution, which lets --exit-on-error stop on the first failure.

On an interactive terminal a spinner reports how many cases have finished. It stays silent in pipes and CI logs, and --no-progress or --accessible turns it off.

Path to the binary file to test, or a command line to be executed.
[required]
Path to a test suite file; its format is taken from the extension (YAML, TOML, JSON, JSON5, JSONC, Hjson). Repeat to run multiple suites in sequence. Without any suite source, a built-in default suite runs.
Name of an environment variable holding a test suite in YAML. Repeat to collect multiple suites.
Only run the cases with these 1-based numbers. Repeat to select several; omit to run them all.
Skip cases on these platforms. Repeat to skip several.
[values: aarch64, aix, alacritty, all_agents, all_architectures, all_arm, all_ci, all_mips, all_platforms, all_shells, all_sparc, all_terminals, all_traits, all_windows, almalinux, alpine, altlinux, amzn, android, apple_terminal, arch, arch_32_bit, arch_64_bit, arm, armv5tel, armv6l, armv7l, armv8l, ash, azure_pipelines, bamboo, bash, big_endian, bourne_shells, bsd, bsd_without_macos, buildkite, buildroot, c_shells, cachyos, centos, chromeos, circle_ci, cirrus_ci, claude_code, clearlinux, cline, cloudlinux, cmd, codebuild, codex, contour, copilot_cli, crush, csh, cursor, cygwin, dash, debian, dragonfly_bsd, endeavouros, exherbo, fedora, fish, foot, freebsd, gemini_cli, generic_linux, gentoo, ghostty, github_ci, gitlab_ci, gnome_terminal, gnu_screen, gpu_terminals, guix, haiku, hermetic_build, heroku_ci, hurd, hyper, i386, i586, i686, ibm_mainframe, ibm_powerkvm, illumos, iterm2, kali, kitty, konsole, ksh, kvmibm, linux, linux_layers, linux_like, linuxmint, little_endian, loongarch, loongarch64, macos, mageia, mandriva, manjaro, midnightbsd, mips, mips64, mips64el, mipsel, multiplexers, native_terminals, netbsd, nixos, nobara, nushell, openbsd, opensuse, openwrt, oracle, os400, other_posix, other_shells, parallels, pi, pidora, pikaos, powerpc, powershell, ppc, ppc64, ppc64le, raspbian, rhel, rio, riscv, riscv32, riscv64, rocky, s390x, scientific, sh, slackware, sles, slitaz, solaris, sourcemage, sparc, sparc64, sunos, system_v, tabby, tcsh, teamcity, tilix, tmux, travis_ci, tuxedo, ubuntu, ultramarine, unix, unix_layers, unix_without_macos, void, vscode_terminal, wasm32, wasm64, web_terminals, webassembly, wezterm, windows, windows_shells, windows_terminal, wsl1, wsl2, x86, x86_64, xenserver, xonsh, xterm, zellij, zsh]
Exit instantly on the first failed case (sequential runs only).
Default timeout for each CLI call, unless the case sets its own.
Directory to run each case's command in. Defaults to the current one. Moves the command under test, not the runner: suite files are read before any case starts.
Show the execution trace of failed cases.
Print the worker summary and the result tally.
Number of parallel jobs. Accepts an integer, auto (the host's logical CPUs minus one) or max (all logical CPUs). --jobs 0 runs sequentially.
Show this message and exit.

Location of the configuration file. Supports local path with glob patterns or remote URL.
Ignore all configuration files and only use command line parameters and environment variables.
Validate the configuration file and exit.
Export the configuration in the selected format to <stdout>, then exit.
[values: toml, yaml, json, json5, jsonc, hjson, xml, plist]

Accessibility mode: disable colors and render tables in a borderless, screen-reader-friendly format.
Colorize the output. A bare --color is the same as --color=always.
Disable colorization (alias of --color=never).
Show progress indicators during long operations. Disabled for non-interactive output (pipes, dumb terminals, CI) and by --accessible.
Color theme used for help screens.
Rendering style of tables.
[values: aligned, asciidoc, colon-grid, csv, csv-excel, csv-excel-tab, csv-unix, double-grid, double-outline, fancy-grid, fancy-outline, github, grid, heavy-grid, heavy-outline, hjson, html, jira, json, json5, jsonc, latex, latex-booktabs, latex-longtable, latex-raw, mediawiki, mixed-grid, mixed-outline, moinmoin, orgtbl, outline, pipe, plain, presto, pretty, psql, rounded-grid, rounded-outline, rst, simple, simple-grid, simple-outline, textile, toml, tsv, unsafehtml, vertical, xml, yaml, youtrack]

Either CRITICAL, ERROR, WARNING, INFO, DEBUG.
[values: CRITICAL, ERROR, WARNING, INFO, DEBUG]
Increase the default WARNING verbosity by one level for each additional repetition of the option.
Decrease the default WARNING verbosity by one level for each additional repetition of the option.
Shorthand for --verbosity DEBUG.

Measure and print elapsed execution time.
Show all CLI parameters, their provenance, defaults and value, then exit.
Show the tree of nested subcommands and exit.
Read the command's manual page and exit.
Render the command in the given format and exit.
Show the version and exit.

Path to the binary file to test, or a command line to be executed.
Path to a test suite file; its format is taken from the extension (YAML, TOML, JSON, JSON5, JSONC, Hjson). Repeat to run multiple suites in sequence. Without any suite source, a built-in default suite runs.
Name of an environment variable holding a test suite in YAML. Repeat to collect multiple suites.
Only run the cases with these 1-based numbers. Repeat to select several; omit to run them all.
Skip cases on these platforms. Repeat to skip several.
Exit instantly on the first failed case (sequential runs only).
Default timeout for each CLI call, unless the case sets its own.
Directory to run each case's command in. Defaults to the current one. Moves the command under test, not the runner: suite files are read before any case starts.
Show the execution trace of failed cases.
Print the worker summary and the result tally.
Number of parallel jobs. Accepts an integer, auto (the host's logical CPUs minus one) or max (all logical CPUs). --jobs 0 runs sequentially.
Show this message and exit.
Location of the configuration file. Supports local path with glob patterns or remote URL.
Validate the configuration file and exit.
Export the configuration in the selected format to <stdout>, then exit.
Accessibility mode: disable colors and render tables in a borderless, screen-reader-friendly format.
Colorize the output. A bare --color is the same as --color=always.
Disable colorization (alias of --color=never).
Show progress indicators during long operations. Disabled for non-interactive output (pipes, dumb terminals, CI) and by --accessible.
Color theme used for help screens.
Rendering style of tables.
Either CRITICAL, ERROR, WARNING, INFO, DEBUG.
Increase the default WARNING verbosity by one level for each additional repetition of the option.
Decrease the default WARNING verbosity by one level for each additional repetition of the option.
Shorthand for --verbosity DEBUG.
Measure and print elapsed execution time.
Show all CLI parameters, their provenance, defaults and value, then exit.
Show the tree of nested subcommands and exit.
Read the command's manual page and exit.
Render the command in the given format and exit.
Show the version and exit.

~/.config/click-extra/{*.toml,*.yaml,*.yml,*.json,*.json5,*.jwcc,*.jsonc,*.hjson,*.ini,*.xml,*.plist,*.sqlite,*.sqlite3,*.conf,pyproject.toml}

0
Success.
1
A runtime error, or an aborted prompt (Ctrl-C, a declined confirmation).
2
A usage error: unknown option, invalid value, missing operand, or an unparsable configuration file.

Kevin Deldycke

2026-09-09 9.1.1.dev0