Logging

The Python’s standard library logging module is a bit tricky to use. Click Extra provides pre-configured helpers with sane defaults to simplify the logging configuration.

Colored verbosity

--verbosity is a pre-configured option which allow users of your CLI to set the log level of a logging.Logger instance.

Default behavior is to:

  • print to <stderr>,

  • send messages to the root logger,

  • show WARNING-level messages and above,

  • use {-style format strings,

  • render logs with the {levelname}: {message} format,

  • color the log’s {levelname} variable.

Integrated option

This option is part of @extra_command and @extra_group by default:

from click_extra import extra_command, echo

@extra_command
def my_cli():
    echo("It works!")

See that --verbosity is featured in the help screen:

$ my-cli --help
Usage: my-cli [OPTIONS]

Options:
  --time / --no-time        Measure and print elapsed execution time.  [default:
                            no-time]
  --color, --ansi / --no-color, --no-ansi
                            Strip out all colors and all ANSI codes from output.
                            [default: color]
  -C, --config CONFIG_PATH  Location of the configuration file. Supports glob
                            pattern of local path and remote URL.  [default:
                            ~/.config/my-cli/*.{toml,yaml,yml,json,ini,xml}]
  --show-params             Show all CLI parameters, their provenance, defaults
                            and value, then exit.
  --verbosity LEVEL         Either CRITICAL, ERROR, WARNING, INFO, DEBUG.
                            [default: WARNING]
  -v, --verbose             Increase the default WARNING verbosity by one level
                            for each additional repetition of the option.
                            [default: 0]
  --version                 Show the version and exit.
  -h, --help                Show this message and exit.

This option can be invoked to display all the gory details of your CLI with the DEBUG level:

$ my-cli --verbosity DEBUG
debug: Set <Logger click_extra (DEBUG)> to DEBUG.
debug: Set <RootLogger root (DEBUG)> to DEBUG.
debug: Load configuration matching /home/runner/.config/my-cli/*.{toml,yaml,yml,json,ini,xml}
debug: Pattern is not an URL: search local file system.
debug: No configuration file found.
debug: Version string template variables:
debug: {module}         : <module 'click_extra.testing' from '/home/runner/work/click-extra/click-extra/click_extra/testing.py'>
debug: {module_name}    : click_extra.testing
debug: {module_file}    : /home/runner/work/click-extra/click-extra/click_extra/testing.py
debug: {module_version} : None
debug: {package_name}   : click_extra
debug: {package_version}: 4.14.3
debug: {exec_name}      : click_extra.testing
debug: {version}        : 4.14.3
debug: {prog_name}      : my-cli
debug: {env_info}       : {'username': '-', 'guid': '7ca23af85f9bba88b6a630afcc5be28', 'hostname': '-', 'hostfqdn': '-', 'uname': {'system': 'Linux', 'node': '-', 'release': '6.8.0-1021-azure', 'version': '#25-Ubuntu SMP Wed Jan 15 20:45:09 UTC 2025', 'machine': 'x86_64', 'processor': 'x86_64'}, 'linux_dist_name': '', 'linux_dist_version': '', 'cpu_count': 4, 'fs_encoding': 'utf-8', 'ulimit_soft': 65536, 'ulimit_hard': 65536, 'cwd': '-', 'umask': '0o2', 'python': {'argv': '-', 'bin': '-', 'version': '3.12.3 (main, Jan 17 2025, 18:03:48) [GCC 13.3.0]', 'compiler': 'GCC 13.3.0', 'build_date': 'Jan 17 2025 18:03:48', 'version_info': [3, 12, 3, 'final', 0], 'features': {'openssl': 'OpenSSL 3.0.13 30 Jan 2024', 'expat': 'expat_2.6.1', 'sqlite': '3.45.1', 'tkinter': '', 'zlib': '1.3', 'unicode_wide': True, 'readline': True, '64bit': True, 'ipv6': True, 'threading': True, 'urandom': True}}, 'time_utc': '2025-02-23 15:34:32.270047+00:00', 'time_utc_offset': 0.0, '_eco_version': '1.1.0'}
It works!
debug: Reset <RootLogger root (DEBUG)> to WARNING.
debug: Reset <Logger click_extra (DEBUG)> to WARNING.

Hint

Notice how, in the output above, each logger’s own level is printed as debug messages.

And how they’re all first set to DEBUG at the beginning of the command, then reset back to their default WARNING at the end.

Important

Besides --verbosity, there is another -v/--verbose option. The later works relative to --verbosity, and can be used to increase the level in steps, simply by repeating it.

In the rest of this documentation, we will mainly focus on the canonical --verbosity option to keep things simple (logging is already complicated enough…).

Standalone option

The verbosity option can be used independently of @extra_command, and you can attach it to a vanilla Click command:

import logging
from click import command, echo
from click_extra import verbosity_option

@command
@verbosity_option
def vanilla_command():
    echo("It works!")
    logging.info("We're printing stuff.")
$ vanilla-command --help
Usage: vanilla-command [OPTIONS]

Options:
  --verbosity LEVEL  Either CRITICAL, ERROR, WARNING, INFO, DEBUG.
  --help             Show this message and exit.
$ vanilla-command
It works!
$ vanilla-command --verbosity INFO
It works!
info: We're printing stuff.

Default logger

The --verbosity option is by default attached to the global root logger.

This allows you to use module-level helpers like logging.debug. That way you don’t have to worry about setting up your own logger. And logging messages can be easily produced with minimal code:

import logging
from click import command
from click_extra import verbosity_option

@command
@verbosity_option
def my_cli():
    # Print a message for each level.
    logging.critical("Complete meltdown!")
    logging.error("Does not compute.")
    logging.warning("Mad scientist at work!")
    logging.info("This is a message.")
    logging.debug("We're printing stuff.")

The --verbosity option print by default all messages at the WARNING level and above:

$ my-cli
critical: Complete meltdown!
error: Does not compute.
warning: Mad scientist at work!

But each level can be selected with the option:

$ my-cli --verbosity CRITICAL
critical: Complete meltdown!
$ my-cli --verbosity ERROR
critical: Complete meltdown!
error: Does not compute.
$ my-cli --verbosity WARNING
critical: Complete meltdown!
error: Does not compute.
warning: Mad scientist at work!
$ my-cli --verbosity INFO
critical: Complete meltdown!
error: Does not compute.
warning: Mad scientist at work!
info: This is a message.
$ my-cli --verbosity DEBUG
debug: Set <Logger click_extra (DEBUG)> to DEBUG.
debug: Set <RootLogger root (DEBUG)> to DEBUG.
critical: Complete meltdown!
error: Does not compute.
warning: Mad scientist at work!
info: This is a message.
debug: We're printing stuff.
debug: Reset <RootLogger root (DEBUG)> to WARNING.
debug: Reset <Logger click_extra (DEBUG)> to WARNING.

Caution

root is the default logger associated with --verbosity. Which means that, if not configured, all loggers will be printed at the verbosity level set by the option:

import logging
from click import command
from click_extra import verbosity_option

@command
@verbosity_option
def multiple_loggers_cli():
   # Use the default root logger.
   root_logger = logging.getLogger()
   root_logger.warning("Root warning message")
   root_logger.info("Root info message")

   # Create a custom logger and use it.
   my_logger = logging.getLogger("my_logger")
   my_logger.warning("My warning message")
   my_logger.info("My info message")

In that case, a normal invocation will only print the default WARNING messages:

$ multiple-loggers-cli
warning: Root warning message
warning: My warning message

And calling --verbosity INFO will print both root and my_logger messages of that level:

$ multiple-loggers-cli --verbosity INFO
warning: Root warning message
info: Root info message
warning: My warning message
info: My info message

To prevent this behavior, you can associate the --verbosity option with your own custom logger. This is explained in the next section.

Custom logger

The preferred way to customize log messages is to create your own logger and attach it to the --verbosity option.

This can be done with new_extra_logger. Here is how we can for example change the format of the log messages:

import logging
from click import command
from click_extra import new_extra_logger, verbosity_option

new_extra_logger(
  name="app_logger",
  format="{levelname} | {name} | {message}"
)

@command
@verbosity_option(default_logger="app_logger")
def custom_logger_cli():
     # Default root logger.
    logging.warning("Root logger warning")
    logging.info("Root logger info")

    # Use our custom logger.
    my_logger = logging.getLogger("app_logger")
    my_logger.warning("Custom warning")
    my_logger.info("Custom info")

That way the root logger keeps its default format, while the custom logger uses the new one:

$ custom-logger-cli
warning: Root logger warning
warning | app_logger | Custom warning

And changing the verbosity level will only affect the custom logger:

$ custom-logger-cli --verbosity INFO
warning: Root logger warning
warning | app_logger | Custom warning
info | app_logger | Custom info

Now if we don’t explicitly pass the custom logger to the --verbosity option, the default root logger will be tied to it instead:

import logging
from click import command
from click_extra import new_extra_logger, verbosity_option

new_extra_logger(
  name="app_logger",
  format="{levelname} | {name} | {message}"
)

@command
@verbosity_option
def root_verbosity_cli():
     # Default root logger.
    logging.warning("Root logger warning")
    logging.info("Root logger info")

    # Use our custom logger.
    my_logger = logging.getLogger("app_logger")
    my_logger.warning("Custom warning")
    my_logger.info("Custom info")

In that case the default behavior doesn’t change and messages are rendered in their own logger’s format, at the default WARNING level:

$ root-verbosity-cli
warning: Root logger warning
warning | app_logger | Custom warning

But changing the verbosity level only affects root, in the opposite of the previous example:

$ root-verbosity-cli --verbosity INFO
warning: Root logger warning
info: Root logger info
warning | app_logger | Custom warning

Important

By design, new loggers are always created as sub-loggers of root. And as such, their messages are propagated back to it.

But new_extra_logger always creates new loggers by setting their propagate attribute to False. This means that messages of new loggers won’t be propagated to their parents.

This is the reason why, in the example above, the root and app_logger loggers are independent.

Let’s experiment with that property and set the propagate attribute to True:

import logging
from click import command
from click_extra import new_extra_logger, verbosity_option

new_extra_logger(
   name="app_logger",
   propagate=True,
   format="{levelname} | {name} | {message}"
)

@command
@verbosity_option
def logger_propagation_cli():
   # Default root logger.
   logging.warning("Root logger warning")
   logging.info("Root logger info")

   # Use our custom logger.
   my_logger = logging.getLogger("app_logger")
   my_logger.warning("Custom warning")
   my_logger.info("Custom info")
$ logger-propagation-cli
warning: Root logger warning
warning | app_logger | Custom warning
warning: Custom warning

Here you can immediately spot the issue with propagation: app_logger’s messages are displayed twice. Once in their custom format, and once in the format of the root logger.

See also

The reason for that hierarchycal design is to allow for dot-separated logger names, like foo.bar.baz. Which allows for even more granular control of loggers by filtering.

Tip

You can creatively configure loggers to produce any kind of messages, like this JSON-like format:

import logging
from click import command
from click_extra import new_extra_logger, verbosity_option

new_extra_logger(
   name="json_logger",
   format='{{"time": "{asctime}", "name": "{name}", "level": "{levelname}", "msg": "{message}"}}',
)

@command
@verbosity_option(default_logger="json_logger")
def json_logs_cli():
   my_logger = logging.getLogger("json_logger")
   my_logger.info("This is an info message.")
$ json-logs-cli --verbosity INFO
{"time": "2025-02-23 15:34:41,557", "name": "json_logger", "level": "info", "msg": "This is an info message."}

Hint

Because loggers are registered in a global registry, you can set them up in one place and use them in another. That is the idiomatic approach, which consist in always referring to them by name, as in all examples above.

But for convenience, you can pass the logger object directly to the option:

import logging
from click import command
from click_extra import new_extra_logger, verbosity_option

my_logger = new_extra_logger(name="app_logger")

@command
@verbosity_option(default_logger=my_logger)
def logger_object_cli():
   # Default root logger.
   logging.warning("Root warning message")
   logging.info("Root info message")

   # My custom logger.
   my_logger.warning("My warning message")
   my_logger.info("My info message")
$ logger-object-cli --verbosity INFO
warning: Root warning message
warning: My warning message
info: My info message

Global configuration

If you want to change the global configuration of all loggers, you can rely on new_extra_logger. Because the latter defaults to the root logger, any default logger propagating their messages to it will be affected:

import logging
from click import command
from click_extra import new_extra_logger, verbosity_option

root_logger = new_extra_logger(format="{levelname} | {name} | {message}")

@command
@verbosity_option(default_logger=root_logger)
def root_format_cli():
     # Default root logger.
    logging.warning("Root logger warning")
    logging.info("Root logger info")

    # Use our custom logger.
    my_logger = logging.getLogger("my_logger")
    my_logger.warning("Custom warning")
    my_logger.info("Custom info")
$ root-format-cli
warning | root | Root logger warning
warning | my_logger | Custom warning
$ root-format-cli --verbosity INFO
warning | root | Root logger warning
info | root | Root logger info
warning | my_logger | Custom warning
info | my_logger | Custom info

Get verbosity level

You can get the name of the current verbosity level from the context or the logger itself:

import logging
from click_extra import command, echo, pass_context, verbosity_option

@command
@verbosity_option
@pass_context
def vanilla_command(ctx):
    level_from_context = ctx.meta["click_extra.verbosity_level"]

    level_from_logger = logging._levelToName[logging.getLogger().getEffectiveLevel()]

    echo(f"Level from context: {level_from_context}")
    echo(f"Level from logger: {level_from_logger}")
$ vanilla-command --verbosity INFO
Level from context: INFO
Level from logger: INFO

Internal click_extra logger

Click Extra has its own logger, named click_extra, which is used to print debug messages to inspect its own internal behavior.

click_extra.logging API

        classDiagram
  ExtraOption <|-- ExtraVerbosity
  ExtraVerbosity <|-- VerboseOption
  ExtraVerbosity <|-- VerbosityOption
  Formatter <|-- ExtraFormatter
  StreamHandler <|-- ExtraStreamHandler
    

Logging utilities.

click_extra.logging.LOG_LEVELS: dict[str, int] = {'CRITICAL': 50, 'DEBUG': 10, 'ERROR': 40, 'INFO': 20, 'WARNING': 30}

Mapping of canonical log level names to their integer level.

That’s our own version of logging._nameToLevel, but:

click_extra.logging.DEFAULT_LEVEL: int = 30

WARNING is the default level we expect any loggers to starts their lives at.

WARNING has been chosen as it is the level at which the default Python’s global root logger is set up.

This value is also used as the default level for VerbosityOption .

click_extra.logging.DEFAULT_LEVEL_NAME: str = 'WARNING'

Name of the DEFAULT_LEVEL.

class click_extra.logging.THandler

Custom types to be used in type hints below.

alias of TypeVar(‘THandler’, bound=Handler)

class click_extra.logging.ExtraStreamHandler(stream=None)[source]

Bases: StreamHandler

A handler to output logs to the console.

Wraps logging.StreamHandler, but use click.echo() to support color printing.

Only <stderr> or <stdout> are allowed as output stream.

If stream is not specified, <stderr> is used by default

Initialize the handler.

If stream is not specified, sys.stderr is used.

property stream: IO[Any]

The stream to which logs are written.

A proxy of the parent logging.StreamHandler’s stream attribute.

Redefined here to enforce checks on the stream value.

emit(record)[source]

Use click.echo() to print to the console.

Return type:

None

class click_extra.logging.ExtraFormatter(fmt=None, datefmt=None, style='%', validate=True, *, defaults=None)[source]

Bases: Formatter

Click Extra’s default log formatter.

Initialize the formatter with specified format strings.

Initialize the formatter either with the specified format string, or a default as described above. Allow for specialized date formatting with the optional datefmt argument. If datefmt is omitted, you get an ISO8601-like (or RFC 3339-like) format.

Use a style parameter of ‘%’, ‘{’ or ‘$’ to specify that you want to use one of %-formatting, str.format() ({}) formatting or string.Template formatting in your format string.

Changed in version 3.2: Added the style parameter.

formatMessage(record)[source]

Colorize the record’s log level name before calling the standard formatter.

Colors are sourced from a click_extra.colorize.HelpExtraTheme. Default colors are configured on click_extra.colorize.default_theme.

Return type:

str

click_extra.logging.extraBasicConfig(*, filename=None, filemode='a', format='{levelname}: {message}', datefmt=None, style='{', level=None, stream=None, handlers=None, force=False, encoding=None, errors='backslashreplace', stream_handler_class=<class 'click_extra.logging.ExtraStreamHandler'>, file_handler_class=<class 'logging.FileHandler'>, formatter_class=<class 'click_extra.logging.ExtraFormatter'>)[source]

Configure the global root logger.

This function is a wrapper around Python standard library’s logging.basicConfig(), but with additional parameters and tweaked defaults.

It sets up the global root logger, and optionally adds a file or stream handler to it.

Differences in default values:

Argument

extraBasicConfig() default

logging.basicConfig() default

style

{

%

format

{levelname}: {message}

%(levelname)s:%(name)s:%(message)s

This function takes the same parameters as logging.basicConfig(), but require them to be all passed as explicit keywords arguments.

Parameters:
  • filename (str | None) – Specifies that a logging.FileHandler be created, using the specified filename, rather than an ExtraStreamHandler.

  • filemode (str) –

    If filename is specified, open the file in this mode.

    Defaults to a.

  • format (str | None) –

    Use the specified format string for the handler.

    Defaults to {levelname}: {message}.

  • datefmt (str | None) – Use the specified date/time format, as accepted by time.strftime().

  • style (Literal['%', '{', '$']) –

    If format is specified, use this style for the format string:

    Defaults to {.

  • level (int | str | None) – Set the root logger level to the specified level.

  • stream (Optional[IO[Any]]) – Use the specified stream to initialize the ExtraStreamHandler. Note that this argument is incompatible with filename - if both are present, a ValueError is raised.

  • handlers (Iterable[Handler] | None) – If specified, this should be an iterable of already created handlers to add to the root logger. Any handlers which don’t already have a formatter set will be assigned the default formatter created in this function. Note that this argument is incompatible with filename or stream - if both are present, a ValueError is raised.

  • force (bool) – If this argument is specified as True, any existing handlers attached to the root logger are removed and closed, before carrying out the configuration as specified by the other arguments.

  • encoding (str | None) – Name of the encoding used to decode or encode the file. To be specified along with filename, and passed to logging.FileHandler for opening the output file.

  • errors (str | None) – Optional string that specifies how encoding and decoding errors are to be handled by the logging.FileHandler. Defaults to backslashreplace. Note that if None is specified, it will be passed as such to open().

Return type:

None

Important

Always keep the signature of this function, the default values of its parameters and its documentation in sync with the one from Python’s standard library.

These new arguments are available for better configurability:

Parameters:

Note

I don’t like the camel-cased name of this function and would have called it extra_basic_config(), but it’s kept this way for consistency with Python’s standard library.

click_extra.logging.new_extra_logger(name='root', *, propagate=False, force=True, **kwargs)[source]

Setup a logger in the style of Click Extra.

By default, this helper will:

  • Fetch the logger registered under the name parameter, or creates a new one with that name if it doesn’t exist,

  • Set the logger’s propagate attribute to False,

  • Force removal of any existing handlers and formatters attached to the logger,

  • Attach a new ExtraStreamHandler with ExtraFormatter,

  • Return the logger object.

This function is a wrapper around extraBasicConfig() and takes the same keywords arguments.

Parameters:
  • name (str) – ID of the logger to setup. If None, Python’s root logger will be used. If a logger with the provided name is not found in the global registry, a new logger with that name will be created.

  • propagate (bool) – Sets the logger’s propagate attribute. Defaults to False.

  • force (bool) – Same as the force parameter from logging.basicConfig() and extraBasicConfig(). Defaults to True.

  • kwargs – Any other keyword parameters supported by logging.basicConfig() and extraBasicConfig().

Return type:

Logger

class click_extra.logging.ExtraVerbosity(param_decls=None, default_logger='root', expose_value=False, is_eager=True, **kwargs)[source]

Bases: ExtraOption

A base class implementing all the common halpers to manipulated logger’s verbosity.

Sets the level of the provided logger. If no logger is provided, sets the level of the global root logger.

Important

The internal click_extra logger will be aligned to the level set through this class.

Caution

This class is not intended to be used as-is. It is an internal place to reconcile the verbosity level selected by the competing logger options implemented below:

  • --verbosity

  • --verbose/-v

Set up a verbosity-altering option.

Parameters:

default_logger (Logger | str) – If a logging.Logger object is provided, that’s the instance to which we will set the level to. If the parameter is a string and is found in the global registry, we will use it as the logger’s ID. Otherwise, we will create a new logger with new_extra_logger() Default to the global root logger.

property all_loggers: Generator[Logger, None, None]

Returns the list of logger IDs affected by the verbosity option.

Will returns click_extra internal logger first, then the option’s logger_name.

reset_loggers()[source]

Forces all loggers managed by the option to be reset to DEFAULT_LEVEL. :rtype: None

Important

Loggers are reset in reverse order to ensure the internal logger is changed last. That way the internal click_extra logger can report its ongoing logger-altering operations while using the logging facilities itself.

Danger

Resetting loggers is extremely important for unittests. Because they’re global, loggers have tendency to leak and pollute their state between multiple test calls.

set_level(ctx, param, value)[source]

Set level of all loggers configured on the option.

All verbosity-related options are attached to this callback, so that’s where we reconcile the multiple values provided by different options. In case of a conflict, the highest versbosity level always takes precedence.

The final reconciled level chosen for the logger will be saved in ctx.meta["click_extra.verbosity_level"]. This context property served as a kind of global state shared by all verbosity-related options.

Return type:

None

default: t.Union[t.Any, t.Callable[[], t.Any]]
type: types.ParamType
is_flag: bool
is_bool_flag: bool
flag_value: t.Any
name: t.Optional[str]
opts: t.List[str]
secondary_opts: t.List[str]
logger_name: str

The ID of the logger to set the level to.

This will be provided to logging.getLogger() to fetch the logger object, and as such, can be a dot-separated string to build hierarchical loggers.

class click_extra.logging.VerbosityOption(param_decls=None, default_logger='root', default='WARNING', metavar='LEVEL', type=Choice(['CRITICAL', 'ERROR', 'WARNING', 'INFO', 'DEBUG']), help='Either CRITICAL, ERROR, WARNING, INFO, DEBUG.', **kwargs)[source]

Bases: ExtraVerbosity

--verbosity option to set the the log level of ExtraVerbosity.

Set up a verbosity-altering option.

Parameters:

default_logger (Logger | str) – If a logging.Logger object is provided, that’s the instance to which we will set the level to. If the parameter is a string and is found in the global registry, we will use it as the logger’s ID. Otherwise, we will create a new logger with new_extra_logger() Default to the global root logger.

set_level(ctx, param, value)[source]

The value passed to --verbosity will be saved in ctx.meta["click_extra.verbosity"].

Return type:

None

class click_extra.logging.VerboseOption(param_decls=None, count=True, **kwargs)[source]

Bases: ExtraVerbosity

--verbose/-v` option to increase the log level of ExtraVerbosity by a number of steps.

If -v is passed to a CLI, then it will increase the verbosity level by one step. The option can be provided multiple times by the user. So if -vv (or -v -v) is passed, the verbosity will be increase by 2 levels.

The default base-level from which we start incrementing is sourced from VerbosityOption.default. So with --verbosity’s default set to WARNING:

  • -v will increase the level to INFO,

  • -vv will increase the level to DEBUG,

  • any number of repetition above that point will be set to the maximum level, so for -vvvvv for example will be capped at DEBUG.

Set up a verbosity-altering option.

Parameters:

default_logger – If a logging.Logger object is provided, that’s the instance to which we will set the level to. If the parameter is a string and is found in the global registry, we will use it as the logger’s ID. Otherwise, we will create a new logger with new_extra_logger() Default to the global root logger.

get_base_level(ctx)[source]

Returns the default base-level from which the option will start incrementing.

We try first to get the default level from any instance of VerbosityOption defined on the current command. If none is found, it’s because the --verbose option is used standalone. In which case we defaults to DEFAULT_LEVEL_NAME.

Return type:

str

get_help_record(ctx)[source]

Dynamiccaly generates the default help message.

We need that patch because get_base_level() depends on the context, so we cannot hard-code the help message as VerboseOption.__init__() default.

Return type:

tuple[str, str] | None

set_level(ctx, param, value)[source]

Translate the number of steps to the target log level.

The value passed to --verbose/-v will be saved in ctx.meta["click_extra.verbose"].

Return type:

None