MDEDUP(1) MDEDUP(1)

mdedup - Deduplicate mail boxes.

mdedup [OPTIONS] MAIL_SOURCE_1 MAIL_SOURCE_2 ...

Deduplicate mails from multiple sources.

Process:
- Step #1: load mails from their sources.
- Step #2: compute the canonical hash of each mail based on their headers (and


           optionally their body), and regroup mails sharing the same hash.
- Step #3: apply a selection strategy on each subset of duplicate mails.
- Step #4: perform an action on all selected mails.
- Step #5: report statistics.
Mail sources to deduplicate. Can be a single mail box or a list of mails.

Start from the option values of a preset. conservative treats as copies only the mails whose bytes are identical, and stands for --raw-headers --hash-body raw --size-threshold -1 --content-threshold -1 --strategy select-one. default keeps the built-in defaults. aggressive trusts the Message-ID header alone to group copies, and stands for --hash-header Message-ID. An option set on the command line, in the environment or in a configuration file overrides the value the preset gives it. conservative turns the thresholds off only with its own --raw-headers and --hash-body raw. When another value or a --hash-header list replaces either one, both thresholds keep their defaults.

Force all provided mail sources to be parsed in the specified format. If not set, auto-detect the format of sources independently. Auto-detection only supports maildir, mbox and eml formats. Use this option to open up other box formats, or bypass unreliable detection.
Remove the lock on mail source opening if one is found.

Headers to use to compute each mail's hash. Must be repeated multiple times to set an ordered list of headers. Header IDs are case-insensitive. Repeating entries are ignored.
Hash the header block of each mail as its box stores it, line endings aside: every header, in its order, with no normalization. Replaces the --hash-header list, so the two are refused together, and no minimal number of headers is required.
Method used to hash the body of mails. Defaults to skip, which doesn't hash the body at all: it is the fastest method and header-based hash should be sufficient to determine duplicate set. raw hashes the body bytes as the box holds them, with no decoding. decoded undoes the transfer encodings and decodes the text first, so copies that differ in their encoding alone hash the same (slow). normalized also strips all whitespace (slowest).
Compute and display the internal hashes used to identify duplicates. Do not performs any selection or action.
Reuse the hashes computed by previous runs, so unchanged mails are not read nor parsed again. An entry is only trusted while the size and modification time of the file backing its mail are unchanged, and the whole cache is discarded as soon as any option feeding the hashes changes. Mails of a file-based box (mbox, babyl, mmdf) all share the box's file, so editing it invalidates every one of them at once. Off by default.
Location of the hash cache database. Implies --cache. Defaults to /home/runner/.cache/mdedup/hashes.db

Process each set of mails sharing the same hash and apply the selection --strategy. Fine-grained checks on size and content are performed beforehand. Mails differing above safety levels are set aside so the rest can still be deduplicated, and the set is skipped if fewer than 2 remain. Limits can be set via the --size-threshold and --content-threshold options.

Selection strategy to apply within a subset of duplicates. Can be repeated multiple times to set an ordered list of fallback strategies: each duplicate set is handed over to the next strategy each time a strategy fails to discriminate its mails, by selecting all of them, none of them, or by missing the timestamps to compare them. Repeating entries are ignored, including aliases of strategies already listed. If not set, every duplicate set is skipped: none of its mails is selected or discarded. The mails without duplicates stay selected, and a copy or a move still carries them. Description of each strategy is available further down that help screen.
Source of a mail's time reference used in time-sensitive strategies.
Regular expression on a mail's file path. Applies to individual mail location for folder-based boxes (maildir, mh, eml). But for file-based boxes (mbox, babyl, mmdf), applies to the whole box's path, as all mails are packed into one single file. Required in discard-matching-path, discard-non-matching-path, select-matching-path and select-non-matching-path strategies.
Maximum difference allowed in size between mails sharing the same hash, in bytes of their decoded bodies. Mails in an offending pair are set aside until the rest all pass. The subset is skipped if fewer than 2 remain. Set to 0 to enforce strictness and apply selection strategy on the subset only if all mails are exactly the same. Set to -1 to allow any difference and apply the strategy whatever the differences.
Maximum difference allowed in content between mails sharing the same hash, in bytes of the diff of their decoded bodies. Mails in an offending pair are set aside until the rest all pass. The subset is skipped if fewer than 2 remain. Set to 0 to enforce strictness and apply selection strategy on the subset only if all mails are exactly the same. Set to -1 to allow any difference and apply the strategy whatever the differences.
Show the unified diff of duplicates not within thresholds.

Action performed on the selected mails. Defaults to copy-selected as it is the safest: it only reads the mail sources and create a brand new mail box with the selection results. Mails that have no duplicate are part of the selection, but delete-selected leaves them in place, as deleting one removes the only copy of it there is.
Location of the destination mail box to where to copy or move deduplicated mails. Required in copy-selected, copy-discarded, move-selected and move-discarded actions.
Format of the mail box to which deduplication mails will be exported to. Only affects copy-selected, copy-discarded, move-selected and move-discarded actions.
If destination mail box already exists, add mails into it instead of interrupting (default behavior). Affect copy-selected, copy-discarded, move-selected and move-discarded actions.
Hardlink discarded mails whose content differs byte for byte from the copy they are linked to, instead of leaving them alone. Their own content is then swapped for that copy's, so whatever was unique to them, like the headers a mail collects on its way to one account, is lost. Only affects the hardlink-discarded action.
Do not perform any action but act as if it was, and report which action would have been performed otherwise.

Number of parallel jobs used to hash mails (step #2) and settle duplicate sets (step #3). Accepts an integer, 'auto' (the host's logical CPUs, minus one when there are three or more) or 'max'. Defaults to 1 (sequential). Only folder-based boxes fan out; higher values speed up --hash-body decoded/normalized and boxes holding many duplicate sets.
Show this message and exit.

Location of the configuration file. Supports local path with glob patterns or remote URL.
Ignore all configuration files and only use command line parameters and environment variables.
Validate the configuration file and exit.
Export the configuration in the selected format to <stdout>, then exit.
[values: toml, yaml, json, json5, jsonc, hjson, xml, plist]

Accessibility mode: disable colors and render tables in a borderless, screen-reader-friendly format.
Colorize the output. A bare --color is the same as --color=always.
Disable colorization (alias of --color=never).
Show progress indicators during long operations. Disabled for non-interactive output (pipes, dumb terminals, CI) and by --accessible.
Color theme used for help screens.
Rendering style of tables.
[values: aligned, asciidoc, colon-grid, csv, csv-excel, csv-excel-tab, csv-unix, double-grid, double-outline, fancy-grid, fancy-outline, github, grid, heavy-grid, heavy-outline, hjson, html, jira, json, json5, jsonc, latex, latex-booktabs, latex-longtable, latex-raw, mediawiki, mixed-grid, mixed-outline, moinmoin, orgtbl, outline, pipe, plain, presto, pretty, psql, rounded-grid, rounded-outline, rst, simple, simple-grid, simple-outline, textile, toml, tsv, unsafehtml, vertical, xml, yaml, youtrack]

Either CRITICAL, ERROR, WARNING, INFO, DEBUG.
[values: CRITICAL, ERROR, WARNING, INFO, DEBUG]
Increase the default WARNING verbosity by one level for each additional repetition of the option.
Decrease the default WARNING verbosity by one level for each additional repetition of the option.
Shorthand for --verbosity DEBUG.

Measure and print elapsed execution time.
Show all CLI parameters, their provenance, defaults and value, then exit.
Show the tree of nested subcommands and exit.
Read the command's manual page and exit.
Render the command in the given format and exit.
Show the version and exit.

Start from the option values of a preset. conservative treats as copies only the mails whose bytes are identical, and stands for --raw-headers --hash-body raw --size-threshold -1 --content-threshold -1 --strategy select-one. default keeps the built-in defaults. aggressive trusts the Message-ID header alone to group copies, and stands for --hash-header Message-ID. An option set on the command line, in the environment or in a configuration file overrides the value the preset gives it. conservative turns the thresholds off only with its own --raw-headers and --hash-body raw. When another value or a --hash-header list replaces either one, both thresholds keep their defaults.
Force all provided mail sources to be parsed in the specified format. If not set, auto-detect the format of sources independently. Auto-detection only supports maildir, mbox and eml formats. Use this option to open up other box formats, or bypass unreliable detection.
Remove the lock on mail source opening if one is found.
Headers to use to compute each mail's hash. Must be repeated multiple times to set an ordered list of headers. Header IDs are case-insensitive. Repeating entries are ignored.
Hash the header block of each mail as its box stores it, line endings aside: every header, in its order, with no normalization. Replaces the --hash-header list, so the two are refused together, and no minimal number of headers is required.
Method used to hash the body of mails. Defaults to skip, which doesn't hash the body at all: it is the fastest method and header-based hash should be sufficient to determine duplicate set. raw hashes the body bytes as the box holds them, with no decoding. decoded undoes the transfer encodings and decodes the text first, so copies that differ in their encoding alone hash the same (slow). normalized also strips all whitespace (slowest).
Compute and display the internal hashes used to identify duplicates. Do not performs any selection or action.
Reuse the hashes computed by previous runs, so unchanged mails are not read nor parsed again. An entry is only trusted while the size and modification time of the file backing its mail are unchanged, and the whole cache is discarded as soon as any option feeding the hashes changes. Mails of a file-based box (mbox, babyl, mmdf) all share the box's file, so editing it invalidates every one of them at once. Off by default.
Location of the hash cache database. Implies --cache. Defaults to /home/runner/.cache/mdedup/hashes.db
Selection strategy to apply within a subset of duplicates. Can be repeated multiple times to set an ordered list of fallback strategies: each duplicate set is handed over to the next strategy each time a strategy fails to discriminate its mails, by selecting all of them, none of them, or by missing the timestamps to compare them. Repeating entries are ignored, including aliases of strategies already listed. If not set, every duplicate set is skipped: none of its mails is selected or discarded. The mails without duplicates stay selected, and a copy or a move still carries them. Description of each strategy is available further down that help screen.
Source of a mail's time reference used in time-sensitive strategies.
Regular expression on a mail's file path. Applies to individual mail location for folder-based boxes (maildir, mh, eml). But for file-based boxes (mbox, babyl, mmdf), applies to the whole box's path, as all mails are packed into one single file. Required in discard-matching-path, discard-non-matching-path, select-matching-path and select-non-matching-path strategies.
Maximum difference allowed in size between mails sharing the same hash, in bytes of their decoded bodies. Mails in an offending pair are set aside until the rest all pass. The subset is skipped if fewer than 2 remain. Set to 0 to enforce strictness and apply selection strategy on the subset only if all mails are exactly the same. Set to -1 to allow any difference and apply the strategy whatever the differences.
Maximum difference allowed in content between mails sharing the same hash, in bytes of the diff of their decoded bodies. Mails in an offending pair are set aside until the rest all pass. The subset is skipped if fewer than 2 remain. Set to 0 to enforce strictness and apply selection strategy on the subset only if all mails are exactly the same. Set to -1 to allow any difference and apply the strategy whatever the differences.
Show the unified diff of duplicates not within thresholds.
Action performed on the selected mails. Defaults to copy-selected as it is the safest: it only reads the mail sources and create a brand new mail box with the selection results. Mails that have no duplicate are part of the selection, but delete-selected leaves them in place, as deleting one removes the only copy of it there is.
Location of the destination mail box to where to copy or move deduplicated mails. Required in copy-selected, copy-discarded, move-selected and move-discarded actions.
Format of the mail box to which deduplication mails will be exported to. Only affects copy-selected, copy-discarded, move-selected and move-discarded actions.
If destination mail box already exists, add mails into it instead of interrupting (default behavior). Affect copy-selected, copy-discarded, move-selected and move-discarded actions.
Hardlink discarded mails whose content differs byte for byte from the copy they are linked to, instead of leaving them alone. Their own content is then swapped for that copy's, so whatever was unique to them, like the headers a mail collects on its way to one account, is lost. Only affects the hardlink-discarded action.
Do not perform any action but act as if it was, and report which action would have been performed otherwise.
Number of parallel jobs used to hash mails (step #2) and settle duplicate sets (step #3). Accepts an integer, 'auto' (the host's logical CPUs, minus one when there are three or more) or 'max'. Defaults to 1 (sequential). Only folder-based boxes fan out; higher values speed up --hash-body decoded/normalized and boxes holding many duplicate sets.
Show this message and exit.
Location of the configuration file. Supports local path with glob patterns or remote URL.
Validate the configuration file and exit.
Export the configuration in the selected format to <stdout>, then exit.
Accessibility mode: disable colors and render tables in a borderless, screen-reader-friendly format.
Colorize the output. A bare --color is the same as --color=always.
Disable colorization (alias of --color=never).
Show progress indicators during long operations. Disabled for non-interactive output (pipes, dumb terminals, CI) and by --accessible.
Color theme used for help screens.
Rendering style of tables.
Either CRITICAL, ERROR, WARNING, INFO, DEBUG.
Increase the default WARNING verbosity by one level for each additional repetition of the option.
Decrease the default WARNING verbosity by one level for each additional repetition of the option.
Shorthand for --verbosity DEBUG.
Measure and print elapsed execution time.
Show all CLI parameters, their provenance, defaults and value, then exit.
Show the tree of nested subcommands and exit.
Read the command's manual page and exit.
Render the command in the given format and exit.
Show the version and exit.

~/.config/mdedup/{*.toml,*.yaml,*.yml,*.json,*.jsonc,*.ini,*.xml,*.plist,*.sqlite,*.sqlite3,*.conf,pyproject.toml}

0
Success.
1
A runtime error, an aborted prompt (Ctrl-D), or a declined confirmation.
2
A usage error: unknown option, invalid value, missing operand, or an unparsable configuration file.
130
Interrupted with Ctrl-C: the process ends by SIGINT, reported as 130.
2026-11-07