Changelog¶
9.3.2.dev0 (unreleased)¶
Warning
This version is not released yet and is under active development.
9.3.1 (2026-08-28)¶
Attach a
mail-deduplicate-manpages.tar.gzasset to every GitHub release, rendered from the same command tree the documentation site builds its man pages from.Size the report’s
Descriptionand the hash table’sHeader valuecolumns to the terminal instead of a fixed width, so neither table overflows an 80-column terminal.Stop embedding line breaks in the report’s descriptions: the structured formats now carry each description as a single string, and
--table-format=verticalaligns wrapped lines under the label column.Stop
--action delete-selectedfrom deleting the mails that have no duplicate, which it has done since8.1.1. Closes #1053.Fix the
--input-formathelp, which still said auto-detection only coveredmaildirandmbox:emlfolders have been detected since9.1.0.Fix the installation guide’s
mdedup --versionexample, which rendered the documentation tool’s version instead of the CLI’s.Fix the API documentation of
normalize_address_header: a mismatched code span leaked backticks into the rendered text, and its issue references are now links.Expand the troubleshooting docs: how to read a strategy against an action, what the
0and-1thresholds do, and why genuine copies can be skipped for size.Surface the deferred work buried in code comments and prose on the documentation’s Todo-list page.
9.3.0 (2026-08-08)¶
Breaking: a dry run now says
DRY RUN: N mails would be acted upon, but none will be altered.once, in place of aDRY RUN: Skip action.line per mail. Scripts matching the old wording need updating.Breaking: Python importers must switch to
Strategy.function/Strategy.apply(),Action.perform()andDuplicateSet.select(), which replacestrategy_function/apply_strategy(),perform_action()andcategorize_candidates(). The CLI is unchanged.Add a
hardlink-discardedaction: each discarded mail becomes a hardlink to the copy kept in its set, reclaiming its space while staying in its own folder. Only byte-identical copies are linked, unless--hardlink-differingis passed. Closes #164.Add
--cacheand--cache-path, off by default, to reuse the hashes of previous runs: a second run over a 20,000-mail maildir takes half the time. Entries expire with their mail or when the hashing options change. Closes #87.Spread hashing and selection over worker processes instead of threads, so
--jobsfinally pays off: a first run over a 20,000-mail maildir drops from3.4sto1.8sat--jobs=4. Only folder-based boxes (maildir,MH,eml) fan out.Speed up hashing by about
2xand halve the file-system calls amaildirrun makes per mail, taking a whole run about2.6xfaster with the default--hash-body skip. Refs #87.Cut the memory retained per mail from about
950to600bytes. Refs #87.Settle the size and content thresholds over the whole duplicate set instead of comparing every pair: a single set of 200 copies took
1.90sand now takes0.04s.Cut what a run prints by seven eighths, from 32,000 lines to 4,000 on a 20,000-mail corpus: mails and duplicate sets are now named at
--verbosity DEBUGonly. The selection step gained a progress bar.Add
platformdirsto the runtime dependencies, which resolves the per-platform location of the hash cache.Fix the documentation’s broken links:
pipx’s installation guide moved, and thectimenote now cites Python’sos.path.getctimedocs. Closes #899.Add a performance page covering where a run spends its time, the memory it holds and how to manage the hash cache, plus tutorial and design sections on hardlinking. Correct the per-mail memory figures, which claimed a few hundred bytes.
9.2.0 (2026-08-07)¶
Keep memory usage flat whatever the size of the mail sources: a 215 MB maildir of 1,500 mails peaks at 48 MB of resident memory instead of 283 MB. Closes #761, and removes the memory pressure motivating #87.
Fix the threshold options’ help, which still claimed a breach skips the whole duplicate set instead of setting the offending mails aside.
Drop the non-existent content-based strategy from the README’s feature list.
9.1.0 (2026-08-01)¶
Breaking: pluralize the repeatable options’ parameter IDs: configuration keys and environment variables become
hash_headers/MDEDUP_HASH_HEADERSandstrategies/MDEDUP_STRATEGIES. The--hash-headerand--strategyflags are unchanged.Skip duplicate sets containing mails with an unparsable or absent
Dateheader instead of crashing on time-based strategies. The warning names the offending mails, and a newSkipped - Timestampmetric counts these sets. Closes #132 and #600.Identify mails from folder-based boxes in logs by the fully-qualified path of their own file, so it can be copy-pasted for direct inspection. Closes #157.
Set aside mails exceeding the size or content thresholds against the rest of their set instead of skipping the whole set, so an outlier no longer prevents the deduplication of the true copies sharing its hash. Closes #851.
Discover nested maildir folders stored as plain directories at any depth, as produced by
isync/mbsync’s Verbatim naming style, even when the root directory holds no mail itself. Closes #973.Add a new
emlsource and export format, reading and writing loose.emlfiles from a folder, walked recursively. Autodetected, and the unrecognized-folder error now points at--input-format. Closes #760.Allow repeating
--strategyto chain fallback strategies: each duplicate set is handed over to the next strategy when one fails to discriminate its mails. Closes #647.Report each copied, moved or deleted mail with a live
✓trail line and a timed summary on interactive terminals.Validate the
--regexpand--exportoption requirements at parse time withcloupconstraints, rewording their error messages.Reject configuration files carrying keys that match no CLI option, and refuse mail sources from configuration files: boxes to deduplicate are always passed on the command line.
Accept kebab-case keys in configuration files, matching the spelling of the CLI flags.
Ship the TOML writer so
--export-config tomlworks out of the box, and name every configurable option in generated templates, commented out when unset, under kebab-case keys.Stop requiring
--exportin-H/--hash-onlymode: constraints from the selection and action steps no longer apply there.Drop
boltonsandwheneverfrom the runtime dependencies (both remain for tests) and requireclick-extra8.7, whose--paramsand--export-confignow reflect the loaded configuration file.Build the prebuilt Linux executables inside a
manylinux_2_28container so they link against a glibc2.28floor instead of the CI runner’s newer glibc, letting them run on older enterprise distributions such as RHEL 8+, Debian 10, Ubuntu 20.04 and openSUSE Leap 15.3+. Closes #759.Fix a spurious metrics-inconsistency exit (code
115) for*-discardedactions and dry runs: the statistics self-check now compares each action counter to the subset of mails the action targets. Closes #841.Fix an unhandled traceback on invalid
--regexpvalues, now rejected with a proper parameter error.Fix the payload memory purge of selected mails, which never triggered for the
move-discardedaction.Add a hands-on tutorial page to the documentation and rework the README quickstart around use cases and example commands. Closes #984.
Execute the tutorial’s commands at documentation build time with
click-extra’sclick:rundirectives, so the rendered outputs always match the current implementation. Closes #23.Rewrite the configuration documentation: precedence, key spelling,
pyproject.tomldiscovery, template export, validation and environment variables.
9.0.0 (2026-07-27)¶
Breaking: upgrade to
click-extra8.1, which renames the--show-paramsoption to--params. The upgrade also drops the removedExtraCommandimport that broke8.1.2against recentclick-extra. Closes #1009.Breaking: remove the
-m/--minimal-headersoption. The per-mail floor is now derived automatically asmin(4, number of --hash-header values), so narrowing the hash below four headers, down to a single one, no longer needs a separate flag. Closes #974.Fix
AttributeErrorcrash in-H/--hash-onlymode when displaying each mail’s canonical headers. Closes #1004.Add
--jobsoption to parallelize mail hashing.Add
--themeoption to select the CLI color theme.Migrate repository tooling, CI and release automation to
repomaticreusable workflows. Replace Dependabot with Renovate.Adopt the PEP 440
.devNdevelopment versioning scheme.Switch to the
uv_buildbuild backend and declare the license as an SPDX expression.Move
test,typinganddocsextras to dependency groups.Replace the
arrowdate library withwhenever.Rename the development documentation page to contributing, convert the API docs to MyST Markdown and add a Claude Code project guide.
Render GitHub-style alerts in the documentation with the native
alertextension ofmyst-parser5.1.Run the test suite in parallel with
pytest-xdist, import test modules viaimportliband restrict collection totests/.Run the CLI test suite in every CI matrix cell and move package-install smoke tests to a dedicated CI job.
Drop the Codecov Test Analytics upload and the
junit.xmlartifact it required.
8.1.2 (2025-12-02)¶
Refactor some code to improve readability.
8.1.1 (2025-12-01)¶
8.1.0 (2025-11-27)¶
Add new
-m/--minimal-headersoption to set the minimal number of headers required to compute a hash. Closes #943.Validate number of
--hash-headeroptions against--minimal-headersvalue.Require at least one header to be provided to the
--hash-headeroption.Add installation instructions for
brewon macOS and Arch Linux.Use mixin classes to share common code between mailbox types and avoid dynamic inheritance.
8.0.0 (2025-11-21)¶
Log options explicitly set by user but ignored because of
--hash-onlymode.Add
--no-config,--table-formatoptions inherited from Click Extra.Table rendering in CLI output is now controlled by
--table-formatoption.Remove direct dependency on
tabulate, useclick-extra’s table utilities instead.Add official support for Python 3.14.
Remove maximum capped version of all dependencies (relax all
~=specifiers to>=). This gives more freedom to downstream and upstream packagers. Document each minimal version choice.Move all typing-related imports behind a hard-coded
TYPE_CHECKINGguard to avoid runtime imports.Produce
mdedup-windows-arm64.exeWindows binary forarm64architecture.Run tests on stable Python 3.14 release.
Run tests on Python 3.15-dev, mark them as unstable.
Skip tests on intermediate Python versions (
3.11,3.12and3.13) to reduce CI load.Run tests on Python
3.14tand3.15tfree-threaded variants.Use
astral-sh/setup-uvaction to installuvinstead of manually installing it withpip.Move
ubuntu-24.04toubuntu-24.04-arm,macos-15tests tomacos-26andwindows-2025tests towindows-11-arm.
7.6.2 (2025-04-20)¶
Ignore line endings when comparing content. Closes #844.
Render failed statistics assertions in plain English.
Exit with error code
115when statistics are inconsistent.Add a new
--verboseoption to increase the verbosity level.Reassign the short
-voption from--verbosityto--verbose.Only run unittests against the oldest and newest major supported version of Python. Remove tests on
3.11and3.12.To speed up the tests, we only test the latest available OS for each platform. Remove tests on
ubuntu-22.04,macos-13andwindows-2019.Upgrade tests from
windows-2022towindows-2025.
7.6.1 (2024-11-30)¶
Fix conflicting
-h/--hash-headerand-h/--helpoptions. Closes #762.
7.6.0 (2024-11-24)¶
Add official support for Python 3.13.
Drop support for Python 3.9. Refs #787.
Add dependency on
extra-platforms. Closes #784.Run tests on stable Python 3.13 release.
Run tests on Python 3.14-dev, mark them as unstable.
Move
macos-14tests tomacos-15.Add tests on
ubuntu-24.04. Remove tests onubuntu-20.04.Run workflows on
ubuntu-24.04.Rely on frozen
uv.lockfor reproducibility.
7.5.0 (2024-07-03)¶
Switch from Poetry to
uv.Drop support for Python 3.8.
Mark Python 3.13-dev tests as stable.
7.4.0 (2024-06-20)¶
Slim down package by moving unit tests out of the main package.
Split
devdependency groups into optionaltest,typinganddocsgroups.Remove direct dependency on
mypy.Run tests on Python
3.13-devon all platforms butmacos.Run tests on
macos-14. Drop tests onmacos-12.Build
arm64binaries onmacos-14.
7.3.0 (2023-11-15)¶
Drop support of Python 3.7.
Reduce memory usage. Addresses #362.
Replace unmaintained
bump2versionbybump-my-version.Test
mdedupbinaries.Run tests and actions on released Python 3.12 version.
Run tests on macos-13. Remove tests on macos-11.
7.2.3 (2023-05-04)¶
Reverts distribution of package via trusted channel.
7.2.2 (2023-05-04)¶
Note
7.2.2 is available on 🐙 GitHub.
Warning
7.2.2 is not available on 🐍 PyPI.
Redo release to fix trusted publisher on PyPI.
7.2.1 (2023-05-04)¶
Note
7.2.1 is available on 🐙 GitHub.
Warning
7.2.1 is not available on 🐍 PyPI.
Produce dependency graph in Mermaid instead of Graphviz. Add new dev dependency on
sphinxcontrib-mermaid.Move all documentation assets to
assetssubfolder.Distribute package on PyPI via a trusted publisher channel.
7.2.0 (2023-02-15)¶
Build standalone executable for macOS, Linux and Windows.
Generates dependency graph in Graphviz format.
Run tests on Python
3.12-dev.Code, comments and documentation style change to conform to new QA workflows based on
ruff.
7.1.0 (2022-12-07)¶
Renumber and rename phases to steps.
Group options per steps.
Add minimal code typing and checking.
Add logo.
Execute all workflows with Python 3.11.
7.0.0 (2022-11-26)¶
Drop Python 3.6 support.
Add support for Python 3.11.
Add new
--time/--no-timeoption to measure elapsed execution time.Add new
--color/--no-colorand--ansi/--no-ansialias options to deactivate CLI color rendering.Add new
--color/--no-colorand--ansi/--no-ansialias options to deactivate CLI color rendering.Add new
-C/--configoption which support local and remote configuration file in TOML, YAML, JSON, INI or XML formats.Add new
--show-paramsoption to debug default parameter value and provenance.Fix incconsistent printing of help screen. Closes #160.
Force linear rendering of options in help screen to improve readability.
Fix run on Python 3.10. Closes #361.
Run unittests on Python 3.10 and Python 3.11.
Add dependency on
click-extraandtyping-extensionsRemove direct dependency on
click-help-colors,click-logandtomlkit.Fix broken selection logic in quantity-based strategies. Refs #146.
Add unittests to cover time-based and size-based selection edge-cases.
Drop unittests on deprecated
ubuntu-18.04andmacos-10.15.Add unittests on
ubuntu-22.04,macos-12andwindows-2022.Run tests on multiple cores.
Simplify project management by abandoning the dual use of
main/developbranches.Migrate to external workflows to automate builds, releases, autofixes, linting, documentation, changelog, mailmap maintenance and label management.
Convert all documentation from RST to Markdown. Closes #368.
Keep CLI output in sync in documentation. Closes #23.
Move
sphinxdependencies to dev requirements.Add citation file.
6.2.0 (2021-09-12)¶
Upgrade to Click 8.x.
Implements all missing
copy-discarded,move-discardedanddelete-discardedactions. Closes #270 and #146.Add
-b/--hash-bodyoption to set the way each email body is hashed.Add
--export-appendoption to allow for the resulting deduplicated email to be appended to an existing mail box.Skip duplicate sets without any matching duplicate candidates. Closes #203.
6.1.3 (2021-04-14)¶
Fix dynamic
mailbox.Messageinheritance. Closes #191.
6.1.2 (2021-01-26)¶
Reconcile
v3branch withdevelop.
6.1.1 (2021-01-26)¶
Simple re-release.
6.1.0 (2021-01-26)¶
Add retroactive support for Python 3.6. Closes #154.
Fix documentation link and generation. Closes #66.
Auto-generate API documentation via a GitHub action workflow.
Add
tomlkitdependency.Add test runs against new OSes and distributions:
ubuntu-18.04andmacos-11.0.Remove
pycodestyle, it brings nothing more now that we rely onblack.
6.0.2 (2020-11-05)¶
Load up all subfolders from
MaildirandMHboxes. Closes #123.
6.0.1 (2020-10-22)¶
Check early that
--exportfile doesn’t exists. Closes #119.Add screenshots.
6.0.0 (2020-10-17)¶
Add new
-a/--actionoption to choose what to do on the final mail selection.Implements new
copy-discarded,copy-selected,delete-discarded,delete-selected,move-discardedandmove-selectedactions.Add new
-E/--exportand-e/--export-formatoptions to support the newcopy-*andmoved-*actions.Rename all
--delete-*strategies to--discard-*.Add
--select-*aliases to all strategies.Add new
discard-all-but-one,discard-one,select-oneandselect-all-but-oneselection strategies.Rename
-f/--sources-formatoption to-i/--input-format.--time-sourceparameter is now optional and defaults todate-header.Add metric description in deduplication end report.
Add detailed strategy description in help screen’s epilog.
Colorize help screen.
Colorize version screen and print environment data for bug reports.
Run tests on Python 3.9.
5.1.0 (2020-10-06)¶
Add new
-h/--hash-headeroption to select which mail headers to use to compute hash.Remove
-i/--message-idoption. Can be emulated with-h Message-IDor--hash-header Message-IDoption.Make all keyword-based choice parameters (
--sources-format,--strategyand--time-source) case-insensitive.
5.0.0 (2020-10-05)¶
Add supports for
Babyl,MHandMMDFmailbox types.Add new
-f/--sources-formatoption to force the type of mail sources on opening.Add new
-u/--force-unlockoption to force the removal of a lock on mailboxes on opening.Split-up the selection of mails candidate for removal and the deletion itself into two distinct batch operations.
Add issue templates to guide users to report bugs and request features.
Upgrade to
Poetry 1.1.0.Test publishing to PyPI in dry-run mode by the way of Poetry.
Auto-optimize images.
Auto-lock closed issues and PRs after a moment of inactivity.
4.0.0 (2020-10-02)¶
Removes the
hashsubcommand. Replaced it with a--hash-onlyparameter to the main dedupe command.Removes
deduplicatesubcommand.mdedupis now a simple CLI.Fix computation of stats. Closes #89.
Drop support for Python 2. Refs #90.
Bump minimal Python requirement to 3.7.
Repackage project around Poetry and
pyproject.toml. Get rid ofsetup.py.Replace unmaintained
bumpversionbybump2version.Drop dependency on
progressbar2and replace it withclick’s. Closes #37.Switch from Travis to GitHub Actions for all CI/CD jobs.
Regularly run unittests against Windows, Linux and macOS.
Removes all copyright dates.
Auto-fix Python format, typos and JSON content.
Let dependabot check for dependencies and update them.
Keep
.gitignoreand.mailmapup to date and in sync.Auto-publish package on tagging events via
twine.Always run test build and check resulting package on each commits.
Use declarative JSON to maintain list of GitHub labels for issues and PRs.
Lint both Python code and YAML content on each commit.
Run unittests in random to order to detect coupling.
3.0.1 (2021-01-25)¶
Note
3.0.1 is available on 🐍 PyPI.
Warning
3.0.1 is not available on 🐙 GitHub.
Add explicit warning in CLI output to warn about 3.x branch deprecation. Refs #180.
3.0.0 (2020-09-03)¶
Caution
As of v3.0.0, the project has been renamed to Mail Deduplicate and is
now available on:
GitHub at https://github.com/kdeldycke/mail-deduplicate
PyPI at https://pypi.org/project/mail-deduplicate
Rename project from
maildir-deduplicatetomail-deduplicate.Rename
masterbranch tomain.Add support for mboxes. Closes #48.
Remove requirement on
-s/--strategyparameter, to let mails being grouped into duplicate sets without any removal action, effectively acting as a second-level dry-run.
2.2.0 (2020-09-03)¶
Warning
This is the last version of the project known under the name Maildir Deduplicate
(a.k.a. maildir-deduplicate).
Note
2.2.0 is the first version available on 🐙 GitHub.
Add disclaimer to prepare project name change.
Fix Header being object instead of string (#61).
Make body_lines conversion more reliable.
Add Message-ID as a header to check.
Bump requirement to
click_log >= 0.2.0. Closes #58, #59 and #60.Replace
nosebypytest.Only notify by mail of test failures.
Drop support of Python 3.3.
2.1.0 (2017-01-13)¶
Fix rendering of changelog link in RST.
Show selected log level in debug mode.
Test builds against Python 3.6 and 3.7-dev, and most recent PyPy targeting Python 2.7 and 3.3.
Bump requirement to
readme_renderer >= 16.0.Skip sets with unparsable mails because of incorrect encoding. Closes #47, #41 and #39.
Require the
-s/--strategyCLI parameter to be set. Closes #44.
2.0.1 (2016-11-28)¶
Reject mails without
Message-IDheaders if--message-idoption is active.Add a class to holds global config.
Print more debug info in unittests when CLI produce tracebacks.
Always check for package metadata in Travis CI jobs.
Fix package’s long description against PyPI rendering constraints.
Add link to full changelog in package’s long description.
2.0.0 (2016-11-13)¶
Refactor and redefine all removal strategies. Closes #40.
Add a new
-t/--time-sourceCLI parameter to point to the canonical source of a mail’s timestamp.Rename
-s/--show-diffsparameter to-d/--show-diff.Rename
-D/--diff-thresholdparameter to-C/--content-threshold.Add new
-sshorthand to--strategyparameter.Re-implement diff threshold options. Closes #43.
Re-implement statistics. Closes #42.
Normalize and canonicalize all paths to deduplicate on the fly mails pointing to the same file.
Build documentation via Sphinx.
Add
testanddevelopdependencies.Move coverage config to
setup.cfg.Replace
pep8package bypycodestyle.Enforce
pycodestylechecks in Travis CI jobs.Remove popularity badge: PyPI download counters are broken and no longer displayed.
Test production of packages in Travis CI jobs.
Move development and packaging documentation to Sphinx. Closes #22.
Make wheels generated under Python 2 environnment available for Python 3 too.
Let unittests generates their own fake and temporary maildirs.
Print CLI output in unittests.
Use generic factory to produce mail fixtures in unittests.
Only show latest changes in the long description of the package instead of the full changelog.
1.3.0 (2016-08-11)¶
1.2.0 (2016-03-29)¶
1.1.0 (2016-01-10)¶
Add Python 3.4 and 3.5 support. Closes #30.
Add default
isortconfiguration.Remove hackish default encoding forcing in main code.
1.0.2 (2015-12-22)¶
Fix
AttributeErroron message instance. Closes #28.Various fixes of references to internal variables introduced by a switch to more Pythonic classes in 1.0.0.
Remove
Message-IDfrom the list of default canonical headers.
1.0.1 (2015-11-23)¶
Switch from coveralls to codecov.
1.0.0 (2015-10-03)¶
0.1.0 (2014-12-15)¶
Package the script for proper distribution.
0.0.1 (2011-04-27)¶
First external contribution.
0.0.0 (2010-06-08)¶
First commit.