1

Topic: Deleting duplicate messages by Message ID

==== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.7
- Linux/BSD distribution name and version: Ubuntu 14.04 LTS
- Store mail accounts in which backend: Mysql
- Web server: nginx
- Manage mail accounts with iRedAdmin-Pro? No

We have our phone system setup to forward all voicemails as wav file attachment’s to 6 or so mailboxes, Once our employees return the voicemails they are really of no use to us and just occupy space on our server.

Does anyone know of a quick way to purge duplicate messages other than connecting to each mailbox via a mail client and deleting them manually or relying on the end users to do it? Since the messages all share the same ESMTP id is there a command that can purge all these duplicate messages by referencing the ESMTP id?

Thanks,
Tim

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: Deleting duplicate messages by Message ID

Dovecot-2.x has this feature:
https://wiki2.dovecot.org/Tools/Doveadm/Deduplicate

NOTE: check its '-m' option please.

3

Re: Deleting duplicate messages by Message ID

ZhangHuangbin wrote:

Dovecot-2.x has this feature:
https://wiki2.dovecot.org/Tools/Doveadm/Deduplicate

NOTE: check its '-m' option please.

Thanks for the reply! I’m having a hard time wrapping my head around the doveadm syntax, could you provide an example.

Here is a header from one such email

Return-Path: <noreply@voipinterface.net>
Delivered-To: tim@domain.com
Received: from mx01.domain.com (localhost [127.0.0.1])
        by mx01.domain.com (Postfix) with ESMTP id 363191251D7
        for <tim@domain.com>; Thu,  1 Feb 2018 14:04:34 -0500 (EST)
X-Virus-Scanned: Debian amavisd-new at mx01.domain.com
X-Spam-Flag: NO
X-Spam-Score: -0.1
X-Spam-Level:
X-Spam-Status: No, score=-0.1 tagged_above=-999 required=4
        tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1]
        autolearn=unavailable autolearn_force=no
Authentication-Results: mx01.domain.com (amavisd-new); dkim=pass (1024-bit key)
        header.d=voipinterface.net
Received: from mx01.domain.com ([127.0.0.1])
        by mx01.domain.com (mx01.domain.com [127.0.0.1]) (amavisd-new, port 10024)
        with ESMTP id S0ZdVHVBOxlj for <tim@domain.com>;
        Thu,  1 Feb 2018 14:04:33 -0500 (EST)
Received: by mx01.domain.com (Postfix, from userid 2000)
        id 63B9A12520D; Thu,  1 Feb 2018 14:04:33 -0500 (EST)
X-Sieve: Pigeonhole Sieve 0.4.2
X-Sieve-Redirected-From: ypcrvoicemail@domain.com
Delivered-To: ypcrvoicemail@domain.com
Received: from mx01.domain.com (localhost [127.0.0.1])
        by mx01.domain.com (Postfix) with ESMTP id 3ACE91251E3
        for <ypcrvoicemail@domain.com>; Thu,  1 Feb 2018 14:04:33 -0500 (EST)
X-Virus-Scanned: Debian amavisd-new at mx01.domain.com
Received: from mx01.domain.com ([127.0.0.1])
        by mx01.domain.com (mx01.domain.com [127.0.0.1]) (amavisd-new, port 10024)
        with ESMTP id N2bQvGpjHB39 for <ypcrvoicemail@domain.com>;
        Thu,  1 Feb 2018 14:04:28 -0500 (EST)
Received: from mail-sf.voip.ms (mail-sf.voip.ms [67.202.71.62])
        by mx01.domain.com (Postfix) with ESMTPS id 6FDF21251DA
        for <ypcrvoicemail@domain.com>; Thu,  1 Feb 2018 14:04:28 -0500 (EST)
Received: from voicemail1.voip.ms (ip154.208-100-22.static.steadfastdns.net [208.100.22.154])
        by mail-sf.voip.ms (Postfix) with ESMTPS id A381530801E8
        for <ypcrvoicemail@domain.com>; Thu,  1 Feb 2018 14:04:27 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=voipinterface.net;
        s=default; t=1517511867;
        bh=pljnYJnI+JWe2/sJ5Tb/7fXKiJP7PTLtQkk0C4qW79g=;
        h=Date:From:To:Subject;
        b=GGnpn0rCcz+lbvz0hkx9oW2NqpMCxvbXer3Dah05oHHjUaVEWiKk9SgjY+6xQT6+J
         Ur1H9yJacdGr91EA93TuoEyMJGk+ciw+CJM30RMqb2WO/QL6FtrWItzCk6zwlHnqCN
         CD7nnKEiwSj/9E2Hkeh6M0ozNR90TQIRAiGjXXJI=
Received: from voicemail1.voip.ms (localhost [127.0.0.1])
        by voicemail1.voip.ms (8.14.7/8.14.7) with ESMTP id w11J4Rlj023486
        for <ypcrvoicemail@domain.com>; Thu, 1 Feb 2018 14:04:27 -0500
Received: (from root@localhost)
        by voicemail1.voip.ms (8.14.7/8.14.7/Submit) id w11J4RnN023476;
        Thu, 1 Feb 2018 14:04:27 -0500
Date: Thu, 01 Feb 2018 14:04:27 -0500
From: "Voicemail System" <noreply@voipinterface.net>
To: "ypcr voicemail" <ypcrvoicemail@domain.com>
Subject: =?ISO-8859-1?Q?New__voicemail_in_mailbox_88842702_from_=22YPCR?=
 =?ISO-8859-1?Q?_Braswell_Eddie=22_=3C4109989222=3E?=
Message-ID: <Asterisk-1-1779153674-88842702-13205@voicemail1.voip.ms>
X-Asterisk-CallerID: 4109989222
X-Asterisk-CallerIDName: YPCR Braswell Eddie
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----voicemail_188842702132051636022834"


This is a multi-part message in MIME format.

I also just notice that each user has different ESMTP id when checking the header of these message(s), I take it this is not the same as GUIDs ? I’m not  even sure why I need to define a single email message when all I want to do is purge duplicate messages across multiple mailboxes?

Could you help me fill in the blanks here and advise how I find the GUIDs from the header?

doveadm deduplicate [-u <user>|-A] [-S <socket_path>] [-m] <search query>

Some clarification would be greatly appreciated 
thanks!

4

Re: Deleting duplicate messages by Message ID

WARNING: Please try it with a testing account, a testing message, or a testing server first.

doveadm deduplicate -u user@domain.com -m ALL