1 (edited by judah.teng 2015-07-22 13:38:13)

Topic: Globlal whitelist not working for amavis bad header

==== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.2
- Linux/BSD distribution name and version: Debian 8 Jessie
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): Apache
- Manage mail accounts with iRedAdmin-Pro?: Yes
- Related log if you're reporting an issue:
====

Hi there, is there a way to use the global whitelist to whitelist mails with bad header? It is not working. I use the iRedAdmin-Pro panel to whitelist domains but mails with bad header from whitelisted domains still get quarantined. I have since disabled bad header checks. Please help.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: Globlal whitelist not working for amavis bad header

Could you please show us mail headers of this bad-header message?

3

Re: Globlal whitelist not working for amavis bad header

There were many different kinds, I did not keep a copy. The issue is mails with bad-headers from whitelisted domains are still being quarantined. I want all mail from whitelisted domains to go through, even those with bad-headers. Is this possible? Is there a configuration for it?

4

Re: Globlal whitelist not working for amavis bad header

We need related log or mail headers for troubleshooting, cannot help without related log/info. so if you're lost, please try to follow our suggestion to get related log/info.

Here's what you should do next:

*) Show us output of commands:

# postconf smtpd_recipient_restrictions
# grep 'plugins' /opt/iredapd/settings.py

*) Turn on debug mode in iRedAPD and try to send an email from whitelisted domain (you can whitelist your Gmail account and send from this Gmail account for testing), paste related log in /var/log/iredapd.log. We need to know whether whitelist/blacklist is working as expected. Reference: http://www.iredmail.org/docs/debug.iredapd.html

*) Wait for new bad-header emails, if it's quarantined again, please go to iRedAdmin-Pro and check the mail headers of this quarantined mail. Paste all its mail headers here so that we can help troubleshoot.

5

Re: Globlal whitelist not working for amavis bad header

I'll give you what I can for now, hang on for the rest

$ sudo postconf smtpd_recipient_restrictions
smtpd_recipient_restrictions = reject_unknown_recipient_domain, reject_non_fqdn_recipient, reject_unlisted_recipient, check_policy_service inet:127.0.0.1:7777, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination

$ sudo grep 'plugins' /opt/iredapd/settings.py
# Enabled plugins.
# - Plugin name is file name which placed under 'plugins/' directory,
# - Plugins are applied in specified order. It's better to list plugins which
#   white/blacklist related plugins, e.g. amavisd_wblist.
# - Suggested order of plugins (if you enable them) for RCPT state:
plugins = ["reject_null_sender", "amavisd_message_size_limit", "amavisd_wblist", "sql_alias_access_policy"]

6

Re: Globlal whitelist not working for amavis bad header

2015-07-24 04:32:04 DEBUG smtp session: request=smtpd_access_policy
2015-07-24 04:32:04 DEBUG smtp session: protocol_state=RCPT
2015-07-24 04:32:04 DEBUG smtp session: protocol_name=ESMTP
2015-07-24 04:32:04 DEBUG smtp session: client_address=10.0.1.240
2015-07-24 04:32:04 DEBUG smtp session: client_name=ip-10-0-1-240.ap-southeast-1.compute.internal
2015-07-24 04:32:04 DEBUG smtp session: reverse_client_name=ip-10-0-1-240.ap-southeast-1.compute.internal
2015-07-24 04:32:04 DEBUG smtp session: helo_name=mr22p34im-asmtp003.me.com
2015-07-24 04:32:04 DEBUG smtp session: sender=aaa@icloud.com
2015-07-24 04:32:04 DEBUG smtp session: recipient=bbb@bbb.com.sg
2015-07-24 04:32:04 DEBUG smtp session: recipient_count=0
2015-07-24 04:32:04 DEBUG smtp session: queue_id=
2015-07-24 04:32:04 DEBUG smtp session: instance=37ee.55b1bfc4.3c9cc.0
2015-07-24 04:32:04 DEBUG smtp session: size=1024
2015-07-24 04:32:04 DEBUG smtp session: etrn_domain=
2015-07-24 04:32:04 DEBUG smtp session: stress=
2015-07-24 04:32:04 DEBUG smtp session: sasl_method=
2015-07-24 04:32:04 DEBUG smtp session: sasl_username=
2015-07-24 04:32:04 DEBUG smtp session: sasl_sender=
2015-07-24 04:32:04 DEBUG smtp session: ccert_subject=
2015-07-24 04:32:04 DEBUG smtp session: ccert_issuer=
2015-07-24 04:32:04 DEBUG smtp session: ccert_fingerprint=
2015-07-24 04:32:04 DEBUG smtp session: ccert_pubkey_fingerprint=
2015-07-24 04:32:04 DEBUG smtp session: encryption_protocol=TLSv1.2
2015-07-24 04:32:04 DEBUG smtp session: encryption_cipher=DHE-RSA-AES128-GCM-SHA256
2015-07-24 04:32:04 DEBUG smtp session: encryption_keysize=128
2015-07-24 04:32:04 DEBUG --> Apply plugin: reject_null_sender
2015-07-24 04:32:04 DEBUG <-- Result: DUNNO
2015-07-24 04:32:04 DEBUG Skip plugin: amavisd_message_size_limit (protocol_state != RCPT)
2015-07-24 04:32:04 DEBUG --> Apply plugin: amavisd_wblist
2015-07-24 04:32:04 DEBUG Possible policy senders: ['@.', 'aaa@icloud.com', '@icloud.com', '@.icloud.com', '@com', '@.com', '10.0.1.240', '10.0.*.240', '10.*.*.*', '*.*.1.240', '10.*.*.240', '10.0.1.*', '*.*.*.240', '*.0.1.240', '10.*.1.240', '10.0.*.*', '*.*.*.*']
2015-07-24 04:32:04 DEBUG Possible policy recipients: ['@.', 'bbb@bbb.com.sg', '@bbb.com.sg', '@.bbb.com.sg', '@com.sg', '@.com.sg', '@sg', '@.sg', 'bbb@*']
2015-07-24 04:32:04 DEBUG SQL: Get policy senders: SELECT id,email FROM mailaddr WHERE email IN ('@.', 'aaa@icloud.com', '@icloud.com', '@.icloud.com', '@com', '@.com', '10.0.1.240', '10.0.*.240', '10.*.*.*', '*.*.1.240', '10.*.*.240', '10.0.1.*', '*.*.*.240', '*.0.1.240', '10.*.1.240', '10.0.*.*', '*.*.*.*') ORDER BY priority DESC
2015-07-24 04:32:04 DEBUG Senders (in sql table: amavisd.mailaddr): [(176L, '@icloud.com')]
2015-07-24 04:32:04 DEBUG SQL: Get policy recipients: SELECT id,email FROM users WHERE email IN ('@.', 'bbb@bbb.com.sg', '@bbb.com.sg', '@.bbb.com.sg', '@com.sg', '@.com.sg', '@sg', '@.sg', 'bbb@*') ORDER BY priority DESC
2015-07-24 04:32:04 DEBUG Recipients (in `amavisd.users`): [(1L, '@.')]
2015-07-24 04:32:04 DEBUG SQL: Get wblist: SELECT rid,sid,wb FROM wblist WHERE sid IN (176) AND rid IN (1)
2015-07-24 04:32:04 DEBUG Found per-recipient white/blacklists: [(1L, 176L, 'W')]
2015-07-24 04:32:04 DEBUG <-- Result: OK wblist=(1, 176, 'W')
2015-07-24 04:32:04 INFO [10.0.1.240] RCPT, aaa@icloud.com -> bbb@bbb.com.sg, OK wblist=(1, 176, 'W')
2015-07-24 04:32:04 DEBUG Session ended

7

Re: Globlal whitelist not working for amavis bad header

iredapd log of quarantined mail:

2015-07-24 06:36:58 DEBUG smtp session: request=smtpd_access_policy
2015-07-24 06:36:58 DEBUG smtp session: protocol_state=RCPT
2015-07-24 06:36:58 DEBUG smtp session: protocol_name=ESMTP
2015-07-24 06:36:58 DEBUG smtp session: client_address=10.0.0.169
2015-07-24 06:36:58 DEBUG smtp session: client_name=ip-10-0-0-169.ap-southeast-1.compute.internal
2015-07-24 06:36:58 DEBUG smtp session: reverse_client_name=ip-10-0-0-169.ap-southeast-1.compute.internal
2015-07-24 06:36:58 DEBUG smtp session: helo_name=cluster-k.mailcontrol.com
2015-07-24 06:36:58 DEBUG smtp session: sender=aaa@aaa.com
2015-07-24 06:36:58 DEBUG smtp session: recipient=bbb@bbb.com.sg
2015-07-24 06:36:58 DEBUG smtp session: recipient_count=0
2015-07-24 06:36:58 DEBUG smtp session: queue_id=
2015-07-24 06:36:58 DEBUG smtp session: instance=490.55b1dd0a.a189d.0
2015-07-24 06:36:58 DEBUG smtp session: size=27152
2015-07-24 06:36:58 DEBUG smtp session: etrn_domain=
2015-07-24 06:36:58 DEBUG smtp session: stress=
2015-07-24 06:36:58 DEBUG smtp session: sasl_method=
2015-07-24 06:36:58 DEBUG smtp session: sasl_username=
2015-07-24 06:36:58 DEBUG smtp session: sasl_sender=
2015-07-24 06:36:58 DEBUG smtp session: ccert_subject=
2015-07-24 06:36:58 DEBUG smtp session: ccert_issuer=
2015-07-24 06:36:58 DEBUG smtp session: ccert_fingerprint=
2015-07-24 06:36:58 DEBUG smtp session: ccert_pubkey_fingerprint=
2015-07-24 06:36:58 DEBUG smtp session: encryption_protocol=TLSv1.2
2015-07-24 06:36:58 DEBUG smtp session: encryption_cipher=ECDHE-RSA-AES256-GCM-SHA384
2015-07-24 06:36:58 DEBUG smtp session: encryption_keysize=256
2015-07-24 06:36:58 DEBUG --> Apply plugin: reject_null_sender
2015-07-24 06:36:58 DEBUG <-- Result: DUNNO
2015-07-24 06:36:58 DEBUG Skip plugin: amavisd_message_size_limit (protocol_state != RCPT)
2015-07-24 06:36:58 DEBUG --> Apply plugin: amavisd_wblist
2015-07-24 06:36:58 DEBUG Possible policy senders: ['@.', 'aaa@aaa.com', '@aaa.com', '@.aaa.com', '@com', '@.com', '10.0.0.169', '10.*.0.169$
2015-07-24 06:36:58 DEBUG Possible policy recipients: ['@.', 'bbb@bbb.com.sg', '@bbb.com.sg', '@.bbb.com.sg', '@com.sg', '@.com.sg', '@sg', '@.sg', 'bbb@*']
2015-07-24 06:36:58 DEBUG SQL: Get policy senders: SELECT id,email FROM mailaddr WHERE email IN ('@.', 'aaa@aaa.com', '@aaa.com', '@.aaa.com$
2015-07-24 06:36:58 DEBUG Senders (in sql table: amavisd.mailaddr): [(210L, '@aaa.com')]
2015-07-24 06:36:58 DEBUG SQL: Get policy recipients: SELECT id,email FROM users WHERE email IN ('@.', 'bbb@bbb.com.sg', '@bbb.com.sg', '@.bbb.com.sg', '@com.sg', '@.$
2015-07-24 06:36:58 DEBUG Recipients (in `amavisd.users`): [(1L, '@.')]
2015-07-24 06:36:58 DEBUG SQL: Get wblist: SELECT rid,sid,wb FROM wblist WHERE sid IN (210) AND rid IN (1)
2015-07-24 06:36:58 DEBUG Found per-recipient white/blacklists: [(1L, 210L, 'W')]
2015-07-24 06:36:58 DEBUG <-- Result: OK wblist=(1, 210, 'W')
2015-07-24 06:36:58 INFO [10.0.0.169] RCPT, aaa@aaa.com -> bbb@bbb.com.sg, OK wblist=(1, 210, 'W')
2015-07-24 06:36:58 DEBUG Session ended

Mail Headers
Content-Type    multipart/mixed; boundary="----=_NextPart_000_1437719817"
Date    24 JUL 15 06:36:57 UT
From    aaa@aaa.com
MIME-Version    1.0
Message-Id    <201507240636.t6O6avNf011361@wm.aaa.com>
Received    from 172.10.138.71 ([10.138.20.52]) by wm.aaa.com (8.12.11/8.12.11) with SMTP id t6O6avNf011361; Fri, 24 Jul 2015 14:36:57 +0800
Received    from cluster-k.mailcontrol.com (ip-10-0-0-169.ap-southeast-1.compute.internal [10.0.0.169]) by mx.bbb.com.sg (Postfix) with ESMTPS id B7FD760752 for <bbb@bbb.com.sg>; Fri, 24 Jul 2015 06:36:58 +0000 (UTC)
Received    from mx.bbb.com.sg ([127.0.0.1]) by mx.bbb.com.sg (mx.bbb.com.sg [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zqe2cwOXqqZm for <bbb@bbb.com.sg>; Fri, 24 Jul 2015 06:36:58 +0000 (UTC)
Received    from wm.aaa.com ([103.228.100.71]) by rly03k.srv.mailcontrol.com (MailControl) with ESMTP id t6O6av3Q016385 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 24 Jul 2015 07:36:57 +0100
Subject    CENSORED
To    bbb@bbb.com.sg
To    ccc@ccc.sg
X-Amavis-Alert    BAD HEADER SECTION, Duplicate header field: "To"
X-Envelope-To    <bbb@bbb.com.sg>
X-Envelope-To-Blocked    <bbb@bbb.com.sg>
X-Quarantine-ID    <zqe2cwOXqqZm>
X-Scanned-By    MailControl 44278.224 (www.mailcontrol .com) on 10.75.0.113
X-Spam-Flag    NO
X-Spam-Level   
X-Spam-Score    0
X-Spam-Status    No, score=x tag=x tag2=x kill=x tests=[] autolearn=unavailable

8

Re: Globlal whitelist not working for amavis bad header

judah.teng wrote:

To    bbb@bbb.com.sg
To    ccc@ccc.sg
X-Amavis-Alert    BAD HEADER SECTION, Duplicate header field: "To"

Duplicate 'To:' headers.

Temporary solution #1: Don't quarantine bad-header email in Amavisd config file. For example:

$final_bad_header_destiny = D_PASS;
$bad_header_quarantine_method = undef;

Do you have '@lookup_sql_dsn' setting enabled in Amavisd config file? Please show me command output:

# grep '@lookup_sql_dsn' /etc/amavis/conf.d/50-user

9 (edited by judah.teng 2015-07-27 10:31:16)

Re: Globlal whitelist not working for amavis bad header

$ grep '@lookup_sql_dsn' /etc/amavis/conf.d/50-user
@lookup_sql_dsn = @storage_sql_dsn;

Yes, I understand why the header is bad. My question is why does the whitelist not allow this email to go through? Yes, I have disabled bad header quarantining because whitelisting is not working for bad headers.

10

Re: Globlal whitelist not working for amavis bad header

I suggest you asking for support in Amavisd mailing list, it depends on how Amavisd uses this whitelist record (just show them you're using the valid whitelist value '@aaa.com' -- as shown in iRedAPD log).

11

Re: Globlal whitelist not working for amavis bad header

Noted, thanks.