1 Topic by bigman

  • bigman
  • Member
  • Offline
  • Registered: 2015-09-14
  • Posts: 41

Topic: Whitelist / Blacklist Priority Problem

==== Required information ====
- iRedMail version (check /etc/iredmail-release):  0.9.3
- Linux/BSD distribution name and version: Ubuntu 14.04
- Store mail accounts in which backend (LDAP/MySQL/PGSQL):  MySQL
- Web server (Apache or Nginx): Apache
- Manage mail accounts with iRedAdmin-Pro? Yes
- Related log if you're reporting an issue:
====

I try to setup whitelist / blacklist per domain and per user as below.
For Global / Domain , I set black list recipient : for example @outlook.com, so all user should be block to send email to outlook.com
But For special user, I set whitelist recipient : @. (I want this user can send email to all, including outlook.com)

The result, this special user is still blocked to send to outlook.com

But if I set the whitelist recipient  : @outlook.com for this user, then he can send email to outlook.com

in the Whitelist / Blacklist menu there are some statement
- Whitelist has higher priority than blacklist.
- Per-user whitelists & blacklists has highest priority, then per-domain setting, then global setting.
So if I give @. in whitelist per user, this user should not be block right ? but it is blocked. How to get this work ?

Actually for final result I want to block all recipient domain except the email list in the whitelist for all users, but for special users I will give @. in the whitelist so that he can send to all.

But for trial, I set like above, but it is fail.

Please help. thanks.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Whitelist / Blacklist Priority Problem

Could you please turn on debug mode in iRedAPD, restart iredapd service, send testing email to reproduce this issue, extract FULL log related to this testing email, and paste the log here. So that others can help troubleshoot.

Reference: http://www.iredmail.org/docs/debug.iredapd.html

3 Reply by bigman

  • bigman
  • Member
  • Offline
  • Registered: 2015-09-14
  • Posts: 41

Re: Whitelist / Blacklist Priority Problem

Hi Zhang thanks for the reply
Here is the iredapd log after I change to debug mode, restart iredapd and send trial email.

2016-01-08 08:57:12 DEBUG Connect from 127.0.0.1, port 59946.
2016-01-08 08:57:12 DEBUG smtp session: request=smtpd_access_policy
2016-01-08 08:57:12 DEBUG smtp session: protocol_state=RCPT
2016-01-08 08:57:12 DEBUG smtp session: protocol_name=ESMTP
2016-01-08 08:57:12 DEBUG smtp session: client_address=127.0.0.1
2016-01-08 08:57:12 DEBUG smtp session: client_name=mail.mydomain.com
2016-01-08 08:57:12 DEBUG smtp session: reverse_client_name=mail.mydomain.com
2016-01-08 08:57:12 DEBUG smtp session: helo_name=10.0.1.8
2016-01-08 08:57:12 DEBUG smtp session: sender=test@mydomain.com
2016-01-08 08:57:12 DEBUG smtp session: recipient=testing@outlook.com
2016-01-08 08:57:12 DEBUG smtp session: recipient_count=0
2016-01-08 08:57:12 DEBUG smtp session: queue_id=
2016-01-08 08:57:12 DEBUG smtp session: instance=384e.568f1778.46b1d.0
2016-01-08 08:57:12 DEBUG smtp session: size=0
2016-01-08 08:57:12 DEBUG smtp session: etrn_domain=
2016-01-08 08:57:12 DEBUG smtp session: stress=
2016-01-08 08:57:12 DEBUG smtp session: sasl_method=LOGIN
2016-01-08 08:57:12 DEBUG smtp session: sasl_username=test@mydomain.com
2016-01-08 08:57:12 DEBUG smtp session: sasl_sender=
2016-01-08 08:57:12 DEBUG smtp session: ccert_subject=
2016-01-08 08:57:12 DEBUG smtp session: ccert_issuer=
2016-01-08 08:57:12 DEBUG smtp session: ccert_fingerprint=
2016-01-08 08:57:12 DEBUG smtp session: ccert_pubkey_fingerprint=
2016-01-08 08:57:12 DEBUG smtp session: encryption_protocol=TLSv1
2016-01-08 08:57:12 DEBUG smtp session: encryption_cipher=ECDHE-RSA-AES256-SHA
2016-01-08 08:57:12 DEBUG smtp session: encryption_keysize=256
2016-01-08 08:57:12 DEBUG --> Apply plugin: reject_null_sender
2016-01-08 08:57:12 DEBUG Local sender.
2016-01-08 08:57:12 DEBUG <-- Result: DUNNO
2016-01-08 08:57:12 DEBUG --> Apply plugin: greylisting
2016-01-08 08:57:12 DEBUG Found SASL username, bypass greylisting for outbound email.
2016-01-08 08:57:12 DEBUG <-- Result: DUNNO
2016-01-08 08:57:12 DEBUG --> Apply plugin: amavisd_wblist
2016-01-08 08:57:12 DEBUG Possible policy senders: ['@.', 'test@mydomain.com', '@mydomain.com', '@.mydomain.com', '@com', '@.com']
2016-01-08 08:57:12 DEBUG Possible policy recipients: ['@.', 'testing@outlook.com', '@outlook.com', '@.outlook.com', '@com', '@.com']
2016-01-08 08:57:12 DEBUG Apply wblist for outbound message.
2016-01-08 08:57:12 DEBUG [SQL] Query local addresses:
SELECT id, email
               FROM users
              WHERE email IN ('@.', 'test@mydomain.com', '@mydomain.com', '@.mydomain.com', '@com', '@.com')
           ORDER BY priority DESC
2016-01-08 08:57:12 DEBUG Local addresses (in `users`): [(201L, 'test@mydomain.com'), (2L, '@mydomain.com'), (1L, '@.')]
2016-01-08 08:57:12 DEBUG [SQL] Query external addresses:
SELECT id, email
               FROM mailaddr
              WHERE email IN ('@.', 'testing@outlook.com', '@outlook.com', '@.outlook.com', '@com', '@.com')
           ORDER BY priority DESC
2016-01-08 08:57:12 DEBUG Addresses (in `mailaddr`): [(2143L, '@outlook.com'), (1651L, '@.')]
2016-01-08 08:57:12 DEBUG [SQL] Query outbound wblist:
SELECT rid, sid, wb
               FROM outbound_wblist
              WHERE sid IN (201, 2, 1) AND rid IN (2143, 1651)
2016-01-08 08:57:12 DEBUG Found wblist: [(1651L, 201L, 'W'), (2143L, 2L, 'B')]
2016-01-08 08:57:12 INFO Blacklisted: wblist=(2143, 2, 'B')
2016-01-08 08:57:12 DEBUG <-- Result: REJECT Blacklisted
2016-01-08 08:57:12 DEBUG INSERT INTO log (admin, ip, msg, timestamp, event)
                          VALUES ('iredapd', '127.0.0.1', 'REJECT Blacklisted (test@mydomain.com -> testing@outlook.com, amavisd_wblist)', NOW(), 'iredapd')
       
2016-01-08 08:57:12 INFO [127.0.0.1] RCPT, => test@mydomain.com -> testing@outlook.com, REJECT Blacklisted
2016-01-08 08:57:12 DEBUG Session ended

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Whitelist / Blacklist Priority Problem

Bug in iRedAPD-1.7.0. Please try attached patch:

diff -r 7a5f0c8fa004 plugins/amavisd_wblist.py
--- a/plugins/amavisd_wblist.py    Mon Jan 04 18:02:47 2016 +0800
+++ b/plugins/amavisd_wblist.py    Fri Jan 08 12:07:47 2016 +0800
@@ -120,7 +120,7 @@
         logger.debug('No wblist found.')
         return SMTP_ACTIONS['default']
 
-    logger.debug('Found wblist: %s' % str(wblists))
+    logger.debug('Found inbound wblist: %s' % str(wblists))
 
     # Check sender addresses
     # rids/recipients are orded by priority
@@ -128,10 +128,10 @@
         # sids/senders are sorted by priority
         for sid in sender_ids:
             if (rid, sid, 'W') in wblists:
-                return SMTP_ACTIONS['accept'] + " wblist=(%d, %d, 'W')" % (rid, sid)
+                return SMTP_ACTIONS['accept'] + " inbound_wblist=(%d, %d, 'W')" % (rid, sid)
 
             if (rid, sid, 'B') in wblists:
-                logger.info("Blacklisted: wblist=(%d, %d, 'B')" % (rid, sid))
+                logger.info("Blacklisted: inbound_wblist=(%d, %d, 'B')" % (rid, sid))
                 return SMTP_ACTIONS['reject_blacklisted']
 
     return SMTP_ACTIONS['default']
@@ -161,18 +161,17 @@
         logger.debug('No wblist found.')
         return SMTP_ACTIONS['default']
 
-    logger.debug('Found wblist: %s' % str(wblists))
+    logger.debug('Found outbound wblist: %s' % str(wblists))
 
     # Check sender addresses
     # rids/recipients are orded by priority
-    for rid in recipient_ids:
-        # sids/senders are sorted by priority
-        for sid in sender_ids:
+    for sid in sender_ids:
+        for rid in recipient_ids:
             if (rid, sid, 'W') in wblists:
-                return SMTP_ACTIONS['accept'] + " wblist=(%d, %d, 'W')" % (rid, sid)
+                return SMTP_ACTIONS['accept'] + " outbound_wblist=(%d, %d, 'W')" % (rid, sid)
 
             if (rid, sid, 'B') in wblists:
-                logger.info("Blacklisted: wblist=(%d, %d, 'B')" % (rid, sid))
+                logger.info("Blacklisted: outbound_wblist=(%d, %d, 'B')" % (rid, sid))
                 return SMTP_ACTIONS['reject_blacklisted']
 
     return SMTP_ACTIONS['default']
Post's attachments

amavisd_wblist.patch 2.06 kb, 10 downloads since 2016-01-08 

You don't have the permssions to download the attachments of this post.

5 Reply by bigman

  • bigman
  • Member
  • Offline
  • Registered: 2015-09-14
  • Posts: 41

Re: Whitelist / Blacklist Priority Problem

hi Zhang thanks for reply,

How to run this patch? Could you please tell me the steps.

thanks

6 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Whitelist / Blacklist Priority Problem

*) Download the patch and upload it to your iRedMail server, let's say it's /root/amavisd_wblist.patch.
*) backup old plugin file and verify the patch:

cd /opt/iredapd
cp plugins/amavisd_wblist.py plugins/amavisd_wblist.py.bak
patch --dry-run -p1 < /root/amavisd_wblist.patch

If no error in output message, it's safe to apply this patch immediately:

patch -p1 < /root/amavisd_wblist.patch

Then restart iRedAPD service.

7 Reply by bigman

  • bigman
  • Member
  • Offline
  • Registered: 2015-09-14
  • Posts: 41

Re: Whitelist / Blacklist Priority Problem

hi Zhang,

Thanks, I've applied the patch and now ok. I have another question :

1. Is there any limitation in whitelist / blacklist field ? How many rows it can handle ? Where it save the data (what table and what column in mysql?)
2. Because we are planning like above scenario, is there any development planning to create a group user, so that we don't need to set in each special user's whitelist, I mean we just need to set a special user to be a member of this group, and we give this group the whitelist, so all user will be block but not special users that are the member of that group. Is it possible ? smile

thanks

8 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Whitelist / Blacklist Priority Problem

bigman wrote:

1. Is there any limitation in whitelist / blacklist field ? How many rows it can handle ? Where it save the data (what table and what column in mysql?)

*) No limitation.
*) Unlimited rows. needless to remind you that toooooo many sql records may cause slow sql query smile
*) Currently, white/blacklists are saved in "amavisd" database. Please check this document for more details:
http://www.iredmail.org/docs/amavisd.sql.db.html

bigman wrote:

2. Because we are planning like above scenario, is there any development planning to create a group user, so that we don't need to set in each special user's whitelist, I mean we just need to set a special user to be a member of this group, and we give this group the whitelist, so all user will be block but not special users that are the member of that group. Is it possible ? smile

No plan yet.

If we need to detect which white/blacklisted address is a mail list, we need to query SQL with many (if you have many white/blacklisted addresses) conditions with "SELECT xxx FROM vmail.alias WHERE address IN (...many conditions here...)", then check whether sender is member of any returned mailing lists. Although i didn't try it, i think it doesn't have good sql query performance.

Maybe we can define some prefix like 'list:somelist@mydomain.com' as white/blacklisted address, and let iRedAPD expands it to all its members and check wblist. [Note: this is not a promise, just an idea.]

9 Reply by bigman

  • bigman
  • Member
  • Offline
  • Registered: 2015-09-14
  • Posts: 41

Re: Whitelist / Blacklist Priority Problem

hi Zhang,
thanks for your great support ! smile

Yes I think it's a good solution if we can set prefix like that in whitelist/blacklist field.
Although you are not promise I will waiting big_smile

thanks alot Zhang