1 (edited by Martin 2017-09-21 22:30:05)

Topic: [SOLVED] Whitelist Greylisting for Office 365

==== Required information ====
- iRedMail version (check /etc/iredmail-release):
- Linux/BSD distribution name and version:
- Store mail accounts in which backend (LDAP/MySQL/PGSQL):
- Web server (Apache or Nginx):
- Manage mail accounts with iRedAdmin-Pro?
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
======== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.7
- Linux/BSD distribution name and version: Debian Jessie
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): LDAP
- Web server (Apache or Nginx): Apache
- Manage mail accounts with iRedAdmin-Pro? Yes
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

Hi,

There are hours of delay to receive mail from mail domains hosted by Microsoft Office 365 due to greylisting.

Servers used by this service are present in greylisting_whitelist_domain_spf table of iRedAPD database, so these servers should be whitelisted?

Example of greylisting_whitelist_domain_spf table:
5256580    @.    104.47.0.0/17    AUTO-UPDATE: ebay.com
Related Postfix log:
NOQUEUE: reject: RCPT from mail-db5eur01on0068.outbound.protection.outlook.com[104.47.2.68]: 451 4.7.1 <martin.leusch@imrcom.fr>: Recipient address rejected: Intentional policy rejection, please try again later;

What is the behavior of SPF whitelist for greylisting? Is sender address must be on the whitelisted domain (@outlook.com) or could be on tiers domain (@company.com) and use Office 365 service?

2

Re: [SOLVED] Whitelist Greylisting for Office 365

Try this:

cd /opt/iredapd/tools/
python spf_to_greylist_whitelists.py --submit outlook.com

It should whitelist all mail servers (for greylisting service) specified in SPF DNS record of mail domain name 'outlook.com'.

----

Does my reply help a little? How about buying me a cup of coffee ($5) as an encouragement?

buy me a cup of coffee

3

Re: [SOLVED] Whitelist Greylisting for Office 365

But outlook.com is in greylisting_whitelist_domains table yet and all IP and CIDR listed in SPF DNS record of mail domain name 'outlook.com' are in greylisting_whitelist_domain_spf table.

Why mails from mail-db5eur01on0068.outbound.protection.outlook.com[104.47.2.68] are greylisted if 104.47.0.0/17 CIDR is present in greylisting_whitelist_domain_spf table?

4

Re: [SOLVED] Whitelist Greylisting for Office 365

Martin wrote:

Why mails from mail-db5eur01on0068.outbound.protection.outlook.com[104.47.2.68] are greylisted if 104.47.0.0/17 CIDR is present in greylisting_whitelist_domain_spf table?

iRedAPD was not up to date, in iRedAPD 2.0, greylisting plugin doesn't look in greylisting_whitelist_domain_spf table for whitelisted servers, upgrading to 2.1 should solve this.

5

Re: [SOLVED] Whitelist Greylisting for Office 365

If you're running iRedMail-0.9.7 (claimed in your first post), you should have iRedAPD-2.1 out of box. If you upgraded iRedMail server from an old version, you should follow our tutorial to upgrade it during upgrading iRedMail:
http://www.iredmail.org/docs/iredmail.releases.html

----

Does my reply help a little? How about buying me a cup of coffee ($5) as an encouragement?

buy me a cup of coffee

6

Re: [SOLVED] Whitelist Greylisting for Office 365

ZhangHuangbin wrote:

Try this:

cd /opt/iredapd/tools/
python spf_to_greylist_whitelists.py --submit outlook.com

It should whitelist all mail servers (for greylisting service) specified in SPF DNS record of mail domain name 'outlook.com'.

This doesn't work anymore, or does it?

I executed the script and added "spf.protection.outlook.com", as this is an SPF record for office365 customers.
The senders domain also has this record set, but the Mails are getting greylisted anyway. and then never reach me, because every retry from office365 is from a different IP.

https://i.imgur.com/7DZVDBh.png

7

Re: [SOLVED] Whitelist Greylisting for Office 365

Hi @lug,

Please do not reply in other's thread, especially it's years old.

lug wrote:

This doesn't work anymore, or does it?

Which iRedAPD are you running? could you upgrade iRedAPD to 2.7 and try again?

----

Does my reply help a little? How about buying me a cup of coffee ($5) as an encouragement?

buy me a cup of coffee

8

Re: [SOLVED] Whitelist Greylisting for Office 365

Okay I see, there may be a misunderstanding.
spf_to_greylist just reads the spf records of a domain and add these entries to the whitelist, right?

The probleme there is, there are thousands of custom domains hosted in the microsoft cloud, but what they all have in common is the spf record

spf.protection.outlook.com

, so I thought you could add this entry to the whitelist.

Now I just added the o365 IP range to the whitelist.
https://i.imgur.com/1qgTqoz.png

9

Re: [SOLVED] Whitelist Greylisting for Office 365

Servers listed in SPF DNS record of Outlook servers (microsoft.com, outlook.com) are whitelisted by default. You can find it in /opt/iredapd/SQL/greylisting_whitelist_domains.sql, it's imported while upgrading iRedAPD.

----

Does my reply help a little? How about buying me a cup of coffee ($5) as an encouragement?

buy me a cup of coffee

10 (edited by lug 2019-04-09 22:12:38)

Re: [SOLVED] Whitelist Greylisting for Office 365

I run the latest iredapd (2.7) and even if I added the ip ranges of office365 to the whitelist in the iredadmin webpanel, the messages still get rejected

Apr  8 17:02:21 mail postfix/smtpd[32753]: NOQUEUE: reject: RCPT from ***.outbound.protection.outlook.com[52.101.130.67]: 451 4.7.1 <lug@mydomain.com>: Recipient address rejected: Intentional policy rejection, please try again later; from=<externalsender@hisdomain.eu> to=<lug@mydomain.com> proto=ESMTP helo=<***.outbound.protection.outlook.com>

52.101.130.67 is in 52.100.0.0/14, which is in my greylisting exceptions.
And yes I see, 52.100.0.0/14 is learned from hotmail.com automatically. But obviously, it doens't work.

I looked into the database, the entries were submitted correctly:
https://i.imgur.com/c1vCZgx.png

Shall I open new topic?

11

Re: [SOLVED] Whitelist Greylisting for Office 365

Hello, the problem is still present, mail from 52.101.130.47 greylisted.

12

Re: [SOLVED] Whitelist Greylisting for Office 365

After adding the senders domain to the whitelist, the mail got accepted from 52.101.134.105...
So every other mail from office365 will be still blocked, if I don't whitelist every senders domain, which is nearly impossible.

13

Re: [SOLVED] Whitelist Greylisting for Office 365

This is not expected. Could you please show me output of commands below?

cd /opt/iredapd/tools/

# Check whether outlook.com is whitelisted for greylisting service.
python greylisting_admin.py --list-whitelist-domains | grep -i 'outlook.com'

# List full IP/networks of the domain
python spf_to_greylist_whitelists.py --debug outlook.com

I can see outlook.com has network range "52.100.0.0/14", it contains IP "52.101.134.105". So it should be whitelisted (for greylisting service).

Btw, which version of iRedAPD are you running? Please make sure you're running the latest iRedAPD-2.7 (at least 2.5).

----

Does my reply help a little? How about buying me a cup of coffee ($5) as an encouragement?

buy me a cup of coffee

14 (edited by lug 2019-04-15 19:53:08)

Re: [SOLVED] Whitelist Greylisting for Office 365

ZhangHuangbin wrote:

python greylisting_admin.py --list-whitelist-domains | grep -i 'outlook.com'

outlook.com
spf.protection.outlook.com (<- the one I added via script)

ZhangHuangbin wrote:

python spf_to_greylist_whitelists.py --debug outlook.com

* 1 mail domains in total.
        + [outlook.com]
                + SPF -> v=spf1 include:spf-a.outlook.com include:spf-b.outlook.com ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.microsoft.com ~all
                + Result: set(['213.199.154.0/24', '207.46.198.0/25', '23.103.128.0/19', '207.46.116.128/29', '23.103.198.0/23', '65.55.81.48/28', '157.56.232.0/21', '23.103.200.0/21', '111.221.23.128/25', '94.245.112.10/31', '213.199.161.128/27', '157.56.248.0/21', '70.37.151.128/25', '65.55.169.0/24', '157.55.225.0/25', '65.55.34.0/24', '157.55.1.128/26', '207.68.176.96/27', '207.46.58.128/25', '65.55.126.0/25', '23.103.191.0/24', '213.199.180.128/26', '23.103.208.0/21', '157.55.157.128/25', '207.46.4.128/25', '40.92.0.0/15', '157.56.110.0/23', '111.221.69.128/25', '157.55.2.0/25', '65.54.241.0/24', '94.245.120.64/26', '157.55.49.0/25', '65.55.88.0/24', '157.55.9.128/25', '207.46.163.0/24', '207.46.50.224', '65.55.178.128/27', '65.54.190.0/24', '157.55.11.0/25', '157.56.112.0/24', '157.56.240.0/20', '157.55.234.0/24', '65.55.52.224/27', '65.55.78.128/25', '207.46.132.128/27', '207.46.51.64/26', '65.55.174.0/25', '111.221.66.0/25', '2a01:111:f400::/48', '52.100.0.0/14', '134.170.140.0/24', '207.46.200.0/27', '65.54.121.120/29', '111.221.112.0/21', '207.46.50.192/26', '104.47.0.0/17', '157.55.0.192/26', '207.46.100.0/24', '65.55.111.0/24', '65.54.51.64/26', '157.56.24.0/25', '65.55.33.64/28', '94.245.112.0/27', '65.55.94.0/25', '157.55.61.0/24', '213.199.177.0/26', '65.55.90.0/24', '65.55.113.64/26', '207.68.176.0/26', '64.4.22.64/26', '65.54.61.64/26', '2001:489a:2202::/48', '65.55.234.192/26', '111.221.26.0/27', '216.32.180.0/23', '207.46.117.0/24', '65.55.116.0/25', '40.107.0.0/16'])

ZhangHuangbin wrote:

Btw, which version of iRedAPD are you running? Please make sure you're running the latest iRedAPD-2.7 (at least 2.5).

I use 2.7 and newest iredmail/iradmin-pro.

Is there any other check besides IP? Because senders Domain is not @outlook.com

15

Re: [SOLVED] Whitelist Greylisting for Office 365

lug wrote:

Is there any other check besides IP? Because senders Domain is not @outlook.com

- Does this real mail domain name have correct SPF record?
- Did you already whitelist @outlook.com (SPF)? If yes, then it's ok if the real mail domain name doesn't have SPF DNS record.

Btw, it will be easier / quicker to troubleshoot if you show us the real mail domain name.

----

Does my reply help a little? How about buying me a cup of coffee ($5) as an encouragement?

buy me a cup of coffee

16

Re: [SOLVED] Whitelist Greylisting for Office 365

Confirmed it's a bug of iRedAPD. Patch for iRedAPD-2.7 here:

--- libs/dnsspf.py    2019-04-17 22:49:29.000000000 +0800
+++ libs/dnsspf.py    2019-04-17 22:50:00.000000000 +0800
@@ -315,8 +315,8 @@
 
     # Get CIDR networks
     if _ip_object.version == 4:
-        # if `ip=a.b.c.d`, ip prefix = `a.b.`
-        _ipv4_prefix = '.'.join(ip.split('.', 2)[:2]) + '.'
+        # if `ip=a.b.c.d`, ip prefix = `a.`
+        _ipv4_prefix = ip.split('.', 1)[0] + '.'
         _cidrs = [i for i in _ips if (i.startswith(_ipv4_prefix) and '.0/' in i)]
     elif _ip_object.version == 6:
         _cidrs = [i for i in _ips if (':' in i and '/' in i)]
--- plugins/greylisting.py    2019-04-17 22:50:15.000000000 +0800
+++ plugins/greylisting.py    2019-04-17 22:51:03.000000000 +0800
@@ -71,14 +71,14 @@
 
     # Gather CIDR networks
     if ip_object.version == 4:
-        # if `ip=a.b.c.d`, ip prefix = `a.b.`
-        _cidr_prefix = '.'.join(client_address.split('.', 2)[:2]) + '.'
+        # if `ip=a.b.c.d`, ip prefix = `a.`
+        _cidr_prefix = client_address.split('.', 1)[0] + '.'
 
         # Make sure _cidr is IPv4 network and in 'same' IP range.
         _cidrs = [_cidr for _cidr in whitelists if (_cidr.startswith(_cidr_prefix) and '/' in _cidr)]
     elif ip_object.version == 6:
-        # if `ip=a:b:c:...`, ip prefix = `a:b:`
-        _cidr_prefix = ':'.join(client_address.split(':', 2)[:2]) + ':'
+        # if `ip=a:b:c:...`, ip prefix = `a:`
+        _cidr_prefix = client_address.split(':', 1)[0] + ':'
 
         _cidrs = [_cidr for _cidr in whitelists if _cidr.startswith(_cidr_prefix) and ':/' in _cidr]
 
@@ -144,7 +144,7 @@
         return False
 
     if ip_object.version == 4:
-        _cidr_prefix = '.'.join(client_address.split('.', 2)[:2]) + '.'
+        _cidr_prefix = client_address.split('.', 1)[0] + '.'
 
     # Found enabled/disabled greylisting setting
     for r in records:

Restarting iRedAPD service is required after applied the patch.

----

Does my reply help a little? How about buying me a cup of coffee ($5) as an encouragement?

buy me a cup of coffee

17

Re: [SOLVED] Whitelist Greylisting for Office 365

That patch works, thanks! smile