1 Topic by Clouseau (edited by Clouseau 2019-03-23 00:19:07)

  • Clouseau
  • Member
  • Offline
  • Registered: 2012-03-18
  • Posts: 74

Topic: Greylisting, SPF detection, parsing problems and wrong result sets

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release): 0.9.7
- Linux/BSD distribution name and version: Debian 8.11
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): LDAP
- Web server (Apache or Nginx): Apache
- Manage mail accounts with iRedAdmin-Pro? No
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

Hello,

it looks like spf_to_greylist_whitelists.py script doesn't return IP addresses of IP's noted in SPF recods when they have: +include:_spf.google.com in it. Seems to me "_" is not parsed so it is not queried further.

python spf_to_greylist_whitelists.py --debug domain.com
* 1 mail domains in total.
        + [domain.com]
                + SPF -> v=spf1 +ip4:X.X.X.X +a +mx +a:somehostname.domain.com +a:someothersmtp.domain2.com +include:_spf.google.com ~all
                + Result: set(['X.X.X.X', 'Y.Y.Y.Y', 'Z.Z.Z.Z'])

There is no parsing of +include:_spf.google.com  which would include further:

_spf.google.com descriptive text "v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all"

EDIT: check third example, error is due to a "+" sign in SPF record and not "_" sign

---------------------------------------------------------------------------------------

In a case you add outlook.com with --debug:

python spf_to_greylist_whitelists.py --debug outlook.com
* 1 mail domains in total.
        + [outlook.com]
                + SPF -> v=spf1 include:spf-a.outlook.com include:spf-b.outlook.com ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.microsoft.com ~all
                + Result: set(['213.199.154.0/24', '207.46.198.0/25', '64.4.22.64', '70.37.151.128', '207.46.58.128', '94.245.112.10', '65.55.33.64', '23.103.198.0/23', '213.199.161.128', '157.56.232.0/21', '23.103.200.0/21', '157.56.248.0/21', '65.55.88.0/24', '65.55.169.0/24', '157.55.225.0/25', '207.46.4.128', '65.55.34.0/24', '207.68.176.0/26', '65.54.51.64', '65.55.126.0/25', '65.54.121.120', '23.103.191.0/24', '23.103.208.0/21', '213.199.177.0/26', '40.92.0.0/15', '157.56.110.0/23', '65.55.78.128', '157.55.2.0/25', '65.54.241.0/24', '207.46.132.128', '157.55.49.0/25', '157.55.157.128', '157.55.1.128', '134.170.140.0/24', '213.199.180.128', '207.46.163.0/24', '207.46.50.224', '65.54.190.0/24', '65.55.234.192', '157.55.11.0/25', '157.56.112.0/24', '207.46.51.64', '157.56.240.0/20', '157.55.9.128', '157.55.234.0/24', '65.55.81.48', '94.245.120.64', '23.103.128.0/19', '111.221.66.0/25', '65.55.178.128', '2a01:111:f400::/48', '52.100.0.0/14', '207.46.200.0/27', '111.221.112.0/21', '104.47.0.0/17', '65.55.113.64', '207.46.100.0/24', '65.55.111.0/24', '65.55.52.224', '157.56.24.0/25', '111.221.69.128', '207.68.169.173', '94.245.112.0/27', '65.55.94.0/25', '65.55.238.129', '157.55.61.0/24', '65.54.61.64', '207.68.176.96', '65.55.90.0/24', '157.55.0.192', '111.221.23.128', '207.46.50.192', '2001:489a:2202::/48', '65.55.174.0/25', '111.221.26.0/27', '216.32.180.0/23', '207.46.116.128', '207.46.117.0/24', '65.55.116.0/25', '40.107.0.0/16'])

iRedApt I use was 2.1 and I noticed this. I have upgraded to 2.4 and it is the same.
-----------------------------------------------------------------------------------------------
Third example works, looks like it is parsed but you have to remove "+include" and just use "include". "+" sign in DNS record is legit syntax though and +a and +ipv4 are parsed but not +include...

# python spf_to_greylist_whitelists.py --debug domain3.com
* 1 mail domains in total.
        + [domain3.com]
                + SPF -> v=spf1 +a +mx +ip4:X.X.X.X include:_spf.mail.iskon.hr ~all
                + Result: set(['213.191.128.72', 'X.X.X.X', '213.191.128.70', '213.191.128.80'])

root@ignored:/opt/iredapd/tools# host -t TXT _spf.mail.iskon.hr
_spf.mail.iskon.hr descriptive text "v=spf1 ip4:213.191.128.70/31 ip4:213.191.128.72/29 ip4:213.191.128.80/29"

And one more issue, result set includes only IP's and not ranges  of spf recors, check result set and check last DNS query for "host -t TXT _spf.mail.iskon.hr"

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: Greylisting, SPF detection, parsing problems and wrong result sets

Please give me a real domain name for testing.

3 Reply by Clouseau (edited by Clouseau 2019-03-25 18:32:10)

  • Clouseau
  • Member
  • Offline
  • Registered: 2012-03-18
  • Posts: 74

Re: Greylisting, SPF detection, parsing problems and wrong result sets

ZhangHuangbin wrote:

Please give me a real domain name for testing.

I can send you privately but I see not PM option to send you a message. Maybe to your email address?

P.S. this is 100% correct, if  SPF records have +include:_spf.google.com in the record, spf_to_greylist_whitelists.py script will not fetch the results from include. If SPF records have include:_spf.google.com all is good.

I can fix this by removing + from include from all domains but "+" is a legit syntax...

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: Greylisting, SPF detection, parsing problems and wrong result sets

Could you try this patch for iRedAPD?

diff -r 452b6535bb89 libs/dnsspf.py
--- a/libs/dnsspf.py    Tue Mar 26 18:09:19 2019 +0800
+++ b/libs/dnsspf.py    Wed Mar 27 11:40:34 2019 +0800
@@ -148,7 +148,7 @@
     for tag in tags:
         v = tag.split(':', 1)[-1]
 
-        if tag.startswith('include:'):
+        if tag.startswith('include:') or tag.startswith('+include:'):
             included_domains.add(v)
         elif tag.startswith('redirect='):
             d = tag.split('=', 1)[-1]

5 Reply by Clouseau

  • Clouseau
  • Member
  • Offline
  • Registered: 2012-03-18
  • Posts: 74

Re: Greylisting, SPF detection, parsing problems and wrong result sets

It works smile