1 Topic by ss2win

  • ss2win
  • Member
  • Offline
  • Registered: 2016-04-30
  • Posts: 15

Topic: random SPAM getting through

==== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.4
- Linux/BSD distribution name and version: CentOS 6.7 in LXC container on Proxmox 4.0
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): LDAP
- Web server (Apache or Nginx): nginx
- Manage mail accounts with iRedAdmin-Pro? no
- Related log if you're reporting an issue:

Apr 29 11:39:35 webmail postfix/smtpd[14095]: AE83161514: client=unknown[85.206.175.41]
Apr 29 11:39:36 webmail postfix/cleanup[13982]: AE83161514: message-id=<7F1CC1F74F311C4EA83711C3F5AEDC.21677427@bannerelkproperties.supernaturalatino.com>
Apr 29 11:39:36 webmail postfix/qmgr[9246]: AE83161514: from=<LiaKun@bannerelkproperties.supernaturalatino.com>, size=19017, nrcpt=1 (queue active)
Apr 29 11:39:40 webmail postfix/smtpd[14145]: 2438C6151B: client=localhost[127.0.0.1]
Apr 29 11:39:40 webmail postfix/cleanup[13982]: 2438C6151B: message-id=<7F1CC1F74F311C4EA83711C3F5AEDC.21677427@bannerelkproperties.supernaturalatino.com>
Apr 29 11:39:40 webmail postfix/qmgr[9246]: 2438C6151B: from=<LiaKun@bannerelkproperties.supernaturalatino.com>, size=19723, nrcpt=1 (queue active)
Apr 29 11:39:40 webmail amavis[14279]: (14279-04) Passed CLEAN {RelayedInbound}, [85.206.175.41]:46804 [85.206.175.41] <LiaKun@bannerelkproperties.supernaturalatino.com> -> <<MYEMAIL>@<MYDOMAIN>>, Message-ID: <7F1CC1F74F311C4EA83711C3F5AEDC.21677427@bannerelkproperties.supernaturalatino.com>, mail_id: LhE1K48yvmFp, Hits: 3.783, size: 18981, queued_as: 2438C6151B, 3788 ms
Apr 29 11:39:40 webmail postfix/smtp[14081]: AE83161514: to=<<MYEMAIL>@<MYDOMAIN>>, relay=127.0.0.1[127.0.0.1]:10024, delay=4.5, delays=0.67/0/0.01/3.8, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 2438C6151B)
Apr 29 11:39:40 webmail postfix/qmgr[9246]: AE83161514: removed
Apr 29 11:39:40 webmail postfix/pipe[14131]: 2438C6151B: to=<<MYEMAIL>@<MYDOMAIN>>, relay=dovecot, delay=0.1, delays=0.02/0/0/0.08, dsn=2.0.0, status=sent (delivered via dovecot service)
Apr 29 11:39:40 webmail postfix/qmgr[9246]: 2438C6151B: removed
====

Hello everyone,

I have just successfully migrated iRedMail from 0.9.0 to 0.9.4 (phew!) and though it's well, everything seeming to work OK, though I am seeing a lot of SPAM that doesn't appear to have been scanned by the system.  SPAM filtering is occurring for the majority of messages but many are delivered to the INBOX with very low scores. The missed positives are flagged as SPAM if I run them through spamassassin manually on the command line like this:

spamassassin -t -D < [email_message]

Here is the message headers as delivered to INBOX:

================================================================================
Return-Path: <LiaKun@bannerelkproperties.supernaturalatino.com>
Delivered-To: <MYEMAIL>@<MYDOMAIN>
Received: from webmail.<MYDOMAIN> (localhost [127.0.0.1])
        by webmail.<MYDOMAIN> (Postfix) with ESMTP id 2438C6151B
        for <<MYEMAIL>@<MYDOMAIN>>; Fri, 29 Apr 2016 11:39:40 -0400 (EDT)
X-Virus-Scanned: amavisd-new at webmail.<MYDOMAIN>
X-Spam-Flag: NO
X-Spam-Score: 3.783
X-Spam-Level: ***
X-Spam-Status: No, score=3.783 tagged_above=-10 required=5.5
        tests=[HTML_MESSAGE=0.001, PYZOR_CHECK=2.5, RDNS_NONE=1.274,
        SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01]
        autolearn=no
Received: from webmail.<MYDOMAIN> ([127.0.0.1])
        by webmail.<MYDOMAIN> (webmail.<MYDOMAIN> [127.0.0.1]) (amavisd-new, port 10024)
        with ESMTP id LhE1K48yvmFp for <<MYEMAIL>@<MYDOMAIN>>;
        Fri, 29 Apr 2016 11:39:36 -0400 (EDT)
Received: from bannerelkproperties.supernaturalatino.com (unknown [85.206.175.41])
        by webmail.<MYDOMAIN> (Postfix) with ESMTP id AE83161514
        for <<MYEMAIL>@<MYDOMAIN>>; Fri, 29 Apr 2016 11:39:35 -0400 (EDT)
Received: by bannerelkproperties.supernaturalatino.com id h4e03g0001gp for <<MYEMAIL>@<MYDOMAIN>>; Fri, 29 Apr 2016 11:34:09 -0400 (envelope-from <LiaKun@bannerelkproperties.supernaturalatino.com>)
To: <<MYEMAIL>@<MYDOMAIN>>
Reply-To: <daniel@supernaturalatino.com>
Message-ID: <7F1CC1F74F311C4EA83711C3F5AEDC.21677427@bannerelkproperties.supernaturalatino.com>
From: Discounted Cruises <daniel@supernaturalatino.com>
Subject: Unbought cruise rooms, 5 night 6 days
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="Hu51Hs7gXrGrQzLc7pAsJsH92sKoZ"
Date: Fri, 29 Apr 2016 11:39:36 -0400

--Hu51Hs7gXrGrQzLc7pAsJsH92sKoZ
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit

I was very surprised when I entered the room,

<CUT>
================================================================================

And here's the same message as scanned on the command line:

================================================================================
Content analysis details:   (16.5 points, 5.0 required)

pts rule name              description
---- ---------------------- --------------------------------------------------
3.6 RCVD_IN_SBL_CSS        RBL: Received via a relay in Spamhaus SBL-CSS
                            [85.206.175.41 listed in zen.spamhaus.org]
-0.0 SPF_HELO_PASS          SPF: HELO matches SPF record
2.5 URIBL_DBL_SPAM         Contains a spam URL listed in the DBL blocklist
                            [URIs: supernaturalatino.com]
1.7 URIBL_BLACK            Contains an URL listed in the URIBL blacklist
                            [URIs: supernaturalatino.com]
0.0 HTML_MESSAGE           BODY: HTML included in message
2.5 PYZOR_CHECK            Listed in Pyzor (http://pyzor.sf.net/)
1.3 RDNS_NONE              Delivered to internal network by a host with no rDNS
5.0 KAM_VERY_BLACK_DBL     Email that hits both URIBL Black and Spamhaus DBL
0.0 T_REMOTE_IMAGE         Message contains an external image
================================================================================

As you see, there's a big difference!

Why would amavisd be bypassed for some messages and not others? Where do I begin to debug this? I'm kinda lost especially since everything else seems to be working so well and there is a LOT of SPAM being blocked otherwise.

Thanks for any insight you can provide.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ss2win

  • ss2win
  • Member
  • Offline
  • Registered: 2016-04-30
  • Posts: 15

Re: random SPAM getting through

For reference, here's a message that was properly dealt with - flagged SPAM!

/var/log/maillog

================================================================================
Apr 29 11:39:08 webmail postfix/smtpd[13519]: 2F7256150B: client=unknown[162.144.151.181]
Apr 29 11:39:08 webmail postfix/cleanup[13916]: 2F7256150B: message-id=<2BA73286988466D949FA847D36F533.66404082@ascoweb-com.setubuh.com>
Apr 29 11:39:08 webmail postfix/qmgr[9246]: 2F7256150B: from=<OmerPreister@ascoweb-com.setubuh.com>, size=18893, nrcpt=1 (queue active)
Apr 29 11:39:12 webmail postfix/smtpd[13978]: DDAF46151F: client=localhost[127.0.0.1]
Apr 29 11:39:12 webmail postfix/cleanup[13982]: DDAF46151F: message-id=<2BA73286988466D949FA847D36F533.66404082@ascoweb-com.setubuh.com>
Apr 29 11:39:12 webmail postfix/qmgr[9246]: DDAF46151F: from=<OmerPreister@ascoweb-com.setubuh.com>, size=19740, nrcpt=1 (queue active)
Apr 29 11:39:12 webmail amavis[14278]: (14278-03) Passed SPAM {RelayedTaggedInbound}, [162.144.151.181]:60032 [162.144.151.181] <OmerPreister@ascoweb-com.setubuh.com> -> <<MYEMAIL>@<MYDOMAIN>>, Message-ID: <2BA73286988466D949FA847D36F533.66404082@ascoweb-com.setubuh.com>, mail_id: YJI_oVYdGmDH, Hits: 9.08, size: 18857, queued_as: DDAF46151F, 4486 ms
Apr 29 11:39:12 webmail postfix/smtp[14240]: 2F7256150B: to=<<MYEMAIL>@<MYDOMAIN>>, relay=127.0.0.1[127.0.0.1]:10024, delay=5.3, delays=0.84/0/0/4.5, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as DDAF46151F)
Apr 29 11:39:12 webmail postfix/qmgr[9246]: 2F7256150B: removed
Apr 29 11:39:12 webmail postfix/pipe[13932]: DDAF46151F: to=<<MYEMAIL>@<MYDOMAIN>>, relay=dovecot, delay=0.05, delays=0.01/0/0/0.03, dsn=2.0.0, status=sent (delivered via dovecot service)
Apr 29 11:39:12 webmail postfix/qmgr[9246]: DDAF46151F: removed
================================================================================

As scored by "spamassassin -t -D < [email_message]"

================================================================================
Content analysis details:   (15.1 points, 5.0 required)

pts rule name              description
---- ---------------------- --------------------------------------------------
3.6 RCVD_IN_SBL_CSS        RBL: Received via a relay in Spamhaus SBL-CSS
                            [162.144.151.181 listed in zen.spamhaus.org]
-0.0 SPF_HELO_PASS          SPF: HELO matches SPF record
2.5 URIBL_DBL_SPAM         Contains a spam URL listed in the DBL blocklist
                            [URIs: setubuh.com]
0.0 URIBL_RED              Contains an URL listed in the URIBL redlist
                            [URIs: setubuh.com]
0.0 HTML_MESSAGE           BODY: HTML included in message
2.4 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
                            above 50%
                            [cf: 100]
0.4 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
                            [cf: 100]
2.5 RAZOR2_CHECK           Listed in Razor2 (http://razor.sf.net/)
2.5 PYZOR_CHECK            Listed in Pyzor (http://pyzor.sf.net/)
0.0 DIGEST_MULTIPLE        Message hits more than one network digest check
1.3 RDNS_NONE              Delivered to internal network by a host with no rDNS
0.0 T_REMOTE_IMAGE         Message contains an external image
================================================================================

Headers:

================================================================================
Return-Path: <OmerPreister@ascoweb-com.setubuh.com>
Delivered-To: <MYEMAIL>@<MYDOMAIN>
Received: from webmail.<MYDOMAIN> (localhost [127.0.0.1])
        by webmail.<MYDOMAIN> (Postfix) with ESMTP id DDAF46151F
        for <<MYEMAIL>@<MYDOMAIN>>; Fri, 29 Apr 2016 11:39:12 -0400 (EDT)
X-Virus-Scanned: amavisd-new at webmail.<MYDOMAIN>
X-Spam-Flag: YES
X-Spam-Score: 9.08
X-Spam-Level: *********
X-Spam-Status: Yes, score=9.08 tagged_above=-10 required=5.5
        tests=[DIGEST_MULTIPLE=0.001, HTML_MESSAGE=0.001, PYZOR_CHECK=2.5,
        RAZOR2_CF_RANGE_51_100=0.365, RAZOR2_CF_RANGE_E8_51_100=2.43,
        RAZOR2_CHECK=2.5, RDNS_NONE=1.274, SPF_HELO_PASS=-0.001,
        SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01, URIBL_RED=0.001] autolearn=spam
Received: from webmail.<MYDOMAIN> ([127.0.0.1])
        by webmail.<MYDOMAIN> (webmail.<MYDOMAIN> [127.0.0.1]) (amavisd-new, port 10024)
        with ESMTP id YJI_oVYdGmDH for <<MYEMAIL>@<MYDOMAIN>>;
        Fri, 29 Apr 2016 11:39:08 -0400 (EDT)
Received: from ascoweb-com.setubuh.com (unknown [162.144.151.181])
        by webmail.<MYDOMAIN> (Postfix) with ESMTP id 2F7256150B
        for <<MYEMAIL>@<MYDOMAIN>>; Fri, 29 Apr 2016 11:39:07 -0400 (EDT)
Received: by ascoweb-com.setubuh.com id h4e01m0001gt for <<MYEMAIL>@<MYDOMAIN>>; Fri, 29 Apr 2016 11:30:21 -0400 (envelope-from <OmerPreister@ascoweb-com.setubuh.com>)
To: <<MYEMAIL>@<MYDOMAIN>>
Reply-To: <daniel@setubuh.com>
Message-ID: <2BA73286988466D949FA847D36F533.66404082@ascoweb-com.setubuh.com>
From: Discounted Cruises <daniel@setubuh.com>
Subject: ***Spam*** Five Night Cruise, Balcony room
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="Pr4hG4Un1uLc47B6ViTfL84kHt81Rt2eP"
Date: Fri, 29 Apr 2016 11:39:07 -0400

--Pr4hG4Un1uLc47B6ViTfL84kHt81Rt2eP
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit

I was very surprised when I entered the room
<CUT>
================================================================================

thanks

3 Reply by deltatango

  • deltatango
  • Member
  • Offline
  • Registered: 2016-04-03
  • Posts: 28

Re: random SPAM getting through

I see similar behavior, but in my case it's all about the timing. When I manually recheck a message that I've already received, enough time has gone by that spamassassin hits many more URI RBL rules than it did upon initial delivery. A long greylist delay would likely help with this, but that causes its own set of problems (in the form of cranky users).

4 Reply by ss2win

  • ss2win
  • Member
  • Offline
  • Registered: 2016-04-30
  • Posts: 15

Re: random SPAM getting through

I thought about timing so these tests were done minutes after the messages were received. It seems like they should be flagged without any BL lookup as they are so obviously spammy. I've used greylisting in the past and it did help but it really upset the users so it's not an option anymore.

5 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,172

Re: random SPAM getting through

I'm afraid that you have to turn on debug mode in Amavisd (with SpamAssassin debug mode enabled too) and check its log file to understand what rules SpamAssassin applied to detect spam on incoming email.

FYI: http://www.iredmail.org/docs/debug.amavisd.html

6 Reply by ss2win

  • ss2win
  • Member
  • Offline
  • Registered: 2016-04-30
  • Posts: 15

Re: random SPAM getting through

Thanks. For some reason the problem is not as bad as it was in the beginning. I've been researching how to improve scanning and I'll report back once I have something to share.

7 Reply by deltatango

  • deltatango
  • Member
  • Offline
  • Registered: 2016-04-03
  • Posts: 28

Re: random SPAM getting through

A couple of things that helped me crack down on spam:

I reworked my postscreen config incorporating techniques from http://rob0.nodns4.us/postscreen.html including the use of whitelists from spamhaus and dnswl.org.  I haven't enabled After-220 deep protocol tests yet, but I probably will, with a loooong local postscreen cache, at least 30 days or more.

I added the KAM ruleset from https://www.pccc.com/downloads/SpamAssa … rib/KAM.cf   That really increased my spam scores and, so far, zero false positives.  I'd rather suffer a dozen spams in my Inbox rather than a single false positive.

I will turn greylisting back on (using a 13 minute forced delay, so that senders that retry in 15 minutes will succeed), but again with a very long cache, 90 days minimum, maybe 180. That, plus spamhaus/dnswl whitelisting, should help with user heartburn.

I haven't tried this yet but it looks promising:  https://github.com/stevejenkins/hardwar … qrdns.pcre

8 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,172

Re: random SPAM getting through

Do you have postscreen and DNSBL service enabled in Postfix to help reduce spam?

9 Reply by ss2win

  • ss2win
  • Member
  • Offline
  • Registered: 2016-04-30
  • Posts: 15

Re: random SPAM getting through

I have KAM, several RBL's configured and many custom rules. I don't have postscreen but will check it out. There are still some spam getting through that look they weren't scanned at all but it's not as big an issue as it was in the beginning - might just be timing as previously mentioned.

10 Reply by ss2win

  • ss2win
  • Member
  • Offline
  • Registered: 2016-04-30
  • Posts: 15

Re: random SPAM getting through

Just an FYI, there are multiple instances of sa_debug= in /etc/amavisd/amavisd.conf. I don't think I added them but it could be possible... Note when turning on debugging, the last one rules smile