1

Topic: Set spam deliver vs. bounce threshold?

==== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.7
- Linux/BSD distribution name and version: Ubuntu 16.04LTS
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): Apache
- Manage mail accounts with iRedAdmin-Pro? Yes v2.8.0
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

I'd like to be able to set a threshold (globally, maybe eventually per-user) for a spam score at which to simply bounce the message vs. delivering it to the user. So for example a score of 3.2 might be delivered, while a score of 16.5 would bounce. Setting a threshold above which we're sure enough the message is really spam would mean less clutter in the users' junk boxes with stuff we're quite sure is junk.

So I guess I'm wondering if:

a) there is a way to do this today (either via iRedAdminPro, or config files)
b) if this could be added
c) if this could be implemented to be settable on a per-user basis.

Thoughts?

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: Set spam deliver vs. bounce threshold?

dts wrote:

a) there is a way to do this today (either via iRedAdminPro, or config files)

Sure.

Find the global spam policy with SQL command below:

sql> USE amavisd;
sql> SELECT * FROM policy WHERE policy_name='@.' LIMIT 1 \G

You can update column "spam_tag2_level" and "spam_kill_level" (this one is managed in iRedAdmin-Pro).

Currently iRedAdmin-Pro sets both to same score.

3

Re: Set spam deliver vs. bounce threshold?

OK, I'm clearly confused. I see a lot of spam that gets passed, and my nightly report says as much, but don't see any blocked. When I see a spam score of 19.36 or something, and it's still delivered, nothing is being rejected for spam score. At present I see:

              spam_tag_level: -100
             spam_tag2_level: 4
             spam_tag3_level: 4
             spam_kill_level: 4

And spam comes through with scores well over 4. What am I missing, or is something malfunctioning that needs to be debugged?

4

Re: Set spam deliver vs. bounce threshold?

Here's an email report received about something that was flagged but nonetheless was delivered:

Content type: Banned
Internal reference code for the message is 27678-02/S-Lc66ypUvlZ

First upstream SMTP client IP address: [174.138.43.71]
According to a 'Received:' trace, the message apparently originated at:
[197.210.226.41], mail.dealer.com unknown [174.138.43.71]

Return-Path: <admin@tententp.cf>
From: "TNT Express" <admin@tententp.cf>
Message-ID: <20180312102431.C16EE120069@acorn.amaranth.net>
Subject: TNT Express //Arrival Notice // AWB #9078013580 3/12/2018
Not quarantined.

The message WILL BE relayed to:
<dts@senie.com>

Spam scanner report:
Spam detection software, running on the system "acorn.amaranth.net",
has identified this incoming email as possible spam.  The original
message has been attached to this so you can view it or label
similar future email.  If you have any questions, see
the administrator of that system for details.

Content preview:  Dear Sir, TNT AWB #9078013580 Tracking details. Your shipment
  listed below arrived our regional Office Today. Check attachment for the
Delivery details of your shipment Thank you for using our services. Shipment
  reference : AWB: Original Documents Description : Courier Express Service
  Priority: Arrival Notification If you would like to find out about the many
  ways TNT helps you to track your shipment, or if you would like to know more
  about the services provided by TNT, simply connect to www.tnt.com and select
  your location at any time. [...]

Content analysis details:   (10.4 points, 5.0 required)

pts rule name              description
---- ---------------------- --------------------------------------------------
1.7 DEAR_SOMETHING         BODY: Contains 'Dear (something)'
2.6 RCVD_IN_SBL            RBL: Received via a relay in Spamhaus SBL
                           [197.210.226.41 listed in zen.spamhaus.org]
3.6 RCVD_IN_SBL_CSS        RBL: Received via a relay in Spamhaus SBL-CSS
-0.0 SPF_PASS               SPF: sender matches SPF record
0.0 HTML_MESSAGE           BODY: HTML included in message
2.5 RDNS_NONE              Delivered to internal network by a host with no rDNS
Return-Path: <admin@tententp.cf>
Received: from mail.dealer.com (unknown [174.138.43.71])
    by acorn.amaranth.net (Postfix) with ESMTP id C16EE120069
    for <dts@senie.com>; Mon, 12 Mar 2018 06:24:29 -0400 (EDT)
Received: from [192.168.43.161] (unknown [197.210.226.41])
    by mail.dealer.com (Postfix) with ESMTPA id B2831CCF168;
    Mon, 12 Mar 2018 10:21:48 +0000 (UTC)

5

Re: Set spam deliver vs. bounce threshold?

Interesting. Turns out there was a policy record for my dts@senie.com account. What is odd is that when making any changes to the policy for a user (at least in the experiments I've done so far, the policy column spam_lover gets set to Y. That seems wrong. What I was trying to do is simply change the spam threshold value. Also, other fields like spam_lover get set to NULL, where the default record sets to N. Wondering if there are some issues with iRedAdminPro-SQL in this area.

6

Re: Set spam deliver vs. bounce threshold?

dts wrote:

What is odd is that when making any changes to the policy for a user (at least in the experiments I've done so far, the policy column spam_lover gets set to Y. That seems wrong.

Did you uncheck option "Enable spam checking" in the spam policy setting page?

7

Re: Set spam deliver vs. bounce threshold?

ZhangHuangbin wrote:
dts wrote:

What is odd is that when making any changes to the policy for a user (at least in the experiments I've done so far, the policy column spam_lover gets set to Y. That seems wrong.

Did you uncheck option "Enable spam checking" in the spam policy setting page?

The "Enable spam checking" was checked.

8

Re: Set spam deliver vs. bounce threshold?

Now seeing that "Virus Lover" column populates with a "Y" if I reset to defaults (causing the record to go away, and change just the spam threshold to 4.0.

BTW, the text to the right of the spam threshold is in light grey, which kind of makes it look like the field is disabled, but it's not. Just a cosmetic thing, but that text "Default is 4.0" doesn't need to be affected by rolling the mouse over. Not clear that amavis really is working with 4.0 as a default if there is no record in the policy table.

9

Re: Set spam deliver vs. bounce threshold?

Could you please show me the value of SQL records, and screenshots of iRedAdmin-Pro spam policy setting pages?
Both global setting and the per-user setting.

it sounds like a bug of iRedAdmin-Pro.

10

Re: Set spam deliver vs. bounce threshold?

OK. Cycled through this again. Used the "Delete policy and use default settings" button to cause iRedAdmin-Pro to delete the only record that was in the policy table:

mysql> SELECT * FROM policy WHERE policy_name='@.' LIMIT 1 \G
Empty set (0.00 sec)

Then I made a single change on the page, putting in 4.0 in the "Classify mail as spam when score is >=" field and saved. Screen shot after doing so. Then dumped the policy record and got this:

mysql> SELECT * FROM policy WHERE policy_name='@.' LIMIT 1 \G
*************************** 1. row ***************************
                          id: 8
                 policy_name: @.
                 virus_lover: Y
                  spam_lover: NULL
             unchecked_lover: NULL
          banned_files_lover: NULL
            bad_header_lover: NULL
         bypass_virus_checks: N
          bypass_spam_checks: N
        bypass_banned_checks: N
        bypass_header_checks: N
         virus_quarantine_to:
          spam_quarantine_to:
        banned_quarantine_to:
     unchecked_quarantine_to: NULL
    bad_header_quarantine_to:
         clean_quarantine_to: NULL
       archive_quarantine_to: NULL
              spam_tag_level: NULL
             spam_tag2_level: 4
             spam_tag3_level: 4
             spam_kill_level: 4
       spam_dsn_cutoff_level: NULL
spam_quarantine_cutoff_level: NULL
        addr_extension_virus: NULL
         addr_extension_spam: NULL
       addr_extension_banned: NULL
   addr_extension_bad_header: NULL
              warnvirusrecip: NULL
             warnbannedrecip: NULL
               warnbadhrecip: NULL
              newvirus_admin: NULL
                 virus_admin: NULL
                banned_admin: NULL
            bad_header_admin: NULL
                  spam_admin: NULL
            spam_subject_tag: NULL
           spam_subject_tag2: NULL
           spam_subject_tag3: NULL
          message_size_limit: NULL
            banned_rulenames: NULL
          disclaimer_options: NULL
              forward_method: NULL
                 sa_userconf: NULL
                 sa_username: NULL
1 row in set (0.00 sec)

11

Re: Set spam deliver vs. bounce threshold?

dts wrote:

Now seeing that "Virus Lover" column populates with a "Y" if I reset to defaults (causing the record to go away, and change just the spam threshold to 4.0.

Bug of iRedAdmin-Pro. Please try patch below:

--- a/libs/amavisd/spampolicy.py    2018-03-19 07:47:40.557436173 +0000
+++ b/libs/amavisd/spampolicy.py    2017-11-07 06:49:54.000000000 +0000
@@ -116,15 +116,19 @@
 
         if 'spam_quarantine_to' in form:
             updates['spam_quarantine_to'] = 'spam-quarantine'
+            updates['spam_lover'] = 'N'
 
         if 'virus_quarantine_to' not in form:
             updates['virus_quarantine_to'] = ''
+            updates['virus_lover'] = 'Y'
 
         if 'banned_quarantine_to' in form:
             updates['banned_quarantine_to'] = 'banned-quarantine'
+            updates['banned_files_lover'] = 'N'
 
         if 'bad_header_quarantine_to' in form:
             updates['bad_header_quarantine_to'] = 'bad-header-quarantine'
+            updates['bad_header_lover'] = 'N'
 
         # Modify spam subject
         if 'modify_spam_subject' in form:

12

Re: Set spam deliver vs. bounce threshold?

Code as already present (didn't apply any patch) seems to look like what you're suggesting:

        if 'spam_quarantine_to' in form:
            updates['spam_quarantine_to'] = 'spam-quarantine'
            updates['spam_lover'] = 'N'

        if 'virus_quarantine_to' not in form:
            updates['virus_quarantine_to'] = ''
            updates['virus_lover'] = 'Y'

        if 'banned_quarantine_to' in form:
            updates['banned_quarantine_to'] = 'banned-quarantine'
            updates['banned_files_lover'] = 'N'

        if 'bad_header_quarantine_to' in form:
            updates['bad_header_quarantine_to'] = 'bad-header-quarantine'
            updates['bad_header_lover'] = 'N'

        # Modify spam subject
        if 'modify_spam_subject' in form:

13

Re: Set spam deliver vs. bounce threshold?

Do you have same content in file "libs/amavisd/spampolicy.py"?

        # Update spam policy
        updates = {'spam_lover': 'N',
                   'virus_lover': 'N',
                   'banned_files_lover': 'N',
                   'bad_header_lover': 'N',
                   'bypass_spam_checks': 'N',
                   'bypass_virus_checks': 'N',
                   'bypass_banned_checks': 'N',
                   'bypass_header_checks': 'N'}

        if 'enable_spam_checks' not in form:
            updates['bypass_spam_checks'] = 'Y'

        if 'enable_virus_checks' not in form:
            updates['bypass_virus_checks'] = 'Y'

        if 'enable_banned_checks' not in form:
            updates['bypass_banned_checks'] = 'Y'

        if 'enable_header_checks' not in form:
            updates['bypass_header_checks'] = 'Y'

The "updates = {" line may be different, but it doesn't matter much, just check the initial values.