1

Topic: User based antispam via Dovecot and Spamassassian (update)

==== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.7
- Linux/BSD distribution name and version: centOS 7.4
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): all
- Web server (Apache or Nginx): all
- Manage mail accounts with iRedAdmin-Pro?



Based on the post http://www.iredmail.org/forum/topic8169 … assin.html
I've an updated walk through. The former used dovecot-antispam plugin has been deprecated, this one is based on antiSpamWithSieve https://wiki2.dovecot.org/Plugins/Antispam.

Feel free to use it.



1. Mod, Add Dovecot Settings:
/etc/dovecot/dovecot.conf

protocol imap {
    mail_plugins = $mail_plugins imap_quota imap_acl imap_sieve
    ...
}
plugin {
   ...
    # Antispam
    sieve_plugins = sieve_imapsieve sieve_extprograms

    # From elsewhere to Spam folder
    imapsieve_mailbox1_name = Junk
    imapsieve_mailbox1_causes = COPY
    imapsieve_mailbox1_before = file:/var/vmail/sieve/report-spam.sieve

    # From Spam folder to elsewhere
    imapsieve_mailbox2_name = *
    imapsieve_mailbox2_from = Junk
    imapsieve_mailbox2_causes = COPY
    imapsieve_mailbox2_before = file:/var/vmail/sieve/report-ham.sieve

    sieve_pipe_bin_dir = /var/vmail/sieve
    sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment +vnd.dovecot.debug
}

2. create sieve scripts
/var/vmail/sieve/report-spam.sieve

require ["vnd.dovecot.debug", "vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];

debug_log "report_spam executed ${1}";

if environment :matches "imap.user" "*" {
  # to use a global user: 
  #set "username" “amavis”;
  set "username" "${1}";
}

pipe :copy "sa-learn-spam.sh" [ "${username}" ];

/var/vmail/sieve/report-ham.sieve

require ["vnd.dovecot.debug", "vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];

debug_log "report_ham executed ${1}";

if environment :matches "imap.mailbox" "*" {
  set "mailbox" "${1}";
}

if string "${mailbox}" "Trash" {
  stop;
}

if environment :matches "imap.user" "*" {
  # to use a global user: 
  #set "username" “amavis”;
  set "username" "${1}";
}

pipe :copy "sa-learn-ham.sh" [ "${username}" ];

3. Create sa-learn executables

/var/vmail/sieve/sa-learn-spam.sh

exec /usr/bin/sa-learn -u ${1} --spam

vi /var/vmail/sieve/sa-learn-ham.sh

exec /usr/bin/sa-learn -u ${1} --ham

4. Change permissions

chown vmail:vmail /var/vmail/sieve/report-*
chown vmail:vmail /var/vmail/sieve/sa-learn-*
chmod +x /var/vmail/sieve/report-*
chmod +x /var/vmail/sieve/sa-learn-*

----- old part -----


5. Edit the local.cf from spamassassin: (NOTICE: Write down the "Data you Setting for MYSQL connection")

use_bayes          1
bayes_auto_learn   1
bayes_auto_expire  1

# Store bayesian data in MySQL
bayes_store_module Mail::SpamAssassin::BayesStore::MySQL
bayes_sql_dsn      DBI:mysql:(db):(host):(port)

# Store bayesian data in MySQL
#bayes_store_module Mail::SpamAssassin::BayesStore::PgSQL
#bayes_sql_dsn      DBI:Pg:database:sql_server:sql_port
#
bayes_sql_username (user)
bayes_sql_password (password)
#
# Override the username used for storing
# data in the database. This could be used to group users together to
# share bayesian filter data. You can also use this config option to
# trick sa-learn to learn data as a specific user.
#
#bayes_sql_override_username vmail

Please change the Plaseholder in : (db), (host), (port), (user), (password)

6. create mysql user, db and scheme for spamassassin bayes

# cd /root/
# wget [url]http://svn.apache.org/repos/asf/spamassassin/tags/spamassassin_release_3_3_2/sql/bayes_mysql.sql[/url]

### NOTICE /root/bayes_mysql.sql must be modded TYPE => ENGINE for correct mysql syntax.


### CREATE DB for SA_BAYES AND IMPORT SCHEME.

# mysql -uroot -p
mysql> CREATE DATABASE sa_bayes;
mysql> USE sa_bayes;
mysql> SOURCE /root/bayes_mysql.sql;

### CREATE USER AND GIVE ACCESS (USERDATA NEEDED IN LOCAL.CF for SETTING UP SPAMASSASSIN)
mysql> GRANT SELECT, INSERT, UPDATE, DELETE ON sa_bayes.* TO sa_user@localhost IDENTIFIED BY 'sa_user_password';
mysql> FLUSH PRIVILEGES;
mysql> EXIT;

7. RESTART DOVECOT, AMAVIS(-NEW)

# systemctl restart  dovecot
# systemctl restart amavisd

8. Done.
Now all emails that are moved to the Junk folder, through IMAP will be pushed to sa-learn --spam. If you get a ham mail in Junk/Spam that is actually not spam, move it to another folder like Inbox and it will be learned as --ham.

-----

Optional:
If you like to push all marked spam mails to sa-learn too, you can modify the following

/var/vmail/sieve/dovecot.sieve

require ["fileinto", "vnd.dovecot.debug", "vnd.dovecot.pipe", "copy", "environment", "variables"];

# rule:[Move Spam to Junk Folder]
if header :is "X-Spam-Flag" "YES"
{
    fileinto "Junk";
    set "username" "amavis";
    pipe :copy "sa-learn-spam.sh" [ "${username}" ];
}

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 (edited by rrosson 2017-11-17 01:54:36)

Re: User based antispam via Dovecot and Spamassassian (update)

Just an FYI. Looks like us Ubuntu users will have to wait till Ubuntu pushes out a newer version than 2.2.22. The above steps require dovecot 2.2.24 or later.

Reverting back to @sherwood earlier mod I had in place before til this can occur.  sad

-Ron

3

Re: User based antispam via Dovecot and Spamassassian (update)

Thanks very much for sharing. We will merge it in iRedMail future release (on linux/bsd distributions which offers required Dovecot version).

DISCUSSION:

We implemented same feature with old 'antispam' plugin before, but we save a copy of spam/clean emails to another directory instead of call 'sa-learn' immediately.

During our testing, if user moves multiple emails at the same time, e.g. 30, the Roundcube webmail will hang/freeze for few seconds, the more emails user moves at the same time, the longer it freezes. This is not ideal.

If we save a copy of moved messages, and run a shell script (via cron) to scan them with 'sa-learn', it's very fast. So it's possible that when we implement this feature, we will use same settings (save first, scan every X minutes and delete scanned messages).

4

Re: User based antispam via Dovecot and Spamassassian (update)

Hello, is there the possibility of modifying the script sa-learn-spam.sh and sa-learn-ham.sh so that they also run wblist_admin.py?
It would be amazing, that when moving an email to the Junk folder, wblist_admin.py add an entry for blacklist that address (only for the user to do so).
Also, when moving an email from Junk to INBOX, wblist_admin.py put that address in whitelist for the user who does it.

5

Re: User based antispam via Dovecot and Spamassassian (update)

User may move email to Junk due to message content, not sender address, so i think running wblist_admin.py might be too risky in this case.

6

Re: User based antispam via Dovecot and Spamassassian (update)

Hi Zhang, thank you for your response.
But in the event that a user "makes a mistake" and moves a message to the SPAM folder, he could simply move it back to INBOX.
It's very difficult for normal users to enter a panel and modify the rules of wblist_admin.py, if it's not in a simple way, they will not use it.
It would be very complicated to modify the scripts to include wblist_admin.py?
For bayes to start being useful, it takes a lot of time (plus user's help to move messages), and also need to use a high score in Spamassassin, which is not good either. Other systems such as Gmail somehow "mark" the message, and you need to just move it once or twice to the SPAM folder and it will never appear again in INBOX, with Bayes this does not happen. I think Gmail uses the address of the sender, just as wblist_admin.py would be doing as I say.

7

Re: User based antispam via Dovecot and Spamassassian (update)

Afro wrote:

It would be very complicated to modify the scripts to include wblist_admin.py?

Not complicate. The problem is, 'sa-learn-spam.sh' is ran by Dovecot daemon user or "vmail" user, but script 'wblist_admin.py' should be ran as 'root'. If you can work out an alternative script, it should be fine.

8

Re: User based antispam via Dovecot and Spamassassian (update)

ZhangHuangbin wrote:
Afro wrote:

It would be very complicated to modify the scripts to include wblist_admin.py?

Not complicate. The problem is, 'sa-learn-spam.sh' is ran by Dovecot daemon user or "vmail" user, but script 'wblist_admin.py' should be ran as 'root'. If you can work out an alternative script, it should be fine.

Any approach or help to do something like that? I'm not a code expert.
Thank you!