Topic: [HOW-TO]DSPAM and SPAM Training
##########################################################################################
######################### DSPAM is Deprecated #########################
#########################################################################################
Below Topic id for history, if you need to train your spam :
See this : http://www.iredmail.org/forum/topic8169 … assin.html
==== Required information ====
- iRedMail version: 0.8.6
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): Mysql
- Linux/BSD distribution name and version: Debian 7.4
- Related log if you're reporting an issue: none
====
I All, First off all thanks for iredmail, it's great and quite easy to use and manage.
But, As non spam lover I want to have DSPAM as complement for spam filtering.
So Here Is a HowTo that worked for me, with debian and mysql backend.
Compilation
aptitude install libmysqlclient-dev
cd ~
wget <DSPAM TAR.GZ ADDRESS>
tar xvzf dspam-3.10.2.tar.gz
cd dspam-3.10.2
./configure --prefix=/usr --disable-dependency-tracking --includedir=/usr/include --with-logdir=/var/log/dspam/ --with-dspam-home=/var/amavis/dspam --sysconfdir=/etc/ --enable-domain-scale --without-delivery-agent --with-mysql-includes=/usr/include/mysql --with-storage-driver=mysql_drv --enable-virtual-users --enable-preferences-extension --enable-daemon --enable-debug
make
make installAlternatives apt-get install dspam
Base de données
mysql -u root -p
create database dspam;
GRANT SELECT, INSERT, UPDATE, DELETE ON dspam.* TO 'dspam'@'localhost' IDENTIFIED BY '******';
FLUSH PRIVILEGES;
quitcd ~
wget /spip/doc/Procmail/dspam/mysql_objects-4.1.sql
mysql -u root -p dspam < mysql_objects-4.1.sqlConfiguration
cd /etc/
mv dspam.conf dspam.conf-orig
wget /spip/doc/Procmail/dspam.conf
sed -i 's/\/var\/amavis\/dspam/\/var\/lib\/amavis\/dspam/g' dspam.conf
sed -i 's/\*\*\*\*\*\*/toto/g' dspam.confcd /etc/init.d/
wget /spip/doc/Procmail/init.d/dspam
chmod +x dspam
insserv -v /etc/init.d/dspamPermissions
chown amavis: /usr/bin/dspam*
chown amavis: /etc/dspam.conf
chmod 750 /usr/bin/dspam*
chmod 640 /etc/dspam.conf
chown amavis: /var/log/dspam
chown -R amavis: /var/lib/amavis/dspamAmAviS
Ajouter ceci à /etc/amavis/conf.d/50-user
$sa_tag_level_deflt = 10
$dspam = 'dspam';
@spam_scanners = (
['SpamAssassin', 'Amavis::SpamControl::SpamAssassin'],
['DSPAM', 'Amavis::SpamControl::ExtProg', $dspam,
[ qw(--client --stdout --process --deliver=innocent,spam
--user), $daemon_user ],
score_factor => 0.39,
],
);Parametre DSPAM
dspam_admin change preference default "dailyQuarantineSummary" "off"
dspam_admin change preference default "enableBNR" "on"
dspam_admin change preference default "enableWhitelist" "on"
dspam_admin change preference default "fallbackDomain" "off"
dspam_admin change preference default "ignoreGroups" "off"
dspam_admin change preference default "ignoreRBLLookups" "off"
dspam_admin change preference default "makeCorpus" "off"
dspam_admin change preference default "optIn" "off"
dspam_admin change preference default "optOut" "on"
dspam_admin change preference default "optOutClamAV" "on"
dspam_admin change preference default "processorBias" "on"
dspam_admin change preference default "showFactors" "off"
dspam_admin change preference default "signatureLocation" "headers"
dspam_admin change preference default "spamAction" "tag"
dspam_admin change preference default "spamSubject" "[SPAM]"
dspam_admin change preference default "statisticalSedation" "6"
dspam_admin change preference default "storeFragments" "off"
dspam_admin change preference default "tagNonspam" "off"
dspam_admin change preference default "tagSpam" "off"
dspam_admin change preference default "trainingMode" "TOE"
dspam_admin change preference default "trainPristine" "off"
dspam_admin change preference default "whitelistThreshold" "10"Id off user can be found via : id -u username
mysql -u root -p dspam
INSERT INTO `dspam_virtual_uids` (`uid`, `username`) VALUES
(108, 'amavis');Activating Vuser
dspam_admin change preference default "optIn" "on"
dspam_admin change preference default "optOut" "off"Demarrage
/etc/init.d/dspam start
We check : ps aux | grep dspam
Amavis restart
/etc/init.d/amavis restart
We check
cat /var/log/mail.info | grep DSPAM
Maintenance
cd /var/lib/amavis/
wget /spip/doc/Procmail/dspam-purge-4.1.sql
wget /spip/doc/Procmail/caremail/cron/dspam-purgesql.sh
sed -i 's/\*\*\*\*\*\*/toto/g' dspam-purgesql.shDovecot Folders
You need to replace in /etc/dovecot/dovecot.conf this parts :
namespace {
type = private
separator = /
prefix =
#location defaults to mail_location.
inbox = yes
}By this parts
namespace {
type = private
separator = /
prefix =
#location defaults to mail_location.
inbox = yes
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Spam {
auto = subscribe
#special_use = \Spam
}
mailbox SpamFalse {
auto = subscribe
#special_use = \SpamFalse
}
}Next thing : You have to comment out all "autocreate =" and "autosubscribe =" lines, AND remove "autocreate" plugin name in "mail_plugins =" (both "protocol lda {}" and "protocol imap {}" blocks).
and restart Dovecot, on login All users will now have all folder listed above.
mail adresses
Create by iredAdmin, iredAdminPro, mysql, or postfix-admin to virtual users,spam@domain.tld and ham@domain.tld give them some space but they won't need to much, since message won't stay there.
Script
sa_trap.sh
This one, go in all mailboxes, copy email in a folder, than add all email found in HAM and SPAM directory to the 2 special adresses spam@domain.tld and ham@domain.tld
#!/bin/sh
mkdir -p /var/tmp/spamtrap/cur
mkdir /var/tmp/spamtrap/new
chown -R vmail: /var/tmp/spamtrap
chown -R vmail: /var/tmp/hamtrap
doveadm search -u "*@*" mailbox Spam ALL |
while read guser guid uid; do
doveadm fetch -u "*@*" text mailbox-guid $guid uid $uid > /var/tmp/spamtrap/cur/msgspam.$uid
## Want to not expunge quote following line
doveadm expunge -u "*@*" mailbox Spam mailbox-guid $guid uid $uid
done
doveadm search -u "*@*" mailbox SpamFalse ALL |
while read guser guid uid; do
doveadm fetch -u "*@*" text mailbox-guid $guid uid $uid > /var/tmp/hamtrap/cur/msgham.$uid
doveadm expunge -u "*@*" mailbox SpamFalse mailbox-guid $guid uid $uid
done
chown -R vmail: /var/tmp/spamtrap
chown -R vmail: /var/tmp/hamtrap
doveadm -v import -u ham@domain.tld maildir:/var/tmp/hamtrap "" mailbox INBOX all
doveadm -v import -u spam@domain.tld maildir:/var/tmp/spamtrap "" mailbox INBOX all
rm /var/tmp/spamtrap/cur/msgspam*
rm /var/tmp/hamtrap/cur/msgham*sa_learn.sh
This one give all messages as SPAM or HAM to spamassassin and DSPAM
#!/bin/bash
# Script par tonio AT starbridge DOT org. (28-07-2012)
DATE=`date '+%s'`
doveadm search -u "spam@domain.tld" mailbox INBOX ALL |
while read guid uid; do
doveadm fetch -u "spam@domain.tld" text mailbox-guid $guid uid $uid > /home/spam/msgspam-$DATE.$uid
#Expuging , but kepping copy in backup.
doveadm expunge -u "spam@domain.tld" mailbox INBOX mailbox-guid $guid uid $uid
done
doveadm search -u "ham@domain.tld" mailbox INBOX ALL |
while read guid uid; do
doveadm fetch -u "ham@domain.tld" text mailbox-guid $guid uid $uid > /home/ham/msgham-$DATE.$uid
doveadm expunge -u "ham@domain.tld" mailbox INBOX mailbox-guid $guid uid $uid
done
chown amavis: /home/spam/* > /dev/null 2>&1
chown amavis: /home/ham/* > /dev/null 2>&1
chmod 666 /home/spam/* > /dev/null 2>&1
chmod 666 /home/ham/* > /dev/null 2>&1
su -c "/usr/bin/sa-learn -D --username='vmail' --spam /home/spam/" amavis
su -c "/usr/bin/sa-learn -D --username='vmail' --ham /home/ham/" amavis
#########DSPAM training########################
echo "Dspam Training"
#SPAM
#on fait le retrain uniquement si le tag est sur Innocent
/usr/bin/find /home/spam/ -type f -exec grep -l "X-DSPAM-Result: Innocent" {} \; | while read foo1
do
su -c "/usr/bin/dspam --client --user amavis --class=spam --source=error < '${foo1}'" amavis
done
#HAM
/usr/bin/find /home/ham/ -type f -exec grep -l "X-DSPAM-Result: Spam" {} \; | while read foo3
do
su -c "/usr/bin/dspam --client --user amavis --class=innocent --source=error < '${foo3}'" amavis
done
#
#########End DSPAM training########################
mv /home/spam/* /home/spam-backup/ > /dev/null 2>&1
mv /home/ham/* /home/ham-backup/ > /dev/null 2>&1
chown vmail: /home/spam-backup/* > /dev/null 2>&1
chown vmail: /home/ham-backup/* > /dev/null 2>&1
chmod 600 /home/spam-backup/* > /dev/null 2>&1
chmod 600 /home/ham-backup/* > /dev/null 2>&1
su -c "/usr/bin/sa-learn --backup" amavis > /var/vmail/bayesrulesReplace all domain.tld by yours.
Crontab
add this line after the mysql backup in crontab root user :
# SA learn from spamtrap every 4h
0 */4 * * * /bin/bash /var/vmail/sa_trap.sh
30 */4 * * * /bin/bash /var/vmail/sa_learn.shsources
/spip/spip.php?article12
Because off Link limitation in post, please replace all missing domains by w.w.w.starbridge.o.r.g
You need to train DSPAM for your needs, and as simple way to do it, I want to make 2 folder in all users mailboxes.
I thought it was needed to add in /etc/dovecot/conf.d/15-mailboxes the 2 folder I want to add, like this :mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { #special_use = \Spam auto=subscribe } mailbox SpamFalse { #special_use = \SpamFalse auto=subscribe } # If you have a virtual "All messages" mailbox: #mailbox virtual/All { # special_use = \All #}Unfortunately it don't seem to work this way, conf.d folder don't even appears to be linked in dovecot.conf
So i added them in the /etc/dovecot/dovecot.conf file with the autocreate, but as I goggled this solution it seem deprecated.
And, must off all It didn't create folder where mailbox already exists.So where i can add folder to existing mailboxes ?
My goal is, that spam who pass the filter (and vice-versa), can be moved by users in the "Spam" folder, which is periodically scanned by a script ( /spip/doc/Procmail/caremail/cron/sa-trap-sdbox like this one, adjust to the needs) who take email off all user to put them in spam@ or ham@. Those spam and ham address are after treated by another script ( /spip/doc/Procmail/caremail/sa-dspam-learn-sdbox ) which train dspam and spamassassin.
Those script run in crontab every X hours suited to your needs.Is there requirements that could not be fit for iredmail ? Because this solution is way better for those who didn't use the webmail, and especially when it comes to explain to the 50years old technological refractors secretary how to handle spam...
DSPAM because the volume of spam that spamassassin let go is really outstanding (but working).
Please pardon my english im trying my best.
----
Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.