iRedMail — [Tutorial] Increasing ClamAV effectiveness

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (Quack66) — Fri, 04 Sep 2020 14:20:17 +0000

I now just realized that a lot of the settings now ships by default with iredmail ! Thanks Zhang !

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (Quack66) — Fri, 04 Sep 2020 14:10:38 +0000

Updated for iredmail 1.3.1 and Ubuntu 20.01 !

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (Quack66) — Wed, 31 May 2017 18:45:30 +0000

Updates:
- Removed SecurityInfo from the list of DBs since they are no longer hosted on SaneSecurity. Check their website here to add their DB. You will need to register

- Removed Malware patrol since the you need to register for free version and they are no longer hosted on SaneSecurity

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (ZhangHuangbin) — Mon, 29 May 2017 05:30:04 +0000

awkpavo wrote:

Would also be nice to an alternative for the Sanesecurity hosted signatures. Just so you have a choice once Sanesecurity decides to charge for it, etc.

Copying the Sanesecurity files and hosting on your own server doesn't change the fact: you use their work, but return nothing.

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (martinveasey) — Fri, 26 May 2017 13:56:16 +0000

Quack66 wrote:

======================== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.6
- Linux/BSD distribution name and version: Ubuntu 16.04 LTS
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): PGSQL
- Web server (Apache or Nginx): Apache
- Manage mail accounts with iRedAdmin-Pro? Yes
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====
As you probably know, Clamav is fine as an AV but is not on-par with paid commercial product and sometime viruses can slip through.

Thanks to Quack66, the OP, for this tutorial - it worked well for me !

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (awkpavo) — Thu, 25 May 2017 13:41:44 +0000

Would also be nice to an alternative for the Sanesecurity hosted signatures. Just so you have a choice once Sanesecurity decides to charge for it, etc.

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (Quack66) — Wed, 24 May 2017 12:42:57 +0000

Hey Zhang,

I think those points are all valid. In my opinion the best way to approach this is maybe not to add it to the base but instead add a doc on how to do so with a disclaimer at the top.

Like Serge said, anyone maintaining their own server can integrate this easily.

Regards

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (SergeAv) — Wed, 24 May 2017 11:41:56 +0000

IMHO people who really want to maintain his servers are able to integrate such services by himself. And IMHO your decision is right.

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (ZhangHuangbin) — Wed, 24 May 2017 09:39:10 +0000

According to the Donate page of Sanesecurity website, I'm hesitate to integrate them:
http://sanesecurity.com/donate/

iRedMail has few hundred deployments every day, i believe 99.9% users will NOT notice that Sanesecurity asks for a donation for personal and commercial use (it's free to use, but a donation is appreciated). So our integration brings them nothing, except network traffic and possible workload.

Any suggestions?

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (SergeAv) — Wed, 24 May 2017 07:33:58 +0000

clamdscan command use ClamAV daemon. In my case to enable SaneSecurity signatures to check I was need to set

OfficialDatabaseOnly  no

in clamd.conf
Thanks a lot to topicstarter for a useful idea!

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (ZhangHuangbin) — Wed, 24 May 2017 03:18:29 +0000

Amavisd is configured to pipe email to ClamAV socket, not running clam* commands directly.

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (SergeAv) — Tue, 23 May 2017 14:41:40 +0000

Dear ZhangHuangbin, is IRedMail use the clamscan command to check mail or clamdscan?
In my system (IRedMail-0.9.6, CentOS 6.7, MySQL) it shows different results:

# clamdscan test.msg
/tmp/test.msg: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 0.001 sec (0 m 0 s)

# clamscan test.msg
test.msg: Sanesecurity.TestSig_Type3_Bdy.4.UNOFFICIAL FOUND

----------- SCAN SUMMARY -----------
Known viruses: 6449849
Engine version: 0.99.2
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 21.308 sec (0 m 21 s)

Re: [Tutorial] Increasing ClamAV effectiveness

null@example.com (ZhangHuangbin) — Fri, 19 May 2017 09:01:14 +0000

Thanks for sharing, i will check the tutorials later to see whether we can integrate it into iRedMail directly.

[Tutorial] Increasing ClamAV effectiveness

null@example.com (Quack66) — Thu, 18 May 2017 15:20:47 +0000

Edit: Currently tested and working on iredmail 1.3.1 under Ubuntu 20.01

As you probably know, Clamav is fine as an AV but is not on-par with paid commercial product and sometime viruses can slip through.

There is a way to add custom signatures (Databases) that are maintained by 3rd parties to clam to increase detection rate on viruses and 0-days. This tutorial is how to configure this.

The folks over at SaneSecurity are kind enough to host a bunch of different signatures that can be used with clam. Please do not abuse this service and consider making a donation.

They do offer a script to automatically configure everything but I don't recommend it since it might break your installation.

All the signatures can be found here. I've created this tutorial using the main ones. Feel free to modify/remove or use different signatures.

First, make sure your clamav is setup correctly. Edit the file /etc/clamav/clamd.conf and make sure OfficialDatabaseOnly is set to false
If not then change it and restart clamav

service clamav restart

Then we need to use freshclam to automatically download the third party databases. Open the following file /etc/clamav/freshclam.conf and add this at the end:

 
# Sanesecurity + Foxhole
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/junk.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/jurlbl.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/phish.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/rogue.hdb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/sanesecurity.ftm
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/sigwhitelist.ign2
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/scam.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/spamimg.hdb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/spamattach.hdb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/blurl.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/foxhole_generic.cdb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/foxhole_filename.cdb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/foxhole_js.cdb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/foxhole_js.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/foxhole_all.cdb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/foxhole_all.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/foxhole_mail.cdb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/malwarehash.hsb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/hackingteam.hsb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/badmacro.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/shelter.ldb

# winnow
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/winnow_malware.hdb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/winnow_malware_links.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/winnow_phish_complete_url.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/winnow_extended_malware.hdb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/winnow.attachments.hdb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/winnow_bad_cw.hdb

# Malware.expert
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/malware.expert.hdb

# bofhland
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/bofhland_cracked_URL.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/bofhland_malware_URL.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/bofhland_phishing_URL.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/bofhland_malware_attach.hdb

# Porcupine
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/porcupine.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/phishtank.ndb
DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/porcupine.hsb

You then need to restart freshclam

service clamav-freshclam restart

To make sure the databases were downloaded check your logs in /var/log/clamav/freshclam.log.
You should not have any errors before continuing.

We need to tell amavis to pass the whole message to clam so it could also scan things like the subject.
To do so, open the following file:
/etc/amavis/conf.d/50-user
and find the following lines:

@keep_decoded_original_maps = (new_RE(
    # let virus scanner (clamav) see full original message (can be slow)
    # this setting is required if we're going to use third-party clamav
    # signatures. for example, Sanesecurity signatures.
    # FYI: http://sanesecurity.com/support/signature-testing/
    #qr'^MAIL$',
    qr'^MAIL-UNDECIPHERABLE$', # same as ^MAIL$ if mail is undecipherable
    qr'^(ASCII(?! cpio)|text|uuencoded|xxencoded|binhex)'i,
    #qr'^Zip archive data',     # don't trust Archive::Zip
));

You need to uncomment the following line so just remove the '#' before:

#qr'^MAIL$',

Clam will now detect everything based on our custom signature. The issue is that right now it will not know what to do with it since some signatures are spam, malware or even whitelist. Still in the same file:
/etc/amavis/conf.d/50-user
check if virus_name_to_spam_score_maps already exist. If not add the following before @av_scanners

# Mark Spam/Virus with third-party clamav signatures: SaneSecurity.
#   *) The order matters, first match wins. Set to 'undef' to keep as infected
#   *) Anything declared as undefined will be marked as a virus
@virus_name_to_spam_score_maps =(new_RE(
    # SaneSecurity + Foxhole
    [ qr'^Sanesecurity\.(Malware|Badmacro|Foxhole|Rogue|Trojan)\.' => undef ],
    [ qr'^Sanesecurity\.MalwareHash\.'    => undef ],
    [ qr'^Sanesecurity.TestSig_'          => undef ],
    [ qr'^Sanesecurity\.'                 => 0.1 ],

    # winnow
    [ qr'^winnow\.(Exploit|Trojan|malware)\.'     => undef ],
    [ qr'^winnow\.(botnet|compromised|trojan)'    => undef ],
    [ qr'^winnow\.(exe|ms|JS)\.'                  => undef ],
    [ qr'^winnow\.phish\.'                        => 3.0 ],
    [ qr'^winnow\.'                               => 0.1 ],

    # bofhland
    [ qr'^Bofhland\.Malware\.'                    => undef ],
    [ qr'^BofhlandMWFile'                         => undef ],
    [ qr'^Bofhland\.Phishing\.'                   => 3.0 ],
    [ qr'^Bofhland\.'                             => 0.1 ],

    # porcupine.ndb
    [ qr'^Porcupine\.(Malware|Trojan)\.'          => undef ],
    [ qr'^Porcupine\.(Junk|Spammer)\.'            => 3.0 ],
    [ qr'^Porcupine\.Phishing\.'                  => 3.0 ],
    [ qr'^Porcupine\.'                            => 0.01 ],

    # phishtank.ndb
    [ qr'^PhishTank\.Phishing\.'                  => 3.0 ],

    # SecuriteInfo
    [ qr'^SecuriteInfo\.com\.Spammer\.'           => 3.0 ],

    # Others
    [ qr'^Structured\.(SSN|CreditCardNumber)\b'            => 0.1 ],
    [ qr'^(Heuristics\.)?Phishing\.'                       => 0.1 ],
    [ qr'^(Email|HTML)\.Phishing\.(?!.*Sanesecurity)'      => 0.1 ],
    [ qr'^Email\.Spam\.Bounce(\.[^., ]*)*\.Sanesecurity\.' => 0   ],
    [ qr'^Email\.Spammail\b'                               => 0.1 ],
    [ qr'^MSRBL-(Images|SPAM)\b'                           => 0.1 ],
    [ qr'^VX\.Honeypot-SecuriteInfo\.com\.Joke'            => 0.1 ],
    [ qr'^VX\.not-virus_(Hoax|Joke)\..*-SecuriteInfo\.com(\.|\z)' => 0.1 ],
    [ qr'^Email\.Spam.*-SecuriteInfo\.com(\.|\z)'          => 0.1 ],
    [ qr'^Safebrowsing\.'                                  => 0.1 ],
    [ qr'^INetMsg\.SpamDomain'                             => 0.1 ],
    [ qr'^Doppelstern\.(Spam|Scam|Phishing|Junk|Lott|Loan)'=> 0.1 ],
    [ qr'^ScamNailer\.'                                    => 0.1 ],
    [ qr'^HTML/Bankish'                                    => 0.1 ],
    [ qr'(-)?SecuriteInfo\.com(\.|\z)'                     => undef ],
    [ qr'^MBL_NA\.UNOFFICIAL'                              => 0.1 ],
    [ qr'^MBL_'                                            => undef ],
));

Note: I'm setting the spam score to 30 to make sure they are set to spam even if they have other valid scores. Anything declared as undefined will be marked as a virus

To apply everything we need to restart amavis

service amavis restart

That's it !

To test everything, follow the instructions here. All 3 tests should be marked as Viruses if everything was done correctly.