Topic: Suggestion to block more junk and alleviate iRedMail load
==== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.2
- Linux/BSD distribution name and version: Ubuntu 14.04 LTS
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): Both
- Manage mail accounts with iRedAdmin-Pro? Yes
- Related log if you're reporting an issue:
Zhang, I would like to share a couple of things for your consideration and I'll share a little of my experience while managing iRedMail boxes to do so.
One of my mailservers (running latest iRedMail as of this post) was struggling to keep up with the load due to resources limitations. There is an easy fix for that (to some extend); to add more CPUs, RAM and faster drives IF possible. The problem is that upgrading is not always an option and there are many things that can be done to better handle iRedMail server load as in my case.
First, for me the auto-configuration which sets max_servers and smtp-amavis to 4 by default is a little to high. It is my opinion (based on my own usage) that these processes shouldn't be set to more than 2 out of the box. The main reasons being that there is really an additional load on the server which might not be necessary unless there are lots of transactions daily (meaning really busy box). This is absolutely debatable and millage may vary, once again, depending on how busy the mailer is. In my case, I'm doing little over 10,000 (clean) email transactions and 2 processes has been more than enough to keep the queue empty almost 100% of the time and the server benefited from the freed resources (specifically RAM).
The other point is postfix postscreen; it needs to make it to iRedMail by default. I really haven't followed the development much so it might be the case that it is already planned. Regardless, the point is that postfix postscreen along with postfix additional checks will help you reduce the load quite a lot in your server… I recovered 1GB of RAM with minor changes.
I'll try to be as straight forward as possible:
1- Reduce max_servers and smtp-amavis to 2 processes. (mentioned above)
2- Enable postscreen. You have a guide available => http://www.iredmail.org/docs/enable.postscreen.html (mentioned above)
3- Warn about the repercussions of adding large amount of records to amavis blacklist:
Adding blacklist records to amavis can really make it slower and resources intensive. I had a list of over 1200 IP blocks blocked there and amavis was literally ripping my box. If another admin happens to host heavily spammed users like some of the ones I host, the problem can grow out of proportions quite fast because amavis has to query the database just a little too much (small VPSs and limited resources boxes have a greater impact here).
The solution: The solution was simple and I wish that this was part of iRedMail by default for those less familiar with administering MailServers (including myself); postfix. Shifting a whole more to postfix instead can make a day and night difference. Postfix uses very little resources compared to amavis and iRedMail default configuration doesn't take any actions up until messages RCPT with smtp_recipient_restrictions anyways as it should.
So, what I wanted to recommend is to include sender_access, header_checks and body_checks by default in iRedMail. A pre-configured setup that doesn't have to carry much configuration.
They could be empty or have some basic (useful regex) to give admins with less experience an idea on how to work with them.
In my case offloading work from amavis to postfix made a huge difference and this could be beneficial to others. I have sender_control and sender_access in smtp_recipient_restrictions like everything else in default iRedMail although postfix does not recommend this as best practice due to the relaxed permissions, but together they seem to do the work beautifully and there is a lot of messages that don't have to be passed back to amavis for processing, which in turn reduces the load on the server substantially since spam is in big part blocked right at the front door:
File /etc/postfix/postscreen_access.cidr
check_sender_access pcre:/etc/postfix/sender_control
check_sender_access hash:/etc/postfix/sender_access (postmap)
header_checks = regexp:/etc/postfix/header_checks (postmap)
body_checks = regexp:/etc/postfix/body_checks (postmap)
Examples: (Please suggest regex rules)
/etc/postfix/postscreen_access.cidr
#### Rules are evaluated in the order as specified.
#
#1.2.3.4 permit
#2.3.4.5 reject
#
# Permit IPs
1.2.3.4 permit
8.9.0.0/16 permit
#
# Reject Ips
#
5.6.7.0/24 reject
9.8.0.0/16 reject
#
#### postscreen_access.cidr ENDcheck_sender_access pcre:/etc/postfix/sender_control
#### Domians listed under ACCEPT section take precedence over REJECT. For example .us and .info TLDs are rejected under REJECT TLDs section but the following two domains are permitted:
#
# Use ACCEPT section to whitelist domains
#
# ACCEPT from
#
/@domain\.us$/ OK
/@domain2\.info$/ OK
#
#### Guide
#
# reject_unknown_reverse_client_hostname - rejects the email if the IP of the sending server does not have a reverse DNS (ptr) record.
# reject_invalid_helo_hostname - rejects the email if the sending server uses invalid characters in the helo/ehlo host name.
# reject_non_fqdn_helo_hostname - rejects the email if the sending server does not use a fully qualified domain name as the helo/ehlo host name. Bad: "mxsrv-13". Good: "mxsrv-13.foo.com"
# reject_unknown_helo_hostname - rejects the email if the sending server uses a helo/ehlo host name that does not resolve to a public IP address. Bad: "mx1.foo.local". Good: "mx1.foo.com"
# reject_unknown_client_hostname - rejects the email if the IP address and host name of the sending server does not have Forward Confirmed reverse DNS (FCrDNS), sometimes called full-circle DNS.
#
/aol\.com$/ reject_unknown_helo_hostname,reject_unknown_client_hostname
/amazon\.com$/ reject_unknown_helo_hostname,reject_unknown_client_hostname
/(apple|mac|me|icloud)\.com$/ reject_unknown_helo_hostname,reject_unknown_client_hostname
/comcast\.net$/ reject_unknown_helo_hostname,reject_unknown_client_hostname
/cox\.com$/ reject_unknown_helo_hostname,reject_unknown_client_hostname
/(facebook|facebookmail)\.com$/ reject_unknown_helo_hostname,reject_unknown_client_hostname
/(gmail|google)\.com$/ reject_unknown_helo_hostname,reject_unknown_client_hostname
/(bing|hotmail|live|msn|microsoft)\.com$/ reject_unknown_helo_hostname,reject_unknown_client_hostname
/rr\.com$/ reject_unknown_helo_hostname,reject_unknown_client_hostname
/namecheap\.com$/ reject_unknown_helo_hostname,reject_unknown_client_hostname
/netflix\.com$/ reject_unknown_helo_hostname,reject_unknown_client_hostname
/dropbox\.com$/ reject_unknown_helo_hostname,reject_unknown_client_hostname
#
#### REJECT TLDs
#
# REJECT from
#
/\.asia$/ REJECT Mail from .asia not accepted
/\.accountant$/ REJECT Mail from .accountant not accepted
/\.bid$/ REJECT Mail from .bid not accepted
/\.cc$/ REJECT Mail from .cc not accepted
/\.cn$/ REJECT Mail from .cn not accepted
/\.cricket$/ REJECT Mail from .cricket not accepted
/\.date$/ REJECT Mail from .date not accepted
/\.download$/ REJECT Mail from .download not accepted
/\.eu$/ REJECT Mail from .eu not accepted
/\.faith$/ REJECT Mail from .faith not accepted
/\.gr$/ REJECT Mail from .gr not accepted
/\.help$/ REJECT Mail from .help not accepted
/\.info$/ REJECT Mail from .info not accepted
/\.jp$/ REJECT Mail from .jp not accepted
/\.link$/ REJECT Mail from .link not accepted
/\.loan$/ REJECT Mail from .loan not accepted
/\.my$/ REJECT Mail from .my not accepted
/\.name$/ REJECT Mail from .name not accepted
/\.ninja$/ REJECT Mail from .ninja not accepted
/\.party$/ REJECT Mail from .party not accepted
/\.pw$/ REJECT Mail from .pw not accepted
/\.racing$/ REJECT Mail from .racing not accepted
/\.party$/ REJECT Mail from .party not accepted
/\.review$/ REJECT Mail from .review not accepted
/\.ro$/ REJECT Mail from .ro not accepted
/\.science$/ REJECT Mail from .science not accepted
/\.sg$/ REJECT Mail from .sg not accepted
/\.space$/ REJECT Mail from .space not accepted
/\.uno$/ REJECT Mail from .uno not accepted
/\.top$/ REJECT Mail from .top not accepted
/\.trade$/ REJECT Mail from .trade not accepted
/\.us$/ REJECT Mail from .us not accepted
/\.vn$/ REJECT Mail from .vn not accepted
/\.wang$/ REJECT Mail from .wang not accepted
/\.webcam$/ REJECT Mail from .webcam not accepted
/\.win$/ REJECT Mail from .win not accepted
/\.work$/ REJECT Mail from .work not accepted
/\.xyz$/ REJECT Mail from .xyz not accepted
#
#### sender_control ENDcheck_sender_access hash:/etc/postfix/sender_access
#### Options: OK or REJECT + comment
#
# Domians listed in ACCEPT take precedence over REJECT
#
# ACCEPT from
friendlydomain.com OK
gooddomain.net OK
#
# Now REJECT from
#
baddomain.it REJECT Blacklisted
spamydomain.xyz REJECT Blacklisted
#
#### sender_access ENDheader_checks = regexp:/etc/postfix/header_checks (postmap)
#### Check headers
#
# Checks are done in order, top to bottom.
#
# IGNORE will cause the particular header to be removed
#
/^Received:.*with ESMTPSA/ IGNORE
/^X-Originating-IP:/ IGNORE
/^X-Mailer:/ IGNORE
/^Mime-Version:/ IGNORE
/^User-Agent:/ IGNORE
#
#### REJECT non-RFC Compliance
#
/[^[:print:]]{7}/ REJECT RFC2047
/^.*=20[a-z]*=20[a-z]*=20[a-z]*=20[a-z]*/ REJECT RFC822
/(.*)?\{6,\}/ REJECT RFC822
/(.*)[X|x]\{3,\}/ REJECT RFC822
#
#### REJECT unreadable NON-acsii un-printable text
/^Subject:.*=\?(GB2312|big5|euc-kr|ks_c_5601-1987|koi8)\?/ REJECT Unreadable
/^Content-Type:.*charset="?(GB2312|big5|euc-kr|ks_c_5601-1987|koi8|iso-2022-jp)/ REJECT Unreadable
#
#### Check and REJECT these Subject
/^Subject:.* / REJECT Space
/^Subject:.*r[ _\.\*\-]+o[ _\.\*\-]+l[ _\.\*\-]+e[ _\.\*\-]+x/ REJECT Hidden Words
/^Subject:.*p[ _\.\*\-]+o[ _\.\*\-]+r[ _\.\*\-]+n/ REJECT Hidden Words
#
#### Character Set Checks
/^(Content-Type:.*|\s+)charset\s*=\s*"?(Windows-1251)\?/ REJECT Bad Content Type
#
#### Attachments
/^Content-(Type|Disposition):.*(file)?name=.*\.(ade|adp|asd|asf|asx|bat|bhx|chm|cil|cmd|com|cpl|dll|elm|exe|gif|hlp|hta|jse|lnk|mda|mdb|mde|mdw|mim|msi|msp|nws|ocx|pif|reg|scr|sct|shb|shm|shs|vb|vbe|vbs|vbx|vxd|wmf|wms|wmz|wmd|wsc|wsf|wsh|wsz)/ REJECT Bad Attachment .${3}
#
# Backscatter mail from virus scanners
#
/^Subject:.*Anti-Virus Notification/ REJECT Virus Notification
/^Subject:.*due to virus/ REJECT Virus Notification
/^Subject:.*email contains VIRUS/ REJECT Virus Notification
/^Subject:.*InterScanMSS/ REJECT Virus Notification
/^Subject:.*ScanMail for Lotus/ REJECT Virus Notification
/^Subject:.*Symantec AntiVirus/ REJECT Virus Notification
/^subject:.*virus found/ REJECT Virus Notification
#
#### Check and don't accept any of these
#
/^Subject:.*(VP[-]?RX|[vV][ j_\-]?[iI1][ j_\-]?[aA4@][ j_\-]?[gG][ j_\-]?[rR][ j_\-]?[aA4@])/ REJECT Message header rejected [073]
/c[i1]al[i1]s/ REJECT cialis
/Tramadol/ REJECT tramadol
/Valium/ REJECT valium
/Xanax/ REJECT xanax
#
##### Check and match messages with a numeric message ID in subject line.
#
if /^Subject:.*(ID|MSG|ID MSG|MSG ID).*:[ ]?([0-9]{5}) /
#
# Dating
/[a-zA-Z]* is online now/ REJECT Message header rejected [200x]
/[a-zA-Z]* sent new (message[s]?|mail) from/ REJECT Message header rejected [201x]
/[yY]ou have ([0-9]{1,4})?[ ]?(unread|new) (message[s]?|mail)[ ]?(from)?[ ]?([a-zA-Z]*)?[ ]?(for you)?[ ]?(from)?/ REJECT Message header rejected [202]
/([0-9]{1,4}) (single|new|lonely)?[ ]?ladies/ REJECT Message header rejected [203]
/[iI] am [a-zA-Z]*?[ ,\.]?[ ]?([0-9]{1,4}) (y\.o\.|year[s]?)/ REJECT Message header rejected [204]
/[sS]till (lonely|single)\?/ REJECT Message header rejected [205x]
/[rR]emember [mM]e[ ?\.]/ REJECT Message header rejected [206x]
#
# Meds
/[gG]et all your [mM][eE3][dD][sz]?([iI\|1][cC][aA4@][tT][iI\|1][oO0][nN])?/ REJECT Message header rejected [301xx]
/[pP]harma(cy|ceutical)/ REJECT Message header rejected [302x]
/[oO]nline ([mM][eE3][dD][sz]?([iI\|1][cC][aA4@][tT][iI\|1][oO0][nN])?|prescription-free)/ REJECT Message header rejected [303]
/[nN]ever have to (see|visit) (a|the) [dD]octor[s ]?/ REJECT Message header rejected [304]
/[Rr]ange [a-zA-Z]* [mM][eE3][dD][sz]?([iI\|1][cC][aA4@][tT][iI\|1][oO0][nN])?/ REJECT Message header rejected [305]
/([tT]he )?[wW]orld[.]?s.*[oO]nline [mM][eE3][dD][sz]?([iI\|1][cC][aA4@][tT][iI\|1][oO0][nN])?/ REJECT Message header rejected [306x]
endif
#
# header_checks ENDbody_checks = regexp:/etc/postfix/body_checks
# Check message body
#
#### First skip over base 64 encoded text to save CPU cycles.
#
~^[[:alnum:]+/]{60,}$~ OK
#
#### BODY FILTER MAP
/Advance Amount: 1,475.00/ REJECT Sender considered spammer
/amount of 1475/ REJECT Sender considered spammer
#
# viagra
#
/(VP[\-]RX|[vV][ j!\-\.]?[iI1][ j!\-\.]?[aA4@][ !j\-\.]?[gG][ !j\-\.]?[rR][ !j\-\.]?[aA4][ !j\-\.]?)/ REJECT Message body rejected [301]
#
# valium
#
/[vV][j\\-\.! ]?[aAA4@][j\\-\.! ]?[lL|1][j\\-\.! ]?[iI\|1][j\\-\.! ]?[uU][j\\-\.! ]?[mM][j\\-\.! ]?/ REJECT Message body rejected [401]
#
# levitra
#
/[lL\|1][j\\-\.! ]?[eE][j\\-\.! ]?[vV][j\\-\.! ]?[iI\|1][j\\-\.! ]?[tT][j\\-\.! ]?[rR][j\\-\.! ]?[aA][j\\-\.! ]?/ REJECT Message body rejected [303]
#
# ciallis
#
/[cC][j\\-\.! ]?[iI\|1][j\\-\.! ]?[aA4@][j\\-\.! ]?[aA]?[lL\|1]*[j\\-\.! ]?[iI\|1][j\\-\.! ]?[sS5][j\\-\.! ]/ REJECT Message body rejected [304]
#
#MaxGain+
#
/[mM][-\. ]?[aA4][-\. ]?[xX][-\. ]?[gG][-\. ]?[aA4][-\. ]?[iI1][-\. ]?[nN][+]?[!\.\? ]?/ REJECT Message body rejected [210]
#
# Replica goods - watches, shoes.
#
/[wW]atch(es)? [rR]eplica[!\.,\?s ]?/ REJECT Message body rejected [013]
#/[rR]eplica(s)? [wW]atch(es)?|[lL]eather|[sS]hoes|[bB]oots|[fF]ootwear[\.\? ]?/ REJECT Message body rejected [049]
/([pP]opular|[eE]xquisit[e])? [rR]eplica(s)?[\.\? ]?/ REJECT Message body rejected [055]
/[lL]ux(ury|urious|)[\., ]? ([lL]eather|[sS]hoes|[bB]oots|[fF]ootwear)[!\.\? ]/ REJECT Message body rejected [043]
#
# caught naked
/[cC]aught( |you|me)?naked[\.\? ]?/ REJECT Message body rejected [038]
#
#### body_checks ENDI hope this helps. Please feel free to improve it or scrutinize my post as long as you have better suggestions to help each other 
----
Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.