1

Topic: Throttle notifications?

==== Required information ====
- iRedMail version (check /etc/iredmail-release): 2.8.0
- Linux/BSD distribution name and version: Centos 7
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): mysql
- Web server (Apache or Nginx): nginx
- Manage mail accounts with iRedAdmin-Pro? yes
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

Is there a way to recieve a notification when someone has been throttled?

What is the behavior when they are throttled and how to 'unlock' them?

Thank you!

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: Throttle notifications?

I have same idea for a long time. Let's dive into this:

*) How do you want to be notified? an email?
*) How long do you want to silent the notification for same user? half an hour? one hour? It's annoying to receive a lot emails with same content for same user in a short time.

3

Re: Throttle notifications?

i'm doing this by snmp.
-----
i created a trigger AFTER UPDATE ON `throttle_tracking` - if user reach a 80% of his limit (sended mails) procedure calls function send_to_nms. This trigger can looks like below (it is more complex in production - there is another table to set flags at 80%, 90% and 100% used quota so there can be only 3 traps sended to nms etc.):

declare max_msg_user INT;
select iredapd.throttle.max_msgs into max_msg_user from iredapd.throttle where iredapd.throttle.id = NEW.tid;
if NEW.cur_msgs > 0.8 * max_msg_user then
DO send_to_nms(cast(NEW.id as char));
end if;

next i wrote a user defined function (send_to_nms) for using in mysql to execute external program (mysql do not allow to run external program for security reason) - it is written in c but its very simple - main routine is just starting external bash script in the background, passing to it id of record and immediately return (to not block table).

this external bash script do query to mysql about details: "SELECT throttle_tracking.account, throttle_tracking.cur_msgs, throttle.max_msgs, throttle.period, throttle_tracking.init_time, throttle_tracking.last_time FROM throttle_tracking INNER JOIN throttle ON throttle_tracking.tid = throttle.id WHERE throttle_tracking.id = $1;" and do snmptrap with all this informations to nms.
------

yes i know that there is better option to do this - snmptrap should be called directly from iredapd and as i heard there is possibility to write plugin to iredapd so maybe some day i do it in more elegant way.

------
btw: i saw that iredmail has registered oid so i think that there was an idea to implement snmp into iredmail.
hmmm... it should be done - we are using iredmail and iredadmin pro from some years and i very like this software but lack of seriously monitoring is not good. i must monitoring my mail server and i'm doing this by background tail and parse postfix log so i know what i should to know about statistics of my mail server but it will be nice to configure and do it by iredadmin.
i'm using opennms as a nms and grafana to visualize collected data and in the case of mail server it looks like in attached image (its 5 min rates).
but grafana or similar can be connected with any kind of software - with iredadmin too, so it can be done even without nms and snmp - data could be collected internal by iredmail and can be shown in iredadmin via some graph engine. i think its worth to think about it.
--------
cheers
leszek.

Post's attachments

mail-stats.jpg 295.93 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

4 (edited by beezel 2018-05-08 00:38:20)

Re: Throttle notifications?

ZhangHuangbin wrote:

I have same idea for a long time. Let's dive into this:

*) How do you want to be notified? an email?
*) How long do you want to silent the notification for same user? half an hour? one hour? It's annoying to receive a lot emails with same content for same user in a short time.

Thanks Zhang, great response.

1) Email notification would be plenty. We usually have it go our ITDept@ alias.
2) I think this is where a 'release' or 'unthrottle' button would come in handy. Our current product (that we're migrating away from into iRed) gives us a notification email everytime an account is locked out (they disable the account). You won't get another notification until you unlock the account and they trigger it again.

I am still not sure what your throttle function does, exactly. Does it solely rate limit to the value set, buildling up a queue and slowly releasing them? Or does it block after value set has been reached? We much preferred the method of account disable, so we were immediately aware of the breach (almost always weak passwords that spammers are abusing). It was also a good way of letting the users know that something was wrong - Outlook would immediately start bothering them about their password.

5

Re: Throttle notifications?

leszek wrote:

i'm doing this by snmp.
-----
i created a trigger AFTER UPDATE ON `throttle_tracking` - if user reach a 80% of his limit (sended mails) procedure calls function send_to_nms. This trigger can looks like below (it is more complex in production - there is another table to set flags at 80%, 90% and 100% used quota so there can be only 3 traps sended to nms etc.):

declare max_msg_user INT;
select iredapd.throttle.max_msgs into max_msg_user from iredapd.throttle where iredapd.throttle.id = NEW.tid;
if NEW.cur_msgs > 0.8 * max_msg_user then
DO send_to_nms(cast(NEW.id as char));
end if;

next i wrote a user defined function (send_to_nms) for using in mysql to execute external program (mysql do not allow to run external program for security reason) - it is written in c but its very simple - main routine is just starting external bash script in the background, passing to it id of record and immediately return (to not block table).

this external bash script do query to mysql about details: "SELECT throttle_tracking.account, throttle_tracking.cur_msgs, throttle.max_msgs, throttle.period, throttle_tracking.init_time, throttle_tracking.last_time FROM throttle_tracking INNER JOIN throttle ON throttle_tracking.tid = throttle.id WHERE throttle_tracking.id = $1;" and do snmptrap with all this informations to nms.
------

yes i know that there is better option to do this - snmptrap should be called directly from iredapd and as i heard there is possibility to write plugin to iredapd so maybe some day i do it in more elegant way.

------
btw: i saw that iredmail has registered oid so i think that there was an idea to implement snmp into iredmail.
hmmm... it should be done - we are using iredmail and iredadmin pro from some years and i very like this software but lack of seriously monitoring is not good. i must monitoring my mail server and i'm doing this by background tail and parse postfix log so i know what i should to know about statistics of my mail server but it will be nice to configure and do it by iredadmin.
i'm using opennms as a nms and grafana to visualize collected data and in the case of mail server it looks like in attached image (its 5 min rates).
but grafana or similar can be connected with any kind of software - with iredadmin too, so it can be done even without nms and snmp - data could be collected internal by iredmail and can be shown in iredadmin via some graph engine. i think its worth to think about it.
--------
cheers
leszek.

That is some impressive work! Is any of that sanitary enough to share with the rest of us? We run an ELK stack and would love to get some kinda metrics out of here. Are you just parsing /var/logs/maillog live and readying off of 'status=' ? I have never really done much custom stuff, usually just whatever plugins are offered already. This looks great though!

6

Re: Throttle notifications?

beezel wrote:

2) I think this is where a 'release' or 'unthrottle' button would come in handy. Our current product (that we're migrating away from into iRed) gives us a notification email everytime an account is locked out (they disable the account). You won't get another notification until you unlock the account and they trigger it again.

*) Do you think locking/disabling the account is a good idea? Or does it work well in your company?
*) If the account was locked, does mail server still deliver email to its mailbox?
*) Will it bring a lot extra workload to sysadmin? End user may call your directly, you need to answer phone calls, then unlock the account. We'd better not bring extra workload to sysadmin - if possible.

beezel wrote:

I am still not sure what your throttle function does, exactly. Does it solely rate limit to the value set, buildling up a queue and slowly releasing them? Or does it block after value set has been reached?

The throttle plugin supports tracking email based on number of inbound/outbound emails, or accumulated message size, etc. If some user reaches the limit, it rejects further emails directly, not hold in queue and send later.

7

Re: Throttle notifications?

ZhangHuangbin wrote:
beezel wrote:

2) I think this is where a 'release' or 'unthrottle' button would come in handy. Our current product (that we're migrating away from into iRed) gives us a notification email everytime an account is locked out (they disable the account). You won't get another notification until you unlock the account and they trigger it again.

*) Do you think locking/disabling the account is a good idea? Or does it work well in your company?
*) If the account was locked, does mail server still deliver email to its mailbox?
*) Will it bring a lot extra workload to sysadmin? End user may call your directly, you need to answer phone calls, then unlock the account. We'd better not bring extra workload to sysadmin - if possible.

beezel wrote:

I am still not sure what your throttle function does, exactly. Does it solely rate limit to the value set, buildling up a queue and slowly releasing them? Or does it block after value set has been reached?

The throttle plugin supports tracking email based on number of inbound/outbound emails, or accumulated message size, etc. If some user reaches the limit, it rejects further emails directly, not hold in queue and send later.

Thanks Zhang,

We really like the disabled account feature. The account still receives email and stores it in the inbox, but is not allowed to send mail or retrieve email via pop/smtp/imap. This could be accomplished as easily as setting a random password, and then an 'unlock' function that would just reset it to something known.

It does increase our overhead a little bit, but we have it automatically email our ticketing system and we usually respond to it before the user knows. Typically it is a compromised common password, as we do allow authenticated SMTP from outside our network as well as logins to our webmail. We host about 2000 mailboxes as an internal department and have one or two tickets a month.

We worry that the current iRed Throttle plugin would leave the user in a state where they 'sometimes don't receive email' and we would have no direct notification of action taken. IE, we'd have to pour through logs to find out they'd be throttled. This does not seem like a great user experience.

8

Re: Throttle notifications?

beezel wrote:

We really like the disabled account feature. The account still receives email and stores it in the inbox, but is not allowed to send mail or retrieve email via pop/smtp/imap. This could be accomplished as easily as setting a random password, and then an 'unlock' function that would just reset it to something known.

It's easy to disable smtp service for outbound but keep imap/pop services running.

Would you like to sponsor me to develop this feature with one or more support tickets? smile

9

Re: Throttle notifications?

ZhangHuangbin wrote:
beezel wrote:

We really like the disabled account feature. The account still receives email and stores it in the inbox, but is not allowed to send mail or retrieve email via pop/smtp/imap. This could be accomplished as easily as setting a random password, and then an 'unlock' function that would just reset it to something known.

It's easy to disable smtp service for outbound but keep imap/pop services running.

Would you like to sponsor me to develop this feature with one or more support tickets? smile

Thanks Zhang,

I would be very interestd - but my manager is another thing! What are we talking about, financially?