1 Topic by poljakov (edited by poljakov 2015-05-28 05:57:13)

  • poljakov
  • Member
  • Offline
  • Registered: 2015-05-28
  • Posts: 25

Topic: Extrem load

We recently set up an iRedMail instance for about 50.000 active users, 10-15.000 web sessions per a day. The server runs with the standard antivirus and antispam settings, the web interface uses SSL. Neither the user interface nor the Admin Panel could be used cause a high MySQL load. We have no errors in log, only the mail.log grows permanently. Extrem slow also the mail delivery from the user interface.

The mail store is 42GB (after 2 days operation), received about 55.000 mails. The server is a Hyper-V VM with dedicated 8GB RAM, 8 Core (Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz). The main CPU load reported by the 'top' command caused by fail2ban and maillogconvert.pl, however this convert runs not constant. (Makes - we think - statistical records.)

Could be that any log to the database slows down the whole system? Who else uses the DB so hard? There is not too much logins and living webmail sessions because the login screen itself could not be rendered, reporting timeouts.


==== Required information ====
- iRedMail version: 0.9.0
- Store mail accounts in: MariaDB
- Web server: Apache
- Linux/BSD distribution name and version: Ubuntu 14.04.2 LTS
====

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by mir

  • mir
  • Member
  • Offline
  • Registered: 2014-05-30
  • Posts: 170

Re: Extrem load

poljakov wrote:

The mail store is 42GB (after 2 days operation), received about 55.000 mails. The server is a Hyper-V VM with dedicated 8GB RAM, 8 Core (Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz). The main CPU load reported by the 'top' command caused by fail2ban and maillogconvert.pl, however this convert runs not constant. (Makes - we think - statistical records.)

Could be that any log to the database slows down the whole system? Who else uses the DB so hard? There is not too much logins and living webmail sessions because the login screen itself could not be rendered, reporting timeouts.

Try changing your mysql settings for optimizing performance. Try starting with increasing the innodb buffer pool size.
Create /etc/mysql/conf.d/mysql_local.cnf with this content
[mysqld]
innodb_buffer_pool_size = 1G

And restart mysql: service mysql restart

After this read this http://dev.mysql.com/doc/refman/5.5/en/ … nnodb.html to find more options to change for increasing performance.

3 Reply by poljakov

  • poljakov
  • Member
  • Offline
  • Registered: 2015-05-28
  • Posts: 25

Re: Extrem load

Thanks but not helped.

The storage has theoratically 1Gbit/sec capacity, but currently works only with 30-80Mbit/sec. The iowait is very high, meanwhile all the processors are waiting.

Try changing your mysql settings for optimizing performance. Try starting with increasing the innodb buffer pool size.
Create /etc/mysql/conf.d/mysql_local.cnf with this content
[mysqld]
innodb_buffer_pool_size = 1G

And restart mysql: service mysql restart

After this read this http://dev.mysql.com/doc/refman/5.5/en/ … nnodb.html to find more options to change for increasing performance.

4 Reply by camel1cz

  • camel1cz
  • camel1cz
  • Member
  • Offline
  • From: Czech Republic
  • Registered: 2013-04-20
  • Posts: 221

Re: Extrem load

Try to find out, what process is generating most of the IO traffic - you can use a tool like iotop... then try to resolve issue in that application.

If you did use default setup of DB server, I would bet, it's the database causing the IO problem.

5 Reply by poljakov

  • poljakov
  • Member
  • Offline
  • Registered: 2015-05-28
  • Posts: 25

Re: Extrem load

camel1cz wrote:

Try to find out, what process is generating most of the IO traffic - you can use a tool like iotop... then try to resolve issue in that application.

If you did use default setup of DB server, I would bet, it's the database causing the IO problem.

The database was not the problem source. We simply changed the iSCSI storage to SAS, and changed the backend server for a stronger one. Must say, that the mail queue is still very large, but reconfiguring the Amavis and Spamassassin processes it will be processed in a couple of hours.

Thank you very much for iRedMail's direct support. Cool developer, provided quick and professional tips and tricks for setting up our service.