1 Topic by yaroslav.hetmanskyi (edited by yaroslav.hetmanskyi 2021-08-05 23:29:54)

  • yaroslav.hetmanskyi
  • Member
  • Offline
  • Registered: 2021-06-10
  • Posts: 5

Topic: High CPU usage

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release): 1.4.0
- Deployed with iRedMail Easy or the downloadable installer? Downloadable installer
- Linux/BSD distribution name and version: Ubuntu 20.04.2 LTS
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): LDAP
- Web server (Apache or Nginx):Nginx
- Manage mail accounts with iRedAdmin-Pro? Yes
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

Hello everyone.
We have some issue with CPU loading. The fresh iRedMail installation, working on the server with 12 virtual cpu and 40gb of RAM. We were migrated from iRedMail Mysql free version, with 3000+ mailboxes.
Before we start it in production mode everything was fine. But after release, in working hours (when employees is used it) cpu load is over 90% all the time (top processes is mysql(always) and apps plugin). iRedAdmin, Roundcube and mail clients working very slow or response with 504 Gate-way timeout.
In the night hours everything is loading fine, CPU usage coming to normal too.

In /var/log/mail.log seems everything is ok, except "mail roundcube: <mu662i8s> IMAP Error: Login failed for .... Empty startup greeting (127.0.0.1:143)"
in /var/log/nginx/error.log we have a lot messages "[error] 255136#255136: *63233 upstream timed out (110: Connection timed out) while reading response header from upstream ..."
/etc/mysql/mariadb.conf.d/50-server.cnf:max_connections        = 1024
/etc/dovecot/dovecot.conf:    mail_max_userip_connections = 1000


Seems like some limits, because in non peak hours cpu loading is fine.
Also, some problems with connecting Outlook 2010 to imap/pop server - got an error (connecting to smtp without problems). At that time, on the others Mail client's we didn't get such error.
Help, please.

Post's attachments

cpu-netdata.jpg
cpu-netdata.jpg 98.26 kb, file has never been downloaded. 

iredmail-cpu.jpg 478.65 kb, file has never been downloaded. 

iredmail-cpu2.jpg 499.4 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: High CPU usage

Search "_limit" in /var/log/dovecot/*.log, any error or warning reported by Dovecot?

3 Reply by yaroslav.hetmanskyi

  • yaroslav.hetmanskyi
  • Member
  • Offline
  • Registered: 2021-06-10
  • Posts: 5

Re: High CPU usage

The problem with the CPU was solved somehow. Unfortunately, I can't say what was the reason. Outlook is already working fine.
Result of search "_limit" - /var/log/dovecot/dovecot.log:Aug  9 12:23:16 mail dovecot: master: Warning: service(stats): client_limit (1000) reached, client connections are being dropped

Post's attachments

netdata-09.08.2021.jpg
netdata-09.08.2021.jpg 111.57 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

4 Reply by yaroslav.hetmanskyi

  • yaroslav.hetmanskyi
  • Member
  • Offline
  • Registered: 2021-06-10
  • Posts: 5

Re: High CPU usage

After the mail server was worked fine all the week, today CPU loading have coming back.

Post's attachments

cpu-usage.jpg
cpu-usage.jpg 110.12 kb, file has never been downloaded. 

htop.jpg 424.22 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

5 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: High CPU usage

yaroslav.hetmanskyi wrote:

Result of search "_limit" - /var/log/dovecot/dovecot.log:Aug  9 12:23:16 mail dovecot: master: Warning: service(stats): client_limit (1000) reached, client connections are being dropped

please follow this warning and Increase "client_limit" value inside "service stats {}" block.

6 Reply by yaroslav.hetmanskyi

  • yaroslav.hetmanskyi
  • Member
  • Offline
  • Registered: 2021-06-10
  • Posts: 5

Re: High CPU usage

limit increased, warning message disappeared, but the problem with CPU didn't solved

7 Reply by camel1cz

  • camel1cz
  • camel1cz
  • Member
  • Offline
  • From: Czech Republic
  • Registered: 2013-04-20
  • Posts: 220

Re: High CPU usage

As you name mysql processes as top consumers of CPU: did you try some performance tuning of mysql?
The default setup of mysql is usually very basic and not optimised for serious traffic.

8 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: High CPU usage

yaroslav.hetmanskyi wrote:

/etc/dovecot/dovecot.conf:    mail_max_userip_connections = 1000

This setting is not necessary, it limits 1000 connections per user from same IP address.

About the high CPU, If MySQL is the top one, could you try to run SQL command "show processlist;" and check which MYSQL process causing the high CPU?

9 Reply by yaroslav.hetmanskyi

  • yaroslav.hetmanskyi
  • Member
  • Offline
  • Registered: 2021-06-10
  • Posts: 5

Re: High CPU usage

After migrating the virtual machine to another node the problem was solved and cpu usage become stable.