1 Topic by olaf (edited by olaf 2013-07-17 17:28:44)

  • olaf
  • Member
  • Offline
  • Registered: 2013-07-17
  • Posts: 3

Topic: How can I limit required System resources?

==== Required information ====
- iRedMail version:
- Store mail accounts in which backend (LDAP/MySQL/PGSQL):
- Linux/BSD distribution name and version:
- Related log if you're reporting an issue:
====
On a VPS I installed iRedMail 0.8.4 on top of Debian Wheezy. I use MariaDB(MySQL) as a backend. I started on a VPS with 512 MB memory. Everything installed and it worked, but was a bit slow. Also some issue arose as RAM was tight. MariaDB did not always start when the server was rebooted due to RAM limitation. OK. I upgraded to 1 GB RAM. But now I still have a resource hungry setup.
ClamD is a hungry beast. But it makes me wonder considering I have limited requirements: just a few users and a handful of domains. All with very low traffic. Looking at top shows:

%Cpu(s):  0.0 us,  0.3 sy,  0.0 ni, 99.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:   1027072 total,   815588 used,   211484 free,    12496 buffers
KiB Swap:        0 total,        0 used,        0 free,   209504 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND   
2660 clamav    20   0  268m 213m 5408 S   0.0 21.3   1:22.79 clamd             
14907 mysql     20   0  820m 101m 8528 S   0.0 10.1   0:00.44 mysqld           
2262 amavis    20   0  218m  92m 1116 S   0.0  9.2   0:03.73 /usr/sbin/amavi   
2657 amavis    20   0  220m  91m  608 S   0.0  9.1   0:00.00 /usr/sbin/amavi   
2658 amavis    20   0  220m  91m  608 S   0.0  9.1   0:00.00 /usr/sbin/amavi   
4200 cluebrin  20   0  106m  18m 1468 S   0.0  1.9   0:00.00 cbpolicyd         
4201 cluebrin  20   0  106m  18m 1364 S   0.0  1.9   0:00.00 cbpolicyd         
4202 cluebrin  20   0  106m  18m 1364 S   0.0  1.9   0:00.00 cbpolicyd         
4203 cluebrin  20   0  106m  18m 1364 S   0.0  1.9   0:00.00 cbpolicyd         
4181 cluebrin  20   0 78988  17m  956 S   0.0  1.7   0:03.32 cbpolicyd         
9491 www-data  20   0  246m 9.9m  792 S   0.0  1.0   0:00.14 apache2           
9521 www-data  20   0  246m 9.9m  788 S   0.0  1.0   0:00.54 apache2           
9327 www-data  20   0  246m 9.8m  772 S   0.0  1.0   0:00.27 apache2           
9617 www-data  20   0  246m 9916  736 S   0.0  1.0   0:00.01 apache2           
4802 iredadmi  20   0  373m 9756  556 S   0.0  0.9   0:06.27 apache2           
9622 www-data  20   0  246m 9744  584 S   0.0  0.9   0:00.01 apache2           
4781 root      20   0  244m 9544 2496 S   0.0  0.9   0:06.75 apache2           
2884 root      20   0  113m 7600 1400 S   0.0  0.7   1:30.54 fail2ban-server

Considering a limited number of (occasional) users and low traffic: can I limit the number of process spawns of amavis and cbpolicy?
Can ClamD be maintained memory wise? Or even replaced by a less resource hungry variety

After playing around with the configuration of the installed components I noticed that limiting for instance amavis in master.cf by reducing it to 1 process and changing max_use from 20 to 10 it still ends up at 100 and still multiple spawns take place. So I assume this is managed elsewhere. Is this documented somewhere?

I noticed that 512MB or 1GB does not matter that much: all daemons eat up about the same percentage of RAM. Some people reported to have Iredmail running on as low as 256mb RAM servers.

To sum up:
Is there any documentation available on how to tweak Iredmail and it's associated components or can you provide any hints/tips on how to commence?
In my current setup I need a full VPS just for mail and cannot use it for other stuff unless I part with extra cash doubling costs. It would be great if about 30-40% RAM could be freed for other stuff.

Any hints appreciated!

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,091

Re: How can I limit required System resources?

Both SpamAssassin + ClamAV are content-based spam/virus scanner, they eat much system resource.

You can try disable them for OUTGOING emails first, track whether they use much system resource:
http://iredmail.org/wiki/index.php?titl … oing.Mails

if you're not satisfied, try completely disable them:
http://iredmail.org/wiki/index.php?titl … amAssassin

3 Reply by olaf (edited by olaf 2013-07-17 18:10:53)

  • olaf
  • Member
  • Offline
  • Registered: 2013-07-17
  • Posts: 3

Re: How can I limit required System resources?

Thank you for the hints. It is not possible to limit the number of processes or the amount of allowed memory?
I also noticed that there is an Apache process running as "iredadmin" and one running as "root". Both are not the default and highly recommended default user "www-data". Is this initiated by Iredmail? Especially the root user is what is worrying me.

I implemented above suggestion by disabling outgoing contentscanning. It does not do a thing. All processes eat up as much memory as before and spawn as before (server fully rebooted):

Tasks:  86 total,   1 running,  81 sleeping,   0 stopped,   4 zombie
%Cpu(s):  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:   1027072 total,   940380 used,    86692 free,     9056 buffers
KiB Swap:        0 total,        0 used,        0 free,   198092 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND   
2663 clamav    20   0  268m 208m  320 S   0.0 20.8   0:00.00 clamd             
2267 amavis    20   0  218m  93m 2948 S   0.0  9.4   0:01.39 /usr/sbin/amavi   
2659 amavis    20   0  220m  92m 1852 S   0.0  9.3   0:00.00 /usr/sbin/amavi   
2660 amavis    20   0  220m  92m 1848 S   0.0  9.3   0:00.00 /usr/sbin/amavi   
4099 cluebrin  20   0  106m  20m 3324 S   0.0  2.1   0:00.00 cbpolicyd         
4098 cluebrin  20   0  106m  20m 3188 S   0.0  2.0   0:00.01 cbpolicyd         
4100 cluebrin  20   0  106m  20m 3188 S   0.0  2.0   0:00.01 cbpolicyd         
4101 cluebrin  20   0  106m  20m 3188 S   0.0  2.0   0:00.00 cbpolicyd
4070 cluebrin  20   0 78988  18m 2216 S   0.0  1.9   0:00.12 cbpolicyd         
2313 root      20   0  244m  14m 7440 S   0.0  1.4   0:00.14 apache2           
2364 iredadmi  20   0  373m  10m 1752 S   0.0  1.1   0:00.07 apache2           
2365 www-data  20   0  246m  10m 1760 S   0.0  1.1   0:00.00 apache2           
2366 www-data  20   0  246m  10m 1760 S   0.0  1.1   0:00.02 apache2           
2367 www-data  20   0  246m  10m 1760 S   0.0  1.1   0:00.01 apache2     

As the server was fully rebooted an other (Jetty) process was started preventing mysql/mariadb to be started. Both require about 20% RAM. My goal is to allow them to run together, but I fail to see how this can be done without limiting the resources eaten by cbpolicyd, amavis and clam as they sum up to 60% of RAM regardless if it is a 512mb or 1 GB RAM VPS.

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,091

Re: How can I limit required System resources?

olaf wrote:

It is not possible to limit the number of processes or the amount of allowed memory?

I have no idea yet, sorry. You already limit process number in Postfix master.cf, you should reduce value of '$max_servers' in Amavisd config, not '$max_use'.

olaf wrote:

I also noticed that there is an Apache process running as "iredadmin" and one running as "root". Both are not the default and highly recommended default user "www-data". Is this initiated by Iredmail? Especially the root user is what is worrying me.

Both are required.

*) We configure iRedAdmin to run as a separate, non-priviliege system user, 'iredadmin' by default, better for security.
*) The root user, i believe it's used to start Apache daemon first, then chroot sub-processes to 'www-data'.

olaf wrote:

I implemented above suggestion by disabling outgoing contentscanning. It does not do a thing.

It's for outgoing emails, if you have lots of incoming emails, they will eat much system resource as usual.

olaf wrote:

As the server was fully rebooted an other (Jetty) process was started preventing mysql/mariadb to be started. Both require about 20% RAM. My goal is to allow them to run together, but I fail to see how this can be done without limiting the resources eaten by cbpolicyd, amavis and clam as they sum up to 60% of RAM regardless if it is a 512mb or 1 GB RAM VPS.

Sorry, what's 'Jetty' process? i don't know it at all.

5 Reply by olaf

  • olaf
  • Member
  • Offline
  • Registered: 2013-07-17
  • Posts: 3

Re: How can I limit required System resources?

ZhangHuangbin wrote:
olaf wrote:

It is not possible to limit the number of processes or the amount of allowed memory?

ZhangHuangbin wrote:

I have no idea yet, sorry. You already limit process number in Postfix master.cf, you should reduce value of '$max_servers' in Amavisd config, not '$max_use'.

Ok. I will look into it. Thank you.

olaf wrote:

I also noticed that there is an Apache process running as "iredadmin" and one running as "root". Both are not the default and highly recommended default user "www-data". Is this initiated by Iredmail? Especially the root user is what is worrying me.

ZhangHuangbin wrote:

Both are required.

*) We configure iRedAdmin to run as a separate, non-priviliege system user, 'iredadmin' by default, better for security.
*) The root user, i believe it's used to start Apache daemon first, then chroot sub-processes to 'www-data'.

Yes, you are right about the chroot. I was just a bit surprised that it was bound to 0:0:0:0 and that there were more Apache processes spawned than specified in the config. (I already limited the number of processes there although Apache uses not that much RAM compared to other processes.)

olaf wrote:

I implemented above suggestion by disabling outgoing contentscanning. It does not do a thing.

ZhangHuangbin wrote:

It's for outgoing emails, if you have lots of incoming emails, they will eat much system resource as usual.

Regardless if outgoing email is being filtered or not, when starting Iredmail and components, the same amount of processes start and the same amount of RAM is consumed. So maybe there is a difference in the working environment and how much RAM is used, but the initial reservation of RAM is the same and therefor this apparently is not the solution I am looking for.

olaf wrote:

As the server was fully rebooted an other (Jetty) process was started preventing mysql/mariadb to be started. Both require about 20% RAM. My goal is to allow them to run together, but I fail to see how this can be done without limiting the resources eaten by cbpolicyd, amavis and clam as they sum up to 60% of RAM regardless if it is a 512mb or 1 GB RAM VPS.

ZhangHuangbin wrote:

Sorry, what's 'Jetty' process? i don't know it at all.

Jetty is a Java App Server like Tomcat, Jboss, Glassfish, etc. However it requires a small footprint compared to others. I can limit it to say 200-250mb. It is not that relevant to my question however. it is just "something"I would like to be able to run aside.

So this leaves say 700mb (with 50-100mb spare for other and temp processes etc) for a LAMP server + IredMail.
As some people claim they have got it running on an even smaller footprint I wonder how (as they did not specify),
I felt it should be doable with 1 GB if performance was not a real issue (and it is not) considering this, however doubling the memory did not resolve much hence my asking.
Obviously the correct answer to everything here mentioned is to upgrade to at least 4 GB RAM. However that would impose spending an awful lot of extra money on a monthly basis. Way too much for simple personal use in my opinion. So either I should stick to tweaking or look for an other mail solution.

Any further remarks much appreciated as I think the questions raised might also be relevant to others.