1 Topic by Dominique

  • Dominique
  • Member
  • Offline
  • Registered: 2014-01-09
  • Posts: 98

Topic: Amavisd blocking all emails

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release):  0.9.8
- Deployed with iRedMail Easy or the downloadable installer?
- Linux/BSD distribution name and version: CentOS 7
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MariaDB
- Web server (Apache or Nginx): Nginx
- Manage mail accounts with iRedAdmin-Pro? Yes
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

It's the second time I'm experiencing this problem.  Every email in the queue gets blocked and "mailq" shows me the following error:

     (delivery temporarily suspended: connect to 127.0.0.1[127.0.0.1]:10024: Connection refused)

After lots of struggling I managed to pinpoint the problem to Amavis and the only way to fix it is by restarting the amavisd service.  Even restarting the server completely didn't help.

The second time (2 days ago) I remembered it was Amavis but the first time I'm pretty sure I found some error in a log file, pointing to Amavis, so if there's any log file I can post let me know

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,135

Re: Amavisd blocking all emails

Dominique wrote:

     (delivery temporarily suspended: connect to 127.0.0.1[127.0.0.1]:10024: Connection refused)

How much RAM does this server have?

If not much, you may want to decrease the concurrently processed messages by following this tutorial, to get a more stable server:
https://docs.iredmail.org/concurrent.processing.html

3 Reply by Dominique

  • Dominique
  • Member
  • Offline
  • Registered: 2014-01-09
  • Posts: 98

Re: Amavisd blocking all emails

The server has 4Gb of RAM (no swap) and during the last 24 hours received 239 and sent 56 emails, so not the busiest server. I'll try lowering the number and hope the problem won't come back!

Cheers!

4 Reply by Dominique (edited by Dominique 2020-11-09 20:35:23)

  • Dominique
  • Member
  • Offline
  • Registered: 2014-01-09
  • Posts: 98

Re: Amavisd blocking all emails

I found this though, it seems like amavisd failed during an update... and does complain about memory

will lowering the concurrent emails processed fix the problem then?

Nov  7 11:02:09 email freshclam[21376]: ClamAV update process started at Sat Nov  7 11:02:09 2020
Nov  7 11:02:09 email freshclam[21376]: main.cld is up to date (version: 59, sigs: 4564902, f-level: 60, builder: sigmgr)
Nov  7 11:02:10 email freshclam[21376]: getfile: daily-25859.cdiff not found on database.clamav.net (IP: 104.16.219.84)
Nov  7 11:02:10 email freshclam[21376]: getpatch: Can't download daily-25859.cdiff from database.clamav.net
Nov  7 11:02:10 email freshclam[21376]: getfile: daily-25859.cdiff not found on database.clamav.net (IP: 104.16.218.84)
Nov  7 11:02:10 email freshclam[21376]: getpatch: Can't download daily-25859.cdiff from database.clamav.net
Nov  7 11:02:11 email freshclam[21376]: Trying host database.clamav.net (2606:4700::6810:da54)...
Nov  7 11:02:11 email freshclam[21376]: nonblock_connect: connect(): fd=4 errno=101: Network is unreachable
Nov  7 11:02:11 email freshclam[21376]: Can't connect to port 80 of host database.clamav.net (IP: 2606:4700::6810:da54)
Nov  7 11:02:11 email freshclam[21376]: Trying host database.clamav.net (2606:4700::6810:db54)...
Nov  7 11:02:11 email freshclam[21376]: nonblock_connect: connect(): fd=4 errno=101: Network is unreachable
Nov  7 11:02:11 email freshclam[21376]: Can't connect to port 80 of host database.clamav.net (IP: 2606:4700::6810:db54)
Nov  7 11:02:11 email freshclam[21376]: getpatch: Can't download daily-25859.cdiff from database.clamav.net
Nov  7 11:02:11 email freshclam[21376]: Incremental update failed, trying to download daily.cvd
Nov  7 11:02:12 email freshclam[21376]: Downloading daily.cvd [100%]
Nov  7 11:02:56 email freshclam[21384]: [LibClamAV] mpool_malloc(): Can't allocate memory (262144 bytes).
Nov  7 11:02:56 email freshclam[21384]: [LibClamAV] cli_parse_add(): Problem adding signature (3).
Nov  7 11:02:56 email freshclam[21384]: [LibClamAV] Problem parsing database at line 160579
Nov  7 11:02:56 email freshclam[21384]: [LibClamAV] Can't load daily.ldb: Can't allocate memory
Nov  7 11:02:56 email freshclam[21384]: [LibClamAV] cli_tgzload: Can't load daily.ldb
Nov  7 11:02:56 email freshclam[21384]: [LibClamAV] Can't load /var/lib/clamav/clamav-d84314ba1e526a6e2c9ff2c9a4c9c8c0.tmp/clamav-ea61501d9f4c34a919fee57af5026646.cvd: Malformed database
Nov  7 11:02:56 email freshclam[21384]: Failed to load new database: Malformed database
Nov  7 11:02:57 email freshclam[21376]: During database load : ERROR: Failed to load new database: Malformed database
Nov  7 11:02:57 email freshclam[21376]: Database load exited with status 55
Nov  7 11:02:57 email freshclam[21376]: Failed to load new database
Nov  7 11:02:57 email systemd: Removed slice User Slice of root.
Nov  7 11:02:57 email systemd: Stopping User Slice of root.

5 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,135

Re: Amavisd blocking all emails

Dominique wrote:

Nov  7 11:02:56 email freshclam[21384]: [LibClamAV] mpool_malloc(): Can't allocate memory (262144 bytes).

I experienced same issue before on CentOS 7, clamav services suddenly requires more memory after package update.

You can try to decrease concurrently processed emails as a testing, it should be more stable in this case.

6 Reply by adrian.L (edited by adrian.L 2024-11-28 19:15:28)

  • adrian.L
  • Member
  • Offline
  • Registered: 2020-10-08
  • Posts: 2

Re: Amavisd blocking all emails

I can confirm that adding RAM works with this problem.  I have a pretty small mail server, until the error returned yesterday it was running 6GB (which I set after this problem last happened nearly two years ago), but setting the minimum to 12Gb has fixed the problem.  My number of concurrently processed messages was already at 1, but over allocating the ram has allowed me to return to 4 concurrent message processors.

What I can't understand is why the ram consumption increases over time. Rebooting has no impact and it is a long time since I upgraded to 1.7.1 - so it is not like I have just upgraded to a new version of Amavisd.

7 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,135

Re: Amavisd blocking all emails

Adding "ConcurrentDatabaseReload no" in clamd.conf should fix the issue.

8 Reply by adrian.L

  • adrian.L
  • Member
  • Offline
  • Registered: 2020-10-08
  • Posts: 2

Re: Amavisd blocking all emails

ZhangHuangbin wrote:

Adding "ConcurrentDatabaseReload no" in clamd.conf should fix the issue.

Thank you for that setting.

I dropped the ram back to 6Gb and it is still running great.  I looks like it is now only using just over 2Gb, but I will keep the 6Gb there and watch it for a few months. The memory consumption has definitely dropped.