1 Topic by lamagra

  • lamagra
  • Member
  • Offline
  • Registered: 2015-01-29
  • Posts: 46

Topic: cleanup_amavisd_db - not working properly

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release): 1.3.2
- Deployed with iRedMail Easy or the downloadable installer? download
- Linux/BSD distribution name and version: Debian 10.7
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): Nginx
- Manage mail accounts with iRedAdmin-Pro? Yes
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====
Hi,
i have problem with cleanup_amavisd - script has not cleaned the database (send and receive msgs). It should delete all older then 7 days, but it doesn't. I tried to start script manually but no effects and no errors.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,082

Re: cleanup_amavisd_db - not working properly

- Did you see older records in Amavisd database, "msgs" table?
- What are the settings in your iRedAdmin-Pro config file:

AMAVISD_REMOVE_MAILLOG_IN_DAYS =
AMAVISD_REMOVE_QUARANTINED_IN_DAYS =

3 Reply by lamagra

  • lamagra
  • Member
  • Offline
  • Registered: 2015-01-29
  • Posts: 46

Re: cleanup_amavisd_db - not working properly

ZhangHuangbin wrote:

- Did you see older records in Amavisd database, "msgs" table?
- What are the settings in your iRedAdmin-Pro config file:

AMAVISD_REMOVE_MAILLOG_IN_DAYS =
AMAVISD_REMOVE_QUARANTINED_IN_DAYS =

Hi yes i have older records in Amavisd database. Database is growing and because of that first page of iredadmin is slowing down every day.

Settings like bellow but in database i have msgs from 18.12.2020.
AMAVISD_REMOVE_MAILLOG_IN_DAYS = 7
AMAVISD_REMOVE_QUARANTINED_IN_DAYS = 7

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,082

Re: cleanup_amavisd_db - not working properly

Seems something wrong with our cleanup script.
As a temporary solution, you can remove the old records manually and keep tracking this issue, i will take a look and try to fix it.

5 Reply by stahr

  • stahr
  • Member
  • Offline
  • Registered: 2017-06-19
  • Posts: 17

Re: cleanup_amavisd_db - not working properly

Any updates on this topic?  Mine is growing out of control as well and it's not clear to me how I would "manually" go about deleting the entries which currently stand at 1189092.

I'm going to poke around and report back if I come up with a command.  What I can say though is that this started for me about a week or 10 days after I upgraded amavisd (CentOS7):

[root@voyageur 2020]# egrep amavis /var/log/yum.log*
/var/log/yum.log-20210101:Sep 15 13:13:56 Installed: amavis-2.12.0-9.el7.noarch
/var/log/yum.log-20210101:Sep 15 13:13:56 Erased: amavisd-new-2.11.1-1.el7.noarch
/var/log/yum.log-20210101:Oct 20 11:54:53 Updated: amavis-2.12.0-10.el7.noarch


[root@voyageur 2020]# ls -alh 10/*/amavis*
-rw-r--r-- 1 root root 5.5M Oct  1 02:06 10/01/amavisd-2020-10-01-02-06-02.sql.bz2
-rw-r--r-- 1 root root 5.7M Oct  2 02:06 10/02/amavisd-2020-10-02-02-06-01.sql.bz2
-rw-r--r-- 1 root root 5.3M Oct  3 02:06 10/03/amavisd-2020-10-03-02-06-01.sql.bz2
-rw-r--r-- 1 root root 4.6M Oct  4 02:06 10/04/amavisd-2020-10-04-02-06-01.sql.bz2
-rw-r--r-- 1 root root 4.6M Oct  5 02:06 10/05/amavisd-2020-10-05-02-06-01.sql.bz2
-rw-r--r-- 1 root root 4.6M Oct  6 02:06 10/06/amavisd-2020-10-06-02-06-01.sql.bz2
-rw-r--r-- 1 root root 5.3M Oct  7 02:06 10/07/amavisd-2020-10-07-02-06-01.sql.bz2
-rw-r--r-- 1 root root 7.0M Oct  8 02:06 10/08/amavisd-2020-10-08-02-06-01.sql.bz2
-rw-r--r-- 1 root root 5.6M Oct  9 02:06 10/09/amavisd-2020-10-09-02-06-01.sql.bz2
-rw-r--r-- 1 root root 5.5M Oct 10 02:06 10/10/amavisd-2020-10-10-02-06-01.sql.bz2
-rw-r--r-- 1 root root 6.5M Oct 11 02:06 10/11/amavisd-2020-10-11-02-06-01.sql.bz2
-rw-r--r-- 1 root root 4.0M Oct 12 02:06 10/12/amavisd-2020-10-12-02-06-01.sql.bz2
-rw-r--r-- 1 root root 4.0M Oct 13 02:06 10/13/amavisd-2020-10-13-02-06-02.sql.bz2
-rw-r--r-- 1 root root 4.7M Oct 14 02:06 10/14/amavisd-2020-10-14-02-06-01.sql.bz2
-rw-r--r-- 1 root root 5.5M Oct 15 02:06 10/15/amavisd-2020-10-15-02-06-02.sql.bz2
-rw-r--r-- 1 root root 6.0M Oct 16 02:06 10/16/amavisd-2020-10-16-02-06-01.sql.bz2
-rw-r--r-- 1 root root 6.0M Oct 17 02:06 10/17/amavisd-2020-10-17-02-06-01.sql.bz2
-rw-r--r-- 1 root root 5.2M Oct 18 02:06 10/18/amavisd-2020-10-18-02-06-01.sql.bz2
-rw-r--r-- 1 root root 4.3M Oct 19 02:06 10/19/amavisd-2020-10-19-02-06-01.sql.bz2
-rw-r--r-- 1 root root 4.2M Oct 20 02:06 10/20/amavisd-2020-10-20-02-06-01.sql.bz2
-rw-r--r-- 1 root root 5.0M Oct 21 02:06 10/21/amavisd-2020-10-21-02-06-01.sql.bz2
-rw-r--r-- 1 root root 5.7M Oct 22 02:06 10/22/amavisd-2020-10-22-02-06-01.sql.bz2
-rw-r--r-- 1 root root 5.8M Oct 23 02:06 10/23/amavisd-2020-10-23-02-06-01.sql.bz2
-rw-r--r-- 1 root root 5.6M Oct 24 02:06 10/24/amavisd-2020-10-24-02-06-01.sql.bz2
-rw-r--r-- 1 root root 5.0M Oct 25 02:06 10/25/amavisd-2020-10-25-02-06-01.sql.bz2
-rw-r--r-- 1 root root 4.4M Oct 26 02:06 10/26/amavisd-2020-10-26-02-06-02.sql.bz2
-rw-r--r-- 1 root root 4.5M Oct 27 02:06 10/27/amavisd-2020-10-27-02-06-01.sql.bz2
-rw-r--r-- 1 root root 5.1M Oct 28 02:06 10/28/amavisd-2020-10-28-02-06-01.sql.bz2
-rw-r--r-- 1 root root 6.9M Oct 29 02:06 10/29/amavisd-2020-10-29-02-06-01.sql.bz2
-rw-r--r-- 1 root root 8.7M Oct 30 02:06 10/30/amavisd-2020-10-30-02-06-01.sql.bz2
-rw-r--r-- 1 root root  11M Oct 31 02:06 10/31/amavisd-2020-10-31-02-06-01.sql.bz2

Current backup size:

[root@voyageur 2020]# ls -alh /var/vmail/backup/mysql/2021/02/13/amavisd-2021-02-13-02-06-01.sql.bz2
-r-------- 1 root root 175M Feb 13 02:07 /var/vmail/backup/mysql/2021/02/13/amavisd-2021-02-13-02-06-01.sql.bz2


When I manually run the cleanup script:


[root@voyageur 2020]# /usr/bin/python3 /var/www/iredadmin/tools/cleanup_amavisd_db.py
* Backend: ldap
* SQL server: 127.0.0.1:3306
* Enable dirty read for querying without locking SQL tables.
* Delete quarantined mails which older than 7 days
* Delete incoming/outgoing emails which older than 3 days
*       [-] Deleting records: 1 - 100 (Sat Feb 13 08:37:28 2021)
* Delete unreferenced records from table `msgrcpt`.
* Delete unreferenced records from table `quarantine`.
* Delete unreferenced records from table `maddr`.
*       [-] Deleting records: 0 - 2 (Sat Feb 13 08:39:19 2021)
* Delete unreferenced records from table `mailaddr`.
*
* Remained records:
*
*       `msgs`: 1189092
* `quarantine`: 7
*      `maddr`: 745988
*   `mailaddr`: 25835
* Log cleanup status.

6 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,082

Re: cleanup_amavisd_db - not working properly

Please download the latest version below, then replace the one on your server and run it manually (with Python v3):
https://github.com/iredmail/iRedAdmin/b … visd_db.py

7 Reply by lamagra

  • lamagra
  • Member
  • Offline
  • Registered: 2015-01-29
  • Posts: 46

Re: cleanup_amavisd_db - not working properly

ZhangHuangbin wrote:

Please download the latest version below, then replace the one on your server and run it manually (with Python v3):
https://github.com/iredmail/iRedAdmin/b … visd_db.py

Hi,
new version works OK. Thanks!