1 Topic by zacek

  • zacek
  • Member
  • Offline
  • From: Czech Republic
  • Registered: 2014-02-08
  • Posts: 19

Topic: deleting files from deleted mailboxes

==== Required information ====
- iRedMail version: 1.8.2
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): iRedAdmin-Pro-MySQL
- Linux/BSD distribution name and version: Debian 7 (SMP Debian 3.2.60-1+deb7u1 x86_64)
- Related log if you're reporting an issue:
====
Hi Zhang,
I'd like to do some maintenance on our mail server. I read older topics about requirement to manually delete vmail directories and files of deleted users. I found your answers about existing sql table "vmail.deleted_mailboxes" which collect records and I understand that I need to write my own cron script which will delete all remaining files from disk.

I list whole table "vmail.deleted_mailboxes" from mysql and compare it against directories on the disk (in /var/vmail/vmail1/....).

I'm not sure if there is bug but some of them are missing, that directories are written in the table but does not exist on the disk. Is there any exception? Or is there any cleaning script or sanitary check? For example between versions, because I did upgrade from previous version of IRPro?

Thank you for explanation.
Josef Zacek

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: deleting files from deleted mailboxes

zacek wrote:

I'm not sure if there is bug but some of them are missing, that directories are written in the table but does not exist on the disk. Is there any exception?

Maybe you created some mail accounts, but they didn't login at all?

zacek wrote:

Or is there any cleaning script or sanitary check? For example between versions, because I did upgrade from previous version of IRPro?

*) iRedMail and iRedAdmin-Pro don't have script (or cron job) to delete record in "vmail.deleted_mailboxes". But i'm planning to provide one in next release of iRedAdmin-Pro.

*) upgrading iRedMail or iRedAdmin-Pro doesn't touch "vmail.deleted_mailboxes".

3 Reply by zacek

  • zacek
  • Member
  • Offline
  • From: Czech Republic
  • Registered: 2014-02-08
  • Posts: 19

Re: deleting files from deleted mailboxes

ZhangHuangbin wrote:
zacek wrote:

I'm not sure if there is bug but some of them are missing, that directories are written in the table but does not exist on the disk. Is there any exception?

ZhangHuangbin wrote:

Maybe you created some mail accounts, but they didn't login at all?

Yes, it is possible. Some of them was deleted immediately after creation. Some of them was in use for some time.
Probably that is the reason why some of them aren't on the disk.


zacek wrote:

Or is there any cleaning script or sanitary check? For example between versions, because I did upgrade from previous version of IRPro?

ZhangHuangbin wrote:

*) iRedMail and iRedAdmin-Pro don't have script (or cron job) to delete record in "vmail.deleted_mailboxes". But i'm planning to provide one in next release of iRedAdmin-Pro.
*) upgrading iRedMail or iRedAdmin-Pro doesn't touch "vmail.deleted_mailboxes".

I'm looking forward. But at this time I need to save some disk space, so I'will write my own dirty script.

Maybe will be nice of you to add one more column to vmail.deleted_mailboxes (for ex. "deleted_from_disk") with boolean value which will indicate if files was already deleted from disk. Cleanin script could work with both of them, date condition and/or 0-1 indicator for "handy" irregular cleaning.

Thank you.
Josef Zacek

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: deleting files from deleted mailboxes

zacek wrote:

Maybe will be nice of you to add one more column to vmail.deleted_mailboxes (for ex. "deleted_from_disk") with boolean value which will indicate if files was already deleted from disk. Cleanin script could work with both of them, date condition and/or 0-1 indicator for "handy" irregular cleaning.

I prefer to delete record in "vmail.deleted_mailboxes" after removed maildir.

5 Reply by zacek

  • zacek
  • Member
  • Offline
  • From: Czech Republic
  • Registered: 2014-02-08
  • Posts: 19

Re: deleting files from deleted mailboxes

ZhangHuangbin wrote:
zacek wrote:

Maybe will be nice of you to add one more column to vmail.deleted_mailboxes (for ex. "deleted_from_disk") with boolean value which will indicate if files was already deleted from disk. Cleanin script could work with both of them, date condition and/or 0-1 indicator for "handy" irregular cleaning.

I prefer to delete record in "vmail.deleted_mailboxes" after removed maildir.

Yes, you are absolutely right.
In case of long runing systems there is assumption that "vmail.deleted_mailboxes" table will have many records. Boolean indicator "deleted_from_disk" will help to recognize which email account and their files was removed from disk and which wasn't (and tons of files left on the disk, wasting disk space).

The first of all, there is deleting email account from web administration. Secondary writing record to "vmail.deleted_mailboxes" with value "deleted_from_disk" set to "0/false". Only after this, immediately or anytime later, you could clean files and directories from disk (cron, manually, system procedure - check box in web administration, etc...). And only after removing files, value"deleted_from_disk" should be changed from "0/false" to "1/true", not before.

But, I suppose...There is one FATAL PROBLEM in case of two email with the same name - for example - creating aaa@aaa.com user, deleting it + writing record to "vmail.deleted_mailboxes" + repeated creating aaa@aaa.com because user really need it. After some time later, you will do cleaning and of course you will remove all living files of aaa@aaa.com user!!!, because old record in  "vmail.deleted_mailboxes" table!

From that reason I prefer to do cleaning procedure (removing files prom disk + seting flag  "deleted_from_disk" to "1") right after deleting user from web administration, not any time later! It is potentially very dangerous.

6 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: deleting files from deleted mailboxes

zacek wrote:

But, I suppose...There is one FATAL PROBLEM in case of two email with the same name - for example - creating aaa@aaa.com user, deleting it + writing record to "vmail.deleted_mailboxes" + repeated creating aaa@aaa.com because user really need it. After some time later, you will do cleaning and of course you will remove all living files of aaa@aaa.com user!!!, because old record in  "vmail.deleted_mailboxes" table!

As you can see the default maildir path created by iRedMail and iRedAdmin, the maildir path has a timestamp. For example, user 'username@domain.com' has this maildir path:

/var/vmail/vmail1/domain.com/u/s/e/username-2014.09.03.23.05.22/

2014.09.03.23.05.22 is a time stamp (Year.Month.Day.Hour.Minute.Second) of when this mail account was created. So there's no such conflict IF AND ONLY IF you use this maildir format.

I know some admin choose to not append timestamp in maildir path, so the script should check whether someone still using this maildir path.

Another idea is allowing to set how long to keep mailboxes of deleted mail accounts. e.g. 90 days. The script will check value of sql column "deleted_mailboxes.timestamp", if it's old than 90 days, remove the mailbox if no conflict like mentioned above.

7 Reply by zacek

  • zacek
  • Member
  • Offline
  • From: Czech Republic
  • Registered: 2014-02-08
  • Posts: 19

Re: deleting files from deleted mailboxes

Yes, you are right. The timestamp in mail path is the solution. I forgot abou it. smile

If you will do changes in next release, please, still add indicator "deleted_from_disk" (0/1). Cleaning script will be much more efficient.

8 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: deleting files from deleted mailboxes

Why not delete the record directly? Why use 'deleted_from_disk=0/1'?

9 Reply by zacek

  • zacek
  • Member
  • Offline
  • From: Czech Republic
  • Registered: 2014-02-08
  • Posts: 19

Re: deleting files from deleted mailboxes

ZhangHuangbin wrote:

Why not delete the record directly? Why use 'deleted_from_disk=0/1'?

If the cleaning will be done after some time or regularly (for ex. every week/ month... from cron), there will be necessary keep track which ones was deleted from disk and which wasn't. It's useless process all records in "deleted_mailboxes" table from begining, every time you run this cleaning procedure.

Of course, if the cleaning from disk will be processed immediately after email/user deleting from web administration, there is no reason to keep this info in the "deleted_mailboxes" table.

10 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: deleting files from deleted mailboxes

I prefer to make it simpler:

*) If records exist in "deleted_mailboxes", it means mailboxes are not deleted.
*) If records removed from "deleted_mailboxes", it means mailboxes were removed.

11 Reply by zacek

  • zacek
  • Member
  • Offline
  • From: Czech Republic
  • Registered: 2014-02-08
  • Posts: 19

Re: deleting files from deleted mailboxes

ZhangHuangbin wrote:

I prefer to make it simpler:

*) If records exist in "deleted_mailboxes", it means mailboxes are not deleted.
*) If records removed from "deleted_mailboxes", it means mailboxes were removed.


This is the solution too. But think about audit purposes. Is there any other info (some global log?) where is writen which email/account was deleted, when and who did it?

12 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: deleting files from deleted mailboxes

zacek wrote:

But think about audit purposes. Is there any other info (some global log?) where is writen which email/account was deleted, when and who did it?

There's log (System -> Admin Log) shows which admin deleted which account at when.