Aug 5, 2024: iRedMail-1.7.1 has been released.

**AndyInNYC** · 2019-05-22 21:52:18

AndyInNYC
Member
Offline

Registered: 2017-08-18
Posts: 145

Topic: Restoring Old emails from backup to new server

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release):
- Deployed with iRedMail Easy or the downloadable installer?
- Linux/BSD distribution name and version:
- Store mail accounts in which backend (LDAP/MySQL/PGSQL):
- Web server (Apache or Nginx):
- Manage mail accounts with iRedAdmin-Pro?
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

Both my old and new server are 0.9.7 with Apache/MySQL and should be configured identically.

I have created the same account names (and passwords). My attempt to restore from an Outlook pst file failed miserably - the sync was based on the new server's data and the pst file had all its emails deleted (and I copied the wrong pst files to a save place as backup).

So, I'm left with an older backup from the original iRedMail server.

*IF* I copy everything from MailDir down (ie the original structure into the new Maildir) will that work correctly?

The original directory structure was:
vmail/vmail1/[domain]\s\t\e\[wife's directory w/date]/Maildir

new structure is the same, except the directory date is different.

So, copying all of Maildir to Maildir moves the data - is this all I need to do to 'poof' you're back up as of the backup date?

No SQL commands, no bad index files, etc.?

Help?

Andrew

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

fr · 2019-05-22 22:07:59

fr
Member
Offline

Registered: 2019-02-01
Posts: 61

Re: Restoring Old emails from backup to new server

have you read this?

https://docs.iredmail.org/backup.restore.html

i think the problem is the sql restore
bye

**AndyInNYC** · 2019-05-22 22:21:01

AndyInNYC
Member
Offline

Registered: 2017-08-18
Posts: 145

Re: Restoring Old emails from backup to new server

I did read the article - it says that the emails can be backed up via rsync. I shouldn't have to mess with the MySQ if my memory is correct from other stuff I've read.

If there is a simple restore procedure, I'm completely missing it - this is really what I need.

The problem i'm having seems to be with file permissions. I'm not sure what the error is with the dovecot-uidlist

So, I'm getting a slew of error messages in /var/log/dovecot/imap.log

May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: open(/var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/dovecot-uidlist) failed: Permission denied
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Logged out in=96 out=732
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: open(/var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/dovecot-uidlist) failed: Permission denied
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: open(/var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/dovecot-uidlist) failed: Permission denied
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Drafts/dovecot.index.log
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Drafts/dovecot.index.log
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Sent/dovecot.index.log
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Sent/dovecot.index.log
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Trash/dovecot.index.log
May 22 09:58:50 mail dovecot: message repeated 2 times: [ imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Trash/dovecot.index.log]
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: file_dotlock_create(/var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Archive/dovecot-uidlist) failed: Permission denied (euid=2000(vmail) egid=2000(vmail) missing +w perm: /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Archive, dir owned by 0:0 mode=0755)
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Archive/dovecot.index.log
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: file_dotlock_create(/var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Archive/dovecot-uidlist) failed: Permission denied (euid=2000(vmail) egid=2000(vmail) missing +w perm: /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Archive, dir owned by 0:0 mode=0755)
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Archive/dovecot.index.log
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: file_dotlock_create(/var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Boxer/dovecot-uidlist) failed: Permission denied (euid=2000(vmail) egid=2000(vmail) missing +w perm: /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Boxer, dir owned by 0:0 mode=0755)
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Boxer/dovecot.index.log
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: file_dotlock_create(/var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Boxer/dovecot-uidlist) failed: Permission denied (euid=2000(vmail) egid=2000(vmail) missing +w perm: /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Boxer, dir owned by 0:0 mode=0755)
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Boxer/dovecot.index.log
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: file_dotlock_create(/var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Boxer.To-do/dovecot-uidlist) failed: Permission denied (euid=2000(vmail) egid=2000(vmail) missing +w perm: /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Boxer.To-do, dir owned by 0:0 mode=0755)
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Boxer.To-do/dovecot.index.log
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: file_dotlock_create(/var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Boxer.To-do/dovecot-uidlist) failed: Permission denied (euid=2000(vmail) egid=2000(vmail) missing +w perm: /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Boxer.To-do, dir owned by 0:0 mode=0755)
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Boxer.To-do/dovecot.index.log
May 22 09:58:50 mail dovecot: imap(steph@wifesdomain.com): Logged out in=587 out=3351
May 22 09:59:01 mail dovecot: imap-login: Login: user=<steph@wifesdomain.com>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=1430, secured, session=<AGsiXnqJOuV/AAAB>
May 22 09:59:01 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Drafts/dovecot.index.log
May 22 09:59:01 mail dovecot: imap(steph@wifesdomain.com): Error: Index is read-only, can't write-lock /var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/.Drafts/dovecot.index.log
May 22 09:59:01 mail dovecot: imap(steph@wifesdomain.com): Logged out in=99 out=732
May 22 09:59:02 mail dovecot: imap-login: Login: user=<steph@wifesdomain.com>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=1432, secured, session=<rvYoXnqJVuV/AAAB>
May 22 09:59:02 mail dovecot: imap(steph@wifesdomain.com): Error: open(/var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/dovecot-uidlist) failed: Permission denied
May 22 09:59:02 mail dovecot: message repeated 3 times: [ imap(steph@wifesdomain.com): Error: open(/var/vmail/vmail1/wifesdomain.com/s/t/e/steph-2019.05.20.13.02.27//Maildir/dovecot-uidlist) failed: Permission denied]

Unfortunately, I don't know what command to issue to set the permissions correctly (and on what file/files).

I was logged in while it was loading from the backup and I saw the count increase as I refreshed SOGo; now, however, I can't get in to see.

Thoughts.

**AndyInNYC** · 2019-05-22 22:24:06

AndyInNYC
Member
Offline

Registered: 2017-08-18
Posts: 145

Re: Restoring Old emails from backup to new server

Found this:

https://forum.iredmail.org/topic11097-i … edure.html

Seems to have worked!

Andrew

**AndyInNYC** · 2019-05-22 22:30:42

AndyInNYC
Member
Offline

Registered: 2017-08-18
Posts: 145

Re: Restoring Old emails from backup to new server

Last question - can I simply copy EVERYTHING from vmail1 ===> and end up with all my emails ready to go from all accounts (assuming the account was recreated on the system)?

I know I'll need to chown/etc. to get the permissions right, but WILL THIS WORK as opposed to copying MailDir on down?

Thanks.

Andrew

**ZhangHuangbin** · 2019-05-23 09:34:14

ZhangHuangbin
iRedMail Developers
Offline

Registered: 2009-05-06
Posts: 31,091

Re: Restoring Old emails from backup to new server

Please read this tutorial:
https://docs.iredmail.org/migrate.to.ne … erver.html

**AndyInNYC** · 2019-05-24 00:41:23

AndyInNYC
Member
Offline

Registered: 2017-08-18
Posts: 145

Re: Restoring Old emails from backup to new server

ZhangHuangbin, thanks for replying.

Unfortunatly, the tutorial doesn't really tell me what I need to know.

I have a new server and have set up new emails (same as the old emails). I only need to move the old emails over so that they can be read/seen correctly.

I only need the physical emails, not the users. The tutorial states:

Copy all mailboxes (in Maildir format) to new iRedMail server with tools like rsync.
Set correct file owner and permission of mailboxes. Default owner is vmail, group is vmail, permission is 0700.

With SQL backends, you can get full maildir path of user with below SQL command:

So Maildir is right below the path - seems I need to copy everything from Maildir down for each of the users, rather than trying to go from vmail down.

Right?

Since I'm copying all the 'old' emails into a new/working system, do I need to delete any index files so they can recreate based on the old+new emails in the directories?

Andrew

**ZhangHuangbin** · 2019-05-24 08:48:30

ZhangHuangbin
iRedMail Developers
Offline

Registered: 2009-05-06
Posts: 31,091

Re: Restoring Old emails from backup to new server

AndyInNYC wrote:

So Maildir is right below the path - seems I need to copy everything from Maildir down for each of the users, rather than trying to go from vmail down.
Right?

Since you created new accounts instead of restoring from old database, the maildir path of each user has been changed. So yes you need to copy for each user, NOT copy the whole vmail1 directory directly.

AndyInNYC wrote:

Since I'm copying all the 'old' emails into a new/working system, do I need to delete any index files so they can recreate based on the old+new emails in the directories?

No need to remove index files. But if you want to, it's safe to remove index files for Maildir format mailbox, Dovecot will re-generate it. WARNING: if you're using mdbox format, index lost will cause email lose.

**AndyInNYC** · 2019-05-26 07:18:52

AndyInNYC
Member
Offline

Registered: 2017-08-18
Posts: 145

Re: Restoring Old emails from backup to new server

I believe the Inbox was restored via a copy. The Sent folder, however, was not. What do i need to do to recover this folder?

I copied Maildir on down from my backup into the current Maildir directory.

When I try to open the webmail (either SOGo or Roundcube, I get an error (on the InBox).

Sogo responds with 'An error occurred while communicating with the mail server'. If I change the URL shown from ../inbox to ../Sent the most recent (pre restore) sent email are shown. The Inbox shows 124028 emails (about right) but won't display them when I click on Inbox.

sogo.log contains this line at the end (date removed; email made anonymous)
sogod [591]: 72.88.163.79 "POST /SOGo/so/my@email.com/Mail/unseenCount HTTP/1.1" 200 46/46 0.497 - - 96K

Roundcube just tells me there are no messages found in this mailbox.

I'll try more googling.

So 2 questions:
1. How from my backup do I restore Sent Folders
2. How do I display my large inbox

Thanks

Andrew

**ZhangHuangbin** · 2019-05-26 12:22:44

ZhangHuangbin
iRedMail Developers
Offline

Registered: 2009-05-06
Posts: 31,091

Re: Restoring Old emails from backup to new server

Please check Dovecot log file to figure out what's going on. We need detailed error message for troubleshooting, your explanation/description is not helpful in this case.

**AndyInNYC** · 2019-05-26 13:05:49

AndyInNYC
Member
Offline

Registered: 2017-08-18
Posts: 145

Re: Restoring Old emails from backup to new server

I'll pull all the logs from dovecot. What's the answer to question #1 - ie restoring Sent folders?

Thank you.

Andrew

**AndyInNYC** · 2019-05-26 14:24:35

AndyInNYC
Member
Offline

Registered: 2017-08-18
Posts: 145

Re: Restoring Old emails from backup to new server

The only error message I can see is this one from /var/logs/dovecot/imap.log

May 26 02:16:17 mail dovecot: imap(myaddr@myserver.com): Fatal: pool_system_malloc(3146424): Out of memory
May 26 02:11:15 mail dovecot: imap(myaddr@myserver.com): Fatal: master: service(imap): child 3313 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump)

Unfortunately, I have already set vsz_limit to 1000M and 'service dovecot restart' several times as well as restarting the server.

**AndyInNYC** · 2019-05-27 21:41:23

AndyInNYC
Member
Offline

Registered: 2017-08-18
Posts: 145

Re: Restoring Old emails from backup to new server

Well, I fixed part of my problem.

I increased vsz_limit to an absurd number - this didn't help, but I left it at 2GB (I'll lower it later).

What did help is that I deleted all dovecot.index* files in all my folders.

Email loaded right up.

UNFORTUNATELY,

I still don't have access to the SENT folder for any of the mailboxes restored from my backup. How do I get access to the SENT folders/how do I check if the backup even copied the SENT folders? The backup I have locally starts with a directory called vmail. Under vmail is vmail1, backup, public, root@IP and sieve.

I really need the SENT folders for the restored emails. Where oh, where, could they be?

Andrew

**AndyInNYC** · 2019-05-28 22:31:13

AndyInNYC
Member
Offline

Registered: 2017-08-18
Posts: 145

Re: Restoring Old emails from backup to new server

Woohoo, back in business.

My backup had 2 sets of directories under Maildir for my account. Each had a different date. I copied the second into the 'operational' directory on the new server into which I had already copied the first.

I then deleted all the dovecot index files and chown'd the files properly after the copy.

I now have my Sent box and other historical emails and access to my InBox.

Everything seems to be working.

Andrew

**AndyInNYC** · 2019-05-30 01:17:02

AndyInNYC
Member
Offline

Registered: 2017-08-18
Posts: 145

Re: Restoring Old emails from backup to new server

Please mark as closed and happy.

Aug 5, 2024: iRedMail-1.7.1 has been released.

[ Closed ] Restoring Old emails from backup to new server

Posts: 15

1 Topic by AndyInNYC 2019-05-22 21:52:18

Topic: Restoring Old emails from backup to new server

2 Reply by fr 2019-05-22 22:07:59

Re: Restoring Old emails from backup to new server

3 Reply by AndyInNYC 2019-05-22 22:21:01

Re: Restoring Old emails from backup to new server

4 Reply by AndyInNYC 2019-05-22 22:24:06

Re: Restoring Old emails from backup to new server

5 Reply by AndyInNYC 2019-05-22 22:30:42

Re: Restoring Old emails from backup to new server

6 Reply by ZhangHuangbin 2019-05-23 09:34:14

Re: Restoring Old emails from backup to new server

7 Reply by AndyInNYC 2019-05-24 00:41:23

Re: Restoring Old emails from backup to new server

8 Reply by ZhangHuangbin 2019-05-24 08:48:30

Re: Restoring Old emails from backup to new server

9 Reply by AndyInNYC 2019-05-26 07:18:52 (edited by AndyInNYC 2019-05-26 07:40:12)

Re: Restoring Old emails from backup to new server

10 Reply by ZhangHuangbin 2019-05-26 12:22:44

Re: Restoring Old emails from backup to new server

11 Reply by AndyInNYC 2019-05-26 13:05:49

Re: Restoring Old emails from backup to new server

12 Reply by AndyInNYC 2019-05-26 14:24:35

Re: Restoring Old emails from backup to new server

13 Reply by AndyInNYC 2019-05-27 21:41:23

Re: Restoring Old emails from backup to new server

14 Reply by AndyInNYC 2019-05-28 22:31:13

Re: Restoring Old emails from backup to new server

15 Reply by AndyInNYC 2019-05-30 01:17:02

Re: Restoring Old emails from backup to new server

Posts: 15