1 Topic by pbf343

  • pbf343
  • Member
  • Offline
  • Registered: 2013-05-30
  • Posts: 243

Topic: Clients Fail to read files - ? Dovecot issue

==== Required information ====
- iRedMail version (check /etc/iredmail-release):
- Linux/BSD distribution name and version:
- Store mail accounts in which backend (LDAP/MySQL/PGSQL):
- Web server (Apache or Nginx):
- Manage mail accounts with iRedAdmin-Pro?
- Related log if you're reporting an issue:
====


CentOS 6.6 install with MySQL backend. 

Performed an rsync from old system to new system.

However, clients are not showing/accessing files even though the roundcube web interface seems to show the subscribed folders on the left side frame, it hangs on the right side frame and never connects.  Configured an Apple Mac mail client and it fails to read mail files as well. 

NFS mount no iSCSI.
Ownership vmail:vmail




Side notes which I do NOT think are relevant because did a test run previous to your updates of scipts and had no problems with the mounts.
Reinstall and rsync and now having issues.
Backend mount is a ZFS share from BSD system mounted via NFS. 

Did do an add of vmail user & group on FreeNas system.
Also did following: 
id -u vmail
2000
Set the both user and group to 2000 on FreeNas.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by pbf343

  • pbf343
  • Member
  • Offline
  • Registered: 2013-05-30
  • Posts: 243

Re: Clients Fail to read files - ? Dovecot issue

From dovecot.conf


# Assign uid to virtual users.
first_valid_uid = 2000
last_valid_uid = 2000


disable_plaintext_auth = yes
# Allow plain text password per IP address/net
#remote 192.168.0.0/24 {
#   disable_plaintext_auth = no
#}

# Mail location and mailbox format.
mail_location = maildir:%Lh/Maildir/:INDEX=%Lh/Maildir/

# Authentication related settings.
# Append this domain name if client gives empty realm.
auth_default_realm =

# Authentication mechanisms.
auth_mechanisms = PLAIN LOGIN

# LMTP server (Local Mail Transfer Protocol).
# Reference: http://wiki2.dovecot.org/LMTP
service lmtp {
    user = vmail

3 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: Clients Fail to read files - ? Dovecot issue

pbf343 wrote:

Performed an rsync from old system to new system.
However, clients are not showing/accessing files even though the roundcube web interface seems to show the subscribed folders on the left side frame, it hangs on the right side frame and never connects.

Did you read our migration tutorial here?
http://www.iredmail.org/docs/migrate.to … dir-format

WARNING: please make sure maildir path stored in SQL/LDAP matches the mailbox path on file system, so that mail clients can find imported emails.

4 Reply by pbf343

  • pbf343
  • Member
  • Offline
  • Registered: 2013-05-30
  • Posts: 243

Re: Clients Fail to read files - ? Dovecot issue

Yes, read it and nothing in it would appear to be the culprit to the issue.  Additionally, cannot be related to maildir format as it is same version of iRedMail systems, not different format.

Remember the path and users should be identical for installs and appear to be such when checking an account on each system.

/var/vmail/vmail1/_domain_tld

NOTE:  Did a test migration of the databases previously (dump production, restore to this machine same versions).  Was working on DKIM Keys and rolled back to the installed version of the databases (i.e. to see the "first" virtual mail account) to see/test some DKIM features.  So currently the only account on the "new" machine is the original first account (not the production accounts to be restored).

5 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: Clients Fail to read files - ? Dovecot issue

Please turn on debug mode in Dovecot to see what the problem is. Reference:
http://www.iredmail.org/docs/debug.dovecot.html

Paste log related to your testing here so that others can help troubleshoot.

6 Reply by pbf343

  • pbf343
  • Member
  • Offline
  • Registered: 2013-05-30
  • Posts: 243

Re: Clients Fail to read files - ? Dovecot issue

r/
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: maildir++: root=/var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir, index=, control=, inbox=/var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir, alt=
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl: initializing backend with data: vfile
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl: acl username = postmaster@domain_name
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl: owner = 1
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl vfile: Global ACL directory: (none)
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: Namespace : type=shared, prefix=Shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=yes location=maildir:%Lh/Maildir/:INDEX=%Lh/Maildir/Shared/%u
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: shared: root=/var/run/dovecot, index=, control=, inbox=, alt=
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl: initializing backend with data: vfile
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl: acl username = postmaster@domain_name
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl: owner = 0
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl vfile: Global ACL directory: (none)
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl vfile: file /var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir/dovecot-acl not found
Jul 29 22:39:32 imap-login: Info: Login: user=<postmaster@domain_name>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=2367, secured, session=</fX3nA4cdwB/AAAB>
Jul 29 22:39:32 imap: Debug: Loading modules from directory: /usr/lib64/dovecot
Jul 29 22:39:32 imap: Debug: Module loaded: /usr/lib64/dovecot/lib01_acl_plugin.so
Jul 29 22:39:32 imap: Debug: Module loaded: /usr/lib64/dovecot/lib02_imap_acl_plugin.so
Jul 29 22:39:32 imap: Debug: Module loaded: /usr/lib64/dovecot/lib10_quota_plugin.so
Jul 29 22:39:32 imap: Debug: Module loaded: /usr/lib64/dovecot/lib11_imap_quota_plugin.so
Jul 29 22:39:32 imap: Debug: Module loaded: /usr/lib64/dovecot/lib20_autocreate_plugin.so
Jul 29 22:39:32 imap: Debug: Added userdb setting: plugin/master_user=postmaster@domain_name
Jul 29 22:39:32 imap: Debug: Added userdb setting: plugin/quota_rule=*:bytes=1073741824
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: Effective uid=2000, gid=2000, home=/var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47/
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: Quota root: name=user backend=dict args=:proxy::quotadict
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: Quota rule: root=user mailbox=* bytes=1073741824 messages=0
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: Quota warning: bytes=912680550 (85%) messages=0 reverse=no command=quota-warning 85 postmaster@domain_name
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: Quota warning: bytes=966367641 (90%) messages=0 reverse=no command=quota-warning 90 postmaster@domain_name
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: Quota warning: bytes=1020054732 (95%) messages=0 reverse=no command=quota-warning 95 postmaster@domain_name
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: dict quota: user=postmaster@domain_name, uri=proxy::quotadict, noenforcing=0
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir/:INDEX=/var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir/
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: maildir++: root=/var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir, index=, control=, inbox=/var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir, alt=
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl: initializing backend with data: vfile
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl: acl username = postmaster@domain_name
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl: owner = 1
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl vfile: Global ACL directory: (none)
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: Namespace : type=shared, prefix=Shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=yes location=maildir:%Lh/Maildir/:INDEX=%Lh/Maildir/Shared/%u
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: shared: root=/var/run/dovecot, index=, control=, inbox=, alt=
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl: initializing backend with data: vfile
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl: acl username = postmaster@domain_name
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl: owner = 0
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl vfile: Global ACL directory: (none)
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl vfile: file /var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir/dovecot-acl not found
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl vfile: file /var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir/.Drafts/dovecot-acl not found
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl vfile: file /var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir/.Sent/dovecot-acl not found
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl vfile: file /var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir/.Junk/dovecot-acl not found
Jul 29 22:39:32 imap(postmaster@domain_name): Debug: acl vfile: file /var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir/.Trash/dovecot-acl not found

7 Reply by pbf343

  • pbf343
  • Member
  • Offline
  • Registered: 2013-05-30
  • Posts: 243

Re: Clients Fail to read files - ? Dovecot issue

-rwx------. 1 vmail vmail   0 Jul 29 14:31 dovecot-acl-list

8 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: Clients Fail to read files - ? Dovecot issue

pbf343 wrote:

location=maildir:/var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir/

Dovecot is accessing this directory, did you migrate old emails to this directory?

9 Reply by pbf343

  • pbf343
  • Member
  • Offline
  • Registered: 2013-05-30
  • Posts: 243

Re: Clients Fail to read files - ? Dovecot issue

ZhangHuangbin wrote:
pbf343 wrote:

location=maildir:/var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47//Maildir/

Dovecot is accessing this directory, did you migrate old emails to this directory?

OK guess it did not post previously. 

Yes, the mail is going to that directory except that the path has double "//" here while the file system would be a single slash as such"

/var/vmail/vmail1/domain_name/p/o/s/postmaster-2015.07.12.13.54.47/Maildir/

Example:
drwx------. 2 vmail vmail   2 Jul 29 14:31 cur
-rwx------. 1 vmail vmail   0 Jul 29 14:31 dovecot-acl-list
-rw-------. 1 vmail vmail 11K Jul 30 05:08 dovecot.index.cache
-rwx------. 1 vmail vmail  40 Jul 29 14:31 dovecot.index.log
-rwx------. 1 vmail vmail 120 Jul 29 20:53 dovecot.mailbox.log
-rwx------. 1 vmail vmail  51 Jul 29 14:31 dovecot-uidlist
-rwx------. 1 vmail vmail   8 Jul 29 20:53 dovecot-uidvalidity
-rwx------. 1 vmail vmail   0 Jul 29 14:31 dovecot-uidvalidity.55b91bfe
drwx------. 5 vmail vmail   8 Jul 30 02:04 .Drafts
drwx------. 5 vmail vmail   8 Jul 30 02:06 .Junk
drwx------. 2 vmail vmail   4 Jul 30 03:39 new
drwx------. 5 vmail vmail  10 Jul 30 02:04 .Sent
-rw-------. 1 vmail vmail  29 Jul 29 20:53 subscriptions
drwx------. 2 vmail vmail   2 Jul 30 05:09 tmp
drwx------. 5 vmail vmail   8 Jul 30 02:04 .Trash

10 Reply by pbf343

  • pbf343
  • Member
  • Offline
  • Registered: 2013-05-30
  • Posts: 243

Re: Clients Fail to read files - ? Dovecot issue

When doing the various parts of the install, I did restore the mysql database at one point without the mount being present.  Think I even logged into the system.

11 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: Clients Fail to read files - ? Dovecot issue

pbf343 wrote:

I did restore the mysql database at one point without the mount being present.  Think I even logged into the system.

If you migrated without the mount, then mails were restored on local file system. If you mount the remote NFS, files will be invisible until you unmount the NFS. Please make sure you have migrated mails under this directory.

Btw, double slashes in maildir doesn't matter in this case.

12 Reply by pbf343

  • pbf343
  • Member
  • Offline
  • Registered: 2013-05-30
  • Posts: 243

Re: Clients Fail to read files - ? Dovecot issue

OK so unmounted the external storage.  Attempted to move some items around from disk and then remount.  Rsynced a production account to system with directory created with first virtual domain.  Then tried to access the production system using an account on the domain rsynced over.  It failed as still got permissions (?) errors or errors in reading/accessing e-mail. 

So restored the databases "all" from initial install.

Next unmounted disk share again.  Shutdown mail services. 
Restored mysql "all database" to install point.
Rsync back the two test accounts from mv of data.
Added disk space to virtual machine. 
Mysql dump data from production to new system.
Rsync all data from production mail system to local /var/vmail/vmail1 storage. 

Still getting acl errors like noted below.   (NOTE: a final mysqldump of database was NOT performed from time of inital export/import of data and new messages rsyncing to system). 


Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/.Deleted Messages/dovecot-acl not found
Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/.Drafts/dovecot-acl not found
Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/.Junk/dovecot-acl not found
Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/.Notes/dovecot-acl not found
Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/.Sent/dovecot-acl not found
Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/.Trash/dovecot-acl not found
Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/dovecot-acl not found

13 Reply by pbf343

  • pbf343
  • Member
  • Offline
  • Registered: 2013-05-30
  • Posts: 243

Re: Clients Fail to read files - ? Dovecot issue

ZhangHuangbin wrote:
pbf343 wrote:

I did restore the mysql database at one point without the mount being present.  Think I even logged into the system.

If you migrated without the mount, then mails were restored on local file system. If you mount the remote NFS, files will be invisible until you unmount the NFS. Please make sure you have migrated mails under this directory.

Btw, double slashes in maildir doesn't matter in this case.

   Thank you. 


FYI: When I use the term migrate. We did not rsync data to local instal prior to the mount.  We did install which creates the local disk space.  Logged in to the "initial domain user account" that is created during the 1st virtual domain section of the install to see if mail worked.  Then tried to mount the external storage with target of vmail.

14 Reply by pbf343

  • pbf343
  • Member
  • Offline
  • Registered: 2013-05-30
  • Posts: 243

Re: Clients Fail to read files - ? Dovecot issue

pbf343 wrote:

OK so unmounted the external storage.  Attempted to move some items around from disk and then remount.  Rsynced a production account to system with directory created with first virtual domain.  Then tried to access the production system using an account on the domain rsynced over.  It failed as still got permissions (?) errors or errors in reading/accessing e-mail. 

So restored the databases "all" from initial install.

Next unmounted disk share again.  Shutdown mail services. 
Restored mysql "all database" to install point.
Rsync back the two test accounts from mv of data.
Added disk space to virtual machine. 
Mysql dump data from production to new system.
Rsync all data from production mail system to local /var/vmail/vmail1 storage. 

Still getting acl errors like noted below.   (NOTE: a final mysqldump of database was NOT performed from time of inital export/import of data and new messages rsyncing to system). 


Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/.Deleted Messages/dovecot-acl not found
Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/.Drafts/dovecot-acl not found
Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/.Junk/dovecot-acl not found
Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/.Notes/dovecot-acl not found
Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/.Sent/dovecot-acl not found
Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/.Trash/dovecot-acl not found
Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/dovecot-acl not found

15 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: Clients Fail to read files - ? Dovecot issue

pbf343 wrote:

Aug 01 11:28:16 imap(email_user@domain_name.tld): Debug: acl vfile: file /var/vmail/vmail1/domain_name.tld/0/3/f/email_user-2014.07.21.12.56.00//Maildir/dovecot-acl not found

ACL vfile not found is safe to ignore.