1

Topic: Migration questions

============ Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.8
- Linux/BSD distribution name and version: CentOS 7
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): Nginx
- Manage mail accounts with iRedAdmin-Pro?: Yes
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

I am preparing for a migration from iRedMail 0.8.5 to 0.9.8. (Yes, big jump, but I have learnt my lesson.) So far the preparations are going well, although they're behind schedule.

I feel that I should first thank Zhang for saving me a bunch of time by taking care of the installation. His help is worth the cost. I'm impatient sometimes, but his documentation is first class, and so is his product.

I have a few questions as I'm going along:

  1. Can I prevent a domain admin from changing the quota of an account at all, or can I only prevent him from going above a certain limit?

  2. Is there somewhere I can change the default quota for a new account, either globally or on a per-domain basis?

  3. I noticed reference in a configuration file somewhere (possibly Nginx) to hosting iRedAdmin and webmail on their own domains. I'm not familiar with Nginx, having used Apache for years. Has anyone done something like put iRedAdmin on admin.example.com and the webmail on webmail.exmple.com?

  4. Speaking of Roundcube, if I import the roundcube database from the old server and then run the current version of Roundcube on the new server, will it automatically upgrade the old database? The information at https://github.com/roundcube/roundcubemail/wiki/Upgrade seems to hint it will, but it's not clear.

Thanks.


Craig

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: Migration questions

craig wrote:
  1. Can I prevent a domain admin from changing the quota of an account at all, or can I only prevent him from going above a certain limit?

  2. Is there somewhere I can change the default quota for a new account, either globally or on a per-domain basis?

  3. I noticed reference in a configuration file somewhere (possibly Nginx) to hosting iRedAdmin and webmail on their own domains. I'm not familiar with Nginx, having used Apache for years. Has anyone done something like put iRedAdmin on admin.example.com and the webmail on webmail.exmple.com?

  4. Speaking of Roundcube, if I import the roundcube database from the old server and then run the current version of Roundcube on the new server, will it automatically upgrade the old database? The information at https://github.com/roundcube/roundcubemail/wiki/Upgrade seems to hint it will, but it's not clear.

1)

The latter one. With iRedAdmin-Pro, you can set a quota for domain, and domain admin is free to balance the mailbox quota for different users, but not exceed the domain quota.

Curious why you want to prevent quota change by domain admin for one account? if you don't mind sharing.

2)

Sure. Login to iRedAdmin-Pro as global admin, go to domain profile page.

3)

It's easy, create a new config file under /etc/nginx/sites-available/ for your web domain "example.com" like this:

server {
    listen 80;
    server_name example.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443;
    server_name example.com;

    root /var/www/html;
    index index.php index.html;

    include /etc/nginx/templates/misc.tmpl;
    include /etc/nginx/templates/ssl.tmpl;
    include /etc/nginx/templates/roundcube-subdomain.tmpl;
    include /etc/nginx/templates/iredadmin-subdomain.tmpl;
}

Don't forget to link this file to /etc/nginx/sites-enabled/, otherwise it's not loaded by Nginx.

4)

Depends on how you upgrade Roundcube. If you follow section "On the command line", and upgrade with command "bin/installto.sh", it will upgrade the SQL database. Note: this requires you have old Roundcube directory and database on new server.

Another way is, import the SQL files under "SQL/mysql/<xxx>.sql" manually. You can query SQL table "roundcubemail.system" on OLD server to know the version (of SQL structure) it runs, then find newer SQL files under "SQL/mysql/" and import them manually (the sql file names contains year, month, day, you won't miss it).

3

Re: Migration questions

Thanks Zhang. Appreciate your thorough response.

ZhangHuangbin wrote:

Curious why you want to prevent quota change by domain admin for one account? if you don't mind sharing.

I am considering changing our rates based on the size of the email account, so the problem I see is this: If a domain user starts with nine 2 GB accounts and one 5 GB account, the "Max quota of mail user" for the domain must be 5 GB. If I set the "Domain quota size" to (in this example, 9x2+5=) 23 GB, this means that the domain user could theoretically reduce the quotas of (let's say) six of his small accounts to 0.5 GB and increase the quotas of three of his accounts to 5 GB. So with the other 5 GB account, he now has four 5 GB accounts (20 GB) and six 0.5 GB accounts (3 GB), which still adds up to ([4x5]+[6x0.5]=) 23 GB, but he gets three "big" accounts without having to pay extra for them.

Of course, I could charge based on total disk usage for the domain, but that's not my first choice. But I see advantages to, as you say, letting the "domain admin ... balance the mailbox quota for different users, but not exceed the domain quota."

Unless, I wonder if the global admin can override the "Max quota of mail user" for a domain to create (in this example) one 5 GB user, while still leaving the "Max quota of mail user" set to 2 GB. (Tried that; it won't work.) Or increase it temporarily, create the 5 GB User, and then set it back again. (That *does* seem to work, as iRedAdmin still shows that the quota for the 5 GB account is still 5 GB when I lower the "Max quota of mail user" to 2 GB.) What do you think?

ZhangHuangbin wrote:

Sure. Login to iRedAdmin-Pro as global admin, go to domain profile page.

Thanks, I see that, but that only does it for one domain. I worded my question wrong; I suppose I'm looking for a way to set the default quota for all new users globally. Or if I want to do it for all existing domains do I just change the value of the vmail.domain.settings column? Then I can set it for each domain in the future as I create it in iRedAdmin.

And would I edit vmail.mailbox.quota to change the quota for existing users? I'm just looking for a way to do this quickly for all users rather than doing it through the web interface.

ZhangHuangbin wrote:

It's easy, create a new config file under /etc/nginx/sites-available/ for your web domain "example.com" like this:

Thanks. I will work on that next.

ZhangHuangbin wrote:

Depends on how you upgrade Roundcube.

OK, well, the current version of Roundcube is installed already on the new server, of course. So I was looking at those SQL files in "SQL/mysql/<xxx>.sql", but I was hoping the current version would recognise the old database and import those automatically. However, if not, I will do it manually if that's the right way to do it.

Thanks again.


Craig

4

Re: Migration questions

Can I assume that I do not need to import the old "amavisd" database to the new server? It just looks like stuff that is related to the processing and tracking of mail on the old server, and on the new server it will be repopulated automatically with data related to the new server.

5

Re: Migration questions

Another question. I'll keep them under this thread.

The "new" (to me) filtering of domains by the first letter is great. Thanks. However, when I'm displaying all domains, what is the sort order? I have multiple pages of domain, but just the first page lists domains running from A to Z, when there are many domains in between and Z domains shouldn't appear until the last page.

Clicking on the "Domain" header changes the sort order for just that page, but even then the order still doesn't make sense. It seems that domains with aliases get sorted at the top, then domains starting with A. And those Z domains are still on the page.

Have I done something wrong?


Craig

6

Re: Migration questions

OK, so I imported the various databases from my old server to the new server, but the initial domain set up during the installation of iRedMail (and iRedAdmin-Pro) on the new server is, of course, missing, because it did not exist on the old server. The mail spool for the domain exists in the file system, of course.

I presume this can be fixed by simply importing certain records from the original databases. I have a back-up copy of the "vmail" database, which seems to be the only database with relevant data; should I just import the records in that?

Thanks.


Craig

7

Re: Migration questions

craig wrote:

Can I assume that I do not need to import the old "amavisd" database to the new server? It just looks like stuff that is related to the processing and tracking of mail on the old server, and on the new server it will be repopulated automatically with data related to the new server.

According to the information at https://docs.iredmail.org/migrate.to.ne … erver.html the "amavisd" database is only important if there is data in the mailaddr, policy, users and wblist tables. Since I don't think Amavisd was ever really properly working on my old server these are empty, so I seem to be on the right track here.

8

Re: Migration questions

craig wrote:

OK, so I imported the various databases from my old server to the new server, but the initial domain set up during the installation of iRedMail (and iRedAdmin-Pro) on the new server is, of course, missing, because it did not exist on the old server. The mail spool for the domain exists in the file system, of course.

An unfortunate side effect of this is that I cannot view the email with all of the information about the new iRedMail installation. I was looking all over for it (as I hadn't saved it somewhere else separately yet) then realised where it is. It's still on the server, just inaccessible in the normal way until I figure out how to put the domain back into or reattach it to the iRedAdmin system.

9

Re: Migration questions

Here is a summary of the outstanding issues I need help addressing:

* Setting a default mail box quota server-wide, not just for one domain. (I'm fine doing it in the database directly, if necessary.)
* A quick way to change the quota for a lot of users at the same time, again, probably directly in the database. Is the right place to change this in vmail.mailbox.quota?
* Seemingly broken sorting of domains when all domains are displayed.
* How to keep the domain that was configured when iRedMail was installed when I import all the domains from the old server.

Details are all in my previous posts in this thread. Thanks in advance for your help with these.

10

Re: Migration questions

craig wrote:

Unless, I wonder if the global admin can override the "Max quota of mail user" for a domain to create (in this example) one 5 GB user, while still leaving the "Max quota of mail user" set to 2 GB. (Tried that; it won't work.) Or increase it temporarily, create the 5 GB User, and then set it back again. (That *does* seem to work, as iRedAdmin still shows that the quota for the 5 GB account is still 5 GB when I lower the "Max quota of mail user" to 2 GB.) What do you think?

While you reduce the "max quota of mail user", it's too risky to reduce quota of existing mail user, if someone's mailbox is larger than 2GB, further inbound mails will be rejected by Dovecot due to quota exceeded.

craig wrote:

I suppose I'm looking for a way to set the default quota for all new users globally. Or if I want to do it for all existing domains do I just change the value of the vmail.domain.settings column?

In early iRedAdmin-Pro release, it's a global setting, but that is not good for a hosting service, different domain may need different default quota, so we made it a per-domain setting.

Yes if you want to change it, just update vmail.domain.settings column.

craig wrote:

And would I edit vmail.mailbox.quota to change the quota for existing users?

Yes. Another ways:

- Update quota through iRedAdmin-Pro RESTful API: https://docs.iredmail.org/iredadmin-pro … l.api.html
- Run script shipped in iRedAdmin-Pro: tools/update_mailbox_quota.py.

11

Re: Migration questions

craig wrote:

Can I assume that I do not need to import the old "amavisd" database to the new server?

FYI: https://docs.iredmail.org/amavisd.sql.db.html

Amavisd has two settings to use its SQL tables:
@storage_sql_dsn uses 4 sql tables (msgs, msgrcpt, maddr, quarantine), and it's used to store:

* basic info of inbound and outbound message. e.g. mail subject, sender address, recipient address, timestamp, etc. Note: no mail body.
* quarantined mails. Note: it stores full message of quarantined mail, including mail body.

@lookup_sql_dsn: uses 4 sql tables (mailaddr, policy, users, wblist), and it's used to store:

* per-account spam policy
* per-account white/blacklists

12

Re: Migration questions

craig wrote:

However, when I'm displaying all domains, what is the sort order?

It's sorted by domain name, alphabetized.

craig wrote:

Clicking on the "Domain" header changes the sort order for just that page, but even then the order still doesn't make sense. It seems that domains with aliases get sorted at the top, then domains starting with A. And those Z domains are still on the page.

Could you help capture some screenshots (with comment text in picture) to help me understand this issue?

13

Re: Migration questions

craig wrote:

* How to keep the domain that was configured when iRedMail was installed when I import all the domains from the old server.

I don't quite understand this question. If you don't delete the sql records related to this domain, then it will be kept.

14

Re: Migration questions

ZhangHuangbin wrote:

While you reduce the "max quota of mail user", it's too risky to reduce quota of existing mail user, if someone's mailbox is larger than 2GB, further inbound mails will be rejected by Dovecot due to quota exceeded.

OK, thanks. I will experiment.

ZhangHuangbin wrote:

In early iRedAdmin-Pro release, it's a global setting, but that is not good for a hosting service, different domain may need different default quota, so we made it a per-domain setting.

Makes sense, but I want to set a baseline default quota (which is more than it was on the old server), and then possibly change it on individual domains later.

ZhangHuangbin wrote:

Yes if you want to change it, just update vmail.domain.settings column.

Perfect, thanks.

ZhangHuangbin wrote:

Could you help capture some screenshots (with comment text in picture) to help me understand this issue?

I have attached screenshots of both the old and new servers, with the same list of domains. As you can see, on the old server the list starts at A and only reaches C on the first page. On the new server there is one domain starting with a J at the top of the list, then the list goes to a domain starting with "ac" (when you can see on the list on the old server that there are domains starting with "ab") and it goes to Z on the first page. Many domains have been skipped. On the second page the list again goes from A all the way to Z; different domains, but again many skipped.

If you want I will give you access to the development server so you can see for yourself.

ZhangHuangbin wrote:

I don't quite understand this question. If you don't delete the sql records related to this domain, then it will be kept.

Yes, that makes sense, except the SQL dumped from the old server included "DROP TABLE IF EXISTS" SQL instructions, so the original domain was lost when the "vmail" database was imported. But it's just a development server so on the production server I will make sure it is not lost when I import the SQL.

Thanks for your help. Hopefully that is all I need to move forward, but I hope the sorting issue can be fixed, or the cause of the problem found.

Post's attachments

new_server_redacted.png 195.46 kb, file has never been downloaded. 

old_server_redacted.png 183.49 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

15

Re: Migration questions

craig wrote:

I have attached screenshots of both the old and new servers, with the same list of domains. As you can see, on the old server the list starts at A and only reaches C on the first page. On the new server there is one domain starting with a J at the top of the list, then the list goes to a domain starting with "ac"

Seems a bug of iRedAdmin-Pro. Could you try patch below?

diff -r 0ef9298fd076 libs/sqllib/domain.py
--- a/libs/sqllib/domain.py    Fri May 25 23:23:50 2018 +0800
+++ b/libs/sqllib/domain.py    Sat May 26 09:00:07 2018 +0800
@@ -1593,6 +1593,7 @@
                              what=sql_what,
                              where=sql_where,
                              limit=settings.PAGE_SIZE_LIMIT,
+                             order='domain',
                              offset=(page - 1) * settings.PAGE_SIZE_LIMIT)
 
         else:
@@ -1615,6 +1616,7 @@
                              what=sql_what,
                              where=sql_where,
                              limit=settings.PAGE_SIZE_LIMIT,
+                             order='domain.domain',
                              offset=(page - 1) * settings.PAGE_SIZE_LIMIT)
 
         if not qr:

After patched, please restart uwsgi service to reload the changed files.

16 (edited by craig 2018-05-27 07:13:02)

Re: Migration questions

ZhangHuangbin wrote:

Seems a bug of iRedAdmin-Pro. Could you try patch below?

Thanks Zhang. That works with the default sort order (loading the page when I click "Domains and Accounts" or when I click a letter), but there are still some domains listed out of order if I then click on the "Domain" heading. The domains listed out of order seem to be domains with aliases, and a domain with one alias is sorted before a domain with two aliases before a domain with three aliases before domains with no aliases, etc. If you look at my screenshot of the new server you'll see that the first domain has a black smudge to the right of the domain name. It's in the blurred out area, but that black smudge is a big "A1", indicating that the first domain ("je*****") has one alias. That is still happening when I manually sort on the domain column.

Otherwise it's working well enough for me at the moment.

17

Re: Migration questions

I changed a setting in /var/www/iredadmin/settings.py and restarted uwsgi per the instructions at https://docs.iredmail.org/customize.maildir.path.html , but the last modified date on settings.pyc remains the same as the installation date.

I also tried restarting nginx per https://docs.iredmail.org/install.ireda … -iredadmin , but that didn't change anything either.

I don't see anything in logs suggesting an error.

18

Re: Migration questions

craig wrote:

but the last modified date on settings.pyc remains the same as the installation date.

You can remove this "settings.pyc" file, it will be re-generated automatically.
But the point is, does the changed parameter work now?

19

Re: Migration questions

ZhangHuangbin wrote:

You can remove this "settings.pyc" file, it will be re-generated automatically.

Thanks. Seems to be regenerated by a cron job at one minute past the hour. I deleted settings.pyc at about half past the hour and it came back about half an hour later.

ZhangHuangbin wrote:

But the point is, does the changed parameter work now?

Don't know. I changed the webmaster email address, but I don't see that displayed in iRedAdmin. The only time I remember seeing that email used is for over quota emails.

20

Re: Migration questions

craig wrote:

Don't know. I changed the webmaster email address, but I don't see that displayed in iRedAdmin. The only time I remember seeing that email used is for over quota emails.

You can see the webmaster address if you login to iRedAdmin-Pro as a normal domain admin instead of global admin, at the footer of page.

21

Re: Migration questions

ZhangHuangbin wrote:

You can see the webmaster address if you login to iRedAdmin-Pro as a normal domain admin instead of global admin, at the footer of page.

Ah, then it worked. smile

I should also add, as a follow-up to an earlier question, that setting up iRedAdmin and Roundcube at the root of two different sub-domains worked perfectly. Thanks.

22 (edited by craig 2018-06-04 03:11:08)

Re: Migration questions

Odd question: For various reasons I have gone against the advice in iRedAdmin-Pro and created domain admins on the default domain of my mail server rather than marking existing users under individual domains as admins. (One reason is that if I set up a domain and the domain owner wants to set up email addresses himself, there is no user on the domain for me to make the domain admin.) However, those admins can still mark themselves as domain admins once they have logged in as the admin user I create, and log into iRedAdmin-Pro using their own email address from that point on.

For example, my domain admins all look like name@mail43.example.com, mail43.example.com being the primary domain on my *old* server.

If I want to make all of those existing users look like name@mail.example.com (without the number 43 in the sub-domain), is it just a matter of just changing all of those user names somewhere in the vmail database?

Thanks.

23

Re: Migration questions

With respect to my previous question, never mind. I decided to delete those users through iRedAdmin-Pro on the old server, and I will recreate them on the new server.