1 (edited by Wraptor 2017-11-02 21:51:50)

Topic: [Guide] External MySQL server using IPv6

==== Required information ====
- iRedMail version (check /etc/iredmail-release): iRedmail v0.9.7
- Linux/BSD distribution name and version: Debian 9.2 Stretch
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): External MySQL
- Web server (Apache or Nginx): Nginx
- Manage mail accounts with iRedAdmin-Pro? No
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

I only tested the following guide on Debian 9.2 (Stretch), but I referred to the files for other Distro's and/or BSD aswell.
Testers for other distro's are appreciated.

These issues can and (should be resolved) in my opinion as moving to IPv6 is essential in the coming years and not having to deal with these issues is important for newer iRedmail users.

If you're trying to setup a mailserver using iRedmail using the following guide in order to use a remote mysql database be aware that iRedmail does not natively support this.
Guide: https://docs.iredmail.org/install.iredm … erver.html

I've posted a topic earlier this week regarding SOGo not fully working, this wasn't only related to SOGo but everything else aswel. The SOGo issue has been resolved as a bugfix and should work for everybody else now accordingly to ZhangHuangbin.

For demonstration purposes only let's say we used this command to install iRedmail:

USE_EXISTING_MYSQL='YES' \
    MYSQL_SERVER_ADDRESS='fd01:2345:6789:1::1' \                 # Your external MySQL server's IP
    MYSQL_SERVER_PORT='3306' \
    MYSQL_ROOT_USER='admin_iredmail' \
    MYSQL_ROOT_PASSWD='admin_password' \
    MYSQL_GRANT_HOST='fd01:2345:6789:1::99' \                       # Your iRedmail server's IP
    bash iRedMail.sh

Keep in mind the remote MySQL server should allow the user 'admin_iredmail' (in this case) all privileges as written in the guide and has been allowed by the grant host ip.
The Grant Host should also be statically assigned so it'll never change.

After you've installed iRedmail nothing will work as nothing supports IPv6 addresses using mysql, and encoding it as IPv6 (Adding brackets around them eg. [fd01:2345:6789:1::1]) does not work for everything (Works for SOGo) meaning you'll need to create a local host entry for the ipv6 mysql address in your /etc/hosts file.
Here's how it would look in this example:

# Default entries
127.0.1.1       server.domain.tld server localhost
::1             server.domain.tld server localhost ip6-localhost ip6-loopback

ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

# Add this entry
fd01:2345:6789:1::1 mysql

You can change 'mysql' to whatever you like, but this name will be used everywhere to refer to the ipv6 address.

Confige Postfix to work:

# Running the following line of code will replace the first line of every file in the /etc/postfix/mysql folder.
# It will replace it to "hosts       = mysql"
# Relace 'mysql' to the name in your /etc/hosts file.

# For all Linux distro's and OpenBSD:
sed -i "1s/.*/hosts       = mysql/" /etc/postfix/mysql/*

# For FreeBSD:
sed -i "1s/.*/hosts       = mysql/" /usr/local/etc/postfix/mysql/*

# Save the file and restart postfix
systemctl restart postfix

Dovecot should work out of the box using ipv6 addresses so no problems here, if you do struggle try the following:
Confige dovecot to work:

# Edit the following file using your favorite editor (Nano in my case)

# For all Linux distro's and OpenBSD:
nano /etc/dovecot/dovecot-mysql.conf

# For FreeBSD
nano /usr/local/etc/dovecot/dovecot-mysql.conf

# Find the line starting with CONNECT
# Change 'host=fd01:2345:6789:1::1' (this example) to your hostname (Specified in /etc/hosts).
# In thise case it would be:
'host=mysql'

# Save the file and restart dovecot
systemctl restart dovecot

Configure iRedAPD to work:

# Edit the following file using your favorite editor (Nano in my case)
nano /opt/iredapd/settings.py

# Find the following lines starting with:
#     vmail_db_server =
#     amavisd_db_server =
#     iredapd_db_server =
# Update all 3 entries to your mysql hostname configured in /etc/hosts
# This example:
vmail_db_sever = "mysql"
amavisd_db_server = "mysql"
iredapd_db_server = "mysql"

# Save the file and restart iredapd
systemctl restart iredapd

Configure Amavisd/Amavis-new to work:

# Edit the following file using your favorite editor (Nano in my case)

# For RHEL/CentOS:
nano /etc/amavisd/amavisd.conf

# For Debian/Ubuntu:
nano /etc/amavis/conf.d/50-user

# For FreeBSD:
nano /usr/local/etc/amavisd.conf

# For OpenBSD:
nano /etc/amavisd.conf


# Change the 'host=fd01:2345:6789:1::1' (this case) to 'host=mysql' where mysql is the hostname you entered in /etc/hosts
#  for the line starting with @storage_sql_dsn so it looks like:
@storage_sql_dsn = (['DBI:mysql:database=amavisd;host=mysql;port=3306', 'LEAVE THIS', 'LEAVE THIS!']);

# Keep in mind not to change the original value to 'LEAVE THIS' like the example as these are the
#  predefined database credentials for amavisd.

# Save the configuration and restart amavisd/amavis-new
systemctl amavisd restart


I do not use iRedadmin (Standard or pro) or Roundcube (Only SOGo and command line administering using custom tools) so I did not require a fix for those.


Post future problems down in this post and I'll be sure to help you out.

Kind Regards.
Wraptor

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: [Guide] External MySQL server using IPv6

Thanks for sharing.

Could you help test the iRedMail installation with SQL server address like this:

    MYSQL_SERVER_ADDRESS='[fd01:2345:6789:1::1]'

IPv6 address is surrounded by "[]" directly. This should fix all issues mentioned in your post.

3 (edited by Wraptor 2017-10-31 17:07:43)

Re: [Guide] External MySQL server using IPv6

I tried using that but then the installer wouldn't be able to connect to the external mysql server to create the databases and users. Also not every service (iredapd and postfix) supports the ipv6 even with the brackets in my previous testing

4

Re: [Guide] External MySQL server using IPv6

Wraptor wrote:

Also not every service (iredapd and postfix) supports the ipv6 even with the brackets in my previous testing

Did a quick test, iRedAPD and Postfix work fine with '[<ipv6>]' SQL server address. (P.S. i use local ipv6 address '[::1]' for testing.)

5

Re: [Guide] External MySQL server using IPv6

Hmmm, I tried using ips with brackets before, didn't work for me using an external ip address. "[::1]" is hardcoded into the os I think.

Did you resolve using brackets in the ip in the installer arguments? That didn't work for me 3days ago

6

Re: [Guide] External MySQL server using IPv6

If you use '[<ipv6>]' on command line directly, no change required in iRedMail installer.

7

Re: [Guide] External MySQL server using IPv6

I'll try reconstruating this issue. Shouldn't work though

8 (edited by Wraptor 2017-11-02 22:05:18)

Re: [Guide] External MySQL server using IPv6

I'm currently reconstruating my previous setup in a test environment with the following installation command:

USE_EXISTING_MYSQL='YES' \
    MYSQL_SERVER_ADDRESS='[fd10::1000:0:0:1]' \
    MYSQL_SERVER_PORT='3306' \
    MYSQL_ROOT_USER='admin_iredmail' \
    MYSQL_ROOT_PASSWD='admin_password' \
    MYSQL_GRANT_HOST='[fd10::2:0:0:1]' \
bash iRedMail.sh

The database server has the admin_iredmail user created:

GRANT ALL PRIVILEGES ON *.* TO 'admin_iredmail'@'fd10::2:0:0:1' IDENTIFIED BY 'admin_password' WITH GRANT OPTION;
FLUSH PRIVILEGES;
FLUSH HOSTS;



This is the installation (As already predicted) that failed:

********************************************************************
* Start iRedMail Configurations
********************************************************************
[ INFO ] Create self-signed SSL certification files (2048 bits).
[ INFO ] Generate Diffie Hellman Group with openssl, please wait.
[ INFO ] Create required system account: vmail, iredadmin, iredapd.
[ INFO ] Configure Nginx web server and uWSGI.
[ INFO ] Configure PHP.
[ INFO ] Configure MariaDB database server.
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)
[ INFO ] Setup daily cron job to backup SQL databases with /var/vmail/backup/backup_mysql.sh
[ INFO ] Configure Postfix (MTA).
[ INFO ] Configure Dovecot (POP3/IMAP/Managesieve/LMTP/LDA).
[ INFO ] Configure ClamAV (anti-virus toolkit).
[ INFO ] Configure Amavisd-new (interface between MTA and content checkers).
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)
[ INFO ] Configure SpamAssassin (content-based spam filter).
[ INFO ] Configure iRedAPD (postfix policy daemon).
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)
[ INFO ] Configure iRedAdmin (official web-based admin panel).
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)
[ INFO ] Configure SOGo Groupware (Webmail, Calendar, Address Book, ActiveSync).
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)


Which is why you need to use no brackets in the installer, then it works just fine. But nothing else does afterwards as they do require that bracket encoding type of deal.


EDIT:
Also Nginx doesn't listen on ipv6 by default (Neither 80 or 443). Might be handy if Nginx would listen on [::]:80 and [::]:443 by the setup aswell, this listens on ipv6 aswell as ipv4.


iRedAPD doesn't know how to contact the ipv6 mysql server with or without the brackets

Traceback (most recent call last):
File "/opt/iredapd/tools/spf_to_greylist_whitelists.py", line 319, in <module>
qr = conn.select('greylisting_whitelist_domains', what='domain')
File "/usr/lib/python2.7/dist-packages/web/db.py", line 699, in select
return self.query(qout, processed=True)
File "/usr/lib/python2.7/dist-packages/web/db.py", line 658, in query
db_cursor = self._db_cursor()
File "/usr/lib/python2.7/dist-packages/web/db.py", line 570, in _db_cursor
return self.ctx.db.cursor()
File "/usr/lib/python2.7/dist-packages/web/db.py", line 511, in _getctx
self._load_context(self._ctx)
File "/usr/lib/python2.7/dist-packages/web/db.py", line 522, in _load_context
ctx.db = self._connect(self.keywords)
File "/usr/lib/python2.7/dist-packages/web/db.py", line 550, in _connect
return self.db_module.connect(**keywords)
File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 204, in __init__
super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (2005, "Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)")

9

Re: [Guide] External MySQL server using IPv6

Also I found out that the spamassassin service doesn't start by default on Debian, simple "update-rc.d spamassassin defaults" fixed the issue though the installer should do this to.

10

Re: [Guide] External MySQL server using IPv6

Wraptor wrote:

Also I found out that the spamassassin service doesn't start by default on Debian, simple "update-rc.d spamassassin defaults" fixed the issue though the installer should do this to.

SpamAssassin is called by Amavisd directly, the "spamassassin" service is not used at all. So you should stop it.

11

Re: [Guide] External MySQL server using IPv6

What about my post before the SpamAssassin post?

12

Re: [Guide] External MySQL server using IPv6

Wraptor wrote:

[ INFO ] Configure MariaDB database server.
ERROR 2005 (HY000): Unknown MySQL server host '[fd10::1000:0:0:1]' (-3)

Could you try to update /root/.my.cnf and work out a working MySQL server address?

I'm sorry that i don't have an remote SQL server with IPv6 address for testing right now, so i need your help.

13

Re: [Guide] External MySQL server using IPv6

My remote MySQL server is just a VM inside Hyper-V with a private vSwitch for the Mailserver and MySQL server vm.
For the MySql /root/.my.cnf you need to use the IPv6 address without brackets, so the installer should set that file to the given IP without brackets. Now iRedAPD doesn't support IPv6 addresses (With or without brackets).

I found out that:
SOGo supports IPv6 with brackets required in the mysql string.
Dovecot supports IPv6 without brackets (Native).
Postfix and iRedAPD support IPv6 using a hostname that only resolves to an ipv6 address.

I would recommend using a static hostname in /etc/hosts and set that as the hostname for all services to use. This should support both ipv4 and ipv6 if the user is using a remote server.

This also allows the changing of 1 file if the mysql server gets an IP change.

14

Re: [Guide] External MySQL server using IPv6

I will fix the SQL server address in iRedMail.

Wraptor wrote:

Postfix and iRedAPD support IPv6 using a hostname that only resolves to an ipv6 address.

Postfix should work with IPv6. Could you please try again to work it out?

Wraptor wrote:

This also allows the changing of 1 file if the mysql server gets an IP change.

This is a good idea. Which (internal) domain name do you suggest for this sql server address?

15 (edited by Wraptor 2017-11-09 14:45:01)

Re: [Guide] External MySQL server using IPv6

I used mysql for my testing but I would try something random that wouldn't interfere with current dns like mysql.iredmail or something


I'll try postfix again tonight

16

Re: [Guide] External MySQL server using IPv6

I used my own tutorial's command:

sed -i "1s/.*/hosts       = fd10::1000:0:0:1/" /etc/postfix/mysql/*

Nov  9 17:51:12 mail-test postfix/proxymap[1382]: warning: connect to mysql server fd10::1000:0:0:1: Can't connect to MySQL server on 'fd10::1000:0:0' (113 "No route to host")
Nov  9 17:51:12 mail-test postfix/trivial-rewrite[1385]: warning: proxy:mysql:/etc/postfix/mysql/transport_maps_domain.cf lookup error for "*"
Nov  9 17:51:12 mail-test postfix/trivial-rewrite[1385]: warning: proxy:mysql:/etc/postfix/mysql/transport_maps_domain.cf lookup error for "*"
Nov  9 17:51:15 mail-test postfix/proxymap[1382]: warning: connect to mysql server fd10::1000:0:0:1: Can't connect to MySQL server on 'fd10::1000:0:0' (113 "No route to host")
Nov  9 17:51:15 mail-test postfix/trivial-rewrite[1385]: warning: virtual_mailbox_domains: proxy:mysql:/etc/postfix/mysql/virtual_mailbox_domains.cf: table lookup problem
Nov  9 17:51:15 mail-test postfix/trivial-rewrite[1385]: warning: virtual_mailbox_domains lookup failure
Nov  9 17:51:15 mail-test postfix/trivial-rewrite[1385]: warning: virtual_mailbox_domains: proxy:mysql:/etc/postfix/mysql/virtual_mailbox_domains.cf: table lookup problem
Nov  9 17:51:15 mail-test postfix/trivial-rewrite[1385]: warning: virtual_mailbox_domains lookup failure
Nov  9 17:51:15 mail-test postfix/trivial-rewrite[1385]: warning: virtual_mailbox_domains: proxy:mysql:/etc/postfix/mysql/virtual_mailbox_domains.cf: table lookup problem
Nov  9 17:51:15 mail-test postfix/trivial-rewrite[1385]: warning: virtual_mailbox_domains lookup failure


Using the brackets:

sed -i "1s/.*/hosts       = [fd10::1000:0:0:1]/" /etc/postfix/mysql/*

Nov  9 17:55:44 mail-test postfix/proxymap[1489]: fatal: unknown service: 1]/tcp
Nov  9 17:55:45 mail-test postfix/smtpd[1488]: warning: private/proxymap socket: service dict_proxy_open: Connection reset by peer
Nov  9 17:55:45 mail-test postfix/qmgr[1447]: warning: private/proxymap socket: service dict_proxy_open: Success
Nov  9 17:55:45 mail-test postfix/master[1065]: warning: process /usr/lib/postfix/sbin/proxymap pid 1489 exit status 1
Nov  9 17:55:45 mail-test postfix/master[1065]: warning: /usr/lib/postfix/sbin/proxymap: bad command startup -- throttling

So it's not running on either address. Hostname is the only solution for now.
I would love to have iRedmail work with IPv6 out of the box to encourage people to use it more.

17

Re: [Guide] External MySQL server using IPv6

Please try to get help from Postfix mailing list, i believe there's a solution:
http://www.postfix.org/lists.html

18

Re: [Guide] External MySQL server using IPv6

Will my hostname solution be implemented into iRedmail?
If so there would be no need to request postfix help as it's an even better solution for management.

19

Re: [Guide] External MySQL server using IPv6

Wraptor wrote:

Will my hostname solution be implemented into iRedmail?
If so there would be no need to request postfix help as it's an even better solution for management.

Using /etc/hosts to define the SQL server address is a good idea, i will try to implement it in next iRedMail release.
But still it's necessary to figure out how to configure Postfix directly.