1 (edited by smiller50 2019-01-10 14:45:50)

Topic: Per-user alias addresses have stopped working after 0.9.9 update

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release): 0.9.9
- Deployed with iRedMail Easy or the downloadable installer? installer
- Linux/BSD distribution name and version: Debian Stretch
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): Nginx
- Manage mail accounts with iRedAdmin-Pro? No
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

Problem: Per-user alias addresses on my system have stopped working, possibly as a result of 0.9.9 update. Messages to the target mailbox are working normally, but messages to any associated aliases are being refused with 'user unknown'.

Background: I had been running on 0.9.8 for about six months and per-user alias addresses had been working normally. However recently messages to any per-user alias addresses has been failing with a 'user unknown' bounce.

Looking at the dovecot log the problem appears to be a database lookup failure however I do not know why the lookup is now failing since nothing has changed in the SQL forwardings table since the prior working config, and furthermore non-forwarded mailboxes are receiving mail normally (only forwards to aliases are failing.)

This is possibly coincident with my recent update to 0.9.9 but not absolutely certain. All (manual) update steps during the update were performed without any problems, and I have confirmed that the two new fields in the SQL vmail database table are present, and the dovecot.conf file was modified as instructed (I do not know if these are a factor in the problem, but are the only things that I can see that would be related to the upgrade.)

Any ideas as to why per-user alias forwarding is now failing after working properly since the initial installation?

Thanks in advance.


(In all examples primary mailbox is 'target@mydomain.com' and alias is 'alias_address@mydomain.com')




Relevant data from logs:

Postfix:

Jan 10 03:05:33 mx1 postfix/pipe[25684]: D63CE425FE: to=<alias_address@mydomain.com>, relay=dovecot, delay=0.05, delays=0/0.02/0/0.02, dsn=5.1.1, status=bounced (user unknown)

Dovecot:

Jan 10 03:05:33 mx1 dovecot: lda(alias_address@mydomain.com): Debug: Loading modules from directory: /usr/lib/dovecot/modules
Jan 10 03:05:33 mx1 dovecot: lda(alias_address@mydomain.com): Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so
Jan 10 03:05:33 mx1 dovecot: lda(alias_address@mydomain.com): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so
Jan 10 03:05:33 mx1 dovecot: lda(alias_address@mydomain.com: Debug: Module loaded: /usr/lib/dovecot/modules/lib15_notify_plugin.so
Jan 10 03:05:33 mx1 dovecot: lda(alias_address@mydomain.com): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mail_log_plugin.so
Jan 10 03:05:33 mx1 dovecot: lda(alias_address@mydomain.com): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mailbox_alias_plugin.so
Jan 10 03:05:33 mx1 dovecot: lda(alias_address@mydomain.com): Debug: Module loaded: /usr/lib/dovecot/modules/lib90_sieve_plugin.so
Jan 10 03:05:33 mx1 dovecot: lda(alias_address@mydomain.com): Debug: Module loaded: /usr/lib/dovecot/modules/lib90_stats_plugin.so
Jan 10 03:05:33 mx1 dovecot: auth: Debug: master in: USER#0111#011alias_address@mydomain.com#011service=lda
Jan 10 03:05:33 mx1 dovecot: auth-worker(25651): Debug: sql(alias_address@mydomain.com): SELECT 'alias_address@mydomain.com' AS master_user, LOWER(CONCAT(mailbox.storagebasedirectory, '/', mailbox.$
Jan 10 03:05:33 mx1 dovecot: auth-worker(25651): sql(alias_address@mydomain.com): unknown user
Jan 10 03:05:33 mx1 dovecot: auth: Debug: userdb out: NOTFOUND#0111

Jan 10 03:05:33 mx1 dovecot: lda(alias_address@mydomain.com): Debug: auth USER input:


Contents of dovecot-mysql.conf:

driver = mysql
default_pass_scheme = CRYPT
connect = host=127.0.0.1 port=3306 dbname=vmail user=vmail password=sql_password

# Required by doveadm tools which require to list all mail users.
iterate_query = SELECT username AS user FROM mailbox

password_query = SELECT mailbox.password, mailbox.allow_nets \
        FROM mailbox,domain \
       WHERE mailbox.username='%u' \
             AND mailbox.`enable%Ls%Lc`=1 \
             AND mailbox.active=1 \
             AND mailbox.domain=domain.domain \
             AND domain.backupmx=0 \
             AND domain.active=1

user_query = SELECT \
            '%u' AS master_user, \
            LOWER(CONCAT(mailbox.storagebasedirectory, '/', mailbox.storagenode, '/', mailbox.maildir)) AS home, \
            CONCAT(mailbox.mailboxformat, ':~/', mailbox.mailboxfolder, '/') AS mail, \
            CONCAT('*:bytes=', mailbox.quota*1048576) AS quota_rule \
        FROM mailbox,domain \
       WHERE mailbox.username='%u' \
             AND mailbox.`enable%Ls%Lc`=1 \
             AND mailbox.active=1 \
             AND mailbox.domain=domain.domain \
             AND domain.backupmx=0 \
             AND domain.active=1


SQL database entries:

MariaDB [vmail]> SELECT * FROM forwardings;
+----+---------------------------+-------------------------+----------------+----------------+-------------+---------+---------------+----------+--------+
| id | address                   | forwarding              | domain         | dest_domain    | is_maillist | is_list | is_forwarding | is_alias | active |
+----+---------------------------+-------------------------+----------------+----------------+-------------+---------+---------------+----------+--------+
|  4 | target@mydomain.com       | target@mydomain.com     | mydomain.com   | mydomain.com   |           0 |       0 |             1 |        0 |      1 |
|  5 | info@mydomain.com         | target@mydomain.com     | mydomain.com   | mydomain.com   |           0 |       0 |             1 |        1 |      1 |
|  6 | contact@mydomain.com      | target@mydomain.com     | mydomain.com   | mydomain.com   |           0 |       0 |             1 |        1 |      1 |
|  7 | postmaster@mydomain.com   | target@mydomain.com     | mydomain.com   | mydomain.com   |           0 |       0 |             1 |        1 |      1 |
|+----+-------------------------+-------------------------+----------------+----------------+-------------+---------+---------------+----------+--------+

Mail to the target mailbox (target@mydomain.com) is working but forwards to all aliases have stopped working. The above config had been working properly since 0.9.8 was originally installed.

2

Re: Per-user alias addresses have stopped working after 0.9.9 update

smiller50 wrote:

Jan 10 03:05:33 mx1 dovecot: auth-worker(25651): Debug: sql(alias_address@mydomain.com): SELECT 'alias_address@mydomain.com' AS master_user, LOWER(CONCAT(mailbox.storagebasedirectory, '/', mailbox.$

Try to run this SQL command manually, if you believe it should return a record but doesn't, please compare the SQL statements with the value in SQL table carefully.

----

Does my reply help a little? How about buying me a cup of coffee ($5) as an encouragement?

buy me a cup of coffee

3 (edited by smiller50 2019-01-10 15:02:41)

Re: Per-user alias addresses have stopped working after 0.9.9 update

I'm sorry, I do not seem to be able to assemble that query successfully (not sure how to derive the correct variable from 'LOWER(CONCAT(mailbox.storagebasedirectory, '/', mailbox.$' )  Could you provide an example query using the standard iredmail 0.9.8 mailbox directory structure? (which is what I am using)

4

Re: Per-user alias addresses have stopped working after 0.9.9 update

smiller50 wrote:

Could you provide an example query using the standard iredmail 0.9.8 mailbox directory structure? (which is what I am using)

Turn on debug mode in Dovecot, then try to send an email to reproduce this per-user alias address issue, you will find the full SQL statement in Dovecot log.

Your previous post didn't paste full SQL statement.

----

Does my reply help a little? How about buying me a cup of coffee ($5) as an encouragement?

buy me a cup of coffee

5 (edited by smiller50 2019-01-10 16:21:57)

Re: Per-user alias addresses have stopped working after 0.9.9 update

Yes, the SQL statement was truncated, sorry. The full dovecot debug line is:


Jan 10 07:51:23 mx1 dovecot: auth-worker(6606): Debug: sql(alias_address@mydomain.com): SELECT 'alias_address@mydomain.com' AS master_user, LOWER(CONCAT(mailbox.storagebasedirectory, '/', mailbox.storagenode, '/', mailbox.maildir)) AS home, CONCAT(mailbox.mailboxformat, ':~/', mailbox.mailboxfolder, '/') AS mail, CONCAT('*:bytes=', mailbox.quota*1048576) AS quota_rule FROM mailbox,domain WHERE mailbox.username='alias_address@mydomain.com' AND mailbox.`enablelda`=1 AND mailbox.active=1 AND mailbox.domain=domain.domain AND domain.backupmx=0 AND domain.active=1



However this can't be the literal query sent to MySQL since it contains several variables (i.e. LOWER(CONCAT(mailbox.storagebasedirectory, '/', mailbox.storagenode, '/', m
ailbox.maildir))
, etc.) which if not populated with the actual path will not be a valid query(?)  What I'm trying to figure out is to how to convert the variables so I can obtain the literal text of the query so I can submit it as a test. Or... if this is literally what is being sent to MySQL no wonder it is failing(?)

Sorry if I am  missing something here.

6

Re: Per-user alias addresses have stopped working after 0.9.9 update

smiller50 wrote:

... WHERE mailbox.username='alias_address@mydomain.com' ...

Wait a second, you sure it queries (per-user) alias addresses instead of user's primary address? This is not right.

Did you modify any postfix settings? especially parameter "smtpd_recipient_maps =" and files under /etc/postfix/mysql/.
Please show us output of command "postconf -n" for troubleshooting.

----

Does my reply help a little? How about buying me a cup of coffee ($5) as an encouragement?

buy me a cup of coffee

7 (edited by smiller50 2019-01-11 02:06:51)

Re: Per-user alias addresses have stopped working after 0.9.9 update

Yes, the query was for the alias mailbox, not the primary.

But with your help pointing me in the right direction I found the problem (or maybe just a workaround to the real problem), although I do not understand it. The following two clauses are from master.cf, and if I remove 'no_address_mappings' from the port 10025 entry the problem is resolved (alias/forwarded mail is received normally and the associated dovecot log entry now queries for the primary mailbox, and the query is formed correctly.)

Working config is below. Any idea why I need to vary from the default in the 0.9.8 and 0.9.9 release? And will removing 'no_address_mappings' cause any harm? I have also included the output from "postconf -n" in case it is helpful.


From master.cf:

# smtp port used by Amavisd to re-inject scanned email back to Postfix
127.0.0.1:10025 inet n  -   n   -   -  smtpd
    -o syslog_name=postfix/10025
    -o content_filter=
    -o mynetworks_style=host
    -o mynetworks=127.0.0.0/8
    -o local_recipient_maps=
    -o relay_recipient_maps=
    -o strict_rfc821_envelopes=yes
    -o smtp_tls_security_level=none
    -o smtpd_tls_security_level=none
    -o smtpd_restriction_classes=
    -o smtpd_delay_reject=no
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_end_of_data_restrictions=
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
#    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_address_mappings


# smtp port used by mlmmj to re-inject scanned email back to Postfix, with
# address mapping support
127.0.0.1:10028 inet n  -   n   -   -  smtpd
    -o syslog_name=postfix/10028
    -o content_filter=
    -o mynetworks_style=host
    -o mynetworks=127.0.0.1
    -o local_recipient_maps=
    -o relay_recipient_maps=
    -o strict_rfc821_envelopes=yes
    -o smtp_tls_security_level=none
    -o smtpd_tls_security_level=none
    -o smtpd_restriction_classes=
    -o smtpd_delay_reject=no
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_end_of_data_restrictions=
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks




Output from postconf -n:

alias_database = hash:/etc/postfix/aliases
alias_maps = hash:/etc/postfix/aliases
allow_min_user = no
allow_percent_hack = no
biff = no
body_checks = pcre:/etc/postfix/body_checks.pcre
bounce_queue_lifetime = 12h
command_directory = /usr/sbin
compatibility_level = 2
content_filter = smtp-amavis:[127.0.0.1]:10024
daemon_directory = /usr/lib/postfix/sbin
data_directory = /var/lib/postfix
debug_peer_level = 2
debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5
disable_vrfy_command = yes
dovecot_destination_recipient_limit = 1
enable_original_recipient = no
header_checks = pcre:/etc/postfix/header_checks
inet_interfaces = all
inet_protocols = all
lmtp_tls_mandatory_protocols = !SSLv2 !SSLv3
lmtp_tls_protocols = !SSLv2 !SSLv3
mail_owner = postfix
mailbox_size_limit = 30000000
mailq_path = /usr/bin/mailq
maximal_queue_lifetime = 12h
message_size_limit = 30000000
mlmmj_destination_recipient_limit = 1
mydestination = $myhostname, localhost, localhost.localdomain
mydomain = thinmail.net
myhostname = mx1.thinmail.net
mynetworks = 127.0.0.1 [::1]
myorigin = thinmail.net
newaliases_path = /usr/bin/newaliases
non_smtpd_milters = inet:localhost:54321
postscreen_access_list = permit_mynetworks cidr:/etc/postfix/postscreen_access.cidr
postscreen_blacklist_action = drop
postscreen_dnsbl_action = drop
postscreen_dnsbl_reply_map = texthash:/etc/postfix/postscreen_dnsbl_reply
postscreen_dnsbl_sites = zen.spamhaus.org=127.0.0.[2..11]*3 b.barracudacentral.org=127.0.0.2*2
postscreen_dnsbl_threshold = 2
postscreen_dnsbl_whitelist_threshold = -2
postscreen_greet_action = drop
proxy_read_maps = $canonical_maps $lmtp_generic_maps $local_recipient_maps $mydestination $mynetworks $recipient_bcc_maps $recipient_canonical_maps $relay_domains $relay_recipient_maps $relocated_maps $sender_bcc_maps $sender_canonical_maps $smtp_generic_maps $smtpd_sender_login_maps $transport_maps $virtual_alias_domains $virtual_alias_maps $virtual_mailbox_domains $virtual_mailbox_maps $smtpd_sender_restrictions $sender_dependent_relayhost_maps
queue_directory = /var/spool/postfix
receive_override_options = no_address_mappings
recipient_bcc_maps = proxy:mysql:/etc/postfix/mysql/recipient_bcc_maps_user.cf proxy:mysql:/etc/postfix/mysql/recipient_bcc_maps_domain.cf
recipient_delimiter = +
relay_domains = $mydestination proxy:mysql:/etc/postfix/mysql/relay_domains.cf
sender_bcc_maps = proxy:mysql:/etc/postfix/mysql/sender_bcc_maps_user.cf proxy:mysql:/etc/postfix/mysql/sender_bcc_maps_domain.cf
sender_dependent_relayhost_maps = proxy:mysql:/etc/postfix/mysql/sender_dependent_relayhost_maps.cf
sendmail_path = /usr/sbin/sendmail
setgid_group = postdrop
smtp-amavis_destination_recipient_limit = 1
smtp_tls_CAfile = $smtpd_tls_CAfile
smtp_tls_CApath = /etc/ssl/certs
smtp_tls_loglevel = 1
smtp_tls_mandatory_protocols = !SSLv2 !SSLv3
smtp_tls_note_starttls_offer = yes
smtp_tls_protocols = !SSLv2 !SSLv3
smtp_tls_security_level = may
smtpd_data_restrictions = reject_unauth_pipelining
smtpd_end_of_data_restrictions = check_policy_service inet:127.0.0.1:7777
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks permit_sasl_authenticated check_helo_access pcre:/etc/postfix/helo_access.pcre reject_non_fqdn_helo_hostname reject_unknown_helo_hostname
smtpd_milters = inet:localhost:54321
smtpd_recipient_restrictions = reject_non_fqdn_recipient reject_unlisted_recipient check_policy_service inet:127.0.0.1:7777 permit_mynetworks permit_sasl_authenticated reject_unauth_destination
smtpd_reject_unlisted_recipient = yes
smtpd_reject_unlisted_sender = yes
smtpd_sasl_path = private/dovecot-auth
smtpd_sasl_type = dovecot
smtpd_sender_login_maps = proxy:mysql:/etc/postfix/mysql/sender_login_maps.cf
smtpd_sender_restrictions = reject_unknown_sender_domain reject_non_fqdn_sender reject_unlisted_sender permit_mynetworks permit_sasl_authenticated check_sender_access pcre:/etc/postfix/sender_access.pcre
smtpd_tls_CAfile = /etc/letsencrypt/live/mx1.thinmail.net/fullchain.pem
smtpd_tls_CApath = /etc/letsencrypt/live/mx1.thinmail.net
smtpd_tls_cert_file = /etc/letsencrypt/live/mx1.thinmail.net/fullchain.pem
smtpd_tls_dh1024_param_file = /etc/ssl/dh2048_param.pem
smtpd_tls_dh512_param_file = /etc/ssl/dh512_param.pem
smtpd_tls_exclude_ciphers = aNULL, eNULL, EXPORT, DES, RC4, MD5, PSK, aECDH, EDH-DSS-DES-CBC3-SHA, EDH-RSA-DES-CDC3-SHA, KRB5-DE5, CBC3-SHA
smtpd_tls_key_file = /etc/letsencrypt/live/mx1.thinmail.net/privkey.pem
smtpd_tls_loglevel = 1
smtpd_tls_mandatory_protocols = !SSLv2 !SSLv3
smtpd_tls_protocols = !SSLv2 !SSLv3
smtpd_tls_security_level = may
swap_bangpath = no
tls_random_source = dev:/dev/urandom
transport_maps = proxy:mysql:/etc/postfix/mysql/transport_maps_user.cf proxy:mysql:/etc/postfix/mysql/transport_maps_maillist.cf proxy:mysql:/etc/postfix/mysql/transport_maps_domain.cf
unknown_local_recipient_reject_code = 550
virtual_alias_domains =
virtual_alias_maps = proxy:mysql:/etc/postfix/mysql/virtual_alias_maps.cf proxy:mysql:/etc/postfix/mysql/domain_alias_maps.cf proxy:mysql:/etc/postfix/mysql/catchall_maps.cf proxy:mysql:/etc/postfix/mysql/domain_alias_catchall_maps.cf
virtual_gid_maps = static:2000
virtual_mailbox_base = /var/vmail
virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql/virtual_mailbox_domains.cf
virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql/virtual_mailbox_maps.cf
virtual_minimum_uid = 2000
virtual_transport = dovecot
virtual_uid_maps = static:2000

8

Re: Per-user alias addresses have stopped working after 0.9.9 update

smiller50 wrote:

if I remove 'no_address_mappings' from the port 10025 entry the problem is resolved

You can see this is the only difference between transport 127.0.0.1:10025 and 127.0.0.1:10028 in master.cf. they're used by different applications and have different purpose.

Actually this is mentioned in iRedMail-0.9.9 upgrade tutorial, but you updated 10025 transport by yourself which is not mentioned by our upgrade tutorial:
https://docs.iredmail.org/upgrade.iredm … iling-list

With "no_address_mappings" ENABLED, (we're talking about transport 127.0.0.1:10025, used by Amavisd) postfix will NOT expand the envelope recipient address to final destination addresses (real alias addresses, or mailing list members, etc) BEFORE hands over email to the transport, so email sent to a mailing list or mail alias account (not per-user alias address) is only scanned once, obviously, this will save a lot system resource used by amavisd + spamassassin + clamav for spam/virus scanning.

With "no_address_mappings" DISABLED, (we're talking about transport 127.0.0.1:10028, used by mlmmj mailing list manager) postfix expands envelope recipient address to final destination addresses before hands over email message to transport, so the transport gets all final addresses directly.

----

Does my reply help a little? How about buying me a cup of coffee ($5) as an encouragement?

buy me a cup of coffee

9 (edited by smiller50 2019-01-11 15:20:32)

Re: Per-user alias addresses have stopped working after 0.9.9 update

Understand, and thank you for the description of the no_address_mappings function. I'm still not sure exactly why I am unable to successfully activate no_address_mappings on the port 10025 (Amavisd) transport (since this is the iredmail default config I assume that it is working on other systems) but since my system is not very busy I can probably stand the additional overhead and will have to leave things as they are I guess.

Thanks so much for pointing me in the right direction, I threw something in your coffee cup  smile

10

Re: Per-user alias addresses have stopped working after 0.9.9 update

Thanks for the coffee. smile

----

Does my reply help a little? How about buying me a cup of coffee ($5) as an encouragement?

buy me a cup of coffee