1 Topic by camel1cz (edited by camel1cz 2015-05-18 21:55:50)

  • camel1cz
  • camel1cz
  • Member
  • Offline
  • From: Czech Republic
  • Registered: 2013-04-20
  • Posts: 221

Topic: Upgrade iRedAPd from 1.4.3 to latest 1.5.0

======== Required information ====
- iRedMail version: 0.9.1
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): pgsql
- Web server (Apache or Nginx): apache
- Linux/BSD distribution name and version: debian 7
- Related log if you're reporting an issue:
====

Hi guys,

some time ago I tried to upgrade from 1.4.3 to 1.4.4 and had some problems and didn't have time to solve it... so I stayed with the 1.4.3... now I decided to upgrade to latest 1.5.0 and would use some help to solve my issues with this upgrade.

I'm seeing in the log following entries which doesn't seem to be correct:

2015-05-18 15:28:13 INFO [client.ip.address] RCPT, user1@domain.com -> user2@domain.com, REJECT Policy rejection not logged in

It's not just one client seeing this, there is more combinations of users in the same domain...

I found out, this reject is triggered by sender_login_mismatch plugin in function rectriction on line 136.

What I don't understand is the meaning of sender_is_forged variable... it gets set when sender_domain is equal to recipient_domain and according to the code this is enough to reject the email??? What's the logic behind this code?

Notes:
- I'm pretty sure the sender is authenticated and doesn't do any sender change (except change of case of letters in the email address),
- the domain is local one,
- it could be some issue with MUA not authenticating when sending to the same domain as the sender domain? (kind of trying to be smart)
- the email should be accepted even if the session wouldn't be authenticated as the domain is local?

Thanks a lot for help!

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,164

Re: Upgrade iRedAPd from 1.4.3 to latest 1.5.0

camel1cz wrote:

I found out, this reject is triggered by sender_login_mismatch plugin in function rectriction on line 136.

Please allow me to explain. Below code in iRedAPD-1.5.0/plugins/reject_sender_login_mismatch.py with comments should be clear:

def restriction(**kwargs):
    ...
    if not sasl_username:                    # <- check whether sender is authenticated (sasl_username)
                                             # <- all code inside this `if not` block is applied to messages sent
                                             # <- by sender which is NOT authenticated, because the smtp
                                             # <- session data `sasl_username=` doesn't have a value (sender email address)
        ...
        sender_is_forged = False                  # <- predefined, set to False (not forged)
        if sender_domain == recipient_domain:     # <- remember, sender is not authenticated, recipient domain
                                                  # <- is hosted locally and all users on iRedMail server are forced
                                                  # <- to perform smtp authentication to send email, so this sender
                                                  # <- is a forged address.
            sender_is_forged = True
        ...

        if sender_is_forged:
            return SMTP_ACTIONS['reject'] + ' not logged in'
        ...
camel1cz wrote:

- it could be some issue with MUA not authenticating when sending to the same domain as the sender domain? (kind of trying to be smart)

This is the problem. Please check your client's MUA setting.

3 Reply by camel1cz

  • camel1cz
  • camel1cz
  • Member
  • Offline
  • From: Czech Republic
  • Registered: 2013-04-20
  • Posts: 221

Re: Upgrade iRedAPd from 1.4.3 to latest 1.5.0

Thank you, Zhang, for detailed explanation. It does make sense now!


ZhangHuangbin wrote:

This is the problem. Please check your client's MUA setting.

That is the core of the problem... I don't see any settings like this nor I remember it it any MUA I saw... acutaly I'm quite surprised, the problem didn't popup yet... I don't thing my users are different than others.

What do you suggest, to solve this issue? I personally downgraded the plugin to version 1.4.3 and am using 1.5.0 iRedAPd... it does work fine, but I would like to be in sync with the mainstream.

Few more notes:
- I run the original 1.5.0 only for few minutes, but got about 30 errors in logs connected to this issue. Most was from the same domain (about 10 different accounts) but there was also 2 other domains affected,
- I'm sure it's MS software affected... and I'm pretty sure, the problematic MUA is MS Outlook 20xx... is there a way how to find out?
- if it's worth checking such breach of policy, shouldn't we check not only sender_domain == recipient_domain but 'sender_domain is local' instead?
- anyhow I see it a bit dangerous to filter out such email - imagine this situation: user sends email from his home computer via SMTP server of home internet connection and with work email address to his chief... such email would be rejected? I think such situations should be left to SPF or similar and not duplicated in iRedAPd... correct me, if I'm wrong...

I can debug/help if you have any idea... thank you!

4 Reply by mir

  • mir
  • Member
  • Offline
  • Registered: 2014-05-30
  • Posts: 170

Re: Upgrade iRedAPd from 1.4.3 to latest 1.5.0

camel1cz wrote:

That is the core of the problem... I don't see any settings like this nor I remember it it any MUA I saw... acutaly I'm quite surprised, the problem didn't popup yet... I don't thing my users are different than others.

This is wrong. All MUA's has the capability to allow users to enter login name and password. What you are facing is a misbehavior from Microsoft and Outlook when users are domain validated it does not ask the users for login and password because it expect any component to understand NTLM or Microsoft Kerberos.

5 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,164

Re: Upgrade iRedAPd from 1.4.3 to latest 1.5.0

camel1cz wrote:

What do you suggest, to solve this issue?

Turn on debug mode in iRedAPD, then check the smtp session data (logged in iRedAPD log file) to see whether iRedAPD incorrectly rejects it.
Reference: http://www.iredmail.org/docs/debug.iredapd.html

Please paste related iRedAPD log (including smtp session data) here, so that others can help troubleshoot. Of course you'd better replace sensitive info in log before pasting, but please keep it understandable.

Also, show us 'postconf -n' too.

camel1cz wrote:

- I run the original 1.5.0 only for few minutes, but got about 30 errors in logs connected to this issue. Most was from the same domain (about 10 different accounts) but there was also 2 other domains affected,
- I'm sure it's MS software affected... and I'm pretty sure, the problematic MUA is MS Outlook 20xx... is there a way how to find out?

Find which clients they are, then check their MUA settings please.

camel1cz wrote:

- if it's worth checking such breach of policy, shouldn't we check not only sender_domain == recipient_domain but 'sender_domain is local' instead?

That might be the problem, iRedAPD doesn't check whether recipient domain is local. So this rejection may apply on relayed email.
Please get above log first, then let's see whether it occurs when recipient domain is local.

camel1cz wrote:

- anyhow I see it a bit dangerous to filter out such email - imagine this situation: user sends email from his home computer via SMTP server of home internet connection and with work email address to his chief... such email would be rejected? I think such situations should be left to SPF or similar and not duplicated in iRedAPd... correct me, if I'm wrong...

Let's check above item first.

6 Reply by camel1cz

  • camel1cz
  • camel1cz
  • Member
  • Offline
  • From: Czech Republic
  • Registered: 2013-04-20
  • Posts: 221

Re: Upgrade iRedAPd from 1.4.3 to latest 1.5.0

mir wrote:

What you are facing is a misbehavior from Microsoft and Outlook when users are domain validated it does not ask the users for login and password because it expect any component to understand NTLM or Microsoft Kerberos.

Sure, all the users have their usernames/passwords set and SMTP AUTH active... and all are able to send emails to external domains via my iredmail server...
Seems to me, that the MUA doesn't sent SMTP AUTH information when sending to the same domain... do you say, this behavior is known and connected to the MS specific authentication? Any more information? Any workaround possible? Unfortunately, I don't care much about MS products anymore...

@Zhang: I turned on debug mode and will come back with logs...

7 Reply by mir

  • mir
  • Member
  • Offline
  • Registered: 2014-05-30
  • Posts: 170

Re: Upgrade iRedAPd from 1.4.3 to latest 1.5.0

camel1cz wrote:

Sure, all the users have their usernames/passwords set and SMTP AUTH active... and all are able to send emails to external domains via my iredmail server...
Seems to me, that the MUA doesn't sent SMTP AUTH information when sending to the same domain... do you say, this behavior is known and connected to the MS specific authentication? Any more information? Any workaround possible? Unfortunately, I don't care much about MS products anymore...

There really is nothing you can do. What you are experiencing is the core idea of SSO (Single Sign On) which is enforced by joining a windows client to the domain. Only trouble is that services which are not capable of doing NTLM or MS Kerberos are second class citizens left out in the cold:(

8 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,164

Re: Upgrade iRedAPd from 1.4.3 to latest 1.5.0

camel1cz wrote:

Seems to me, that the MUA doesn't sent SMTP AUTH information when sending to the same domain... do you say, this behavior is known and connected to the MS specific authentication? Any more information? Any workaround possible? Unfortunately, I don't care much about MS products anymore...

All MUA should perform SMTP AUTH to send email, no matter send to which recipient.
All i can do is analyzing iRedAPD/Postfix log file to find out unexpected things, but i have no idea whether or not it is an Outlook issue. So, i have to wait for your log.

9 Reply by camel1cz (edited by camel1cz 2015-05-21 04:32:59)

  • camel1cz
  • camel1cz
  • Member
  • Offline
  • From: Czech Republic
  • Registered: 2013-04-20
  • Posts: 221

Re: Upgrade iRedAPd from 1.4.3 to latest 1.5.0

ZhangHuangbin wrote:
camel1cz wrote:

Seems to me, that the MUA doesn't sent SMTP AUTH information when sending to the same domain... do you say, this behavior is known and connected to the MS specific authentication? Any more information? Any workaround possible? Unfortunately, I don't care much about MS products anymore...

All MUA should perform SMTP AUTH to send email, no matter send to which recipient.

I know this, but IT SEEMS they doesn't!

See attached log - it's the log when I use the iRedAPd v. 1.5.0 with sender_login_mismatch plugin from v. 1.4.3 - the result is DUNNO, but the same session ends in "REJECT Policy rejection not logged in" when using the current plugin version:

2015-05-20 13:15:24 DEBUG smtp session: request=smtpd_access_policy
2015-05-20 13:15:24 DEBUG smtp session: protocol_state=RCPT
2015-05-20 13:15:24 DEBUG smtp session: protocol_name=ESMTP
2015-05-20 13:15:24 DEBUG smtp session: client_address=<REMOVED_CLIENT_IP>
2015-05-20 13:15:24 DEBUG smtp session: client_name=<REMOVED_CLIENT_HOSTNAME>
2015-05-20 13:15:24 DEBUG smtp session: reverse_client_name=<REMOVED_CLIENT_HOSTNAME>
2015-05-20 13:15:24 DEBUG smtp session: helo_name=<REMOVED_CLIENT_HOSTNAME>
2015-05-20 13:15:24 DEBUG smtp session: sender=<USER1@DOMAIN.COM>
2015-05-20 13:15:24 DEBUG smtp session: recipient=<USER1@DOMAIN.COM - yes, mail to themself>
2015-05-20 13:15:24 DEBUG smtp session: recipient_count=0
2015-05-20 13:15:24 DEBUG smtp session: queue_id=
2015-05-20 13:15:24 DEBUG smtp session: instance=b0b.555c6ccc.adc32.0
2015-05-20 13:15:24 DEBUG smtp session: size=5186
2015-05-20 13:15:24 DEBUG smtp session: etrn_domain=
2015-05-20 13:15:24 DEBUG smtp session: stress=
2015-05-20 13:15:24 DEBUG smtp session: sasl_method=
2015-05-20 13:15:24 DEBUG smtp session: sasl_username=
2015-05-20 13:15:24 DEBUG smtp session: sasl_sender=
2015-05-20 13:15:24 DEBUG smtp session: ccert_subject=
2015-05-20 13:15:24 DEBUG smtp session: ccert_issuer=
2015-05-20 13:15:24 DEBUG smtp session: ccert_fingerprint=
2015-05-20 13:15:24 DEBUG smtp session: ccert_pubkey_fingerprint=
2015-05-20 13:15:24 DEBUG smtp session: encryption_protocol=
2015-05-20 13:15:24 DEBUG smtp session: encryption_cipher=
2015-05-20 13:15:24 DEBUG smtp session: encryption_keysize=0
2015-05-20 13:15:24 DEBUG --> Apply plugin: sql_alias_access_policy
2015-05-20 13:15:24 DEBUG SQL: query access policy: SELECT accesspolicy, goto, moderators
            FROM alias
            WHERE
                address='<USER1@DOMAIN.COM'
                AND address <> goto
                AND active=1
            LIMIT 1
    
2015-05-20 13:15:24 DEBUG <!> Error: 'Connection' object has no attribute 'fetchone'
2015-05-20 13:15:24 DEBUG --> Apply plugin: reject_sender_login_mismatch
2015-05-20 13:15:24 DEBUG Allowed SASL username: <LIST_OF_ALLOWED_SENDER_MISSMATCH>
2015-05-20 13:15:24 DEBUG Sender: <USER1@DOMAIN.COM>, SASL username: 
2015-05-20 13:15:24 DEBUG <-- Result: DUNNO
2015-05-20 13:15:24 INFO [90.182.221.3] RCPT, <USER1@DOMAIN.COM -> USER1@DOMAIN.COM, DUNNO
2015-05-20 13:15:24 DEBUG Session ended

It's clear from the log, that the MUA didn't send any SMTP AUTH information... but the same setup does work fine for other emails... strage, very strange actually sad

Zhang, also see what @mir says about the authentication from Windows which are part of MS domain... what do you think?

10 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,164

Re: Upgrade iRedAPd from 1.4.3 to latest 1.5.0

There's value in `sasl_username=`, that means it doesn't perform SMTP AUTH. That's weird...
@mir's reply explains why it acts so weird. I have no idea how to solve this issue since it's now a client issue. Maybe you should disable this plugin (or modify its source code to not check forged domain).