1 Topic by cf (edited by cf 2020-07-04 16:03:25)

  • cf
  • Member
  • Offline
  • Registered: 2020-05-16
  • Posts: 10

Topic: ireadpd problem with whitelist after uprade to 4.1

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release): 1.2.2 originally, manually updated to 1.3
- Deployed with iRedMail Easy or the downloadable installer? downloadable installer
- Linux/BSD distribution name and version: CentOS 8.2
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): PGSQL
- Web server (Apache or Nginx): Nginx
- Manage mail accounts with iRedAdmin-Pro? No
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

I recently upgraded iredapd from version 3.6 to 4.0 and then to 4.1. Prior to the upgrade, I had only one entry for one sender address in the whitelist (added with wblist_admin.py --add --whitelist whitelisted@sender.domain), and it was working as intended.

After the upgrade, it seems that mails from a whitelisted sender are rejected and that the whitelist cannot be managed with the wblist_admin.py script.

When a mail was received from the whitelisted sender, I had the following in the logs:

Jul 04 07:31:55 mx postfix/postscreen[1227882]: CONNECT from [40.107.94.67]:36599 to [aaa.xxx.yyy.zzz]:25
Jul 04 07:32:01 mx postfix/postscreen[1227882]: PASS NEW [40.107.94.67]:36599
Jul 04 07:32:02 mx postfix/smtpd[1227913]: connect from mail-mw2nam10on2067.outbound.protection.outlook.com[40.107.94.67]
Jul 04 07:32:03 mx postfix/smtpd[1227913]: Anonymous TLS connection established from mail-mw2nam10on2067.outbound.protection.outlook.com[40.107.94.67]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Jul 04 07:32:04 mx python3[1153943]: iredapd Whitelisted: wblist=(1, 1, 'W')
Jul 04 07:32:04 mx python3[1153943]: iredapd [40.107.94.67] RCPT, whitelisted@sender.domain -> recipient@my.domain, OK [sasl_username=, sender=whitelisted@sender.domain, client_name=mail-mw2nam10on2067.outbound.protection.outlook.com, reverse_client_name=mail-mw2nam10on2067.outbound.protection.outlook.com, helo=NAM10-MW2-obe.outbound.protection.outlook.com, encryption_protocol=TLSv1.2, encryption_cipher=ECDHE-RSA-AES256-GCM-SHA384, server_port=25, process_time=0.1028s]
Jul 04 07:32:04 mx postfix/smtpd[1227913]: 49zL782vHcz7cvsk: client=mail-mw2nam10on2067.outbound.protection.outlook.com[40.107.94.67]
Jul 04 07:32:04 mx postfix/cleanup[1227931]: 49zL782vHcz7cvsk: message-id=<DNDMK7YX1BU4.DCGSA0IRR7NC2@RD0004FFDC6C00>
Jul 04 07:32:06 mx postfix/smtpd[1227913]: warning: problem talking to server 127.0.0.1:7777: Success
Jul 04 07:32:06 mx postfix/smtpd[1227913]: 49zL782vHcz7cvsk: reject: END-OF-MESSAGE from mail-mw2nam10on2067.outbound.protection.outlook.com[40.107.94.67]: 451 4.3.5 <END-OF-MESSAGE>: End-of-data rejected: Server configuration problem; from=<whitelisted@sender.domain> to=<recipient@my.domain> proto=ESMTP helo=<NAM10-MW2-obe.outbound.protection.outlook.com>
Jul 04 07:32:06 mx postfix/smtpd[1227913]: disconnect from mail-mw2nam10on2067.outbound.protection.outlook.com[40.107.94.67] ehlo=2 starttls=1 mail=1 rcpt=1 data=0/1 quit=1 commands=6/7

Trying to check the whitelist produced an error(?) 

[root@mx /opt/iredapd/tools]# python3 wblist_admin.py --list --whitelist
* Establishing SQL connection.
* List all inbound whitelist for account: @.
<memory at 0x7f3163dcfb88>

Doing the same with the old iredapd 3.6 succeeds:

[root@mx /opt/iRedAPD-3.6/tools]# python2 wblist_admin.py --list --whitelist
* Establishing SQL connection.
* List all inbound whitelist for account: @.
whitelisted@sender.domain

I can also delete the entry using python2/iredapd 3.6:

[root@mx /opt/iRedAPD-3.6/tools]# python2 wblist_admin.py --delete --whitelist whitelisted@sender.domain
* Establishing SQL connection.
* Delete inbound whitelist for account: @.
- Deleted: whitelisted@sender.domain

But adding an entry to the whitelist with python3/iredapd 4.1 results in an error:

[root@mx /opt/iredapd/tools]# python3 wblist_admin.py --add --whitelist test@test.com
* Establishing SQL connection.
* Add inbound whitelist for account: @.
* Add senders: test@test.com
'memoryview' object has no attribute 'decode'

Thanks and please let me know if any more info is needed to help fix the issue.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: ireadpd problem with whitelist after uprade to 4.1

cf wrote:

Jul 04 07:32:06 mx postfix/smtpd[1227913]: 49zL782vHcz7cvsk: reject: END-OF-MESSAGE from mail-mw2nam10on2067.outbound.protection.outlook.com[40.107.94.67]: 451 4.3.5 <END-OF-MESSAGE>: End-of-data rejected: Server configuration problem; from=<whitelisted@sender.domain> to=<recipient@my.domain> proto=ESMTP helo=<NAM10-MW2-obe.outbound.protection.outlook.com>

This is caused by an iRedAPD programming bug, we're still working on it since it's hard to reproduce locally, we're contacting iRedMail users who can give me direct ssh access with root privilege for further debugging. Recently get access to one server, but mail traffic is not busy and hard to reproduce it, not yet reproduced till now.

3 Reply by cf

  • cf
  • Member
  • Offline
  • Registered: 2020-05-16
  • Posts: 10

Re: ireadpd problem with whitelist after uprade to 4.1

I just upgraded to iRedAPD 4.2 and it seems to have fixed the message rejection.

However, the wblist_admin.py script still seems to be broken:

python3 /opt/iredapd/tools/wblist_admin.py --add --whitelist test@example.net
* Establishing SQL connection.
* Add inbound whitelist for account: @.
* Add senders: test@example.net
AttributeError("'memoryview' object has no attribute 'decode'",)

4 Reply by cf

  • cf
  • Member
  • Offline
  • Registered: 2020-05-16
  • Posts: 10

Re: ireadpd problem with whitelist after uprade to 4.1

cf wrote:

I just upgraded to iRedAPD 4.2 and it seems to have fixed the message rejection.

However, the wblist_admin.py script still seems to be broken:

python3 /opt/iredapd/tools/wblist_admin.py --add --whitelist test@example.net
* Establishing SQL connection.
* Add inbound whitelist for account: @.
* Add senders: test@example.net
AttributeError("'memoryview' object has no attribute 'decode'",)

Upgraded to 4.3. wblist_admin.py is still not working.

5 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: ireadpd problem with whitelist after uprade to 4.1

Fixed: https://github.com/iredmail/iRedAPD/com … 6f80f37f47
Thanks for the feedback. smile

6 Reply by cf

  • cf
  • Member
  • Offline
  • Registered: 2020-05-16
  • Posts: 10

Re: ireadpd problem with whitelist after uprade to 4.1

Upgraded to 4.4, but it is still not working.

Listing whitelist/blacklist entries results in an error:

python3 /opt/iRedAPD-4.4/tools/wblist_admin.py --list --whitelist
* Establishing SQL connection.
* List all inbound whitelist for account: @.
TypeError("'<' not supported between instances of 'memoryview' and 'memoryview'",)

The --add and --delete commands do not result in errors and APPEAR to work:

python3 /opt/iRedAPD-4.4/tools/wblist_admin.py --add --whitelist test@example.net
* Establishing SQL connection.
* Add inbound whitelist for account: @.
* Add senders: test@example.net

However, neither --add nor --delete do anything to the actual white/blacklist. I.e. the wblist and outbound_wblist tables in the amavisd db are unchanged.

(Using python2 and wblist_admin.py from iRedAPD 3.6 still works as intended)

7 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: ireadpd problem with whitelist after uprade to 4.1

You're right, it was not completely fixed.
Just committed one more fix, tested locally and works for me.
https://github.com/iredmail/iRedAPD/com … 714245878e

Could you help verify it with steps below?

- Download the latest development edition with this link: https://github.com/iredmail/iRedAPD/archive/master.zip
- Unarchive, rename it to iRedAPD-4.4.
- Upgrade it as usual:

cd iRedAPD-4.4/tools/
bash upgrade_iredapd.sh

Then try to reproduce the issue again.

8 Reply by cf

  • cf
  • Member
  • Offline
  • Registered: 2020-05-16
  • Posts: 10

Re: ireadpd problem with whitelist after uprade to 4.1

Both --add and --delete work now.

Still the same error message, however, with the --list argument.

9 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: ireadpd problem with whitelist after uprade to 4.1

Should be fixed with the latest development edition. Could you help download the latest development edition and upgrade to give it one more try?

Note: after downloaded and unarchived, please rename the directory to something like "iRedAPD-4.4", otherwise the upgrade script doesn't consider the directory as an iRedAPD source code directory.

10 Reply by cf

  • cf
  • Member
  • Offline
  • Registered: 2020-05-16
  • Posts: 10

Re: ireadpd problem with whitelist after uprade to 4.1

Now everything seems to be working. Thanks.

11 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: ireadpd problem with whitelist after uprade to 4.1

Thanks for the feedback. smile Topic closed.