1 Topic by derek1776

  • derek1776
  • Member
  • Offline
  • Registered: 2021-12-03
  • Posts: 21

Topic: Help needed with stopping relaying spam

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release): 1.51
- Deployed with iRedMail Easy or the downloadable installer?
- Linux/BSD distribution name and version:  Ubuntu 20.04
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): Nginx
- Manage mail accounts with iRedAdmin-Pro?  No
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====


Not sure what happened yet, mail stopped flowing so I jumped on the server to find a bunch of spam trying to relay out.  I changed the suss account password, ran the find_top_sasl_username and find_sasl_login_ip  to try to stop the bleeding.   Is the a command to delete these messages from the queue?  I tried postsuper with the queue-ID with no luck.  Not sure how to stop these, the server has been running fine since 1-2022.   


Here is the tail of maillog.  Its ugly and been going on since 7-2-2022.  Any guidance would be greatly appreciated.  Thanks!

maillog
Jul  4 12:44:52 mail postfix/bounce[3584]: 4Lc7hK0ZQtzrfn: sender non-delivery notification: 4LcBVX3LP5z4FtV
Jul  4 12:44:52 mail postfix/qmgr[1465]: 4Lc7hK0ZQtzrfn: removed
Jul  4 12:44:52 mail postfix/qmgr[1465]: 4LbZ5G5Xkxz1WHc: from=<>, size=7493, nrcpt=1 (queue active)
Jul  4 12:44:52 mail postfix/smtp[3270]: Trusted TLS connection established to smtp.sendgrid.net[52.204.68.213]:587: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Jul  4 12:44:54 mail postfix/smtp[3293]: 4Lbqnk2hQzz3cx5: to=<hukset2@aol.com>, relay=smtp.sendgrid.net[167.89.123.82]:587, delay=50590, delays=47821/2738/0.16/30, dsn=5.0.0, status=bounced (host smtp.sendgrid.net[167.89.123.82] said: 550 The from address does not match a verified Sender Identity. Mail cannot be sent until this error is resolved. Visit https://sendgrid.com/docs/for-developer … -identity/ to see the Sender Identity requirements (in reply to end of DATA command))
Jul  4 12:44:54 mail postfix/cleanup[3555]: 4LcBVZ6lc9z4FpK: message-id=<4LcBVZ6lc9z4FpK@mail.mydomain.com>
Jul  4 12:44:54 mail postfix/bounce[3584]: 4Lbqnk2hQzz3cx5: sender non-delivery notification: 4LcBVZ6lc9z4FpK
Jul  4 12:44:54 mail postfix/qmgr[1465]: 4Lbqnk2hQzz3cx5: removed
Jul  4 12:44:54 mail postfix/qmgr[1465]: 4LbdcB056sz2Pj9: from=<>, size=7469, nrcpt=1 (queue active)
Jul  4 12:44:54 mail postfix/smtp[3330]: 4LbM1P1Lcrzs5b: to=<jesela85@yahoo.com>, relay=smtp.sendgrid.net[54.146.218.5]:587, delay=117583, delays=114815/2738/0.02/30, dsn=5.0.0, status=bounced (host smtp.sendgrid.net[54.146.218.5] said: 550 The from address does not match a verified Sender Identity. Mail cannot be sent until this error is resolved. Visit https://sendgrid.com/docs/for-developer … -identity/ to see the Sender Identity requirements (in reply to end of DATA command))
Jul  4 12:44:54 mail postfix/cleanup[3555]: 4LcBVZ6q6Mz4FtW: message-id=<4LcBVZ6q6Mz4FtW@mail.mydomain.com>
Jul  4 12:44:54 mail postfix/bounce[3584]: 4LbM1P1Lcrzs5b: sender non-delivery notification: 4LcBVZ6q6Mz4FtW
Jul  4 12:44:54 mail postfix/qmgr[1465]: 4LbM1P1Lcrzs5b: removed
Jul  4 12:44:54 mail postfix/qmgr[1465]: 4LbTSQ6RXQz3Xww: from=<>, size=7502, nrcpt=1 (queue active)
Jul  4 12:44:54 mail postfix/smtp[3243]: Trusted TLS connection established to smtp.sendgrid.net[34.237.250.201]:587: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Jul  4 12:44:54 mail postfix/smtp[3293]: Trusted TLS connection established to smtp.sendgrid.net[52.0.142.242]:587: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Jul  4 12:44:56 mail postfix/smtp[3295]: 4LbyRl45QCz4Mrb: to=<laquetta1119@gmail.com>, relay=smtp.sendgrid.net[167.89.123.97]:587, delay=32597, delays=29827/2739/0.16/30, dsn=5.0.0, status=bounced (host smtp.sendgrid.net[167.89.123.97] said: 550 The from address does not match a verified Sender Identity. Mail cannot be sent until this error is resolved. Visit https://sendgrid.com/docs/for-developer … -identity/ to see the Sender Identity requirements (in reply to end of DATA command))
Jul  4 12:44:56 mail postfix/cleanup[3555]: 4LcBVc0bTpz4FpL: message-id=<4LcBVc0bTpz4FpL@mail.mydomain.com>
Jul  4 12:44:56 mail postfix/bounce[3584]: 4LbyRl45QCz4Mrb: sender non-delivery notification: 4LcBVc0bTpz4FpL
Jul  4 12:44:56 mail postfix/qmgr[1465]: 4LbyRl45QCz4Mrb: removed
Jul  4 12:44:56 mail postfix/qmgr[1465]: 4Lbxny3fKMz4Jg2: from=<info@bone.go.id>, size=5007, nrcpt=1 (queue active)
Jul  4 12:44:56 mail postfix/smtp[3330]: Trusted TLS connection established to smtp.sendgrid.net[34.237.250.201]:587: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by Cthulhu

  • Cthulhu
  • Member
  • Offline
  • Registered: 2019-06-04
  • Posts: 907

Re: Help needed with stopping relaying spam

mailq | tail +2 | grep -v '^ *(' | awk  'BEGIN { RS = "" } { if ($7 == "comproised@mail.com") print $1 }' | tr -d '*!' | postsuper -d -

3 Reply by derek1776

  • derek1776
  • Member
  • Offline
  • Registered: 2021-12-03
  • Posts: 21

Re: Help needed with stopping relaying spam

Update:  I cleaned out the queue with:

postqueue -p  (saw thousands of spam messages)

postsuper -d ALL

Had to run it a few times to get them all off.

This seems to have stopped the endless log entries for the moment.  Hoping by changing the offending account password has stopped it.   WIll update as I discover more.

4 Reply by derek1776

  • derek1776
  • Member
  • Offline
  • Registered: 2021-12-03
  • Posts: 21

Re: Help needed with stopping relaying spam

Thanks for the quick reply Cthulhu,  I ran this command on the suspect account and saw no output. Queue is still empty.

5 Reply by derek1776

  • derek1776
  • Member
  • Offline
  • Registered: 2021-12-03
  • Posts: 21

Re: Help needed with stopping relaying spam

Update 2:

Now I seem to be getting SASL authentication errors now.  May be bigger troubles afoot.


Jul  4 13:16:24 mail postfix/submission/smtpd[2064]: warning: mail.mydomain.com[127.0.0.1]: SASL PLAIN authentication failed:
Jul  4 13:16:26 mail postfix/submission/smtpd[2064]: warning: mail.mydomain.com[127.0.0.1]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
Jul  4 13:16:26 mail postfix/submission/smtpd[2064]: lost connection after AUTH from mail.mydomain.com[127.0.0.1]
Jul  4 13:16:26 mail postfix/submission/smtpd[2064]: disconnect from mail.mydomain.com[127.0.0.1] ehlo=2 starttls=1 auth=0/2 commands=3/5
Jul  4 13:18:02 mail postfix/submission/smtpd[2064]: connect from mail.mydomain.com[127.0.0.1]
Jul  4 13:18:02 mail postfix/submission/smtpd[2064]: lost connection after UNKNOWN from mail.mydomain.com[127.0.0.1]
Jul  4 13:18:02 mail postfix/submission/smtpd[2064]: disconnect from mail.mydomain.com[127.0.0.1] unknown=0/1 commands=0/1
Jul  4 13:18:02 mail postfix/submission/smtpd[2064]: connect from mail.mydomain.com[127.0.0.1]
Jul  4 13:18:02 mail postfix/submission/smtpd[2064]: lost connection after CONNECT from mail.mydomain.com[127.0.0.1]
Jul  4 13:18:02 mail postfix/submission/smtpd[2064]: disconnect from mail.mydomain.com[127.0.0.1] commands=0/0
Jul  4 13:18:02 mail postfix/submission/smtpd[2064]: connect from mail.mydomain.com[127.0.0.1]
Jul  4 13:18:02 mail postfix/submission/smtpd[2064]: Anonymous TLS connection established from mail.mydomain.com[127.0.0.1]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Jul  4 13:18:04 mail postfix/submission/smtpd[2064]: warning: mail.mydomain.com[127.0.0.1]: SASL PLAIN authentication failed:
Jul  4 13:18:10 mail postfix/submission/smtpd[2064]: warning: mail.mydomain.com[127.0.0.1]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
Jul  4 13:18:10 mail postfix/submission/smtpd[2064]: lost connection after AUTH from mail.mydomain.com[127.0.0.1]
Jul  4 13:18:10 mail postfix/submission/smtpd[2064]: disconnect from mail.mydomain.com[127.0.0.1] ehlo=2 starttls=1 auth=0/2 commands=3/5
Jul  4 13:18:10 mail postfix/submission/smtpd[2064]: connect from mail.mydomain.com[127.0.0.1]
Jul  4 13:18:10 mail postfix/submission/smtpd[2064]: Anonymous TLS connection established from mail.mydomain.com[127.0.0.1]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Jul  4 13:18:16 mail postfix/submission/smtpd[2064]: warning: mail.mydomain.com[127.0.0.1]: SASL PLAIN authentication failed:
Jul  4 13:18:22 mail postfix/submission/smtpd[2064]: warning: mail.mydomain.com[127.0.0.1]: SASL LOGIN authentication failed: UGFzc3dvcmQ6
Jul  4 13:18:22 mail postfix/submission/smtpd[2064]: lost connection after AUTH from mail.mydomain.com[127.0.0.1]
Jul  4 13:18:22 mail postfix/submission/smtpd[2064]: disconnect from mail.mydomain.com[127.0.0.1] ehlo=2 starttls=1 auth=0/2 commands=3/5

6 Reply by derek1776

  • derek1776
  • Member
  • Offline
  • Registered: 2021-12-03
  • Posts: 21

Re: Help needed with stopping relaying spam

I think this new wave of error is from the mail client,  will change passwords and should fix this.

7 Reply by Cthulhu

  • Cthulhu
  • Member
  • Offline
  • Registered: 2019-06-04
  • Posts: 907

Re: Help needed with stopping relaying spam

well, if the account was compromised, then i guess the errors are now due to trying to send emails trough this compromised acc

8 Reply by derek1776

  • derek1776
  • Member
  • Offline
  • Registered: 2021-12-03
  • Posts: 21

Re: Help needed with stopping relaying spam

Cthulhu wrote:

well, if the account was compromised, then i guess the errors are now due to trying to send emails trough this compromised acc

Very likely the case. Is there a way to find out what IP they are coming in?

I tried the scripts in the tools directory.

bash find_top_sasl_usernames.sh
      4 mylogin@mydomain.com
      4 otherlogin@mydomain2.com
bash find_sasl_login_ip.sh /path/to/maillog mylogin@mydomain.com
      4 client=mail.jewettfarm.com[127.0.0.1],
bash find_sasl_login_ip.sh /path/to/maillog mylogin@mydomain.com
      4 client=mail.mydomain.com[127.0.0.1],
root@mail:/home/iRedMail-1.5.1/tools# postsuper -p
root@mail:/home/iRedMail-1.5.1/tools#

9 Reply by Cthulhu

  • Cthulhu
  • Member
  • Offline
  • Registered: 2019-06-04
  • Posts: 907

Re: Help needed with stopping relaying spam

fail2ban should handle this on its own

10 Reply by derek1776

  • derek1776
  • Member
  • Offline
  • Registered: 2021-12-03
  • Posts: 21

Re: Help needed with stopping relaying spam

Cthulhu wrote:

fail2ban should handle this on its own

Should, but the offenders seem to come in on 127.0.0.1  [ie warning: mail.mydomain.com[127.0.0.1]: SASL PLAIN authentication failed:]

and fail22ban is set to ignore this address: Ignore 127.0.0.1 by ignoreself rule. So not sure how to track them down,  I did find odd IP's in the iredapd log and banned those with ipttables,  they still come in.

11 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,082

Re: Help needed with stopping relaying spam

Try to check old Postfix log file, e.g. /var/log/mail.log.1:

bash find_top_sasl_usernames.sh /var/log/mail.log.1
derek1776 wrote:

Update:  I cleaned out the queue with:
postqueue -p  (saw thousands of spam messages)
postsuper -d ALL

This is dangerous, you may remove legal emails with `postsuper -d ALL`.

If queue is full of spams, legal email may be stalled in queue and waiting for Postfix to pick it up.

12 Reply by derek1776

  • derek1776
  • Member
  • Offline
  • Registered: 2021-12-03
  • Posts: 21

Re: Help needed with stopping relaying spam

ZhangHuangbin wrote:

Try to check old Postfix log file, e.g. /var/log/mail.log.1:

bash find_top_sasl_usernames.sh /var/log/mail.log.1
derek1776 wrote:

Update:  I cleaned out the queue with:
postqueue -p  (saw thousands of spam messages)
postsuper -d ALL

This is dangerous, you may remove legal emails with `postsuper -d ALL`.

If queue is full of spams, legal email may be stalled in queue and waiting for Postfix to pick it up.

Old log file shows same compromised account. Passwords have been changed. The queue shows empty now, I had to resend a few messages, so the risk was minimal.

I still see bogus sends in the logs,  like this thing is still active, even after changing login credentials for the effected account that showed up in find_top_sasl_usernames.sh /var/log/mail.log.1. 

Sample log here:
Jul  5 15:46:36 mail postfix/smtp[44800]: 4LctTm47CMz1KYb: to=<m.koeberl@hiway.at>, relay=smtp.sendgrid.net[167.89.115.53]:587, delay=0.21, delays=0.01/0.01/0.16/0.03, dsn=5.0.0, status=bounced (host smtp.sendgrid.net[167.89.115.53] said: 550 The from address does not match a verified Sender Identity. Mail cannot be sent until this error is resolved. Visit https://sendgrid.com/docs/for-developer … -identity/ to see the Sender Identity requirements (in reply to end of DATA command))
Jul  5 15:46:36 mail postfix/smtp[44799]: Trusted TLS connection established to smtp.sendgrid.net[167.89.123.95]:587: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Jul  5 15:46:36 mail postfix/smtp[44797]: 4LctTm4t1Cz1KYc: to=<m.koeberl@hiway.at>, relay=smtp.sendgrid.net[107.20.8.136]:587, delay=0.25, delays=0.02/0/0.2/0.03, dsn=5.0.0, status=bounced (host smtp.sendgrid.net[107.20.8.136] said: 550 The from address does not match a verified Sender Identity. Mail cannot be sent until this error is resolved. Visit https://sendgrid.com/docs/for-developer … -identity/ to see the Sender Identity requirements (in reply to end of DATA command))
Jul  5 15:46:37 mail postfix/smtp[44799]: 4LctTm5B1Wz1Kgm: to=<m.koeberl@hiway.at>, relay=smtp.sendgrid.net[167.89.123.95]:587, delay=0.27, delays=0.01/0/0.16/0.1, dsn=5.0.0, status=bounced (host smtp.sendgrid.net[167.89.123.95] said: 550 The from address does not match a verified Sender Identity. Mail cannot be sent until this error is resolved. Visit https://sendgrid.com/docs/for-developer … -identity/ to see the Sender Identity requirements (in reply to end of DATA command))
Jul  5 15:46:44 mail postfix/smtp[44800]: Trusted TLS connection established to smtp.sendgrid.net[167.89.123.97]:587: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Jul  5 15:46:44 mail postfix/smtp[44797]: Trusted TLS connection established to smtp.sendgrid.net[167.89.123.97]:587: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Jul  5 15:46:44 mail postfix/smtp[44800]: 4LctTw1XtLz1KYk: to=<necfgn@gmail.com>, relay=smtp.sendgrid.net[167.89.123.97]:587, delay=0.24, delays=0.01/0.02/0.17/0.04, dsn=5.0.0, status=bounced (host smtp.sendgrid.net[167.89.123.97] said: 550 The from address does not match a verified Sender Identity. Mail cannot be sent until this error is resolved. Visit https://sendgrid.com/docs/for-developer … -identity/ to see the Sender Identity requirements (in reply to end of DATA command))

My outbound mail relay (sendgrid) is stopped and not allowing mail to flow out due to all the spam attempts.  Kinda lost as to how to stop it.  I have limited control over the firewall as it is running in a cloud instance until I can move it back to on-premise behind my firewall with ip blocks.   Never had so much trouble in the 2 years this has been running.  Will try to keep the post updated as I fight through it.

Thanks

13 Reply by derek1776

  • derek1776
  • Member
  • Offline
  • Registered: 2021-12-03
  • Posts: 21

Re: Help needed with stopping relaying spam

Update:  Mail is again flowing in both directions fine now.  All I did was reload postfix and it seems to be fine,  at least from the logs.  Have no idea what was going on,  but for the moment it seems to have stopped and everything is working. Mail-tester is giving me an 8.2/10, so the damage is not too bad.