1 Topic by HAL9000

  • HAL9000
  • Member
  • Offline
  • Registered: 2014-08-05
  • Posts: 12

Topic: Postfix acting as spam server. Looks like it's part of a botnet..

==== Required information ====
- iRedMail version: mail_version = 0.8.6
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Linux/BSD distribution name and version: CentOS 6.5
- Related log if you're reporting an issue: http://pastebin.com/raw.php?i=cJzjTZ46
====

Let me first start out by saying I am no mail guru. If you need more info to help me just let me know. I am sorry if I have not provided the correct information but please be patient with me and I will do the best I can.

This week the mail server has been having all sorts of strange issues. It has been duplicating emails last week it wouldn't even let me connect to imap with thunderbird until postfix was restarted. I never did determine why this was. At any rate a few nights ago I was searching around in the mail log trying to search for an answer as to why this might be causing these issues. With no real leads I simply began looking for anomalies. Doing this I stumbled upon thousands of email messages that are being routed through the mail system. So far as I can tell my main.cf file is correct and postfix should not be acting as an open relay. I do not understand how mail is being sent through the system. Does anyone know how this is possible?

The spam messages seem to be coming from hundreds of different domains that from all over the world. Most of them seem point to CentOS servers running apache, mail, and ssh. The only thing that's set up on them is the apache test page that comes with CentOS. The mails that are being sent through our system are mostly being sent to cornerstone-valuation.com (little website with a submit email form that does not have a captcha) although there are some being sent from [CUSTOMER EMAIL] to a disposable email service(10minutemail.com) and the rest from [REDACTED]@cfm-valuation.com. This leads me to think these machines are actually part of a botnet and that our machine is as well.

From there things get even more wild. The IP’s that is sending spam though our system to [REDACTED]@cornerstone-valuation.com and drdrb.net are also sending spam many of our customers using our system. Maybe all of them I’m not sure. This means the people doing this have access to all of our customer email accounts which are located in /var/vmail/vmail1 (which can’t even be viewed unless root) and have distributed this information all across the internet. Does anyone here know how the spammer got this information?

Here are examples of log files and a huge list of domains that probably point at a bonnet of CentOS servers. Just follow the link below.

http://pastebin.com/raw.php?i=cJzjTZ46

Another issue is that A lot of the messages that are being kicked back at the server, are not originating from the server in the first place. They are coming from outside servers that are using [OUR MAIL DOMAIN] in the EHLO statements.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,094

Re: Postfix acting as spam server. Looks like it's part of a botnet..

Please show us output of command "postconf -n" to help troubleshoot.

3 Reply by HAL9000

  • HAL9000
  • Member
  • Offline
  • Registered: 2014-08-05
  • Posts: 12

Re: Postfix acting as spam server. Looks like it's part of a botnet..

ZhangHuangbin wrote:

Please show us output of command "postconf -n" to help troubleshoot.

Sure thing. 

alias_database = hash:/etc/postfix/aliases
alias_maps = hash:/etc/postfix/aliases
allow_min_user = no
allow_percent_hack = no
biff = no
bounce_queue_lifetime = 4h
broken_sasl_auth_clients = yes
command_directory = /usr/sbin
config_directory = /etc/postfix
content_filter = smtp-amavis:[127.0.0.1]:10024
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
debug_peer_level = 2
debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5
delay_warning_time = 0h
disable_vrfy_command = yes
dovecot_destination_recipient_limit = 1
enable_original_recipient = no
home_mailbox = Maildir/
html_directory = no
inet_interfaces = all
inet_protocols = ipv4
mail_owner = postfix
mailbox_command = /usr/libexec/dovecot/deliver
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
maximal_backoff_time = 4000s
maximal_queue_lifetime = 4h
message_size_limit = 104857600
minimal_backoff_time = 300s
mydestination = $myhostname, localhost, localhost.localdomain, localhost.$myhostname
mydomain = {REDACTED]
myhostname = smtp.{REDACTED]
mynetworks = 127.0.0.0/8 10.0.0.0/24
mynetworks_style = subnet
myorigin = $mydomain
newaliases_path = /usr/bin/newaliases.postfix
proxy_read_maps = $canonical_maps $lmtp_generic_maps $local_recipient_maps $mydestination $mynetworks $recipient_bcc_maps $recipient_canonical_maps $relay_domains $relay_recipient_maps $relocated_maps $sender_bcc_maps $sender_canonical_maps $smtp_generic_maps $smtpd_sender_login_maps $transport_maps $virtual_alias_domains $virtual_alias_maps $virtual_mailbox_domains $virtual_mailbox_maps $smtpd_sender_restrictions
queue_directory = /var/spool/postfix
queue_run_delay = 300s
readme_directory = /usr/share/doc/postfix-2.11.0/README_FILES
recipient_bcc_maps = proxy:mysql:/etc/postfix/mysql/recipient_bcc_maps_user.cf, proxy:mysql:/etc/postfix/mysql/recipient_bcc_maps_domain.cf
recipient_delimiter = +
relay_domains = $mydestination, proxy:mysql:/etc/postfix/mysql/relay_domains.cf
sample_directory = /usr/share/doc/postfix-2.11.0/samples
sender_bcc_maps = proxy:mysql:/etc/postfix/mysql/sender_bcc_maps_user.cf, proxy:mysql:/etc/postfix/mysql/sender_bcc_maps_domain.cf
sendmail_path = /usr/sbin/sendmail.postfix
setgid_group = postdrop
smtp-amavis_destination_recipient_limit = 1
smtp_data_init_timeout = 240s
smtp_data_xfer_timeout = 600s
smtp_destination_concurrency_limit = 5
smtp_destination_rate_delay = 1s
smtpd_banner = smtp.{REDACTED] ESMTP $mail_name
smtpd_data_restrictions = reject_unauth_pipelining
smtpd_end_of_data_restrictions = check_policy_service inet:127.0.0.1:10031
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_helo_hostname, reject_invalid_helo_hostname, check_helo_access pcre:/etc/postfix/helo_access.pcre
smtpd_recipient_restrictions = check_sender_access hash:/etc/postfix/sender_access reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unlisted_recipient, permit_mynetworks, permit_sasl_authenticated, check_policy_service inet:127.0.0.1:7777, check_policy_service inet:127.0.0.1:10031, reject_unauth_destination, reject_rbl_client b.barracudacentral.org, reject_rbl_client bl.spamcop.net, reject_rbl_client cbl.abuseat.org, reject_rbl_client zen.spamhaus.org, permit
smtpd_reject_unlisted_recipient = yes
smtpd_reject_unlisted_sender = yes
smtpd_sasl_auth_enable = yes
smtpd_sasl_authenticated_header = no
smtpd_sasl_local_domain =
smtpd_sasl_path = ./dovecot-auth
smtpd_sasl_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_sender_login_maps = proxy:mysql:/etc/postfix/mysql/sender_login_maps.cf
smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated
smtpd_tls_CAfile = /etc/pki/tls/certs/wildcard_{REDACTED].pem
smtpd_tls_cert_file = /etc/pki/tls/certs/wildcard_{REDACTED].pem
smtpd_tls_key_file = /etc/pki/tls/private/wildcard_{REDACTED].key
smtpd_tls_loglevel = 0
smtpd_tls_security_level = may
swap_bangpath = no
tls_random_source = dev:/dev/urandom
transport_maps = proxy:mysql:/etc/postfix/mysql/transport_maps_user.cf, proxy:mysql:/etc/postfix/mysql/transport_maps_domain.cf
unknown_local_recipient_reject_code = 550
virtual_alias_domains =
virtual_alias_maps = proxy:mysql:/etc/postfix/mysql/virtual_alias_maps.cf, proxy:mysql:/etc/postfix/mysql/domain_alias_maps.cf, proxy:mysql:/etc/postfix/mysql/catchall_maps.cf, proxy:mysql:/etc/postfix/mysql/domain_alias_catchall_maps.cf
virtual_gid_maps = static:2000
virtual_mailbox_base = /var/vmail
virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql/virtual_mailbox_domains.cf
virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql/virtual_mailbox_maps.cf
virtual_minimum_uid = 2000
virtual_transport = dovecot
virtual_uid_maps = static:2000

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,094

Re: Postfix acting as spam server. Looks like it's part of a botnet..

HAL9000 wrote:

smtpd_recipient_restrictions = ..., permit_mynetworks, permit_sasl_authenticated, check_policy_service inet:127.0.0.1:7777, check_policy_service inet:127.0.0.1:10031, ...
smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated

Obviously, you did some change to Postfix config file, and default settings configured by iRedMail can prevent this issue.

Here's default setting configured by iRedMail:

smtpd_recipient_restrictions = ..., check_policy_service inet:127.0.0.1:7777, check_policy_service inet:127.0.0.1:10031, permit_mynetworks, permit_sasl_authenticated,  ...
smtpd_sender_restrictions = permit_mynetworks, reject_sender_login_mismatch, permit_sasl_authenticated

1: Place 'check_policy_service' in front of 'permit_mynetworks' and 'permit_sasl_authenticated' will enforce throttling checking before sending out email, including sasl authentcated user.

2: with 'reject_sender_login_mismatch', if sasl authentication username is different than the 'From:' address, Postfix will reject this email.

5 Reply by HAL9000 (edited by HAL9000 2014-08-15 02:16:43)

  • HAL9000
  • Member
  • Offline
  • Registered: 2014-08-05
  • Posts: 12

Re: Postfix acting as spam server. Looks like it's part of a botnet..

ZhangHuangbin wrote:

1: Place 'check_policy_service' in front of 'permit_mynetworks' and 'permit_sasl_authenticated' will enforce throttling checking before sending out email, including sasl authentcated user.

2: with 'reject_sender_login_mismatch', if sasl authentication username is different than the 'From:' address, Postfix will reject this email.

Okay the out put of postconf -n now looks like this now:

alias_database = hash:/etc/postfix/aliases
alias_maps = hash:/etc/postfix/aliases
allow_min_user = no
allow_percent_hack = no
biff = no
bounce_queue_lifetime = 4h
broken_sasl_auth_clients = yes
command_directory = /usr/sbin
config_directory = /etc/postfix
content_filter = smtp-amavis:[127.0.0.1]:10024
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
debug_peer_level = 2
debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5
delay_warning_time = 0h
disable_vrfy_command = yes
dovecot_destination_recipient_limit = 1
enable_original_recipient = no
home_mailbox = Maildir/
html_directory = no
inet_interfaces = all
inet_protocols = ipv4
mail_owner = postfix
mailbox_command = /usr/libexec/dovecot/deliver
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
maximal_backoff_time = 4000s
maximal_queue_lifetime = 4h
message_size_limit = 104857600
minimal_backoff_time = 300s
mydestination = $myhostname, localhost, localhost.localdomain, localhost.$myhostname
mydomain = [REDACTED]
myhostname = smtp.[REDACTED]
mynetworks = 127.0.0.0/8 10.0.0.0/24
mynetworks_style = subnet
myorigin = $mydomain
newaliases_path = /usr/bin/newaliases.postfix
proxy_read_maps = $canonical_maps $lmtp_generic_maps $local_recipient_maps $mydestination $mynetworks $recipient_bcc_maps $recipient_canonical_maps $relay_domains $relay_recipient_maps $relocated_maps $sender_bcc_maps $sender_canonical_maps $smtp_generic_maps $smtpd_sender_login_maps $transport_maps $virtual_alias_domains $virtual_alias_maps $virtual_mailbox_domains $virtual_mailbox_maps $smtpd_sender_restrictions
queue_directory = /var/spool/postfix
queue_run_delay = 300s
readme_directory = /usr/share/doc/postfix-2.11.0/README_FILES
recipient_bcc_maps = proxy:mysql:/etc/postfix/mysql/recipient_bcc_maps_user.cf, proxy:mysql:/etc/postfix/mysql/recipient_bcc_maps_domain.cf
recipient_delimiter = +
relay_domains = $mydestination, proxy:mysql:/etc/postfix/mysql/relay_domains.cf
sample_directory = /usr/share/doc/postfix-2.11.0/samples
sender_bcc_maps = proxy:mysql:/etc/postfix/mysql/sender_bcc_maps_user.cf, proxy:mysql:/etc/postfix/mysql/sender_bcc_maps_domain.cf
sendmail_path = /usr/sbin/sendmail.postfix
setgid_group = postdrop
smtp-amavis_destination_recipient_limit = 1
smtp_data_init_timeout = 240s
smtp_data_xfer_timeout = 600s
smtp_destination_concurrency_limit = 5
smtp_destination_rate_delay = 1s
smtpd_banner = smtp.[REDACTED] ESMTP $mail_name
smtpd_data_restrictions = reject_unauth_pipelining
smtpd_end_of_data_restrictions = check_policy_service inet:127.0.0.1:10031
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_helo_hostname, reject_invalid_helo_hostname, check_helo_access pcre:/etc/postfix/helo_access.pcre
smtpd_recipient_restrictions = check_sender_access hash:/etc/postfix/sender_access reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_non_fqdn_sender, reject_non_fqdn_recipient, check_policy_service inet:127.0.0.1:7777, check_policy_service inet:127.0.0.1:10031, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject_rbl_client b.barracudacentral.org, reject_rbl_client bl.spamcop.net, reject_rbl_client cbl.abuseat.org, reject_rbl_client zen.spamhaus.org, permit
smtpd_reject_unlisted_recipient = yes
smtpd_reject_unlisted_sender = yes
smtpd_sasl_auth_enable = yes
smtpd_sasl_authenticated_header = no
smtpd_sasl_local_domain =
smtpd_sasl_path = ./dovecot-auth
smtpd_sasl_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_sender_login_maps = proxy:mysql:/etc/postfix/mysql/sender_login_maps.cf
smtpd_sender_restrictions = permit_mynetworks, reject_sender_login_mismatch, permit_sasl_authenticated
smtpd_tls_CAfile = /etc/pki/tls/certs/wildcard_[REDACTED].pem
smtpd_tls_cert_file = /etc/pki/tls/certs/wildcard_[REDACTED].pem
smtpd_tls_key_file = /etc/pki/tls/private/wildcard_[REDACTED].key
smtpd_tls_loglevel = 0
smtpd_tls_security_level = may
swap_bangpath = no
tls_random_source = dev:/dev/urandom
transport_maps = proxy:mysql:/etc/postfix/mysql/transport_maps_user.cf, proxy:mysql:/etc/postfix/mysql/transport_maps_domain.cf
unknown_local_recipient_reject_code = 550
virtual_alias_domains =
virtual_alias_maps = proxy:mysql:/etc/postfix/mysql/virtual_alias_maps.cf, proxy:mysql:/etc/postfix/mysql/domain_alias_maps.cf, proxy:mysql:/etc/postfix/mysql/catchall_maps.cf, proxy:mysql:/etc/postfix/mysql/domain_alias_catchall_maps.cf
virtual_gid_maps = static:2000
virtual_mailbox_base = /var/vmail
virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql/virtual_mailbox_domains.cf
virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql/virtual_mailbox_maps.cf
virtual_minimum_uid = 2000
virtual_transport = dovecot
virtual_uid_maps = static:2000

So thank you for helping me correct that issue. However, there does seem to be some sort of problem that continues to persists. This is the out put of

"sudo tail -f /var/log/maillog | grep '> <' | grep -vf domains.txt "

where domains.txt is a list of all the domains we manage. 

Aug 14 11:37:22 [REDACTED] amavis[9661]: (09661-01) Passed SPAM {RelayedOutbound,Quarantined}, LOCAL [198.41.122.235]:49926 [198.41.122.235] <3FreeCreditScores@yourcreditscorechanged.info> -> <dwatts@dixonwells.com>, quarantine: b07rFxZov3E3, Queue-ID: 19007824BD, Message-ID: <25185934662510101115@score.yourcreditscorechanged.info>, mail_id: b07rFxZov3E3, Hits: 8.508, size: 3583, queued_as: 9218A82565, 4487 ms
Aug 14 11:37:34 [REDACTED] amavis[9661]: (09661-02) Passed CLEAN {RelayedOutbound}, LOCAL [103.241.0.200]:37466 [103.241.0.200] <Ford@17trout.eu> -> <janicedgrimes@gmail.com>, Queue-ID: 40A11824CB, Message-ID: <89933652865963899344939811832222@z1to6.17trout.eu>, mail_id: hazXzZWsJYqv, Hits: 2.716, size: 6087, queued_as: 45CE3824BD, 4399 ms
Aug 14 11:37:34 [REDACTED] amavis[9251]: (09251-11) Passed CLEAN {RelayedOutbound}, LOCAL [103.241.0.200]:37466 [103.241.0.200] <Ford@17trout.eu> -> <xyandme@netins.net>, Queue-ID: 40A11824CB, Message-ID: <89933652865963899344939811832222@z1to6.17trout.eu>, mail_id: vrG2FcLYi0gD, Hits: 2.716, size: 6087, queued_as: 4FCA98259B, 4418 ms
Aug 14 11:38:46 [REDACTED] amavis[9499]: (09499-10) Passed CLEAN {RelayedOutbound}, LOCAL [206.221.181.55]:60617 [206.221.181.55] <notification@searchbestrates-local.us> -> <pokeno60@gmail.com>, Queue-ID: EE2BD824BD, Message-ID: <alrt3589029.12072345@searchbestrates-local.us-notice554>, mail_id: BHt1RttnZRUe, Hits: 4.391, size: 1776, queued_as: 80115824CB, 4479 ms
Aug 14 11:38:47 [REDACTED] amavis[9661]: (09661-05) Passed CLEAN {RelayedOutbound}, LOCAL [206.221.181.55]:60617 [206.221.181.55] <notification@searchbestrates-local.us> -> <c7-w7@hotmail.com>, Queue-ID: EE2BD824BD, Message-ID: <alrt3589029.12072345@searchbestrates-local.us-notice554>, mail_id: ZN1z8nc7wsXl, Hits: 4.391, size: 1776, queued_as: 02CA2824CB, 4957 ms
Aug 14 11:40:40 [REDACTED] amavis[9499]: (09499-17) Passed CLEAN {RelayedOutbound}, LOCAL [208.75.123.166]:37260 [208.75.123.166] <AffERXNU5RpyOTlaToAvOtQ==_1107990572325_4vn4MByiEeS3+dSuUoBpBQ==@in.constantcontact.com> -> <smarshall@dixonwells.com>, Queue-ID: D18D88259D, Message-ID: <1118188781461.1107990572325.1813382902.0.131138JL.1002@scheduler.constantcontact.com>, mail_id: zbuYJRwtpmaX, Hits: -3.861, size: 21492, queued_as: 90846825A2, dkim_sd=1000073432:auth.ccsend.com, 5665 ms
Aug 14 11:41:01 [REDACTED] amavis[9704]: (09704-14) Passed CLEAN {RelayedOutbound}, LOCAL [173.244.147.136]:34316 [173.244.147.136] <MikeWard@bluestarop.info> -> <c7-w7@hotmail.com>, Queue-ID: B77E1824CB, Message-ID: <1099159157739010992231837081336@fan.bluestarop.info>, mail_id: 2Y2l17TCpKfJ, Hits: 1.595, size: 6548, queued_as: 901FF82551, 4700 ms
Aug 14 11:41:01 [REDACTED] amavis[9499]: (09499-18) Passed CLEAN {RelayedOutbound}, LOCAL [173.244.147.136]:34316 [173.244.147.136] <MikeWard@bluestarop.info> -> <pokeno60@gmail.com>, Queue-ID: B77E1824CB, Message-ID: <1099159157739010992231837081336@fan.bluestarop.info>, mail_id: HYq9UIsBmgRi, Hits: 1.595, size: 6548, queued_as: 9B9848259D, 4736 ms
Aug 14 11:41:50 [REDACTED] amavis[9704]: (09704-16) Passed CLEAN {RelayedOutbound}, LOCAL [63.204.4.39]:51834 [63.204.4.39] <IlanaK@sfjcf.org> -> <maddy@maddyleader.com>, Queue-ID: 20A1A8259B, Message-ID: <FB3821E3-46BF-4E6C-9ADC-FDF70F1891BA@sfjcf.org>, mail_id: aqDZE666DUSM, Hits: -2.558, size: 16022, queued_as: 1B71D825A6, 5673 ms
Aug 14 11:42:11 [REDACTED] amavis[9704]: (09704-18) Passed SPAM {RelayedOutbound,Quarantined}, LOCAL [206.221.181.55]:33035 [206.221.181.55] <notification@searchbestrates-local.us> -> <smarshall@dixonwells.com>, quarantine: XNMiWWP5RYkc, Queue-ID: B1B29824BD, Message-ID: <alrt23062246.17782691@searchbestrates-local.us-notice554>, mail_id: XNMiWWP5RYkc, Hits: 11.742, size: 1760, queued_as: 56DAF824CB, 4534 ms
Aug 14 11:43:04 [REDACTED] amavis[9486]: (09486-15) Passed CLEAN {RelayedOutbound}, LOCAL [205.234.223.191]:47201 [205.234.146.124] <rec@coaches-corner-newsletter.com> -> <c7-w7@hotmail.com>, Queue-ID: C8989824CB, Message-ID: <ed3712d38144e8760c277db31251f88e@coaches-corner-newsletter.com>, mail_id: at8CkPYPThFK, Hits: -1.074, size: 19846, queued_as: F0CAE82590, dkim_sd=default:coaches-corner-newsletter.com, 5077 ms
Aug 14 11:43:04 [REDACTED] amavis[9727]: (09727-17) Passed CLEAN {RelayedOutbound}, LOCAL [205.234.223.191]:47201 [205.234.146.124] <rec@coaches-corner-newsletter.com> -> <pokeno60@gmail.com>, Queue-ID: C8989824CB, Message-ID: <ed3712d38144e8760c277db31251f88e@coaches-corner-newsletter.com>, mail_id: oba_anC8jl-G, Hits: -1.074, size: 19846, queued_as: EFF6382551, dkim_sd=default:coaches-corner-newsletter.com, 5076 ms

Those messages are not supposed to be going through our system.

Also something else that is deeply troubling is there has been a new domain that has been set up on our system that we did not put there and I found that it was the one that was forwarding all mail to [REDACTED]@cornerstone-valuation.com ([REDACTED] in this case being the supposed pen-testers online handle).

Edit: Many of the domains sending the mail through the system still point back to CentOS server with the apache test pages.

Example: http://bluestarop.info/

I'm still rather concerned. Is there any reason I shouldn't be?

6 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,094

Re: Postfix acting as spam server. Looks like it's part of a botnet..

Sorry, i don't quite understand what your new issue/question is. sad

7 Reply by HAL9000 (edited by HAL9000 2014-08-15 13:03:51)

  • HAL9000
  • Member
  • Offline
  • Registered: 2014-08-05
  • Posts: 12

Re: Postfix acting as spam server. Looks like it's part of a botnet..

ZhangHuangbin wrote:

Sorry, i don't quite understand what your new issue/question is. sad

None of the messages in the previous log were to or from any of our users. Why is my server still being used to send out spam? Or am I somehow misinterpreting what is going on? Is it compromised and if it is can it be fixed?

I'm sorry my questions are not more specific, but I'm unsure of what is wrong with it. sad

8 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,094

Re: Postfix acting as spam server. Looks like it's part of a botnet..

1) Which email address is the sender of these spams?
2) Try to find out log entries related to sender email address in 1) in Postfix log file /var/log/maillog.
3) Try to find out which sasl username this spam sender used to send email in Postfix log file.

Usually, above 3 steps will help you find out which mail user was used to send spam, and what you need to do is asking this user to reset password to a strong one.

And, don't forget to apply fixes mentioned in my previous reply (improper Postfix settings):
http://www.iredmail.org/forum/post32588.html#p32588

9 Reply by HAL9000

  • HAL9000
  • Member
  • Offline
  • Registered: 2014-08-05
  • Posts: 12

Re: Postfix acting as spam server. Looks like it's part of a botnet..

ZhangHuangbin wrote:

1) Which email address is the sender of these spams?

Whoever is doing this is using hundreds of different addresses to send spam through the system. But for the sake of figureing out what's wrong I will use one as an example.

ramonp@overwhelms.desiro.asia

ZhangHuangbin wrote:

2) Try to find out log entries related to sender email address in 1) in Postfix log file /var/log/maillog.

Here is the log from just one of the emails that this address has sent.

"sudo grep 37AA882410 /var/log/maillog"

Aug 15 08:37:48 [REDACTED] postfix/smtpd[9971]: 37AA882410: client=[REDACTED][127.0.0.1]
Aug 15 08:37:48 [REDACTED] postfix/cleanup[8215]: 37AA882410: message-id=<F908E09D.2C58.E93B.8F4E@desiro.asia>
Aug 15 08:37:48 [REDACTED] postfix/qmgr[2388]: 37AA882410: from=<ramonp@overwhelms.desiro.asia>, size=415598, nrcpt=1 (queue active)
Aug 15 08:37:48 [REDACTED] amavis[9412]: (09412-07) Passed CLEAN {RelayedOutbound}, LOCAL [93.158.215.139]:48633 [93.158.215.139] <ramonp@overwhelms.desiro.asia> -> <c7-w7@hotmail.com>, Queue-ID: 26379802F0, Message-ID: <F908E09D.2C58.E93B.8F4E@desiro.asia>, mail_id: skmMeFMU6b0y, Hits: 1.396, size: 415146, queued_as: 37AA882410, 4500 ms
Aug 15 08:37:48 [REDACTED] postfix/smtp[9206]: 26379802F0: to=<c7-w7@hotmail.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=8.7, delays=4.1/0/0.01/4.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 37AA882410)
Aug 15 08:37:49 [REDACTED] postfix/smtp[10098]: 37AA882410: to=<c7-w7@hotmail.com>, relay=mx3.hotmail.com[65.55.33.135]:25, delay=1.4, delays=0.11/0/0.25/1, dsn=2.0.0, status=sent (250  <F908E09D.2C58.E93B.8F4E@desiro.asia> Queued mail for delivery)
Aug 15 08:37:49 [REDACTED] postfix/qmgr[2388]: 37AA882410: removed

Here are all the messages there are from that sender in the /var/log/maillog

"sudo grep ramonp@overwhelms.desiro.asia /var/log/maillog"

Aug 15 08:37:43 [REDACTED] postfix/qmgr[2388]: 26379802F0: from=<ramonp@overwhelms.desiro.asia>, size=415146, nrcpt=3 (queue active)
Aug 15 08:37:48 [REDACTED] postfix/qmgr[2388]: 37AA882410: from=<ramonp@overwhelms.desiro.asia>, size=415598, nrcpt=1 (queue active)
Aug 15 08:37:48 [REDACTED] amavis[9412]: (09412-07) Passed CLEAN {RelayedOutbound}, LOCAL [93.158.215.139]:48633 [93.158.215.139] <ramonp@overwhelms.desiro.asia> -> <c7-w7@hotmail.com>, Queue-ID: 26379802F0, Message-ID: <F908E09D.2C58.E93B.8F4E@desiro.asia>, mail_id: skmMeFMU6b0y, Hits: 1.396, size: 415146, queued_as: 37AA882410, 4500 ms
Aug 15 08:37:48 [REDACTED] postfix/qmgr[2388]: 41BA682596: from=<ramonp@overwhelms.desiro.asia>, size=415620, nrcpt=1 (queue active)
Aug 15 08:37:48 [REDACTED] postfix/qmgr[2388]: 4744882597: from=<ramonp@overwhelms.desiro.asia>, size=415600, nrcpt=1 (queue active)
Aug 15 08:37:48 [REDACTED] amavis[8758]: (08758-20) Passed CLEAN {RelayedOutbound}, LOCAL [93.158.215.139]:48633 [93.158.215.139] <ramonp@overwhelms.desiro.asia> -> <pokeno60@gmail.com>, Queue-ID: 26379802F0, Message-ID: <F908E09D.2C58.E93B.8F4E@desiro.asia>, mail_id: 4uWCLUmCnReh, Hits: 1.396, size: 415146, queued_as: 4744882597, 4565 ms
Aug 15 08:37:48 [REDACTED] amavis[8873]: (08873-15) Passed CLEAN {RelayedOutbound}, LOCAL [93.158.215.139]:48633 [93.158.215.139] <ramonp@overwhelms.desiro.asia> -> <[CUSTOMER EMAIL]>, Queue-ID: 26379802F0, Message-ID: <F908E09D.2C58.E93B.8F4E@desiro.asia>, mail_id: oeqVijJVYxky, Hits: 1.396, size: 415146, queued_as: 41BA682596, 4567 ms[REDACTED]

Here are all logs related to that IP

sudo grep 93.158.215.139 /var/log/maillog"

Aug 15 08:37:39 redmail02 postfix/smtpd[8434]: warning: hostname tubert.pw does not resolve to address 93.158.215.139
Aug 15 08:37:39 redmail02 postfix/smtpd[8434]: connect from unknown[93.158.215.139]
Aug 15 08:37:42 redmail02 postfix/smtpd[8434]: 26379802F0: client=unknown[93.158.215.139]
Aug 15 08:37:44 redmail02 postfix/smtpd[8434]: disconnect from unknown[93.158.215.139]
Aug 15 08:37:48 redmail02 amavis[9412]: (09412-07) Passed CLEAN {RelayedOutbound}, LOCAL [93.158.215.139]:48633 [93.158.215.139] <ramonp@overwhelms.desiro.asia> -> <c7-w7@hotmail.com>, Queue-ID: 26379802F0, Message-ID: <F908E09D.2C58.E93B.8F4E@desiro.asia>, mail_id: skmMeFMU6b0y, Hits: 1.396, size: 415146, queued_as: 37AA882410, 4500 ms
Aug 15 08:37:48 redmail02 amavis[8758]: (08758-20) Passed CLEAN {RelayedOutbound}, LOCAL [93.158.215.139]:48633 [93.158.215.139] <ramonp@overwhelms.desiro.asia> -> <pokeno60@gmail.com>, Queue-ID: 26379802F0, Message-ID: <F908E09D.2C58.E93B.8F4E@desiro.asia>, mail_id: 4uWCLUmCnReh, Hits: 1.396, size: 415146, queued_as: 4744882597, 4565 ms
Aug 15 08:37:48 redmail02 amavis[8873]: (08873-15) Passed CLEAN {RelayedOutbound}, LOCAL [93.158.215.139]:48633 [93.158.215.139] <ramonp@overwhelms.desiro.asia> -> <[CUSTOMER EMAIL]>, Queue-ID: 26379802F0, Message-ID: <F908E09D.2C58.E93B.8F4E@desiro.asia>, mail_id: oeqVijJVYxky, Hits: 1.396, size: 415146, queued_as: 41BA682596, 4567 ms

ZhangHuangbin wrote:

3) Try to find out which sasl username this spam sender used to send email in Postfix log file.

There are no lines in /var/log/maillog that concern a sasl username for this sender.

"sudo egrep '93.158.215.139|desiro.asia' /var/log/maillog | grep sasl"

This command returns nothing. sad

ZhangHuangbin wrote:

Usually, above 3 steps will help you find out which mail user was used to send spam, and what you need to do is asking this user to reset password to a strong one.

There is no user to go to as far as I can tell. Neither ramonp@overwhelms.desiro.asia or c7-w7@hotmail.com are users in the system. Neither have a sasl username associated with them. This is why I'm so confused.

ZhangHuangbin wrote:

And, don't forget to apply fixes mentioned in my previous reply (improper Postfix settings):
http://www.iredmail.org/forum/post32588.html#p32588

"sudo postconf -n | egrep 'smtpd_recipient_restrictions|smtpd_sender_restrictions' | egrep -v 'proxy_read_maps'"

smtpd_recipient_restrictions = check_sender_access hash:/etc/postfix/sender_access reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_non_fqdn_sender, reject_non_fqdn_recipient, check_policy_service inet:127.0.0.1:7777, check_policy_service inet:127.0.0.1:10031, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject_rbl_client b.barracudacentral.org, reject_rbl_client bl.spamcop.net, reject_rbl_client cbl.abuseat.org, reject_rbl_client zen.spamhaus.org, permit
smtpd_sender_restrictions = permit_mynetworks, reject_sender_login_mismatch, permit_sasl_authenticated

The policy was applied yesterday when I restart postfix, amavisd, and dovecot. If these setting are not correct I cannot tell it. I wasn't the one who set up these configurations and unfortunately the person who did didn't leave comments explaining why these setting are the way they are. sad

If this sender (ramonp@overwhelms.desiro.asia) and the receiver (c7-w7@hotmail.com) are not users of our system and my mail.cf file is now correct and has been applied why is our system handling their mail? Why is ramonp@overwhelms.desiro.asia not being blocked like he should be and why is there no sasl username associated with it?

10 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,094

Re: Postfix acting as spam server. Looks like it's part of a botnet..

HAL9000 wrote:

There are no lines in /var/log/maillog that concern a sasl username for this sender.

Please don't grep this way. Postfix logs sasl username without sender address in same line, so you cannot find it this way.

You should extract log in the same day (or just the time spam appears), then check the log line by line.

11 Reply by HAL9000 (edited by HAL9000 2014-08-19 04:50:27)

  • HAL9000
  • Member
  • Offline
  • Registered: 2014-08-05
  • Posts: 12

Re: Postfix acting as spam server. Looks like it's part of a botnet..

Perhaps my logging is more verbose than default, but every time the word "sasl" appears in a line it appears with an email address and a Queue-ID.

Example:

Aug 13 10:27:08 [REDACTED] postfix/smtpd[16441]: 002B082568: client=c-[MY DOMAIN][IP ADDRESS], sasl_method=PLAIN, sasl_username=[MY USERNAME]
Aug 13 11:50:19 [REDACTED] postfix/smtpd[27412]: 405DE80273: client=c-[MY DOMAIN][IP ADDRESS], sasl_method=PLAIN, sasl_username=[MY USERNAME]
Aug 13 12:35:53 [REDACTED] postfix/smtpd[5727]: 693CF824FC: client=c-[MY DOMAIN][IP ADDRESS], sasl_method=PLAIN, sasl_username=[MY USERNAME]
Aug 13 13:33:31 [REDACTED] postfix/smtpd[4727]: 5970F82581: client=c-[MY DOMAIN][IP ADDRESS], sasl_method=PLAIN, sasl_username=[MY USERNAME]
Aug 14 09:30:15 [REDACTED] postfix/smtpd[24214]: 806EC81A44: client=c-[MY DOMAIN][IP ADDRESS], sasl_method=PLAIN, sasl_username=[MY USERNAME]
Aug 15 09:32:27 [REDACTED] postfix/smtpd[18100]: 5FE908256B: client=c-[MY DOMAIN][IP ADDRESS], sasl_method=PLAIN, sasl_username=[MY USERNAME]
Aug 15 13:11:46 [REDACTED] postfix/smtpd[12178]: D6948825B4: client=c-[MY DOMAIN][IP ADDRESS], sasl_method=PLAIN, sasl_username=[MY USERNAME]
Aug 15 13:25:39 [REDACTED] postfix/smtpd[21525]: F15F782592: client=c-[MY DOMAIN][IP ADDRESS], sasl_method=PLAIN, sasl_username=[MY USERNAME]

In fact, the following command returns no results at all.

"sudo egrep sasl maillog-20140817 | grep -v @"

That being said, and correct me if I am wrong, I believe there should have been a log entry containing “overwhelms.desiro.asia” and “sasl” and it should also contain the Queue-ID (26379802F0) just like it does for all other valid users of the system.

Here is what a normal sample of my log looks like for a message being routed through the system tracking both Queue-ID and queued_as. It is clear that the first entry contains both Queue-ID and sasl username.

“sudo egrep 'AE7A882592|232ED82593' /var/log/maillog”

sudo egrep 'AE7A882592|232ED82593' /var/log/maillog
Aug 18 08:31:12 [REDACTED] postfix/smtpd[18693]: AE7A882592: client=[REDACTED].[OUR DOMAIN][127.0.0.1], sasl_method=LOGIN, sasl_username=[CUSTOMER EMAIL]
Aug 18 08:31:12 [REDACTED] postfix/cleanup[18713]: AE7A882592: message-id=<ebe709f4cd698b92777997a0d05d9437@[CUSTOMER DOMAIN]>
Aug 18 08:31:12 [REDACTED] postfix/qmgr[11844]: AE7A882592: from=<[CUSTOMER EMAIL]>, size=1939, nrcpt=1 (queue active)
Aug 18 08:31:12 [REDACTED] roundcube: User [CUSTOMER EMAIL] [[CUSTOMER IP]]; Message for [RECIPIENT]; 250: 2.0.0 Ok: queued as AE7A882592
Aug 18 08:31:18 [REDACTED] postfix/smtpd[17890]: 232ED82593: client=[REDACTED].[OUR DOMAIN][127.0.0.1]
Aug 18 08:31:18 [REDACTED] postfix/cleanup[18713]: 232ED82593: message-id=<ebe709f4cd698b92777997a0d05d9437@[CUSTOMER DOMAIN]>
Aug 18 08:31:18 [REDACTED] postfix/qmgr[11844]: 232ED82593: from=<[CUSTOMER EMAIL]>, size=2401, nrcpt=1 (queue active)
Aug 18 08:31:18 [REDACTED] amavis[20010]: (20010-08) Passed CLEAN {RelayedOutbound}, MYNETS LOCAL [127.0.0.1]:46242 [127.0.0.1] <[CUSTOMER EMAIL]> -> <[RECIPIENT]>, Queue-ID: AE7A882592, Message-ID: <ebe709f4cd698b92777997a0d05d9437@[CUSTOMER DOMAIN]>, mail_id: JIQBGtbIfO8R, Hits: -11.899, size: 1939, queued_as: 232ED82593, 5411 ms
Aug 18 08:31:18 [REDACTED] postfix/smtp[19487]: AE7A882592: to=<[RECIPIENT]>, relay=127.0.0.1[127.0.0.1]:10024, delay=5.5, delays=0.09/0/0.01/5.4, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 232ED82593)
Aug 18 08:31:18 [REDACTED] postfix/qmgr[11844]: AE7A882592: removed
Aug 18 08:33:46 [REDACTED] postfix/smtp[20377]: 232ED82593: to=<[RECIPIENT]>, relay=newclients-com.mail.protection.outlook.com[[RECIPIENT IP]]:25, delay=148, delays=0.04/0.01/4.3/144, dsn=2.6.0, status=sent (250 2.6.0 <ebe709f4cd698b92777997a0d05d9437@[CUSTOMER DOMAIN]> [InternalId=38220913971934, Hostname=BN1PR05MB342.namprd05.prod.outlook.com] Queued mail for delivery)
Aug 18 08:33:46 [REDACTED] postfix/qmgr[11844]: 232ED82593: removed

Now, in contrast, here’s what the message that ramonp@overwhelms.desiro.asia sent through our system looks like:

“sudo egrep '26379802F0|37AA882410' /var/log/maillog-20140817”

Aug 15 08:37:42 [REDACTED] postfix/smtpd[8434]: 26379802F0: client=unknown[93.158.215.139]
Aug 15 08:37:42 [REDACTED] postfix/cleanup[8215]: 26379802F0: message-id=<F908E09D.2C58.E93B.8F4E@desiro.asia>
Aug 15 08:37:43 [REDACTED] postfix/qmgr[2388]: 26379802F0: from=<ramonp@overwhelms.desiro.asia>, size=415146, nrcpt=3 (queue active)
Aug 15 08:37:48 [REDACTED] postfix/smtpd[9971]: 37AA882410: client=[REDACTED].[OUR DOMAIN][127.0.0.1]
Aug 15 08:37:48 [REDACTED] postfix/cleanup[8215]: 37AA882410: message-id=<F908E09D.2C58.E93B.8F4E@desiro.asia>
Aug 15 08:37:48 [REDACTED] postfix/qmgr[2388]: 37AA882410: from=<ramonp@overwhelms.desiro.asia>, size=415598, nrcpt=1 (queue active)
Aug 15 08:37:48 [REDACTED] amavis[9412]: (09412-07) Passed CLEAN {RelayedOutbound}, LOCAL [93.158.215.139]:48633 [93.158.215.139] <ramonp@overwhelms.desiro.asia> -> <c7-w7@hotmail.com>, Queue-ID: 26379802F0, Message-ID: <F908E09D.2C58.E93B.8F4E@desiro.asia>, mail_id: skmMeFMU6b0y, Hits: 1.396, size: 415146, queued_as: 37AA882410, 4500 ms
Aug 15 08:37:48 [REDACTED] postfix/smtp[9206]: 26379802F0: to=<c7-w7@hotmail.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=8.7, delays=4.1/0/0.01/4.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 37AA882410)
Aug 15 08:37:48 [REDACTED] postfix/qmgr[2388]: 26379802F0: removed
Aug 15 08:37:49 [REDACTED] postfix/smtp[10098]: 37AA882410: to=<c7-w7@hotmail.com>, relay=mx3.hotmail.com[65.55.33.135]:25, delay=1.4, delays=0.11/0/0.25/1, dsn=2.0.0, status=sent (250  <F908E09D.2C58.E93B.8F4E@desiro.asia> Queued mail for delivery)
Aug 15 08:37:49 [REDACTED] postfix/qmgr[2388]: 37AA882410: removed

The line that is supposed to contain the sasl username (the line with “postfix/smtpd” in it, the first one) only contains this:

“Aug 15 08:37:42 [REDACTED] postfix/smtpd[8434]: 26379802F0: client=unknown[93.158.215.139]”

Both entries consists of the same information. In my thinking the information provided should contain all of the same fields but it doesn’t. I also went and checked around each entry going through around 1000 lines of logs and turned up nothing. Maybe I'm looking at this wrong but I cannot explain any better what I'm finding.

Why doesn’t the message that ramonp@overwhelms.desiro.asia sent have any sasl username associated with it? Why didn’t they have to login to send it through the system? How is something like this possible?

Sorry to keep pressing you but this is a real problem for us over here. Our server is doing this 1000's of times a day.

Edit: I even have another example where the line that should cantain the sasl username doesn't not but actually contains something other than "unknown". lgamerica.lgamerica.com is not valid user of our system ye they are using our system to send their mail with.

"sudo egrep '506DA82590|01320825C0' /var/log/maillog" ( grepping for queue_id and queued_as) 

Aug 18 16:33:05 [REDACTED] postfix/smtpd[16588]: 506DA82590: client=lgamerica.lgamerica.com[63.139.123.170]
Aug 18 16:33:05 [REDACTED] postfix/cleanup[20530]: 506DA82590: message-id=<>
Aug 18 16:33:05 [REDACTED] postfix/qmgr[11844]: 506DA82590: from=<md-uwteam-d@lgamerica.com>, size=4636, nrcpt=1 (queue active)
Aug 18 16:33:08 [REDACTED] postfix/smtpd[19146]: 01320825C0: client=[REDACTED].[OUR DOMAIN][127.0.0.1]
Aug 18 16:33:08 [REDACTED] postfix/cleanup[20977]: 01320825C0: message-id=<20140818203308.01320825C0@smtp.[OWR DOMAIN]>
Aug 18 16:33:08 [REDACTED] postfix/qmgr[11844]: 01320825C0: from=<md-uwteam-d@lgamerica.com>, size=5279, nrcpt=1 (queue active)
Aug 18 16:33:08 [REDACTED] amavis[21130]: (21130-07) Passed BAD-HEADER-7 {RelayedOutbound}, LOCAL [63.139.123.170]:61197 [63.139.123.170] <md-uwteam-d@lgamerica.com> -> <lboughman@dixonwells.com>, Queue-ID: 506DA82590, mail_id: 59GmrNn_cBLw, Hits: -0.691, size: 4635, queued_as: 01320825C0, 2618 ms
Aug 18 16:33:08 [REDACTED] postfix/smtp[21129]: 506DA82590: to=<lboughman@dixonwells.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=5.8, delays=3.2/0/0.01/2.7, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 01320825C0)
Aug 18 16:33:08 [REDACTED] postfix/qmgr[11844]: 506DA82590: removed
Aug 18 16:33:13 [REDACTED] postfix/smtp[21059]: 01320825C0: to=<lboughman@dixonwells.com>, relay=dixonwells-com.mail.eo.outlook.com[207.46.163.215]:25, delay=5.5, delays=0.04/0.01/2.1/3.4, dsn=2.6.0, status=sent (250 2.6.0 <20140818203308.01320825C0@smtp.[OWR DOMAIN]> [InternalId=17274358472984, Hostname=BLUPR04MB837.namprd04.prod.outlook.com] Queued mail for delivery)
Aug 18 16:33:13 [REDACTED] postfix/qmgr[11844]: 01320825C0: removed

12 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,094

Re: Postfix acting as spam server. Looks like it's part of a botnet..

HAL9000 wrote:

That being said, and correct me if I am wrong, I believe there should have been a log entry containing “overwhelms.desiro.asia” and “sasl” and it should also contain the Queue-ID (26379802F0) just like it does for all other valid users of the system.

Wrong assumption.

You overused 'grep/egrep'. All i can suggest is:

ZhangHuangbin wrote:

You should extract log in the same day (or just the time spam appears), then check the log line by line.