Topic: How To Fill Queue Faster?
==== Provide required information ====
- iRedMail version and backend (LDAP/MySQL/PGSQL):
- Linux/BSD distribution name and version:
- Any related log? Log is helpful for troubleshooting.
==== ==== Provide required information ====
- iRedMail version and backend (LDAP/MySQL/PGSQL): 0.8.0 MySQL
- Linux/BSD distribution name and version: Ubuntu 12.04
- Any related log? Log is helpful for troubleshooting.
Here is my postconf -n:
root@murph:~# postconf -n
alias_database = hash:/etc/postfix/aliases
alias_maps = hash:/etc/postfix/aliases
allow_min_user = no
append_dot_mydomain = no
biff = no
bounce_queue_lifetime = 1d
broken_sasl_auth_clients = yes
config_directory = /etc/postfix
content_filter = smtp-amavis:[127.0.0.1]:10024
default_process_limit = 1000
delay_warning_time = 0h
disable_vrfy_command = yes
dovecot_destination_recipient_limit = 10
enable_original_recipient = no
home_mailbox = Maildir/
inet_interfaces = all
inet_protocols = ipv4
mailbox_command = /usr/lib/dovecot/deliver
mailbox_size_limit = 0
maximal_backoff_time = 1000s
maximal_queue_lifetime = 1d
message_size_limit = 20480000
minimal_backoff_time = 60s
mydestination = $myhostname, localhost, localhost.localdomain, localhost.$myhostname
mydomain = miridiatech.com
myhostname = mail.miridiatech.com
mynetworks = 127.0.0.0/8
mynetworks_style = subnet
myorigin = mail.miridiatech.com
proxy_read_maps = $canonical_maps $lmtp_generic_maps $local_recipient_maps $mydestination $mynetworks $recipient_bcc_maps $recipient_canonical_maps $relay_domains $relay_recipient_maps $relocated_maps $sender_bcc_maps $sender_canonical_maps $smtp_generic_maps $smtpd_sender_login_maps $transport_maps $virtual_alias_domains $virtual_alias_maps $virtual_mailbox_domains $virtual_mailbox_maps $smtpd_sender_restrictions
qmgr_message_recipient_limit = 60000
queue_run_delay = 60s
readme_directory = no
recipient_delimiter = +
smtp-amavis_destination_recipient_limit = 10
smtp_data_init_timeout = 240s
smtp_data_xfer_timeout = 600s
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
smtpd_data_restrictions = reject_unauth_pipelining
smtpd_end_of_data_restrictions = check_policy_service inet:127.0.0.1:10031
smtpd_enforce_tls = no
smtpd_error_sleep_time = 0
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks,permit_sasl_authenticated, check_helo_access pcre:/etc/postfix/helo_access.pcre
smtpd_null_access_lookup_key = null@sender.com
smtpd_recipient_restrictions = reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unlisted_recipient, check_policy_service inet:127.0.0.1:7777, check_policy_service inet:127.0.0.1:10031, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject_non_fqdn_helo_hostname, reject_invalid_helo_hostname
smtpd_reject_unlisted_recipient = yes
smtpd_reject_unlisted_sender = yes
smtpd_sasl_auth_enable = yes
smtpd_sasl_authenticated_header = no
smtpd_sasl_local_domain =
smtpd_sasl_path = ./dovecot-auth
smtpd_sasl_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_sender_login_maps = proxy:mysql:/etc/postfix/mysql/sender_login_maps.cf
smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated
smtpd_tls_CAfile = /etc/ssl/certs/iRedMail_CA.pem
smtpd_tls_cert_file = /etc/ssl/certs/iRedMail_CA.pem
smtpd_tls_key_file = /etc/ssl/private/iRedMail.key
smtpd_tls_loglevel = 0
smtpd_tls_security_level = may
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_use_tls = yes
tls_random_source = dev:/dev/urandom
transport_maps = proxy:mysql:/etc/postfix/mysql/transport_maps_user.cf, proxy:mysql:/etc/postfix/mysql/transport_maps_domain.cf
virtual_alias_domains =
virtual_alias_maps = proxy:mysql:/etc/postfix/mysql/virtual_alias_maps.cf, proxy:mysql:/etc/postfix/mysql/domain_alias_maps.cf, proxy:mysql:/etc/postfix/mysql/catchall_maps.cf, proxy:mysql:/etc/postfix/mysql/domain_alias_catchall_maps.cf
virtual_gid_maps = static:1001
virtual_mailbox_base = /var/vmail
virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql/virtual_mailbox_domains.cf
virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql/virtual_mailbox_maps.cf
virtual_minimum_uid = 1001
virtual_transport = dovecot
virtual_uid_maps = static:1001
====
Ok, here's the issue: I have one server we'll call Jake. Jake runs a mailing list manager from Interspire, and we regularly send out a newsletter to 20k+ recipients. (each monday am) Our old mail server (we'll call him "elwood") was able to send to the entire list in under an hour. Elwood was an old slouch of a machine with wheezy, dying drives and poopy ram, and was running a homegrown medusa configuration of Postfix, Amavis, SA, etc. He'd been in service for 5ish years and was doing just fine - but worries of hard disk failure due to the age of the machine prompted to go ahead and move to a new box.
This past weekend I configured a new email server (named "murph") that is much beefier than elwood was. I used iRedMail based on all the great feedback I found online, and other than a few small dns snags that were completely my fault, the setup of the new mail server went perfectly. I'm impressed.
However, this morning we tried to send our newsletter and it is taking FOREVER to send. After some analysis, it appears that my issue is NOT the typical "my queue is filling up with thousands of messages help!" error... In fact, my queue stays mostly empty most of the time. It appears that jake (the server trying to send mail) is either unable, unwilling, or too depressed to try to stuff mail into the outbound queue on murph just as fast as it used to.
Murph seems to handle the delivery of outgoing mail just fine, but jake does not seem to want to try to send it as fast anymore.
I have not made any changes on jake - I suspect there is either some smtp client restriction going on, or some throttling, or maybe mysql performance is sucking to the point that jake just can't submit mail very fast for murph to deliver.
The new server is an Opteron with 4 gb of ram, and the system load sits around .7 all the time. Thus I suspect the issue is not related to the performance of murph, but rather something preventing jake from injecting mail into the queue very fast.
Where do I begin to troubleshoot this?
Thanks!
-- Kimball
----
Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.