1

Topic: Amavis / MySQL Timeouts (Still happening)

==== Required information ====
- iRedMail version (check /etc/iredmail-release): 0.9.2
- Linux/BSD distribution name and version: Debian 8.1
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): Apache
- Manage mail accounts with iRedAdmin-Pro? No
- Related log if you're reporting an issue: /var/log/mail.log
====

It happened again this morning. I'm not sure what setting that I can tweak to prevent this from happening...

Aug 11 12:13:04 mail amavis[27951]: (27951-02) NOTICE: reconnecting in response to: err=2013, HY000, timed out
Aug 11 12:13:04 mail amavis[27951]: (27951-02) (!)lookup_sql: timed out at (eval 95) line 200., ,
Aug 11 12:13:04 mail amavis[27951]: (27951-02) (!)Requesting process rundown, task exceeded allowed time

I've been trying to research the Amavis error, but I'm coming up with nothing. Apparently MySQL is timing out, but I'm not sure why. Is my server dealing with too little activity? It's just a low-traffic mail server with 4 virtual domains and about 8 users. I've tried adapting my.cnf to "max_connections = 100". Is there something else that I can do? As I've stated before, this appears to happen during periods of inactivity. Last night, the server ran without incident until this afternoon.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: Amavis / MySQL Timeouts (Still happening)

1) Low traffic mail server will have this issue, because MySQL connection (open by Amavisd) timed out.
2) Restarting MySQL service will cause the same issue, because existing MySQL connection (open by Amavisd) cannot connect MySQL server.

To fix 1), please try set below settings in Amavisd config file (http://www.iredmail.com/docs/file.locat … ml#amavisd):

$max_servers = 1;
$max_requests = 1;

3

Re: Amavis / MySQL Timeouts (Still happening)

I made the changes that you specified, Zhang. I'm keeping an eye on this. Hopefully this will keep things working. I'll keep you posted. Thank you for your help!

4

Re: Amavis / MySQL Timeouts (Still happening)

Hey Zhang. I made the changes that you specified, and all throughout last night, and today, I didn't receive any disconnect notices in my log files. It appears that it was related to the "mysql has gone away" error, but instead related to amavis timing out instead of mysql, as mysql seems to continue to run, even though Amavis seemed to die. I did receive a notice in my logwatch report this morning that said:

4   *Warning: Virus scanner timeout
4   Miscellaneous warnings

as well as:

55   Connections             
        9   Connections lost (inbound)
       55   Disconnections         
       37   Removed from queue     
       15   Delivered               
       19   Sent via SMTP           
        3   Forwarded               
        2   Deferred               
        4   Deferrals

The lost connections are related to Suricata working in IPS mode on the machine, and filtering out generic SMTP decode commands. However, I believe that the deferrals, as well as the Amavis timeouts to be from before I had made the changes, as since I have made the changes, I haven't noticed any disruptions, or deferrals.

One question: Since we have a mailing list of approximately 1,500 subscribers for our record label, with these changes to Amavis, will I notice any performance degradation when sending out to our mailing list, if I try to send to all 1,500+ subscribers? Should I return the server back to the original 4 Amavis servers setting before an email blast goes out? I'm just wondering if this is something that I'm going to have to change back before we send out an email blast to our mailing list? Just curious. I'm still keeping an eye on this issue, but so far, so good. I'd love to hear your thoughts on the performance if we send out an email blast to our mailing list with the current settings...

Many thanks!

5

Re: Amavis / MySQL Timeouts (Still happening)

SteveLuxe wrote:

4   *Warning: Virus scanner timeout

Is ClamAV running? How much RAM do you have on this server?

SteveLuxe wrote:

One question: Since we have a mailing list of approximately 1,500 subscribers for our record label, with these changes to Amavis, will I notice any performance degradation when sending out to our mailing list, if I try to send to all 1,500+ subscribers? Should I return the server back to the original 4 Amavis servers setting before an email blast goes out?

Yes there's slight performance different.
Depends on how much RAM you have on this server, you can increase the $max_servers setting. But i suggest you try once first, it should be fine. 1500 emails is not that many.

6

Re: Amavis / MySQL Timeouts (Still happening)

Hey Zhang! Thanks so much for the reply. This morning's logwatch showed no errors, and I'm looking through the log files this morning, and noticing nothing out of the ordinary. No disconnects, no Antivirus alerts, and no Amavis errors. It's been up for almost 2 days now with no noticeable disconnects that I can see. To answer your question, I'm running 2gb of RAM on this machine, with about 200mb free RAM most of the time, with everything running on it. Yes, ClamAV is running on this machine as well. It's been running ClamAV updates and restarting the service as normally. I'm hoping that the issue is fixed, now. Thank you so much for advising me, and helping with this issue. I'll keep monitoring the issue for the next couple of days. Hopefully all remains good! smile Thank you!

7

Re: Amavis / MySQL Timeouts (Still happening)

SteveLuxe wrote:

Yes, ClamAV is running on this machine as well. It's been running ClamAV updates and restarting the service as normally.

Are you running clamscan or clamd? And if you are running clamd then how many instances?

Remember clamscan is single threaded and each request for virus scan will start a new instance of clamscan. This could very well be the reason for your problems.

8

Re: Amavis / MySQL Timeouts (Still happening)

Hey Mir! Yes, I'm running clamd. I haven't modified any of the ClamAV settings from the initial iRedMail setup, from when I put this machine together with 0.9.2. It appears that there is only one instance of clamd running on this machine. I believe that the antivirus warnings were related to Amavis, not ClamAV (I was under the impression that Amavis launched ClamAV when it's needed, anyway, right?).

9

Re: Amavis / MySQL Timeouts (Still happening)

The problem gets weirder. We attempted to send an email blast to 750 recipients tonight and I'm getting this error, and continually having to restart postfix:

Aug 13 19:55:28 mail postfix/qmgr[1868]: warning: problem talking to service rewrite: Connection timed out

Any advice would be welcome. I can't restart postfix all night... I'm not sure what to do. This is a bit of a disaster.

10

Re: Amavis / MySQL Timeouts (Still happening)

I'm also getting this error:

Aug 13 20:09:06 mail amavis[5959]: (05959-01) (!)rw_loop: leaving rw loop, no progress, last event (select) 35.040 s ago

Please help if someone can!

11

Re: Amavis / MySQL Timeouts (Still happening)

I've come to the conclusion that my OS is completely screwed up, and I'm going to have to restore from a backup. So much for the problem being solved... Back to the drawing board... Thanks for your help guys!

12

Re: Amavis / MySQL Timeouts (Still happening)

If you need to post log, please post log context also. otherwise it's hard to help troubleshoot.

13

Re: Amavis / MySQL Timeouts (Still happening)

Hey Zhang, I was trying to copy and paste as much as I could yesterday evening when we were in the middle of trying to send out the email campaign. Anything and everything was failing / timing out. I believe that my problem is more of an operating system problem at this point, than it is an iRedMail problem. I'm just going to use the nuclear option and restore to a known good operating system this coming week when I can take the server down. Is there by chance a change log that's been posted since the initial 0.9.2 update, so that I can apply patches / fixes / etc?