1 Topic by kent@kamar.nz (edited by kent@kamar.nz 2024-01-02 05:35:11)

  • kent@kamar.nz
  • Member
  • Offline
  • Registered: 2018-08-28
  • Posts: 30

Topic: 504 Gateway Timeout when using Webmail

- iRedMail version: 2023122901
- Deployed with: iRedMail Easy
- Linux/BSD distribution name and version: Debian / Buster 10
- Store mail accounts in which backend: MySQL
- Web server: Nginx
- Manage mail accounts with iRedAdmin-Pro: Yes
====
Since the previous update, we have been getting a gateway timeout when using Webmail.  I've applied the latest update hoping it would resolve the issue but still happening.

I'm seeing the following error in the nginx logs right as the web page times out.
----
upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.41.40, server: _, request: "POST /?_task=login HTTP/2.0", upstream: "fastcgi://127.0.0.1:9001", host: "{{dns}}", referrer: "https://{{dns}}/"
----

This is happening for all email accounts, including a brand new test email account I just created.

I can see (usint netstat -an) that 127.0.0.1:9001  is listening and using wget I get a connection:
----
tcp        0      0 127.0.0.1:9001          0.0.0.0:*               LISTEN
----
wget 127.0.0.1:9001
--2024-01-02 10:13:15--  http://127.0.0.1:9001/
Connecting to 127.0.0.1:9001... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
----

Any ideas on where I look need to resolve this issue ?

Thanks
Kent.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by KNERD

  • KNERD
  • Member
  • Offline
  • Registered: 2019-01-25
  • Posts: 68

Re: 504 Gateway Timeout when using Webmail

Look at firewall

3 Reply by kent@kamar.nz

  • kent@kamar.nz
  • Member
  • Offline
  • Registered: 2018-08-28
  • Posts: 30

Re: 504 Gateway Timeout when using Webmail

Thanks for the reply - it's not the firewall.  This is a localhost only port.

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,121

Re: 504 Gateway Timeout when using Webmail

- Port 9001 is php-fpm service configured by iRedMail Easy.
- Is php-fpm service running? Any related error in /var/log/php-fpm/*.log?

5 Reply by kent@kamar.nz

  • kent@kamar.nz
  • Member
  • Offline
  • Registered: 2018-08-28
  • Posts: 30

Re: 504 Gateway Timeout when using Webmail

Hi Zhang,

/var/log/php-fpm/php-fpm.log
----
Jan 15 07:47:27 mail php-fpm[919]: [WARNING] [pool inet] child 7107, script '/opt/www/roundcubemail/index.php' (request: "POST /index.php?_task=login") executing too slow (22.609977 sec), logging


/var/log/php-fpm/slow.log
----
[15-Jan-2024 07:47:27]  [pool inet] pid 7107
script_filename = /opt/www/roundcubemail/index.php
[0x00007fe8bc41d940] fgets() /opt/www/roundcubemail-1.6.5/program/lib/Roundcube/rcube_imap_generic.php:212
[0x00007fe8bc41d8a0] readLine() /opt/www/roundcubemail-1.6.5/program/lib/Roundcube/rcube_imap_generic.php:332
[0x00007fe8bc41d7b0] readReply() /opt/www/roundcubemail-1.6.5/program/lib/Roundcube/rcube_imap_generic.php:772
[0x00007fe8bc41d5c0] authenticate() /opt/www/roundcubemail-1.6.5/program/lib/Roundcube/rcube_imap_generic.php:991
[0x00007fe8bc41d4c0] connect() /opt/www/roundcubemail-1.6.5/program/lib/Roundcube/rcube_imap.php:172
[0x00007fe8bc41d390] connect() /opt/www/roundcubemail-1.6.5/program/include/rcmail.php:780
[0x00007fe8bc41d1e0] login() /opt/www/roundcubemail-1.6.5/index.php:119

6 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,121

Re: 504 Gateway Timeout when using Webmail

Any related error in Dovecot? seems login toke too long.

7 Reply by kent@kamar.nz

  • kent@kamar.nz
  • Member
  • Offline
  • Registered: 2018-08-28
  • Posts: 30

Re: 504 Gateway Timeout when using Webmail

ZhangHuangbin wrote:

Any related error in Dovecot? seems login toke too long.


Nothing in the dovecot.log, but I did see this in the imap.log file right when the timeout occurred.
------------
Jan 15 21:33:40 mail dovecot: imap-login: Login: user=<kent@kamar.nz>, method=LOGIN, rip=127.0.0.1, lip=127.0.0.1, TLS, TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)
Jan 15 21:33:40 mail dovecot: imap(kent@kamar.nz)<30955><c8RY2/cOdpN/AAAB>: Connection closed (No commands sent) in=0 out=441 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0

8 Reply by kent@kamar.nz

  • kent@kamar.nz
  • Member
  • Offline
  • Registered: 2018-08-28
  • Posts: 30

Re: 504 Gateway Timeout when using Webmail

Hi Zhang,

Users using regular clients, connecting via iMap don't appear to be having any issues. I have another php script which checks email and I can also see it reporting the same error in the logs (though it does complete eventually but has a longer timeout.

Agree, the delay does appear to be during the logon stage.

Kent.

9 Reply by kent@kamar.nz

  • kent@kamar.nz
  • Member
  • Offline
  • Registered: 2018-08-28
  • Posts: 30

Re: 504 Gateway Timeout when using Webmail

Hi Zhang,

Any ideas ?

Kent.

10 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,121

Re: 504 Gateway Timeout when using Webmail

Is this a busy IMAP server and reached max client limit (search "_limit" in /var/log/dovecot/*.log)?

11 Reply by kent@kamar.nz (edited by kent@kamar.nz 2024-01-22 10:11:43)

  • kent@kamar.nz
  • Member
  • Offline
  • Registered: 2018-08-28
  • Posts: 30

Re: 504 Gateway Timeout when using Webmail

Hi Zhang,
I wouldn't say it's that busy. ~20 users using iMap.

No log entry in dovecot.log, imap.log, lda.log or pop3.log with  '_limit'


Also - this only appears to be affecting php logons (eg. WebMail).  No issues with desktop clients checking via imap.

Kent.

12 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,121

Re: 504 Gateway Timeout when using Webmail

I received reports from other iRedMail server admins before, few times, but for different Roundcube versions. I didn't figure out the cause and no solution at the end.
Some admins solved the issue by upgrading OS.

We may need Roundcube developers to help troubleshoot, but it's not what i can do.

13 Reply by kent@kamar.nz

  • kent@kamar.nz
  • Member
  • Offline
  • Registered: 2018-08-28
  • Posts: 30

Re: 504 Gateway Timeout when using Webmail

Hi Zhang,

I'm getting the same timeout with a php script we are using to check a specific email address - nothing to do with WebMail.

This is a basic php script which logs on and then pulls down any emails sent to that account.

$server = '{mail.kamar.nz:995/pop3/ssl/novalidate-cert}INBOX';
$connection = imap_open( $server, $user, $pswd );
$info = imap_check( $connection );
if( $info->Nmsgs == 0 ) { imap_close( $connection ); }
etc.

Error:
-----------
upstream timed out (110: Connection timed out) while reading response header from upstream, client, request: "GET /checkmail.php", upstream: "fastcgi://127.0.0.1:9001"

Kent.

14 Reply by kent@kamar.nz

  • kent@kamar.nz
  • Member
  • Offline
  • Registered: 2018-08-28
  • Posts: 30

Re: 504 Gateway Timeout when using Webmail

Hi Zhang,

I'm running on Debian 10 (Buster) - any issues updating to Debian 12 (Bookworm) ?

Kent.

15 Reply by RikuS

  • RikuS
  • Member
  • Offline
  • Registered: 2016-06-10
  • Posts: 116

Re: 504 Gateway Timeout when using Webmail

ZhangHuangbin wrote:

I received reports from other iRedMail server admins before, few times, but for different Roundcube versions. I didn't figure out the cause and no solution at the end.
Some admins solved the issue by upgrading OS.

We may need Roundcube developers to help troubleshoot, but it's not what i can do.

Hi Zhang and kent@kamar.nz,

I had the same issue and reverting couple Roundcube settings fixed it for me. Never figured out why this actually happens, as for many the new config version works but few are having this same issue.

See my post here https://forum.iredmail.org/post86738.html#p86738

I would suggest to reverting the relevant configs and see how it goes.

16 Reply by kent@kamar.nz

  • kent@kamar.nz
  • Member
  • Offline
  • Registered: 2018-08-28
  • Posts: 30

Re: 504 Gateway Timeout when using Webmail

Hi Zhang and @RikuS

Thanks for the link.  My Roundcube is working again :-)

Something about php / localhost and dovecot login via ssl is broken !

It does mean anyone can now authenticate with plain text login I assume however.


My custom php script is still not working reliably (it works sometimes, but times out at logon stage 1 out of 3 times. However it's still using ssl to connect to dovecot, so I'll do some research on how to get that connecting using plain text.

I ended up changing:

1)  /opt/iredmail/custom/dovecot/conf-enabled/ssl.conf
...
ssl = yes
disable_plaintext_auth = no
...

2) /opt/www/roundcubemail/config/config.inc.php
...
// IMAP
$config['default_host'] = '127.0.0.1';
$config['default_port'] = 143;
//$config["imap_host"] = "";
...
// SMTP
$config['smtp_server'] = 'tls://127.0.0.1';
$config['smtp_port'] = 587;
//$config["smtp_host"] = "";
...


I then restarted dovecot and all go.


Note: I tried adding the above settings to: /opt/iredmail/custom/roundcube/custom.inc.php
However, this introduced a new 'mail server' field at logon and I wasn't able to send emails.
I guess I'll need to check this after doing an update.


Kent.

17 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,121

Re: 504 Gateway Timeout when using Webmail

Thanks for sharing. smile