1 Topic by lopez-iredmail (edited by lopez-iredmail 2020-04-30 23:07:21)

  • lopez-iredmail
  • Member
  • Offline
  • Registered: 2014-06-24
  • Posts: 18

Topic: iredadmin and roundcube (php-fpm) keep dying

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release): 1.2
- Deployed with iRedMail Easy or the downloadable installer? Downloadable
- Linux/BSD distribution name and version:  FreeBSD 12.1 (jailed)
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): LDAP
- Web server (Apache or Nginx): Nginx
- Manage mail accounts with iRedAdmin-Pro? Yes
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

Hi,

I have actually two issues - don't know if they are related yet.
But both services accessed over ngnix (iredadmin and php-fpm/roundcube) keep dying every few hours, sometimes earlier.

I keep restarting php-fpm regularly, but that's not a solution.

Any hint on which log files I should post or places I should look at?

Thanx,

Best Regards,

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Online
  • Registered: 2009-05-06
  • Posts: 31,088

Re: iredadmin and roundcube (php-fpm) keep dying

Your description is not clear.

- What does "dying" mean here?
- Any related error in Nginx / uwsgi / iredadmin / php-fpm log files?

3 Reply by lopez-iredmail (edited by lopez-iredmail 2020-05-05 23:24:47)

  • lopez-iredmail
  • Member
  • Offline
  • Registered: 2014-06-24
  • Posts: 18

Re: iredadmin and roundcube (php-fpm) keep dying

ZhangHuangbin wrote:

Your description is not clear.

- What does "dying" mean here?
- Any related error in Nginx / uwsgi / iredadmin / php-fpm log files?

Hi Zhang,

Sorry for the lazy description. By dying, I really mean that both php-fpm and uwsgi (iredadmin-related) processes are no more running, and nginx displays, on both http endpoints respectively (/mail and /iredadmin)  "502 Bad Gateway".

Please consider this problem on hold however, for now. After restarting the jail, for now, I haven't seen iredadmin "die" again. Php-fpm, on the other side, does still die, in the sense that after a while the "master" process (ps/top description: php-fpm: master process (/usr/local/etc/php-fpm.conf) (php-fpm)) is missing from ps, but the "inet" pool processes are still there and roundcube seems to work.

There is nothing in the php-fpm or messages log showing anything useful, or I haven't found it yet.

I'll be monitoring this for a while.
I'll give it a few more days.

Thank You for your dedication.

Best,

Lorenzo

4 Reply by lopez-iredmail (edited by lopez-iredmail 2020-05-20 06:15:50)

  • lopez-iredmail
  • Member
  • Offline
  • Registered: 2014-06-24
  • Posts: 18

Re: iredadmin and roundcube (php-fpm) keep dying

Hi, I have an update on this issue.

The breakdowns of php-fpm (for roundcube) keep exiting regularly (about daily). php-fpm.log has nothing to say, while ngnix log shows, regularly, as a last entry, this (probably automated) attempt to exploit a known issue with a different php product:

2020/05/19 06:31:26 [error] 5539#100766: *60968 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: c.c.c.c, server: _, request: "GET /index.php?s=/Index/\think\app/invokefunction&function=call_user_func_array&vars[0]=md5&vars[1][]=HelloThinkPHP HTTP/1.1", upstream: "fastcgi://x.x.x.x:9999", host: "y.y.y.y", referrer: "http://y.y.y.y:80/index.php?s=/Index/\think\app/invokefunction&function=call_user_func_array&vars[0]=md5&vars[1][]=HelloThinkPHP"

Note:
- c.c.c.c is an external client ip
- x.x.x.x is the (rfc1918) IP of the jail, where iRedMail is running
- y.y.y.y is the external IP of the service (requests to y.y.y.y are redirected to x.x.x.x via a firewall on a different machine).

I wonder why these requests are handled by PHP anyway, as only the ones in /mail/ should be in this environment.
Haven't digged in the details of your nginx.conf, but there must be something that leads to this situation.
Any insights or patch prosposals from Your side are of course welcome.

Best Regards,

--- UPDATE:

I commented out the line

#include /usr/local/etc/nginx/templates/php-catchall.tmpl;

in 

/usr/local/etc/ngnix/sites-available/00-default-ssl.conf

Roundcube seems to work still. I hope I didn't break anything, or expose any other PHP running, but I don't believe to have PMA or Adminer installed (will still double check).

5 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Online
  • Registered: 2009-05-06
  • Posts: 31,088

Re: iredadmin and roundcube (php-fpm) keep dying

lopez-iredmail wrote:

I hope I didn't break anything

Commenting out the php_catchall.tmpl is ok.

iRedMail configures few directories with php support, for example, /mail/ (Roundcube). But it's common that sysadmin may write some php scripts or host other PHP applications on same server, so we have php_catchall.tmpl enabled by default to serve any "*.php" files (of course the file must exist on your server under web server document root).

6 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Online
  • Registered: 2009-05-06
  • Posts: 31,088

Re: iredadmin and roundcube (php-fpm) keep dying

By the way, which Roundcube release are you running?

7 Reply by lopez-iredmail

  • lopez-iredmail
  • Member
  • Offline
  • Registered: 2014-06-24
  • Posts: 18

Re: iredadmin and roundcube (php-fpm) keep dying

ZhangHuangbin wrote:

By the way, which Roundcube release are you running?

Hi, I'm currently running roundcube-php74-1.4.2,1

The way I organize it, is - to keep packages up do date quicker with the current releases - I create a poudriere setting with all the port options (and make.conf) generated by iRedMail's packages_freebsd.sh, and have a daily build updating it.

However: The above setting didn't help much. It took a bit longer than usual (actually, around a day), but then php-fpm was down again, showing no process at all....

8 Reply by lopez-iredmail

  • lopez-iredmail
  • Member
  • Offline
  • Registered: 2014-06-24
  • Posts: 18

Re: iredadmin and roundcube (php-fpm) keep dying

ZhangHuangbin wrote:

By the way, which Roundcube release are you running?

Hi, I'm currently running roundcube-php74-1.4.2,1
However: The above setting didn't help much. It took a bit longer than usual (actually, around a day), but then php-fpm was down again, showing no process at all....

9 Reply by lopez-iredmail

  • lopez-iredmail
  • Member
  • Offline
  • Registered: 2014-06-24
  • Posts: 18

Re: iredadmin and roundcube (php-fpm) keep dying

Just to give an update. It still happens around once a day. I'm helping myself out with a cronjob, but that's really an ugly workaround. I wonder whether anyone else is experiencing this. I have several hostings running php-fpm (but with Apache), none of them has these stability issues.

10 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Online
  • Registered: 2009-05-06
  • Posts: 31,088

Re: iredadmin and roundcube (php-fpm) keep dying

Cannot help much without related error log. sad