1 Topic by marcelomartinatti

  • marcelomartinatti
  • Member
  • Offline
  • Registered: 2013-06-27
  • Posts: 30

Topic: Operation and integration of cluebringer and IredAPD

==== Required information ====
- iRedMail version: 0.9.0
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Linux/BSD distribution name and version: Debian 7
- Related log if you're reporting an issue:
====

Hi Zhang,

I switched a mail server (hardware), updating the Debian (6/squeeze for 7/wheezy) and has also updated the version of iRedMail and iRedAdmin-Pro-Msqyl.

All settings and migration vmail and roundcube databases and also the mailboxes with no problems.

However, in old version the postfix-policy was 1.82 and the new is cluebringer and I'm having trouble with it.

I started having various problems and complaints from almost all external emails being returned in greylist function, including Gmail, Hotmail, MSN, etc:

===
Technical details of permanent failure:
Google tried to deliver your message, but it was rejected by the server for the recipient domain xxx.com by xxx.xxx.com. [xxx.xxx.xxx.xxx].

The error that the other server returned was:
554 5.7.1 <xxx@xxx.xxx>: Recipient address rejected: Greylisting in effect, sending server blacklisted
===

As complaints began to increase much and I removed "check_policy_service inet:127.0.0.1:10031" on smtpd_recipient_restrictions in main.cf configuration of the Posfix and also I disabled the "smtpd_end_of_data_restrictions = check_policy_service inet:127.0.0.1:10031".


Then, external mails started coming normally.

But I want (and need) to use the Cluberginger but it is denying everything by default. How I  adjust it?

Another question is about the address/domains or ips configured at whitelists/blacklists in old iRedAdmin. The information were stored in tables blacklist_dnsname, blacklist_helo, blacklist_sender, whitelist_dnsname, whitelist_sender of  postfixpolicyd database. I set up some address that were on the old server but can not find where tables/database these are stored. Were are? There are checked by the system even whith cluebringer disable?

Another problem  were errors that began to emerge with the iRedPad:

13/01/2015 09:20:38 ERROR Error while creating database connection: (1040, 'Too many connections')

To restore the mail service I also disable "check_policy_service inet: 127.0.0.1: 7777" on smtpd_recipient_restrictions. Where can I correct or increase this setting?

Tks, I await your help.

M Martinatti

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Operation and integration of cluebringer and IredAPD

marcelomartinatti wrote:

But I want (and need) to use the Cluberginger but it is denying everything by default. How I  adjust it?

You can enable greylisting training mode for some time (e.g. 1 or 2 weeks), then disable training mode and enable greylisting directly.

During training mode, Cluebringer will gather some information to understand which senders should be bypassed.
Reference: http://wiki.policyd.org/greylisting

Note: training mode is supported in Cluebringer r378+ (revision 378) and v2.1.x.

marcelomartinatti wrote:

13/01/2015 09:20:38 ERROR Error while creating database connection: (1040, 'Too many connections')
To restore the mail service I also disable "check_policy_service inet: 127.0.0.1: 7777" on smtpd_recipient_restrictions. Where can I correct or increase this setting?

Are you sure this is caused by iRedAPD? If so, we have to improve iRedAPD to use SQL connection pool instead of direct connection.

Currently, you can try to increase max connections in MySQL server to solve this issue. Check /etc/mysql/my.cnf.

3 Reply by marcelomartinatti

  • marcelomartinatti
  • Member
  • Offline
  • Registered: 2013-06-27
  • Posts: 30

Re: Operation and integration of cluebringer and IredAPD

Hi Zhang, Tks!

ZhangHuangbin wrote:

Are you sure this is caused by iRedAPD? If so, we have to improve iRedAPD to use SQL connection pool instead of direct connection.

Currently, you can try to increase max connections in MySQL server to solve this issue. Check /etc/mysql/my.cnf.

I understante whith yes, see the maillog :

Jan 13 09:20:43 smtp02 postfix/smtpd[53048]: connect from unknown[xxx.xxx.xx.xxx]
Jan 13 09:20:43 smtp02 postfix/smtpd[51466]: warning: connect to 127.0.0.1:7777: Connection refused
Jan 13 09:20:43 smtp02 postfix/smtpd[51466]: warning: problem talking to server 127.0.0.1:7777: Connection refused
Jan 13 09:20:43 smtp02 postfix/smtpd[51406]: lost connection after EHLO from unknown[xxx.xxx.xx.xxx]
Jan 13 09:20:43 smtp02 postfix/smtpd[51406]: disconnect from unknown[xxx.xxx.xx.xxx]
Jan 13 09:20:43 smtp02 postfix/smtpd[51912]: warning: connect to 127.0.0.1:7777: Connection refused
Jan 13 09:20:43 smtp02 postfix/smtpd[51912]: warning: problem talking to server 127.0.0.1:7777: Connection refused
Jan 13 09:20:43 smtp02 postfix/smtpd[53890]: lost connection after EHLO from [xxx.xxx.xx.xxx]
Jan 13 09:20:43 smtp02 postfix/smtpd[53890]: disconnect from [xxx.xxx.xx.xxx]
Jan 13 09:20:43 smtp02 postfix/smtpd[50152]: warning: connect to 127.0.0.1:7777: Connection refused
Jan 13 09:20:43 smtp02 postfix/smtpd[50152]: warning: problem talking to server 127.0.0.1:7777: Connection refused

And iRedPAD log show:

2015-01-13 09:20:37 ERROR Error while creating database connection: (1040, 'Too many connections')
2015-01-13 09:20:37 ERROR Error while creating database connection: (1040, 'Too many connections')
2015-01-13 09:20:37 ERROR Error while creating database connection: (1040, 'Too many connections')
2015-01-13 09:20:37 ERROR Error while creating database connection: (1040, 'Too many connections')

But, is possible to increase mysql max connections solve.

Please, tell me about it:

marcelomartinatti wrote:

Another question is about the address/domains or ips configured at whitelists/blacklists in old iRedAdmin. The information were stored in tables blacklist_dnsname, blacklist_helo, blacklist_sender, whitelist_dnsname, whitelist_sender of  postfixpolicyd database. I set up some address that were on the old server but can not find where tables/database these are stored. Were are? There are checked by the system even whith cluebringer disable?

My question is whether what is set directly on iredadmin is also managed by cluebringer and where they are stored this information in the MySQL.

Tks

M Martinatti

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Operation and integration of cluebringer and IredAPD

marcelomartinatti wrote:

Another question is about the address/domains or ips configured at whitelists/blacklists in old iRedAdmin. The information were stored in tables blacklist_dnsname, blacklist_helo, blacklist_sender, whitelist_dnsname, whitelist_sender of  postfixpolicyd database. I set up some address that were on the old server but can not find where tables/database these are stored. Were are? There are checked by the system even whith cluebringer disable?

In the latest iRedAdmin-Pro, whitelist/blacklists are stored in Amavisd database (3 tables: mailaddr, users, wblist). This way, we can use iRedAPD (with plugin 'amavisd_wblist') to reject blacklisted senders or bypass whitelisted senders during smtp session, or, if you don't want to reject them during smtp session, Amavisd can still read white/blacklists and tag emails sent from blacklisted sender with high spam score, this way, you have these emails marked as SPAM, or quarantined into SQL server directly.

The latest iRedAdmin-Pro ships script "tools/migrate_cluebringer_wblist_to_amavisd.py" to migrate white/blacklists stored in Cluebringer to Amavisd database, and optionally delete them from Cluebringer database after migrated (it will ask for your confirm).

5 Reply by marcelomartinatti

  • marcelomartinatti
  • Member
  • Offline
  • Registered: 2013-06-27
  • Posts: 30

Re: Operation and integration of cluebringer and IredAPD

ZhangHuangbin wrote:
marcelomartinatti wrote:

Another question is about the address/domains or ips configured at whitelists/blacklists in old iRedAdmin. The information were stored in tables blacklist_dnsname, blacklist_helo, blacklist_sender, whitelist_dnsname, whitelist_sender of  postfixpolicyd database. I set up some address that were on the old server but can not find where tables/database these are stored. Were are? There are checked by the system even whith cluebringer disable?

In the latest iRedAdmin-Pro, whitelist/blacklists are stored in Amavisd database (3 tables: mailaddr, users, wblist). This way, we can use iRedAPD (with plugin 'amavisd_wblist') to reject blacklisted senders or bypass whitelisted senders during smtp session, or, if you don't want to reject them during smtp session, Amavisd can still read white/blacklists and tag emails sent from blacklisted sender with high spam score, this way, you have these emails marked as SPAM, or quarantined into SQL server directly.

The latest iRedAdmin-Pro ships script "tools/migrate_cluebringer_wblist_to_amavisd.py" to migrate white/blacklists stored in Cluebringer to Amavisd database, and optionally delete them from Cluebringer database after migrated (it will ask for your confirm).

So even if you do not use the Cluebringer the policies set on IredAdmin-Pro (whitelists/blacklits) will work normally? Can I only work with the IredAPD without Cluberginger?

Another question is whether the lists (whitelists / blacklists) set by the global administrator on iRedAdmin-Pro has priority over user lists, that is, will be applied to all accounts.

Very tks.

M Martinatti

6 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Operation and integration of cluebringer and IredAPD

marcelomartinatti wrote:

So even if you do not use the Cluebringer the policies set on IredAdmin-Pro (whitelists/blacklits) will work normally?

We just moved whitelist/blacklists from Cluebringer to Amavisd, other features provided by Cluebringer still working, for example, throttling.

marcelomartinatti wrote:

Can I only work with the IredAPD without Cluberginger?

iRedAPD doesn't have the same features as Cluebringer, so you can use only iRedAPD.

marcelomartinatti wrote:

Another question is whether the lists (whitelists / blacklists) set by the global administrator on iRedAdmin-Pro has priority over user lists, that is, will be applied to all accounts.

Per-user wblist has the highest priority, then per-domain wblist, and global wblist has the lowest priority.