1 Topic by Finch (edited by Finch 2013-05-22 16:34:21)

  • Finch
  • Member
  • Offline
  • Registered: 2012-04-16
  • Posts: 8

Topic: LDAP backup issues: slapcat and 80 columns.

======== Required information ====
- iRedMail version: 0.8.4 and 0.8.4
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): LDAP
- Linux/BSD distribution name and version: Scientific Linux 6.2 and CentOS 6.4, both x86_64.
====

Hi,

I am attempting to migrate our email hosting from one server to another, and I'm having a problem with the LDIF files created by the backup_openldap.sh script. I am using the version from 2012-09-05 from BitBucket. As part of the move, I have to edit the LDIF file to reflect the new server details. This can be done with find/replace - easy!

However, I am encountering a problem with the format of the original LDIF file.

For example, if we have a line that should read:

dn: ou=Users,domainName=mydomain.com.xx,o=domains,dc=serverhostname,dc=serverdomain,dc=xx

It will be split over two lines, as follows:

dn: ou=Users,domainName=mydomain.com.xx,o=domains,dc=serverhostname,dc=serverdomain,dc
=xx

A line such as:

homeDirectory: /var/vmail/vmail1/mydomain.com.xx/f/i/n/finch-2012.05.30.16.49.44/

Will appear as:

homeDirectory: /var/vmail/vmail1/mydomain.com.xx/f/i/n/finch-2012.05.30.
16.49.44/

Some investigation has shown me that this is due to the line being 80 columns, and a new line being formed for the 81st. This is true for the .ldiff on the server, as well as those copied via SFTP.

Slapdd does not like this, and fails when restoring from the ldif file.

Short of manually editing all 9000 lines of the document, does anybody have any tips as to how fix this problem? I'm not a programmer (at all) but I'm willing to bet there's a single line of Perl or something that will make the lines whole again.

Thanks!

-Finch.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,090

Re: LDAP backup issues: slapcat and 80 columns.

Hi Finch,

Long line in exported LDIF file will be split into two or several lines, the second line will start with space, which means it's part of above line. It should be ok.

*) Could you show us part of the original LDIF data you exported? Of course you must replace sensitive info in it, but don't modify the format (e.g. remove spaces).

*) Please show us how you export and import LDIF data, with detailed commands you used.

*) Show us the error message while importing LDIF file.

3 Reply by Finch

  • Finch
  • Member
  • Offline
  • Registered: 2012-04-16
  • Posts: 8

Re: LDAP backup issues: slapcat and 80 columns.

Hi,

Thanks for the help. The problem is now resolved.

Here's how I did it:

After editing (via find/replace) the ldif to reflect my new hostname (old: "dc=hostname1,dc=domain1,dc=net" to new: "dc=hostname2,dc=domain1,dc=net") and the dc: and o: lines to relect the hostname, here we go.

[21:59 root@hostname2 /var/lib/ldap/hostname2.domain1.net] slapadd -f /etc/openldap/slapd.conf -l /home/finch/2013.05.22.21.23.09.ldif
bdb_monitor_db_open: monitoring disabled; configure monitor database to enable
slapadd: line 372: database #1 (dc=hostname2,dc=domain1,dc=net) not configured to hold "ou=Users,domainName=virtualdomain.com.xx,o=domains,dc=hostname2,dc=domain1,dc=net"; no database configured for that naming context
_###                   15.40% eta    02s elapsed            none spd  43.1 k/s
Closing DB...

And here's line 372:

dn: ou=Users,domainName=virtualdomain.com.xx,o=domains,dc=hostname2,dc=domain1,dc=n
 et

I changed this to:

dn: ou=Users,domainName=virtualdomain.com.xx,o=domains,dc=hostname2,dc=domain1,dc=net

And the import fell apart at the next one. Strange... but it makes sense, because I am moving from hostname1.domain1.net to hostname2.domain1.net. In reality, hostname1 is mail, and hostname2 is an extra character long - so let's call it hostname22. So each line I edit isn't 80 columns, but 81 (or more).

But interestingly, rows over 80 columns are imported just fine.

The solution was to leave the directories and passwords alone, and edit only the dn: lines to be complete rather than to be an 80 column maximum.

This done,

[22:41 root@hostname2 /var/lib/ldap/hostname2.domain1.net] slapadd -f /etc/openldap/slapd.conf -l /home/finch/2013.05.22.21.23.09.ldif
bdb_monitor_db_open: monitoring disabled; configure monitor database to enable
.#################### 100.00% eta   none elapsed                 spd  86.3 k/s
Closing DB...

Success!

After a chown and an LDAP restart, we're back in business. It appears as though mail directories and passwords were successfully imported, and so were the longer dn: lines.

This does raise an issue, though: how do other people move from one server to another without going through something similar? This 80 column thing is fairly... obsolete, surely. I know it's not your software, though.

Thank you for your prod in the right direction.

Best regards (and as always, thanks for the great product!),

--Finch.

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,090

Re: LDAP backup issues: slapcat and 80 columns.

Finch wrote:

After editing (via find/replace) the ldif to reflect my new hostname (old: "dc=hostname1,dc=domain1,dc=net" to new: "dc=hostname2,dc=domain1,dc=net") and the dc: and o: lines to relect the hostname, here we go.
...
This does raise an issue, though: how do other people move from one server to another without going through something similar? This 80 column thing is fairly... obsolete, surely. I know it's not your software, though.

The LDAP root dn (dc=xx,dc=xx) doesn't matter at all, you can use the same root dn on another server. Or, you can use dc=example,dc=com if you like. Nothing matters (except you want to use your own domain name (or whatever) as root dn).

iRedMail uses tree under o=domains,dc=xx,dc=xx, if you changed the root dn, you have to update related config files which has ldap root dn hard-coded to use new root dn.