1

Topic: Batch create users from txt file

==== Required information ====
- iRedMail version: 0.8.7 (Pro 1.8.2)
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Linux/BSD distribution name and version: Debian Wheezy
- Related log if you're reporting an issue:
====

Hi, we are planning to batch-createsome users in to our mail-server, the first domain consists of 196 users, the I have a txt file with all the accounts (mail-address and password), TAB-separated. I have seen the script to create users in a batch, but we find it a bit tricky to see how we are going to do this. Any tip?

The TXT-file look like this:
user@mydomain.com     Dj(pZ3054
etc.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2

Re: Batch create users from txt file

iRedMail ships a shell script for bulk adding mail users: iRedMail-0.8.7/tools/create_mail_user_SQL.sh.
If you have password as second field in each line of this text file, please update 'tools/create_mail_user_SQL.sh' with below steps:

1: Find below lines in 'tools/create_mail_user_SQL.sh':

DEFAULT_PASSWD='88888888'                                                       
USE_DEFAULT_PASSWD='NO'

echo '' > ${SQL}

2: Change them to:

DEFAULT_PASSWD="$3"   # WARNING: Must use double quote ("), not single quote (').
USE_DEFAULT_PASSWD='YES'

echo '' >> ${SQL}     # Two '>'

3: Remove '@domain.ltd' in your TXT file. For example:

- Original line: "user@mydomain.com     Dj(pZ3054"
- After modified: "user     Dj(pZ3054"

4: Execute this script with your txt file:

# cd iRedMail-0.8.7/tools/
# bash create_mail_user_SQL.sh YOUR_DOMAIN_NAME $(awk '{print $1,$2}' YOUR_TXT_FILE)

It will generate a new file 'output.sql' under current directory, please verify it first, if it's ok then import it into SQL server. NOTE: you need to create domain first manually.

3 (edited by Energia 2014-05-20 15:36:46)

Re: Batch create users from txt file

Nice, it is working partly, but it seems like the script fetch the password column as a user name and generate this output when the first line of the input txt-file looks like this: ole.olsen       It)mW7577



INSERT INTO mailbox (username, password, name, storagebasedirectory,storagenode, maildir, quota, domain, active, local_part, created)
    VALUES ('ole.olsen@mydomain.com', '$1$ROn.Rtdf$cf9KF..EAeeCWCUO/mcKx0', ole.olsen', '/var','vmail', 'mydomain.com/f/o/l/e/ole.olsen-2014.05.20.08.42.23/', '100', $
INSERT INTO alias (address, goto, domain, created, active) VALUES ('ole.olsen@mydomain.com', 'ole.olsen@mydomain.com','mydomain.com', NOW(), 1);
INSERT INTO mailbox (username, password, name, storagebasedirectory,storagenode, maildir, quota, domain, active, local_part, created)

    VALUES ('It)mW7577^M@mydomain.com', '$1$ROn.Rtdf$cf9KF..EAeeCWCUO/mcKx0', 'It)mW7577^M', '/var','vmail', 'mydomain.com/I/t/)/It)mW7577^M-2014.05.20.08.42.23/', '100', 'nesse$
INSERT INTO alias (address, goto, domain, created, active) VALUES ('It)mW7577^M@myadomain.com', 'It)mW7577^M@mydomain.com','mydomain.com', NOW(), 1);
INSERT INTO mailbox (username, password, name, storagebasedirectory,storagenode, maildir, quota, domain, active, local_part, created)

(I first changed the storage_base_directory to /var/vmail, but changed it back again afterwards)

ZhangHuangbin wrote:

iRedMail ships a shell script for bulk adding mail users: iRedMail-0.8.7/tools/create_mail_user_SQL.sh.
If you have password as second field in each line of this text file, please update 'tools/create_mail_user_SQL.sh' with below steps:

1: Find below lines in 'tools/create_mail_user_SQL.sh':

DEFAULT_PASSWD='88888888'                                                       
USE_DEFAULT_PASSWD='NO'

echo '' > ${SQL}

2: Change them to:

DEFAULT_PASSWD="$3"   # WARNING: Must use double quote ("), not single quote (').
USE_DEFAULT_PASSWD='YES'

echo '' >> ${SQL}     # Two '>'

3: Remove '@domain.ltd' in your TXT file. For example:

- Original line: "user@mydomain.com     Dj(pZ3054"
- After modified: "user     Dj(pZ3054"

4: Execute this script with your txt file:

# cd iRedMail-0.8.7/tools/
# bash create_mail_user_SQL.sh YOUR_DOMAIN_NAME $(awk '{print $1,$2}' YOUR_TXT_FILE)

It will generate a new file 'output.sql' under current directory, please verify it first, if it's ok then import it into SQL server. NOTE: you need to create domain first manually.

4

Re: Batch create users from txt file

My mistake. Please modify below lines based on the one you already modified.

1: find below lines inside function 'generate_sql()':

generate_sql()
{
    DOMAIN="$1"
    shift 1

    for i in $@; do
        username="$i"

        [...snip...]

    done
}

2: Update them to below one:

generate_sql()                                                                     
{                                                                                  
    # Get domain name.                                                             
    DOMAIN="$1"                                                                                                    
    username="$2"                                                                  
                                                                                   
    #for i in $@; do                                                               
    #    username="$i"

    [...snip...]

    #done
}

Then execute command again.

5

Re: Batch create users from txt file

FYI: Below is the complete patch for the original file shipped in iRedMail-0.8.7:

diff -r 69e7860e9119 iRedMail/tools/create_mail_user_SQL.sh
--- a/iRedMail/tools/create_mail_user_SQL.sh    Mon May 19 08:09:39 2014 +0800
+++ b/iRedMail/tools/create_mail_user_SQL.sh    Tue May 20 23:54:26 2014 +0800
@@ -40,8 +40,8 @@
 
 # Password setting.
 # Note: password will be crypted in MD5.
-DEFAULT_PASSWD='88888888'
-USE_DEFAULT_PASSWD='NO'     # If set to 'NO', password is the same as username.
+DEFAULT_PASSWD="$3"
+USE_DEFAULT_PASSWD='YES'     # If set to 'NO', password is the same as username.
 
 # Default mail quota.
 DEFAULT_QUOTA='100'   # 100 -> 100M
@@ -69,7 +69,7 @@
 
 # Path to SQL template file.
 SQL="output.sql"
-echo '' > ${SQL}
+echo '' >> ${SQL}
 
 # Cyrpt the password.
 if [ X"${USE_DEFAULT_PASSWD}" == X"YES" ]; then
@@ -82,10 +82,10 @@
 {
     # Get domain name.
     DOMAIN="$1"
-    shift 1
+    username="$2"
 
-    for i in $@; do
-        username="$i"
+    #for i in $@; do
+    #    username="$i"
         mail="${username}@${DOMAIN}"
 
         if [ X"${USE_DEFAULT_PASSWD}" != X"YES" ]; then
@@ -131,7 +131,7 @@
     VALUES ('${mail}', '${CRYPT_PASSWD}', '${username}', '${STORAGE_BASE}','${STORAGE_NODE}', '${maildir}', '${DEFAULT_QUOTA}', '${DOMAIN}', '1','${username}', NOW());
 INSERT INTO alias (address, goto, domain, created, active) VALUES ('${mail}', '${mail}','${DOMAIN}', NOW(), 1);
 EOF
-    done
+    #done
 }
 
 if [ $# -lt 2 ]; then

6

Re: Batch create users from txt file

Hm, after I did the last changes, it only process the first line in my user/password file.

ZhangHuangbin wrote:

FYI: Below is the complete patch for the original file shipped in iRedMail-0.8.7:

diff -r 69e7860e9119 iRedMail/tools/create_mail_user_SQL.sh
--- a/iRedMail/tools/create_mail_user_SQL.sh    Mon May 19 08:09:39 2014 +0800
+++ b/iRedMail/tools/create_mail_user_SQL.sh    Tue May 20 23:54:26 2014 +0800
@@ -40,8 +40,8 @@
 
 # Password setting.
 # Note: password will be crypted in MD5.
-DEFAULT_PASSWD='88888888'
-USE_DEFAULT_PASSWD='NO'     # If set to 'NO', password is the same as username.
+DEFAULT_PASSWD="$3"
+USE_DEFAULT_PASSWD='YES'     # If set to 'NO', password is the same as username.
 
 # Default mail quota.
 DEFAULT_QUOTA='100'   # 100 -> 100M
@@ -69,7 +69,7 @@
 
 # Path to SQL template file.
 SQL="output.sql"
-echo '' > ${SQL}
+echo '' >> ${SQL}
 
 # Cyrpt the password.
 if [ X"${USE_DEFAULT_PASSWD}" == X"YES" ]; then
@@ -82,10 +82,10 @@
 {
     # Get domain name.
     DOMAIN="$1"
-    shift 1
+    username="$2"
 
-    for i in $@; do
-        username="$i"
+    #for i in $@; do
+    #    username="$i"
         mail="${username}@${DOMAIN}"
 
         if [ X"${USE_DEFAULT_PASSWD}" != X"YES" ]; then
@@ -131,7 +131,7 @@
     VALUES ('${mail}', '${CRYPT_PASSWD}', '${username}', '${STORAGE_BASE}','${STORAGE_NODE}', '${maildir}', '${DEFAULT_QUOTA}', '${DOMAIN}', '1','${username}', NOW());
 INSERT INTO alias (address, goto, domain, created, active) VALUES ('${mail}', '${mail}','${DOMAIN}', NOW(), 1);
 EOF
-    done
+    #done
 }
 
 if [ $# -lt 2 ]; then

7

Re: Batch create users from txt file

OK, try this shell command instead:

while read line; do user=$(echo $line | awk '{print $1}'); pass=$(echo $line | awk '{print $2}'); bash create_mail_user_SQL.sh YOUR_DOMAIN_NAME $user $pass; done < YOUR_TXT_FILE

8

Re: Batch create users from txt file

This command gave me the same result as the first try: the script generates a user based on the password.

ZhangHuangbin wrote:

OK, try this shell command instead:

while read line; do user=$(echo $line | awk '{print $1}'); pass=$(echo $line | awk '{print $2}'); bash create_mail_user_SQL.sh YOUR_DOMAIN_NAME $user $pass; done < YOUR_TXT_FILE

9

Re: Batch create users from txt file

Are you sure? It works for me here.

10

Re: Batch create users from txt file

ZhangHuangbin wrote:

Are you sure? It works for me here.

What changes do I need to do in the script before the shell command?

Only this one?
"
DEFAULT_PASSWD="$3"   # WARNING: Must use double quote ("), not single quote (').
USE_DEFAULT_PASSWD='YES'

echo '' >> ${SQL}     # Two '>'
"

11

Re: Batch create users from txt file

This is the patch for the original file shipped in iRedMail-0.8.7: tools/create_mail_user_SQL.sh.
http://www.iredmail.org/forum/post30299.html#p30299

12

Re: Batch create users from txt file

I got it working now, I got the output.sql and imported it, I`m now able to see the users in iRedMail admin, but the users are not able to log in through Roundcube, from /var/log/mail.log I get:
May 24 20:13:01 post roundcube: IMAP Error: Login failed for firstname.lastname@mydomain.com from MY.IP.ADD.RESS. LOGIN: Authentication failed. in /usr/share/apache2/roundcubemail-1.0.1/program/lib/Roundcube/rcube_imap.php on line 184 (POST /mail/?_task=login?_task=login&_action=login)

Could it be something with the hashing of the password?

13

Re: Batch create users from txt file

Could you please turn on debug mode in Dovecot, then try to login to Roundcube, track Dovecot log file (/var/log/dovecot.log) and paste related log here?

Search "debug" on iRedMail FAQ page will show you link of tutorial to turn on debug mode in Dovecot:
http://www.iredmail.org/faq.html

14 (edited by Energia 2014-05-25 20:12:43)

Re: Batch create users from txt file

It seems to be a password mismatch, I have duble checked that I have the correct password (a bit strange since the hashing seems to be the same?)

From my TXT input file used to generate password: firstname.lastname    Ks*eJ7610
From the output.sql:  VALUES ('firstname.lastname@mydomain.com', '$1$a76EIy2o$cTIB.SDOWWaOeSjky1ZlZ.', 'firstname.lastname',

From the dovecot.log:
May 25 13:43:27 auth-worker(13902): Debug: sql(firstname.lastname@mydomain.com,127.0.0.1): query: SELECT password FROM mailbox WHERE username='firstname.lastname@mydomain.com' AND active='1'
May 25 13:43:27 auth-worker(13902): Info: sql(firstname.lastname@mydomain.com,127.0.0.1): Password mismatch (given password: Ks*eJ7610)
May 25 13:43:27 auth-worker(13902): Debug: sql(firstname.lastname@mydomain.com,127.0.0.1): CRYPT(Ks*eJ7610) != '$1$a76EIy2o$cTIB.SDOWWaOeSjky1ZlZ.'
May 25 13:43:29 auth: Debug: client out: FAIL    1    user=firstname.lastname@mydomain.com
May 25 13:43:29 imap-login: Info: Disconnected (auth failed, 1 attempts in 2 secs): user=<firstname.lastname@mydomain.com>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, session=<4pKb+Df6RwB/AAAB>
May 25 13:44:40 imap-login: Info: Disconnected (disconnected before greeting, waited 0 secs): user=<>, rip=XX.XX.XX.66, lip=192.168.18.20, session=<mQnv/Df6sQBYVD5C>
May 25 13:44:40 auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth
May 25 13:44:40 auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so
May 25 13:44:40 auth: Debug: passwd-file /etc/dovecot/dovecot-master-users-password: Read 0 users in 0 secs
May 25 13:44:40 pop3-login: Info: Disconnected (no auth attempts in 0 secs): user=<>, rip=XX.XX.XX.66, lip=192.168.18.20, session=<ypDv/Df6sgBYVD5C>
May 25 13:44:40 auth: Debug: auth client connected (pid=13919)

15

Re: Batch create users from txt file

Energia wrote:

May 25 13:43:27 auth-worker(13902): Debug: sql(firstname.lastname@mydomain.com,127.0.0.1): CRYPT(Ks*eJ7610) != '$1$a76EIy2o$cTIB.SDOWWaOeSjky1ZlZ.'

Dovecot tries to use 'CRYPT' password hash.
Does it work if you prepend '{MD5-CRYPT}' in generated password hash? For example:

- Old one: $1$a76EIy2o$cTIB.SDOWWaOeSjky1ZlZ.
- New one: {MD5-CRYPT}$1$a76EIy2o$cTIB.SDOWWaOeSjky1ZlZ.

16

Re: Batch create users from txt file

Where do I change that?

17

Re: Batch create users from txt file

Energia wrote:

Where do I change that?

Change the password stored in SQL. (e.g. change it with SQL 'UPDATE' command)

18

Re: Batch create users from txt file

I have added {MD5-CRYPT} in front of the user password via phpMyAdmin , but I am still not able to login:

Jun 01 13:28:34 auth-worker(6161): Debug: sql(firstname.lastname@mydomain.com,127.0.0.1): query: SELECT password FROM mailbox WHERE username='firstname.lastname@mydomain.com' AND active='1'
Jun 01 13:28:34 auth-worker(6161): Info: sql(firstname.lastname@mydomain.com,127.0.0.1): Password mismatch (given password: Ks*eJ7610)
Jun 01 13:28:34 auth-worker(6161): Debug: sql(firstname.lastname@mydomain.com,127.0.0.1): MD5-CRYPT(Ks*eJ7610) != '$1$a76EIy2o$cTIB.SDOWWaOeSjky1ZlZ.'
Jun 01 13:28:37 auth: Debug: client out: FAIL   1       user=firstname.lastname@mydomain.com
Jun 01 13:28:37 imap-login: Info: Disconnected (auth failed, 1 attempts in 3 secs): user=<firstname.lastname@mydomain.com>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, session=<cdxFlMT61AB/AAAB>

19

Re: Batch create users from txt file

No idea yet.
According to Dovecot wiki site, it should work as expected: http://wiki2.dovecot.org/Authentication/PasswordSchemes

20

Re: Batch create users from txt file

Is it possible to use the "CRYPT" hash when generating users instead of "MD5-CRYPT" (to prevente the mismatch)? The server is a VM so I am able to roll-back and import the users again...

21

Re: Batch create users from txt file

Energia wrote:

Is it possible to use the "CRYPT" hash when generating users instead of "MD5-CRYPT" (to prevente the mismatch)? The server is a VM so I am able to roll-back and import the users again...

Why not give i a try?

22

Re: Batch create users from txt file

Nice, it works when I change "openssl passwd -1" to "openssl passwd -crypt" in the script