1 Topic by PaulCahill (edited by PaulCahill 2020-04-04 05:01:51)

  • PaulCahill
  • PaulCahill
  • Member
  • Offline
  • From: Northern KY, USA
  • Registered: 2018-06-02
  • Posts: 49

Topic: Pro script - import_users.py errors adding users to groups

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release): 1.0 MARIADB edition
- Deployed with iRedMail Easy or the downloadable installer? downloaded
- Linux/BSD distribution name and version: Ubuntu 18.04 LTS
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): MySQL
- Web server (Apache or Nginx): nginx
- Manage mail accounts with iRedAdmin-Pro? Yes
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====
run:
python  import_users.py /opt/users.list

Error:
[OK] Created user: willb@domain.info
Traceback (most recent call last):
  File "import_users.py", line 206, in <module>
    'require_confirm': 'no'})
  File "import_users.py", line 113, in __put
    return r.json()
  File "/usr/lib/python2.7/dist-packages/requests/models.py", line 892, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

contents of /opt/user.list:
willb@domain.info, sdf4fsa323FAS@, 1024, Will Bark, sysdev@domain.info

I can add users no problem but adding them to groups causes a problem.

Also, is there similar python script for adding aliases to mailboxes? Or a way to do that during the account creation with import_users.py? I don't see anything mentioned.

***edit***
I found I can work around the maillist/group problem by using the tools in:
/opt/mlmmjadmin/tools
python maillist_admin.py create webtest@domain.com only_subscriber_can_post=no disable_archive=yes
python maillist_admin.py add_subscribers  webtest@domain.com paulc@domain.com jons@domain.com

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

PaulCahill's Website

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Pro script - import_users.py errors adding users to groups

PaulCahill wrote:

ValueError: No JSON object could be decoded

Is "mlmmjadmin" service running?

PaulCahill wrote:

Also, is there similar python script for adding aliases to mailboxes? Or a way to do that during the account creation with import_users.py?

FYI: https://docs.iredmail.org/user.alias.address.html

3 Reply by PaulCahill

  • PaulCahill
  • PaulCahill
  • Member
  • Offline
  • From: Northern KY, USA
  • Registered: 2018-06-02
  • Posts: 49

Re: Pro script - import_users.py errors adding users to groups

ZhangHuangbin wrote:
PaulCahill wrote:

ValueError: No JSON object could be decoded

Is "mlmmjadmin" service running?

PaulCahill wrote:

Also, is there similar python script for adding aliases to mailboxes? Or a way to do that during the account creation with import_users.py?

FYI: https://docs.iredmail.org/user.alias.address.html

mlmmjadmin is running:
mlmmj     1392     1  0 Apr02 ?        00:00:19 /usr/bin/uwsgi --ini /opt/mlmmjadmin/rc_scripts/uwsgi/debian.ini --pidfile /var/run/mlmmjadmin/mlmmjadmin.pid

PaulCahill's Website

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Pro script - import_users.py errors adding users to groups

PaulCahill wrote:

mlmmjadmin is running:

Still error while importing user with groups?

5 Reply by PaulCahill

  • PaulCahill
  • PaulCahill
  • Member
  • Offline
  • From: Northern KY, USA
  • Registered: 2018-06-02
  • Posts: 49

Re: Pro script - import_users.py errors adding users to groups

Still error while importing user with groups?

No it does not.

[OK] Created user: willb@domain.com
Traceback (most recent call last):
  File "import_users.py", line 206, in <module>
    'require_confirm': 'no'})
  File "import_users.py", line 113, in __put
    return r.json()
  File "/usr/lib/python2.7/dist-packages/requests/models.py", line 892, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

The user is added to the server, but not to the group. It also exits out and doesn't add any accounts after it errors on that user.

PaulCahill's Website

6 Reply by PaulCahill

  • PaulCahill
  • PaulCahill
  • Member
  • Offline
  • From: Northern KY, USA
  • Registered: 2018-06-02
  • Posts: 49

Re: Pro script - import_users.py errors adding users to groups

Would using a self-signed cert matter?

PaulCahill's Website

7 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Pro script - import_users.py errors adding users to groups

- Self-signed ssl cert doesn't matter in this case.
- Did you enable iRedAdmin-Pro RESTful API? FYI: https://docs.iredmail.org/iredadmin-pro … l.api.html

8 Reply by PaulCahill

  • PaulCahill
  • PaulCahill
  • Member
  • Offline
  • From: Northern KY, USA
  • Registered: 2018-06-02
  • Posts: 49

Re: Pro script - import_users.py errors adding users to groups

ZhangHuangbin wrote:

Did you enable iRedAdmin-Pro RESTful API? FYI: https://docs.iredmail.org/iredadmin-pro … l.api.html

Yes I edited settings.py and added:
ENABLE_RESTFUL_API = True
I've rebooted the box since this was added as well.  Also, I am able to use the script import_users.py to add accounts, just not add accounts AND add to the maillists.

PaulCahill's Website

9 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Pro script - import_users.py errors adding users to groups

Please try to import one more user with groups, i expect some log in mlmmjadmin log file /var/log/mlmmjadmin/mlmmjadmin.log or /var/log/syslog. Please paste related error log here for troubleshooting.

10 Reply by PaulCahill

  • PaulCahill
  • PaulCahill
  • Member
  • Offline
  • From: Northern KY, USA
  • Registered: 2018-06-02
  • Posts: 49

Re: Pro script - import_users.py errors adding users to groups

I know the api is working because I can pull this up in a browser:
https://server.domain.com/iredadmin/api/mls/domain.com
gets me all the relevant info on the mailing lists.

I can run the import_user with just a user:
jons@domain.com, adg4fsa323XAS@, 1024, Jon John,

If I run with just one (existing) mailling list and a new user, we get the errors:
willb@domain.com, sdf4fsXc4FAS@, 1024, Will Bauer, sysdev@domain.com

python  import_users.py /opt/users.list
[OK] Created user: willb@domain.com
Traceback (most recent call last):
  File "import_users.py", line 206, in <module>
    'require_confirm': 'no'})
  File "import_users.py", line 113, in __put
    return r.json()
  File "/usr/lib/python2.7/dist-packages/requests/models.py", line 892, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

It DOES create the user, but fails to add them to the group. Output from /var/log/syslog:
Apr 15 14:16:58 servernamehere iredadmin: [69.24.30.122] POST /iredadmin/api/login 200 170 "-"
Apr 15 14:16:58 servernamehere iredadmin: [69.24.30.122] POST /iredadmin/api/user/willb@domain.com 200 170 "-"
Apr 15 14:16:58 servernamehere iredadmin: [69.24.30.122] PUT /iredadmin/api/ml/group 404 161 "-"

The file  /var/log/mlmmjadmin/mlmmjadmin.log is zero bytes after running the script:
-rw-r----- 1 syslog adm 0 Apr 14 15:03 /var/log/mlmmjadmin/mlmmjadmin.log

So am I using the wrong URL for the tool?
One thing I noticed - in the import_users.py one of the variables I have to change is the api_endpoint. The 'default' is
http://u18:8080/api
I had to take the 8080 off the URL when I added my server:
https://server.domain/iredadmin/api/
The server isn't listening on 8080 for anything.
I notice on here:
https://docs.iredmail.org/network.ports.html
7790 is listed as the API port. Should I be using that in the import_users.py?

Thanks for your help! I bought a copy of the LDAP pro version so I could compare it with the MySQL one side by side.

PaulCahill's Website

11 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Pro script - import_users.py errors adding users to groups

Confirmed it's a bug of tools/import_users.py. Here's patch to fix it:

diff --git a/tools/import_users.py b/tools/import_users.py
index 42a08960..b72bb9ac 100644
--- a/tools/import_users.py
+++ b/tools/import_users.py
@@ -171,7 +171,7 @@ for line in f.readlines():
     # Get mail address(es) of assigned mailing list(s)
     groups = d.pop('groups')
     groups.lower()
-    groups = [addr.lower() for addr in groups.split(':') if iredutils.is_email(addr)]
+    groups = [addr.lower().strip() for addr in groups.split(':') if iredutils.is_email(addr)]
 
     # Create user
     res = __post('/user/' + mail,
@@ -199,11 +199,11 @@ for line in f.readlines():
 
     if groups:
         for group in groups:
-            res = __put('/ml/group',
+            res = __put('/ml/' + group,
                         data={'add_subscribers': mail,
                               'require_confirm': 'no'})
 
             if res['_success']:
                 print("Subscribed user to mailing list: {} -> {}".format(mail, group))
             else:
-                print('<<< ERROR >>> failed to subscribe user to mailing list: {} -> {}, error: '.format(mail, group, res))
+                print('<<< ERROR >>> failed to subscribe user to mailing list: {} -> {}, error: {}'.format(mail, group, res))

12 Reply by PaulCahill

  • PaulCahill
  • PaulCahill
  • Member
  • Offline
  • From: Northern KY, USA
  • Registered: 2018-06-02
  • Posts: 49

Re: Pro script - import_users.py errors adding users to groups

Great - works like a charm. Not sure if it matters to you, but if you specify more than one mail list group, but separate them with a ; instead of a :, the script doesn't complain and doesn't add them to any group. Thanks for the help!

PaulCahill's Website

13 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Pro script - import_users.py errors adding users to groups

PaulCahill wrote:

Not sure if it matters to you, but if you specify more than one mail list group, but separate them with a ; instead of a :, the script doesn't complain and doesn't add them to any group.

You have to follow the rule, If the document says "multiple addresses must be separated by ":"", then just use ":" and not ";" or "," or any other separator you prefer. sad

14 Reply by PaulCahill

  • PaulCahill
  • PaulCahill
  • Member
  • Offline
  • From: Northern KY, USA
  • Registered: 2018-06-02
  • Posts: 49

Re: Pro script - import_users.py errors adding users to groups

Oh I get it - I had a typo, wasn't trying to break the rules smile

PaulCahill's Website