1 Topic by taco

  • taco
  • Member
  • Offline
  • Registered: 2013-11-02
  • Posts: 13

Topic: Quarantine mail body not displaying correctly in v2.0

==== Required information ====
- iRedMail version: 0.8.5
- iRedAdmin-Pro: 2.0
- Store mail accounts in which backend: LDAP
- Linux/BSD distribution name and version: CentOS 6.4
- Related log if you're reporting an issue: no log available
====

Displaying the mail body of a quarantined message is not working, it shows a garbled up page but the mail body is not displayed.
I can provide a screenshot if this is required.

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Quarantine mail body not displaying correctly in v2.0

Please post a screenshot, it can help me understand this issue.

3 Reply by taco

  • taco
  • Member
  • Offline
  • Registered: 2013-11-02
  • Posts: 13

Re: Quarantine mail body not displaying correctly in v2.0

Hi,

Please see screenshot attached, this screenshot is taken after clicking the expand button next to a quarantined message.

Post's attachments

Quarantined Mails.png 141.77 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

4 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Quarantine mail body not displaying correctly in v2.0

Does the error message "invalid request data" shows after you clicked the expand button? This error message means you submitted a invalid mail_id (not exists in SQL database - deleted or not stored in SQL at all).

5 Reply by taco

  • taco
  • Member
  • Offline
  • Registered: 2013-11-02
  • Posts: 13

Re: Quarantine mail body not displaying correctly in v2.0

Below is the html source of a spam message from the quarantine page (email addresses obscured for privacy)

<tr class="box-slide-head clickable" id="u8gUpiMsggxT" mailid="u8gUpiMsggxT">
                        <td class="checkbox"><input type="checkbox" name="record" value="u8gUpiMsggxT\r\njWy-Taa2GsdU" class="clickable"></td>
                        <td>gtube</td>
                        <td><a href="/iredadmin/activities/quarantined/user/XXXXXXXX@gmail.com">XXXXXXXX@gmail.com</a></td>
                        <td><a href="/iredadmin/activities/quarantined/user/XXXXX@XXXXXXXXXXX.com">XXXXX@XXXXXXXXXXX.com</a></td>
                        
                            <td>Spam</td>
                        

                        <td>1 KB</td>
                        <td>2013-11-03 08:44:10</td>
                        <td class="vcenter slide-but checkbox"><span>more</span></td>
                    </tr>

The following query from the database shows that the message is available:

 select * from quarantine where mail_id = 'u8gUpiMsggxT'\G;
*************************** 1. row ***************************
partition_tag: 0
      mail_id: u8gUpiMsggxT
    chunk_ind: 1
    mail_text: X-Envelope-From: <XXXXXXX@gmail.com>
X-Envelope-To: <XXXXX@XXXXXXXXXXX.com>
X-Envelope-To-Blocked: <XXXXX@XXXXXXXXXXX.com>
X-Quarantine-ID: <u8gUpiMsggxT>
X-Spam-Flag: YES
X-Spam-Score: 900.726
X-Spam-Level: ****************************************************************
X-Spam-Status: Yes, score=900.726 tag=2 tag2=6.2 kill=6.9
    tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
    FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, GTUBE=1000,
    HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RDNS_NONE=1.274,
    SPF_PASS=-0.001, TVD_SPACE_RATIO=0.001, USER_IN_SPF_WHITELIST=-100]
    autolearn=no
Authentication-Results: mail1.private-pages.com (amavisd-new); dkim=pass
    header.i=@gmail.com
Received: from mail1.private-pages.com ([127.0.0.1])
    by localhost (mail1.private-pages.com [127.0.0.1]) (amavisd-new, port 10024)
    with ESMTP id u8gUpiMsggxT for <XXXXX@XXXXXXXXXXX.com>;
    Sun,  3 Nov 2013 09:44:10 +0100 (CET)
Received: from mail-pb0-f46.google.com (unknown [209.85.160.46])
    by mail1.private-pages.com (Postfix) with ESMTPS id EE0504316
    for <taco@private-pages.com>; Sun,  3 Nov 2013 09:44:09 +0100 (CET)
Received: by mail-pb0-f46.google.com with SMTP id un15so994722pbc.5
        for <taco@private-pages.com>; Sun, 03 Nov 2013 01:45:33 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:date:message-id:subject:from:to:content-type;
        bh=nViY6AAT1SCA8fKADsH4GZM0N/VGyXUtsx6yGSaqpfM=;
        b=YV3yh2B8N8XmutmLNwtghLXoMUKr2uDCO67H33peXO8Zq4BujKBeMExaw0N+5HsmUf
         y9lurTva4jNdhW4zfXEbHu5gLHAObQNz0if98qAsAOn5zdyIJUG3OC7v0LMz2kIOcXO9
         z5UD032yArtcpqkDy4l54D1FlEfuaftl+FASkePBIhqzPF506oYlq2La2CoXv7FHY0bs
         isZWRdzRdei9kFT92vDJuIxFbcyTjuT69V9yOY6W8rqW9uUgOAr/28H/IPqBHMUT5S3G
         GUyvblUjuwmRZgR7HxCHVFf6+iVekkX4ALwESguISQ4ogCecTvCVeQ63fo2aUsZAcmhE
         C+vg==
MIME-Version: 1.0
X-Received: by 10.66.155.36 with SMTP id vt4mr11814923pab.93.1383466742216;
 Sun, 03 Nov 2013 01:19:02 -0700 (PDT)
Received: by 10.68.218.195 with HTTP; Sun, 3 Nov 2013 01:19:02 -0700 (PDT)
Date: Sun, 3 Nov 2013 09:19:02 +0100
Message-ID: <CAEdK=5VSNyixPRDs28q2K7Wp1CMXu6DztZ231aS7nRvCwHQYiQ@mail.gmail.com>
Subject: gtube
From: Taco Scheltema <XXXXX@XXXXXXXXXXX.com>
To: taco@private-pages.com
Content-Type: multipart/alternative; boundary=047d7bacc01aba7ea704ea41732b

--047d7bacc01aba7ea704ea41732b
Content-Type: text/plain; charset=ISO-8859-1

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

--047d7bacc01aba7ea704ea41732b
Content-Type: text/html; charset=ISO-8859-1

<div dir="ltr"><pre style="color:rgb(0,0,0)">XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X</pre></div>

--047d7bacc01aba7ea704ea41732b--

1 row in set (0.00 sec)

So it seems the message is saved correctly in the database (as far as I can tell).

6 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Quarantine mail body not displaying correctly in v2.0

I don't quite understand your pasted SQL result and screenshot.

The quarantined mail in your screenshot image is a "VIRUS" email with subject "eicar test", but the one you extracted from SQL database is a "SPAM" email with subject "gtube".

I need the SQL records of the one which doesn't work for you.

7 Reply by taco

  • taco
  • Member
  • Offline
  • Registered: 2013-11-02
  • Posts: 13

Re: Quarantine mail body not displaying correctly in v2.0

Hi,

This happens for every message in quarantine, not just for the one in the screenshot. I merely sent you the html source of one with it's matching database record.

8 Reply by taco

  • taco
  • Member
  • Offline
  • Registered: 2013-11-02
  • Posts: 13

Re: Quarantine mail body not displaying correctly in v2.0

Hi again,

I've enabled query logging in mysql to see what's happening on the database level;
The query that I see for a message (again, a different message as I have cleared the quarantine a few times now) is as follows:

SELECT mail_text FROM quarantine WHERE mail_id = 'ck5ujtwfepea' ORDER BY chunk_ind ASC;

when I look in the database I see the following:

mysql> select mail_id from quarantine;
+--------------+
| mail_id      |
+--------------+
| ck5UJtwFepeA |
+--------------+
1 row in set (0.00 sec)

So it appears the mail_id is somehow converted to lowercase after the expand button on the quarantine page is clicked (the mail_id is appearing in the correct case in the html source).

one workaround would be to change the query to:

SELECT mail_text FROM quarantine WHERE LOWER(mail_id) = 'ck5ujtwfepea' ORDER BY chunk_ind ASC;

but better would be to prevent the conversion to lowercase in the first place.

hope this helps.

9 Reply by taco

  • taco
  • Member
  • Offline
  • Registered: 2013-11-02
  • Posts: 13

Re: Quarantine mail body not displaying correctly in v2.0

I just tried the query below but since the mail_id isn't a string (it's a varbinary) it doesn't convert it to lowercase 

SELECT mail_text FROM quarantine WHERE LOWER(mail_id) = 'ck5ujtwfepea' ORDER BY chunk_ind ASC;

I'll have another look to see if I can find where the mail_id is converted to lowercase in the code.

10 Reply by taco (edited by taco 2013-11-05 18:17:51)

  • taco
  • Member
  • Offline
  • Registered: 2013-11-02
  • Posts: 13

Re: Quarantine mail body not displaying correctly in v2.0

I have a workaround (not pretty but it works);

in the file 

libs/amavisd/quarantine.py

I changed line 38 from 

where='mail_id = %s' % web.sqlquote(mail_id),

to

where='LOWER(CONVERT(mail_id using UTF8)) = %s' % web.sqlquote(mail_id),

That works.
Better would be still to prevent the mail_id to be converted to lowercase.

Next issue is that once the body of a message is displayed, clicking the expand button again will not close the message body, instead it shows it twice. clicking multiple times will show a new instance of the message every time.

I can live with that I guess but it would be nice to see that fixed in a future release.

11 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Quarantine mail body not displaying correctly in v2.0

I checked source code, no code to convert mail_id to lower cases at all. And i didn't receive similar issues before:

- Cannot view quarantined mail
- Expand button will not close the message body
- Click multiple times will show a new instance of the message every time

So it really confuses me.

*) Is there any special setting in your MySQL side?
*) Which version of web.py (micro python web framework used in iRedAdmin-Pro) do you use? You can check it with command "rpm -qa | grep 'web'".
*) Is it possible to let me login to your server for further debug? I cannot reproduce this issue on my machine.

12 Reply by taco

  • taco
  • Member
  • Offline
  • Registered: 2013-11-02
  • Posts: 13

Re: Quarantine mail body not displaying correctly in v2.0

Hi,

I have python-webpy-0.36-2.el6.noarch, is that the correct one?

My server is a production server so unfortunately I can't give access to it. let me know what information you need and I'll collect it for you.

Cheers,
Taco

13 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 31,088

Re: Quarantine mail body not displaying correctly in v2.0

The problem is, i cannot reproduce this issue on my machines, and i didn't receive similar bug report before, so i have no idea which part causes this issue, i have to modify some iRedAdmin code for further debug. If i cannot login to server remotely, i'm afraid that i cannot help solve this issue. Sorry about this.

14 Reply by taco

  • taco
  • Member
  • Offline
  • Registered: 2013-11-02
  • Posts: 13

Re: Quarantine mail body not displaying correctly in v2.0

Not a problem, I can live with this issue for the moment.

I will try to install a second instance with a sample domain to see if I can reproduce the issue, if I can I'll give you access to that.

Cheers,
Taco