1 Topic by ionutb

  • ionutb
  • Member
  • Offline
  • Registered: 2020-05-15
  • Posts: 8

Topic: SOGO XML parsing failures

==== REQUIRED BASIC INFO OF YOUR IREDMAIL SERVER ====
- iRedMail version (check /etc/iredmail-release): 1.4.1
- Deployed with iRedMail Easy or the downloadable installer? installer
- Linux/BSD distribution name and version: Debian 10
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): mysql
- Web server (Apache or Nginx):nginx
- Manage mail accounts with iRedAdmin-Pro? yes
- [IMPORTANT] Related original log or error message is required if you're experiencing an issue.
====

Hi,

Starting this morning SoGO ActiveSync fails with the following erros:

===
Feb 23 09:21:18 sogod [4313]: localhost "POST /SOGo/Microsoft-Server-ActiveSync?User=aaa@aa.aa&DeviceId=aaa&DeviceType=iPhone&Cmd=FolderSync HTTP/1.0" 200 0/34 0.281 - - 264K - 19
Feb 23 09:21:22 sogod [4313]: localhost "OPTIONS /SOGo/Microsoft-Server-ActiveSync HTTP/1.0" 200 0/0 0.004 - - 0 - 19
Feb 23 09:21:23 sogod [4313]: [ERROR] <0x0x555a439f17d0[NSDataMalloc]> xml2wbxmlFromContent: failed: Parsing of XML Document Failed
===

I have searched the forums and I noticed 2 years ago someone had a similar issue which was (auto)resolved by SoGO nightly update. Is there anyone else having this problem at this time, or it is something in my setup?

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ericjthorsen

  • ericjthorsen
  • Member
  • Offline
  • Registered: 2018-01-04
  • Posts: 2

Re: SOGO XML parsing failures

I am having the same issue

3 Reply by alec (edited by alec 2022-02-24 16:21:40)

  • alec
  • Member
  • Offline
  • Registered: 2022-02-24
  • Posts: 16

Re: SOGO XML parsing failures

Same problem here. Also ActiveSync is no longer working from yesterday.
Tried to edit configs of Sogo as reported on the guide of Fine Tuning Sogo. I reset device cache with sogo-tool. I restarted the whole machine but nothing changed.

Sogo v5.5.1.20220209-1
Sogo ActiveSync v5.5.1.20220209-1

4 Reply by mict

  • mict
  • Member
  • Offline
  • Registered: 2022-02-24
  • Posts: 1

Re: SOGO XML parsing failures

same problem.
- tried rollback  sogo-activesync / sogo
- eas resetdevice
- flushed memcached
- sogo.conf tunings

Feb 24 06:37:29 sogod [30308]: localhost "OPTIONS /SOGo/Microsoft-Server-ActiveSync HTTP/1.0" 200 0/0 0.002 - - 0 - 15
Feb 24 06:37:29 sogod [30308]: <0x0x555555a35270[SOGoActiveSyncDispatcher]> EAS - request for device IOT4T66BE93UT5EKQD4E63JKNC: <?xml version="1.0"?>
<!DOCTYPE ActiveSync PUBLIC "-//MICROSOFT//DTD ActiveSync//EN" "http://www.microsoft.com/">
<Sync xmlns="AirSync:">
 <Collections>
  <Collection>
   <SyncKey>0</SyncKey>
   <CollectionId>mail%2FTemplates</CollectionId>
   <Options>
    <MIMETruncation>1</MIMETruncation>
    <MIMESupport>0</MIMESupport>
    <BodyPreference xmlns="AirSyncBase:">
     <Type>1</Type>
     <TruncationSize>500</TruncationSize>
    </BodyPreference>
   </Options>
  </Collection>
 </Collections>
</Sync>

Feb 24 06:37:29 sogod [30310]: <0x0x5555561eb950[SOGoActiveSyncDispatcher]> EAS - request for device IOT4T66BE93UT5EKQD4E63JKNC: <?xml version="1.0"?>
<!DOCTYPE ActiveSync PUBLIC "-//MICROSOFT//DTD ActiveSync//EN" "http://www.microsoft.com/">
<FolderSync xmlns="FolderHierarchy:">
 <SyncKey>saft_aeb7_27b992a8_0</SyncKey>
</FolderSync>
Feb 24 06:37:29 sogod [30310]: [ERROR] <0x0x55555621d5a0[NSDataMalloc]> xml2wbxmlFromContent: failed: Parsing of XML Document Failed
Feb 24 06:37:29 sogod [30310]: [ERROR] <0x0x55555621d5a0[NSDataMalloc]> Original data written to: /tmp/saft_7666_27c6ddf1_286.data

xmlparse /tmp/saft_7666_27c6ddf1_286.data
error : Unknown IO error
/tmp/saft_7666_27c6ddf1_286.data:1: validity error : Validation failed: no DTD found !
ActiveSync//EN" "http://www.microsoft.com/"><FolderSync xmlns="FolderHierarchy:"
                                                                               ^
2022-02-24 06:39:15.347 xmlparse[72398:72398] Document is FolderSync

xmllint --format /tmp/saft_7666_27c6ddf1_286.data
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE ActiveSync PUBLIC "-//MICROSOFT//DTD ActiveSync//EN" "http://www.microsoft.com/">
<FolderSync xmlns="FolderHierarchy:">
  <Status>9</Status>
</FolderSync>

cat /tmp/saft_7666_27c6ddf1_286.data
<?xml version="1.0" encoding="utf-8"?><!DOCTYPE ActiveSync PUBLIC "-//MICROSOFT//DTD ActiveSync//EN" "http://www.microsoft.com/"><FolderSync xmlns="FolderHierarchy:"><Status>9</Status></FolderSync>

Currently new version arrived:

Get:1 https://packages.inverse.ca/SOGo/nightly/5/debian buster/buster amd64 sogo-activesync amd64 5.5.1.20220224-1 [1,441 kB]
Get:2 https://packages.inverse.ca/SOGo/nightly/5/debian buster/buster amd64 sogo amd64 5.5.1.20220224-1 [19.9 MB]


=> Still same problem

5 Reply by alec

  • alec
  • Member
  • Offline
  • Registered: 2022-02-24
  • Posts: 16

Re: SOGO XML parsing failures

I tried to remove some EAS clients, flushed cache but nothing changes.

For now I'm reverting my clients to IMAP

6 Reply by Johan Badenhorst

  • Johan Badenhorst
  • Member
  • Offline
  • Registered: 2022-02-24
  • Posts: 3

Re: SOGO XML parsing failures

Having the same issue,

Reverted back to a backup and simply re-imported my complete vmail folder to get the system up and running.

The following packages was the last working packages I had

sogo - 5.5.1.20220221-1
sogo-activesync - 5.5.1.20220221-1

For me also no ActiveSync Finetuning changes had any effect.

7 Reply by ionutb

  • ionutb
  • Member
  • Offline
  • Registered: 2020-05-15
  • Posts: 8

Re: SOGO XML parsing failures

I'm seriously considering a strategy where I would rely on Debian-maintained SoGO packages instead of nightly builds. The packages in Buster are quite old however (4.0.7-1+deb10u1). Upgrading to Bullseye would get me 5.0.1-4.

8 Reply by coralsnake

  • coralsnake
  • Member
  • Offline
  • Registered: 2021-09-20
  • Posts: 5

Re: SOGO XML parsing failures

Bug Ticket is already open: https://www.sogo.nu/bugs/view.php?id=5486

9 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: SOGO XML parsing failures

The bug report mentions a solution: Downgrade libexpat1 package. https://www.sogo.nu/bugs/view.php?id=5486

FYI: https://www.mail-archive.com/users%40so … 31208.html

10 Reply by ionutb (edited by ionutb 2022-02-24 23:13:32)

  • ionutb
  • Member
  • Offline
  • Registered: 2020-05-15
  • Posts: 8

Re: SOGO XML parsing failures

I can confirm that it works after downgrading that package. No apparent side effects for now.

BTW: Thanks Zhang, both for the heads-up and for following the ticket on sogo.nu lists.

11 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: SOGO XML parsing failures

For Ubuntu 20.04 users, please downgrade to libexpat1_2.2.9-1build1, then restart sogo service.
For example:

wget http://us.archive.ubuntu.com/ubuntu/pool/main/e/expat/libexpat1_2.2.9-1build1_amd64.deb
dpkg -i libexpat1_2.2.9-1build1_amd64.deb
service sogo restart

For Debian 11 users, please downgrade to 2.2.6-2+deb10u1.

12 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: SOGO XML parsing failures

User "FoZo" shared the downgrade steps with "apt-get" command here:
https://www.sogo.nu/bugs/view.php?id=5486

13 Reply by ionutb (edited by ionutb 2022-02-24 23:31:50)

  • ionutb
  • Member
  • Offline
  • Registered: 2020-05-15
  • Posts: 8

Re: SOGO XML parsing failures

For Debian 10 users, steps to downgrade:

===
sudo apt install libexpat1=2.2.6-2+deb10u1
sudo apt-mark hold libexpat1
===

Please note that the second line will hold the package at this specific version until hold is removed.

14 Reply by alec

  • alec
  • Member
  • Offline
  • Registered: 2022-02-24
  • Posts: 16

Re: SOGO XML parsing failures

ZhangHuangbin wrote:

For Ubuntu 20.04 users, please downgrade to libexpat1_2.2.9-1build1, then restart sogo service.
For example:

wget http://us.archive.ubuntu.com/ubuntu/pool/main/e/expat/libexpat1_2.2.9-1build1_amd64.deb
dpkg -i libexpat1_2.2.9-1build1_amd64.deb
service sogo restart

For Debian 11 users, please downgrade to 2.2.6-2+deb10u1.

I tried that but after restarting sogo, nginx was responding with Error 502 to users that were trying to login from browser.

15 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: SOGO XML parsing failures

alec wrote:

I tried that but after restarting sogo, nginx was responding with Error 502 to users that were trying to login from browser.

Any error in SOGo log file /var/log/sogo/sogo.log?

16 Reply by coralsnake (edited by coralsnake 2022-03-01 20:08:52)

  • coralsnake
  • Member
  • Offline
  • Registered: 2021-09-20
  • Posts: 5

Re: SOGO XML parsing failures

to summarize up:

Keep in mind that when you are downgrading, you are doing rollback to a package version with known security vulnerabilities CVE-2022-25236 (NIST Score 9.8/critical):
(Description: xmlparse.c in Expat (aka libexpat) before 2.4.5 allows attackers to insert namespace-separator characters into namespace URIs)

Patching of this security vulnerability ended up in incompatibilty between libwbxml and Expat >=2.4.5
"libwbxml is using : (colon) for a namespace separator"  & "Expat disallows use of namespace separator characters in xmlns attribute values for security since 2.4.5"

Therefore new bug tracking for this issue is openend on github and needs resolving, you can follow up here if you like:
https://github.com/libwbxml/libwbxml/issues/78

17 Reply by plantagenet82

  • plantagenet82
  • Member
  • Offline
  • Registered: 2020-03-04
  • Posts: 14

Re: SOGO XML parsing failures

Can confirm same issue with AWS EC2 Ubuntu 18.04. Sogo v4.

#apt-get install libexpat1=2.2.5-3 libexpat1-dev=2.2.5-3
#apt-mark hold libexpat1

Back up and working.

Upgraded to Sogov5 and iRedmail 1.5.2.

18 Reply by alm

  • alm
  • Member
  • Offline
  • Registered: 2021-12-27
  • Posts: 39

Re: SOGO XML parsing failures

plantagenet82 wrote:

Upgraded to Sogov5 and iRedmail 1.5.2.

There is no iRedMail 1.5.2, I guess you meant RoundCubeMail 1.5.2 ?

19 Reply by plantagenet82

  • plantagenet82
  • Member
  • Offline
  • Registered: 2020-03-04
  • Posts: 14

Re: SOGO XML parsing failures

My bad. Yes iRedmail 1.5.1 which includes Roundcube upgrade to version 1.5.2.

20 Reply by coralsnake

  • coralsnake
  • Member
  • Offline
  • Registered: 2021-09-20
  • Posts: 5

Re: SOGO XML parsing failures

libwbxml author released a patched version libwbxml 0.11.8
therefore the new version should be available in the official packages in Debian soon i guess

in the meantime, if you like, the libwbxml.c author provides a community repository with packages for the latest version of SOGo
https://www.axis.cz/linux/debian/pool/sogo-v5/w/wbxml2/
be sure to pick the correct distro

i tested on my debian 11 system and it works fine with the latest libexpat1:
wget ..... libwbxml2-0_0.11.8-0debian11+0_amd64.deb
dpkg -i libwbxml2-0_0.11.8-0debian11+0_amd64.deb

apt-mark unhold libexpat1
apt install libexpat1

21 Reply by atl198

  • atl198
  • Member
  • Offline
  • Registered: 2021-11-26
  • Posts: 6

Re: SOGO XML parsing failures

Can confirm that downgrading libexpat1 works. Should have found this post earlier, would have saved me a couple of hours.

22 Reply by Buck

  • Buck
  • Member
  • Offline
  • Registered: 2020-09-14
  • Posts: 11

Re: SOGO XML parsing failures

Is there a time frame that EAS will come back to alive?

23 Reply by coralsnake

  • coralsnake
  • Member
  • Offline
  • Registered: 2021-09-20
  • Posts: 5

Re: SOGO XML parsing failures

New packages are available from https://packages.inverse.ca/SOGo/

24 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,770

Re: SOGO XML parsing failures

Just run "yum" or "apt" tool to upgrade sogo and its dependent packages will fix the issue.