Update: so I found the files of the lost messages (whoo-hoo!) but when I tried to import them using the command I found on the wiki: http://wiki2.dovecot.org/Tools/Doveadm/Import nothing happened.

This is the command line I used: doveadm -Dv import -u user@example.com maildir:/tmp/store INBOX all

I also tried copying the message files to maildir without the import command, the emails showed up in the inbox but they were greyed out.

What am I doing wrong?

Thanks Zhang, do you have any idea why would any MUAs mark these emails as deleted? Our users definitely didn't delete these emails as they didn't know about them or couldn't even open them. Our users never use RoundCube or the web-interface to check email, only on their iPhone, Android, Macbook and Outlook.

Even more strange... The same emails were in different folders in RoundCube and SOGo. In RoundCube they were in the inbox and grayed out but in SOGo they were in the Junk folder and not greyed out.

With this specific user in the example and screenshot, the user is using and iPhone and mail app on Macbook to read emails. The configuration on the Macbook is IMAP and Exchange on the iPhone.

Yes Zhang, this user does have 'Enable junk mail filtering' enabled in the Apple Mail.app preferences. See screenshot: http://screencast.com/t/inCbQkvZMRnN

This seem to narrow down the options to the Apple Mail.app. Worth mentioning that the user reported that he could see the emails on his iPhone when he was out of the office but when returned to the office and opened the Apple Mail.app on his computer the emails were not in his inbox. He then checked his iPhone again and the emails disappeared!

The user then checked his inbox in RoundCube and this is what it looked like: http://awesomescreenshot.com/0bf5y94hf9
The user then checked his Junk mail folder in the Apple Mail.app on his computer and the missing emails were there
The user then selected all and marked as 'Not Junk'
The emails were still not in his inbox so he moved them to his inbox and everything looks fine on the Apple Mail.app but now his inbox in RoundCube looks like this: http://awesomescreenshot.com/0db5y96pff

What are your thoughts Zhang? We can certainly disable the filter for a day or two. What should the user do next if disabling 'Enable junk mail filtering' will indeed indicate that iRedMail was behaving in this way as a result of the filter?

The user has also other non-iRedMail accounts configured with his Apple Mail.app. Can iRedMail work with Apple Mail.app without such side-effects?

Since Apple Mail.app is one of the world's most popular mail clients, it's working smoothly with many and world's largest email providers, I can't ask users not to use it. They'll look at me like I'm crazy and let me go long before they'll even remotely consider not using their their Apple products that they love so much

Is there any way to fix iRedMail so it'll work without side effects with Apple Mail.app?

Hi Zhang,

Turns out Apple Mail.app isn't the cause for the greyed out emails as we thought. We disabled 'Enable junk filter' two days ago and new emails are still grey in RoundCube or completely missing from the inbox in Apple Mail.app and iPhone. The missing emails are in the junk folder in Apple Mail.app and iPhone. So if Apple Mail.app isn't the problem, what is? Any clue?

Hello,

I have this problem also and here is my discovery story :-)

1) What I see and tested out is that the problem is only for mac users.

2) I work with rules on my mac to move the email to a subfolder -> for example newsletters
    -- When a rule is affected on an email the mail is in gray in "Inbox folder"
    -- And i can find the original mail in the subfolder as expected.

3) There is a way to remove all emails that are marked "deleted"
     -> Open mac mail
     -> Select "mailbox"
     -> Select "Delete Removed parts"
     -> For all accounts (Shift + cmd + backspace)
     -> you get popup " Are you sure ... to remove"
     -> Delete and your webmail has a normal layout.

One important piece -> this is only possible if trash has these settings
-> fn + cmd + ,
-> select your mailbox
-> select tab mailbox settings
-> trash -> both checked and dropdown is never selected.

So here you have some details to reproduce.

I will keep my eye on this post ;-)

Thank you very much for your input Kris. I reproduced the steps you mentioned and have a few questions:

2) The user has only a few rules for all his email accounts and zero rules that related to the one of the affected inboxes.
  -- Any clue why would Apple Mail.app move these emails to the junk folder? Screenshot: http://screencast.com/t/JlqDanpI
  -- The user doesn't have any sub-folders for this mailbox. See screenshot: http://screencast.com/t/Xkb8ItQ5T

3) Tried to reproduce but couldn't reproduce step 3
     -- Couldn't find "Delete Removed parts" in the "mailbox" menu. See screenshot: http://screencast.com/t/2HAmYpDWsl

Couldn't understand the last piece:

"One important piece -> this is only possible if trash has these settings
-> fn + cmd + ,
-> select your mailbox
-> select tab mailbox settings
-> trash -> both checked and dropdown is never selected."

Thank you very much for your help!

I think is logical delete dat apple mail enforces. (State change)

When you do (shift + cmd + backspace) you enforece fysical delete. (File remove from server)

I think you have a combination of problems.

I have a behavior that my emails are in grayed out because a rule moves them.

behavior of a rule ->  Copy mail to folder and do Logical delete.

You in other hand have emails in junk folder. -> this can be also triggerd on the server side.

Please check if your settings or configuration is working properly.

I did experiment with upgrade proces in the past and i remember i had the same result. (Mail keep stuck on server)

-> My solution was easy, i reverted my environment do start the upgrade all over again.

Hope this little bit helps

(Zhang correct me if i am wrong)