message marked as deleted, but could not expunge

[ftoledo]

Bug Report

I try to identify the source of GLPI mail collector BUG (glpi-project/glpi#9893) that use Laminas mail Library

Current behavior

Get exception and the message still hold in source folder

Expected behavior

The message must be move to destination folder.

Summary

GLPI collect mail from IMAP office 365 mailbox, but does not move the message from one folder to another after processing it

I first see a Exception "unique id not found"

[2024-03-04 15:24:37] glpiphplog.CRITICAL:   *** Uncaught Exception Laminas\Mail\Storage\Exception\InvalidArgumentException: unique id not found in /glpi/vendor/laminas/laminas-mail/src/Storage/Imap.php at line 314
  Backtrace :
  src/MailCollector.php:2074                         Laminas\Mail\Storage\Imap->getNumberByUniqueId()
  src/MailCollector.php:1092                         MailCollector->deleteMails()
  front/mailcollector.form.php:103                   MailCollector->collect()
  public/index.php:82                                require()

but after further analysis, I see that the error occurs when trying to call EXPUNGE

I try to print the exception and discover that laminas raise this error:

trace: Laminas\Mail\Storage\Exception\RuntimeException Object
(
    [message:protected] => message marked as deleted, but could not expunge
    [string:Exception:private] => 
    [code:protected] => 0
    [file:protected] => /glpi/vendor/laminas/laminas-mail/src/Storage/Imap.php
    [line:protected] => 270
)

What could be the problem that running EXPUNGE returns with an error?
The server is office 365. Could it be some policy or configuration on the server side?
Any idea to solve it would be useful to me.
Thank you

[Ocramius]

getNumberByUniqueId is effectively just saying "can't find by that ID" - would rather check what your src/MailCollector.php is doing :-)

[poojaajudia]

@ftoledo
I am facing the same issue. But the config is a bit different. Instead of moving the email to a different folder, I need to delete emails from the mailbox. Two mailboxes are affected.

So once the email is processed, it should be deleted automatically (as per app config) from the mailbox. But it’s not deleting and keeping the email in the ‘inbox’ folder. When try to delete it manually from the app, it displayed error message “message marked as deleted, but could not expunge”.

(Not sure if this is relevant here ->) When I delete email from the ‘inbox’ folder exchange mailbox, it goes to ‘deleted items’. But permanently deleting from the ‘deleted items’ to ‘recoverable items’ does not allow.

First, I thought this might be due to some email retention policy in the exchange mailbox.

Upon further investigation with MS, when I reduced the size of ‘TotalDeletedItemSize’ (in the O365 exchange portal), the 'expunge' error was resolved for one mailbox. However, I am unable to reduce the ‘TotalDeletedItemSize” for second mailbox and that still show this ‘expunge’ error while I manually delete emails from the app. Currently working with MS for further investigation.

May be you can try reducing "TotalDeletedItemSize" from the exchange. Please let me know if this works.

Also, for some other mailboxes, it shows this could not expunge error while processing. And does not even complete ticket creation process. See below screenshot. Please share your input if have seen something similar in your app.

[ftoledo]

Hello @poojaajudia, your analysis is very interesting. I think it is also an issue related to retention policies.

The EXPUNGE command is expected to return true/ok but in this case it is returning error.

I don't know if this is violating the RFC/protocol, anyway maybe a quirk needs to be added for this case. Because it only happens with office 365 mailboxes via imap. (using OWA I don't see this behavior)

Unfortunately we opted to switch to a new mailbox for now (because it was a production server) and obviously we no longer have this problem.

But whether Laminas or GLPI must take this behavior into account.

Please share if you get more information from MS, I will continue monitoring this issue and if you find anything else I will share it.

Thanks!

[poojaajudia]

Finally, 'could not expunge' issue was resolved for second mailbox by reducing 'TotalDeletedItemSize'.
And with that I was able to relate all the info I found during the investigation.

TotalDeletedItemSize - Items in the Recoverable Items folder are retained for 14 days by default and then purged by Microsoft Exchange. When a mailbox is put on litigation hold, the Recoverable Items folder isn't purged and items in this folder are retained indefinitely.

If a user does a hard delete on an email (deleted the email from the deleted items folder) it will then move to the Recoverable Items folder and due to the litigation hold will remain there and be counted in the TotalDeletedItemSize property.

For second mailbox, once the Litigation hold was removed and reduced TotalDeletedItemSize (which reached max size 100gb), the 'message marked as deleted, but could not expunge' was resolved.