Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Two groups accidentally created while fetching pending messages after backup restore #5385

Closed
gerryfrancis opened this issue Mar 29, 2024 · 5 comments · Fixed by #5779
Closed
Assignees
Labels
bug Something is not working

Comments

@gerryfrancis
Copy link
Contributor

  • Operating System (Linux/Mac/Windows/iOS/Android):
    Android 11.

  • Delta Chat Version:
    1.44.0 (nightly built 2024-03-26).

  • Expected behavior:
    When pending messages are fetched from the Inbox and Sent IMAP folders after a backup has been restored, emails that were sent by MUAs, e.g. Webmail clients, and contain Re: in the first place of the subject, e.g. in replies, do not trigger creating a new group named Re: [Subject].

  • Actual behavior:
    When pending messages are fetched from the Inbox and Sent IMAP folders after a backup has been restored, emails that were sent by MUAs, e.g. Webmail clients, and contain Re: in the first place of the subject, e.g. in replies, trigger creating a new group named Re: [Subject].

  • Steps to reproduce the problem:
    -- Create a backup of your Delta Chat account.
    -- Un- and reinstall Delta Chat, but do not launch the app, yet.
    -- Receive a classic email that has been addressed to you and a second recipient. (The email should remain in the Inbox folder of your IMAP email account.)
    -- Reply to that email by using a MUA client, e.g. Webmail. (Your email should be stored in the Sent folder with Re: [Subject] as the subject.)
    -- Launch Delta Chat and restore the backup of your account. (Result: Two groups are created, named Re: [Subject] (which contains the reply message) and [Subject] (which contains the first message), while pending messages are being fetched.)

  • Screenshots:
    2024-03-26-10-51

  • Logs:
    N/A.

@iequidoo iequidoo self-assigned this Apr 10, 2024
@iequidoo iequidoo added the bug Something is not working label Apr 10, 2024
@gerryfrancis
Copy link
Contributor Author

FYI, it happens with version 1.45.0 (beta), too.

@iequidoo
Copy link
Collaborator

iequidoo commented May 16, 2024

A fix i can imagine here is to sort by INTERNALDATE not only messages from a particular folder, but from all folders, before passing them to receive_imf_inner(). The problem is that folders are processed by different threads, so it will be actually not sorting, but some kind of synchronisation using e.g. tokio::sync::Notify + Mutex. Not too difficult to implement, but still quite a big change so it's a question whether fixing this is worth it. Maybe also a topo-sorting using the References header is a way.

@iequidoo
Copy link
Collaborator

iequidoo commented May 16, 2024

Another possible fix is to leave this unordered processing of folders as is, but search Message-ID in the already known references, this way we can insert an unordered ("missed previously") message to the already existing chat.

EDIT: the problem is that several messages may be unordered, in this case we know the proper chat for them only when processing the last unordered message. Merging chats looks too complicated to fix this, some INTERNALDATE sorting-based solution could be better.

EDIT: A better possible fix: even if a message doesn't reference any known message, but has the same subject (without prefix) and "group members" as an already existing ad-hoc group, use that group for the new message.

iequidoo added a commit that referenced this issue Jun 16, 2024
Delta Chat -style groups have names w/o prefixes like "Re: " even if the user is added to an already
existing group, so let's remove prefixes from ad-hoc group names too. Usually it's not very
important that the group is a classic email thread existed before, this info just eats up screen
space. Also this way a group name is likely to preserve if the first message was missed.
iequidoo added a commit that referenced this issue Jun 18, 2024
Delta Chat -style groups have names w/o prefixes like "Re: " even if the user is added to an already
existing group, so let's remove prefixes from ad-hoc group names too. Usually it's not very
important that the group is a classic email thread existed before, this info just eats up screen
space. Also this way a group name is likely to preserve if the first message was missed.
iequidoo added a commit that referenced this issue Jun 18, 2024
Delta Chat -style groups have names w/o prefixes like "Re: " even if the user is added to an already
existing group, so let's remove prefixes from ad-hoc group names too. Usually it's not very
important that the group is a classic email thread existed before, this info just eats up screen
space. Also this way a group name is likely to preserve if the first message was missed.
iequidoo added a commit that referenced this issue Jun 19, 2024
Delta Chat -style groups have names w/o prefixes like "Re: " even if the user is added to an already
existing group, so let's remove prefixes from ad-hoc group names too. Usually it's not very
important that the group is a classic email thread existed before, this info just eats up screen
space. Also this way a group name is likely to preserve if the first message was missed.
iequidoo added a commit that referenced this issue Jul 16, 2024
…5385)

This should fix ad-hoc groups splitting when messages are fetched out of order from different
folders or otherwise reordered, or some messages are missing so that the messages reference chain is
broken, or a member was removed from the thread and readded later, etc. Even if this way two
different threads are merged, it looks acceptable, having many threads with the same name/subject
and members isn't a common use case.
iequidoo added a commit that referenced this issue Jul 16, 2024
…5385)

This should fix ad-hoc groups splitting when messages are fetched out of order from different
folders or otherwise reordered, or some messages are missing so that the messages reference chain is
broken, or a member was removed from the thread and readded later, etc. Even if this way two
different threads are merged, it looks acceptable, having many threads with the same name/subject
and members isn't a common use case.
iequidoo added a commit that referenced this issue Jul 17, 2024
…5385)

This should fix ad-hoc groups splitting when messages are fetched out of order from different
folders or otherwise reordered, or some messages are missing so that the messages reference chain is
broken, or a member was removed from the thread and readded later, etc. Even if this way two
different threads are merged, it looks acceptable, having many threads with the same name/subject
and members isn't a common use case.
iequidoo added a commit that referenced this issue Aug 3, 2024
…5385)

This should fix ad-hoc groups splitting when messages are fetched out of order from different
folders or otherwise reordered, or some messages are missing so that the messages reference chain is
broken, or a member was removed from the thread and readded later, etc. Even if this way two
different threads are merged, it looks acceptable, having many threads with the same name/subject
and members isn't a common use case.
iequidoo added a commit that referenced this issue Sep 16, 2024
…5385)

This should fix ad-hoc groups splitting when messages are fetched out of order from different
folders or otherwise reordered, or some messages are missing so that the messages reference chain is
broken, or a member was removed from the thread and readded later, etc. Even if this way two
different threads are merged, it looks acceptable, having many threads with the same name/subject
and members isn't a common use case.
iequidoo added a commit that referenced this issue Sep 20, 2024
…5385)

This should fix ad-hoc groups splitting when messages are fetched out of order from different
folders or otherwise reordered, or some messages are missing so that the messages reference chain is
broken, or a member was removed from the thread and readded later, etc. Even if this way two
different threads are merged, it looks acceptable, having many threads with the same name/subject
and members isn't a common use case.
iequidoo added a commit that referenced this issue Sep 26, 2024
…5385)

This should fix ad-hoc groups splitting when messages are fetched out of order from different
folders or otherwise reordered, or some messages are missing so that the messages reference chain is
broken, or a member was removed from the thread and readded later, etc. Even if this way two
different threads are merged, it looks acceptable, having many threads with the same name/subject
and members isn't a common use case.
iequidoo added a commit that referenced this issue Sep 26, 2024
…5385)

This should fix ad-hoc groups splitting when messages are fetched out of order from different
folders or otherwise reordered, or some messages are missing so that the messages reference chain is
broken, or a member was removed from the thread and readded later, etc. Even if this way two
different threads are merged, it looks acceptable, having many threads with the same name/subject
and members isn't a common use case.
iequidoo added a commit that referenced this issue Sep 26, 2024
…5385)

This should fix ad-hoc groups splitting when messages are fetched out of order from different
folders or otherwise reordered, or some messages are missing so that the messages reference chain is
broken, or a member was removed from the thread and readded later, etc. Even if this way two
different threads are merged, it looks acceptable, having many threads with the same name/subject
and members isn't a common use case.
@gerryfrancis
Copy link
Contributor Author

gerryfrancis commented Sep 27, 2024

@iequidoo I did the test with the latest nightly build: Those messages are assigned to one group now - thank you very much! 👍 :) The only problem left is the chronological order, as the reply/replies (stored in the Sent folder) seem(s) to be fetched/assigned to the group first, but this issue might not be solved easily. Anyway, thanks a lot for the great work! :)

Edit: Oh wow, I have found the PR in regard to the chronological ordering: #5800
Thank you so much for the heads up! :)

@iequidoo
Copy link
Collaborator

iequidoo commented Sep 27, 2024

Edit: Oh wow, I have found the PR in regard to the chronological ordering: #5800

Unfortunately, this PR won't help in your scenario, it only sorts new received outgoing messages down which is useful for multi-device, particularly when an account is shared by users. There's another PR, #5813, that adds a test on your scenario, this test fails currently. But this can be fixed by introducing a new state for received outgoing messages so that they are sorted like fresh incoming ones, which i plan to do once #5800 is merged, mostly to avoid self-conflicts as that doesn't affect many users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something is not working
Projects
None yet
2 participants