Datastore produces old/duplicate records when running app on multiple devices

[ethan021021]

Describe the bug

When running my app on 3 different devices mutations to a record on device 1 and 2 don't seem to sync correctly to device 3.

Scenario:

App is open on device 1 and 2 while the app is force closed on device 3. I then create a record on device 2 and then delete it shortly after. When opening the force closed app on device 3 the deleted record from device 2 is appearing in device 3's SQLite DB. (After sync finishes)

Note: device 1 and device 2 are still correctly "in sync" after reproducing this issue, but device 3 falls "out of sync".

Please let me know if you need more context on this issue

Steps To Reproduce

Steps to reproduce the behavior:
1. Run an app that creates/deletes records on 3 different devices (Simulator works fine)
2. Force close the app on device #3
3. On device #2 create a new record and then delete it 5-10 seconds later
4. Reopen the app on device #3
5. Notice that the deleted record from device #2 is in the SQLite DB of device #3

Expected behavior

The record device 2 created and then deleted should not be in the SQLite DB on device 3

Amplify Framework Version

1.26.1

Amplify Categories

DataStore

Dependency manager

Swift PM

Swift version

5.6

CLI version

8.4.0

Xcode version

13.4 (13F17a)

Relevant log output

No response

Is this a regression?

No

Regression additional context

No response

Device

Simulators: iPhone 13 Pro, iPhone 13, iPhone 13 mini

iOS Version

iOS 15.5

Specific to simulators

No response

Additional context

No response

[ethan021021]

I should also note after reproducing this issue and checking my DynamoDB tables all records are correctly deleted on my model table and are "in sync".

[ethan021021]

I've also realized that if I run dataStore.clear() and then dataStore.start() device 3 is back "in sync"

[atierian]

Thanks for opening this issue @ethan021021. We'll look into it and respond here with any updates.

[alharris-at]

@chrisbonifacio will work on reproducing. For context, we have multiple SQLite DBs, a Model table, and data tables per type. This could be expected as temporary metadata in the model table, but Chris will work to confirm. If a model is deleted in DataStore we perform a soft delete, so the syncQuery will receive a delete mutation (assuming the delete is within the record TTL, which it sounds like from the repro steps) and be gone from the relevant data table, but continue to exist for a time in the Model table.

[ethan021021]

Hey all,

Just wondering if you were able to reproduce this issue? We had a big launch this morning and this issue hit us pretty hard. If you need anything else from me to try and reproduce this I'm more than happy to provide additional materials! I'm also free to hop on a call and chat about it anytime. Thanks a lot for your hard work!

[chrisbonifacio]

Hi @ethan021021 apologies for the delay. We have been able to reproduce this issue internally and are actively looking into it.

[ethan021021]

@chrisbonifacio thanks for the update Chris!

[chrisbonifacio]

@chrisbonifacio thanks for the update Chris!

No problem! Just to follow up, can you tell us a little bit more about the DataStore operations that you are running?

For example, can you share the code examples of how you are calling DataStore APIs to perform create/delete mutations?

Also, when you say

Notice that the deleted record from device #2 is in the SQLite DB of device #3

Are you using the DataStore query API to see the record in the DB or are you querying the SQLite DB directly?

[ethan021021]

@chrisbonifacio thanks for the update Chris!

No problem! Just to follow up, can you tell us a little bit more about the DataStore operations that you are running?

For example, can you share the code examples of how you are calling DataStore APIs to perform create/delete mutations?

Also, when you say

Notice that the deleted record from device #2 is in the SQLite DB of device #3

Are you using the DataStore query API to see the record in the DB or are you querying the SQLite DB directly?

Sadly I don't have the queries anymore as this is a pretty old issue and I had to implement a workaround myself. Sorry about that :(.

What I do remember when I filed this issue I was using the Datastore API to query/mutate. When I saw duplicate records I manually pulled the SQLite DB from the simulators/devices I was testing with and looked in there to see the duplicate records hanging out.

[chrisbonifacio]

@ethan021021 would you mind sharing what you had to do as a workaround? We were able to reproduce this internally and are looking into solutions.

[ethan021021]

@ethan021021 would you mind sharing what you had to do as a workaround? We were able to reproduce this internally and are looking into solutions.

I believe my workaround was to stop Datastore when the app moved to the background then started it back up again when the app came back to the foreground

Edit: I also believe I switched from datastore to API (GQL) directly for this particular model I was seeing the issue with

[lawmicha]

I wonder if this is related to #2982 (comment)