Crash during mobile db migration for large fast sync dbs

[petmongrels]

No description provided.

[mahalakshme]

@petmongrels
how long does it take for large databases to complete migration?

[mahalakshme]

Vivek replied upto a minute

[mahalakshme]

Some of the Cases to test:

• Should not probably take more than one minute
• Pressing back/killing the app when upgrading
• old fast sync
• new upgradation from old version app
• with different large data sets
• Avni icon display when upgrading
• Make sure all data is there as expected after migration - observations, location, regex validations in first name etc.,

[AchalaBelokar]

• I login with AchalaB@Calcutta_kids and then set a fast sync and then upgrade the app. I started upgrading and then it is sowing mw upgrading data please do not close the app. 3.21 to 3.38 still showing this msg.
• After killing the app same message is showing for long time.
• I Login with AchalaB@sncu and updated the app ...then I kill the app and after open it is showing me white screen app is not responding

[petmongrels]

Can you measure how long it taking you. It will finish.

…

On Thu, Feb 22, 2024, 3:42 PM AchalaBelokar ***@***.***> wrote: - I login with ***@***.***_kids and then set a fast sync and then upgrade the app. I started upgrading and then it is sowing mw upgrading data please do not close the app. 3.21 to 3.38 still showing this msg. - After killing the app same message is showing for long time. — Reply to this email directly, view it on GitHub <#1309 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAZ3C73QV76YEKXRHT5U7DYU4KYRAVCNFSM6AAAAABDQSZOVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJZGEYTMNBXGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[petmongrels]

It will be better perhaps to implement a progress bar on upgrade screen and similar on fast sync restore. Other wise it may cause lot of confusion. Keep awake may be required if not there

…

On Fri, Feb 23, 2024, 9:00 AM Vivek Singh ***@***.***> wrote: Can you measure how long it taking you. It will finish. On Thu, Feb 22, 2024, 3:42 PM AchalaBelokar ***@***.***> wrote: > > - I login with ***@***.***_kids and then set a fast sync and > then upgrade the app. I started upgrading and then it is sowing mw > upgrading data please do not close the app. 3.21 to 3.38 still showing this > msg. > - After killing the app same message is showing for long time. > > — > Reply to this email directly, view it on GitHub > <#1309 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAAZ3C73QV76YEKXRHT5U7DYU4KYRAVCNFSM6AAAAABDQSZOVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJZGEYTMNBXGE> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

[mahalakshme]

Yeah agreed it will be good to show progress bar else users might think the app is stuck.

[petmongrels]

Re Compaction

113k ProgEnc in all cases.

Server obs count queries

`select count() from individual i
join address_level on i.address_id = address_level.id and address_level.title = '1'
cross join lateral jsonb_object_keys(i.observations); -- 14255
select count() from program_encounter i
join address_level on i.address_id = address_level.id and address_level.title = '1'
cross join lateral jsonb_object_keys(i.observations); -- 786550
select count() from program_encounter i
join address_level on i.address_id = address_level.id and address_level.title = '1'
cross join lateral jsonb_object_keys(i.cancel_observations); -- 18670
select count() from program_enrolment i
join address_level on i.address_id = address_level.id and address_level.title = '1'
cross join lateral jsonb_object_keys(i.observations); -- 12023
select count() from program_enrolment i
join address_level on i.address_id = address_level.id and address_level.title = '1'
cross join lateral jsonb_object_keys(i.program_exit_observations); -- 2624
select count() from encounter i
join address_level on i.address_id = address_level.id and address_level.title = '1'
cross join lateral jsonb_object_keys(i.observations); -- 46860
select count() from encounter i
join address_level on i.address_id = address_level.id and address_level.title = '1'
cross join lateral jsonb_object_keys(i.cancel_observations); -- 0
select count() from checklist_item i
join checklist on i.checklist_id = checklist.id
join program_enrolment on checklist.program_enrolment_id = program_enrolment.id
join address_level on program_enrolment.address_id = address_level.id and address_level.title = '1'
cross join lateral jsonb_object_keys(i.observations); -- 18828

-- 14255+786550+18670+12023+2624+0+18828 = 852950`

DB file sizes

avni/db$ ls -ltS
total 1544328
-rw-r--r--@ 1 vsingh staff 312098816 Feb 26 13:21 default.7.0.without.compact.realm
-rw-r--r--@ 1 vsingh staff 187318272 Feb 26 13:21 default.6.1.without.compact.realm
-rw-r--r--@ 1 vsingh staff 185421824 Feb 26 13:58 default.6.1.with.compact.realm
-rw-r--r--@ 1 vsingh staff 102535168 Feb 26 13:44 default.7.0.with.compact.realm

Outcome

1. Running migration is increasing the database size due to copying I think (going to little less than double the size)
2. Compacting after migration reduces size due to two reasons
   • 7.0-without-compact is taking more space because the freed up space has not been reclaimed yet so it includes the space taken by the deleted schemas + the new embedded objects
   • secondly due to reduced number of observations overall. Number of observations for this catchment from server side = 8,52,950. In 6.1 realm file the number of observations are = 18,55,479.
3. So the net net gain is due to 10,00,000 less observations.

[himeshr]

Shouldn't we compact only when there is lot of unused space in the file.?
https://www.mongodb.com/docs/realm/sdk/react-native/realm-files/compact-realm/#realm-configuration-file

[AchalaBelokar]

• After downloading the 6.1 apk try to login with AchalaB@Calcutta_kids it is showing this message.. After performing slow sync app is crashed .. blank screen is coming

• And after killing the app and reopen app still app is showing white screen

[himeshr]

Deleted fast-sync file, fresh sync should work now.

[AchalaBelokar]

• I login with AchalaB@sncu and created one individual and after that set up fast sync and update the app. for updating the app it is taking more than 5 min its like 6,7 min .. so this msg is misleading to user.

• 1st time login with the AchalaB@calcutta_kids and shift to another app and after that come to avni app it is showing me same msg may take up to 5 min start time is 4.22 and end it 4.30

[AchalaBelokar]

• While updating the aap I kill the app then it will start from time we reopen the app.
• After pressing back button it is not working as expected
• I created a individual and sync app after that I updated the app all data is visible on app.
• Back button is not working while updating the app
• I am able to see the avni logo with message while updating the app.
• observation which I created it is reflected as we are expected .