Add client migration to remove the individual outside of user's catchment

[mahalakshme]

Whats the issue:

The individual with uuid, 572b1b77-f886-4770-949b-d95a3959d232 is in some of the user's device though it doesn't belong to the users' catchment. Hence its lowestAddressLevel in realmdb is null. This is causing app crash when the user clicks on the individual card from say, the subject search results page.

Analysis:

• The subject doesn't have any subject migrations
• All functionalities related to reset sync are working as expected.
• The individual is a valid entry and belongs to a different catchment.
• The issue is not happening with any other individuals - identified from user's fast sync db.

To do:

• Add a client migration to remove the individual with the above mentioned uuid, when its lowestAddressLevel is null.

Out of scope:

Identifying the root cause of the issue since enough time has been spent and now we have a non-null check added on realmdb tables to identify such issues ahead.

[ashusvnath]

based on discussions with @mahalakshme . making the migration independent of uuid.
all individuals with lowestAddressLevel = null will be removed.

[vinayvenu]

I thought General.logDebug does not work in avni-models. In any case, not sure if it is a good idea to log in a loop.

Also, what happens to related objects when the individual is deleted?

[1t5j0y]

logging inside loop not addressed.

[ashusvnath]

@1t5j0y : I've left logging in place to aid debugging. i don't expect the number of such records to be high, and there's one log per record.

[himeshr]

Test Details

---

The approaches tried to create additional individuals with "null" lowestAddressLevels where as follows:

1. Update server to set the addressLevel as null

-- //Allow empty location
select * from individual where uuid = '212c4e48-3060-403c-91a2-63690d98cdd4'; --335080
update individual set address_id = null, last_modified_date_time = now() where uuid = '212c4e48-3060-403c-91a2-63690d98cdd4';

2. Use playground to manually update an Individual to set the addressLevel as null

async function executeIndividualAddressVoidQuery(type, setOutput) {
    //waitForNotNULL(getDb)
    console.log("executeIndividualAddressVoidQuery");
    setOutput("Preparing db");
    let db = await RealmFactory.createRealm()
    if(db == null) {
        return -1000;
    }
    try {
        if (db.objects(Individual.schema.name).filtered('uuid = $0', '212c4e48-3060-403c-91a2-63690d98cdd4').length > 0) {
            console.log("Clearing Lowest Address Level for Individual with uuid \'212c4e48-3060-403c-91a2-63690d98cdd4\'");
            let individual = db.objects(Individual.schema.name).filtered('uuid = $0', '212c4e48-3060-403c-91a2-63690d98cdd4')[0];
            console.log("individual.lowestAddressLevel", individual.lowestAddressLevel);
            let clonedIndividual = individual.cloneForEdit();
            clonedIndividual.lowestAddressLevel = null;
            db.write(() => {
                db.create(Individual.schema.name, clonedIndividual, true);
            })
            console.log("clonedIndividual.lowestAddressLevel", clonedIndividual.lowestAddressLevel);
        }
        return 0;
    } catch (e) {
        console.error(e);
        return setOutput(-1);
    }
}

---

Above scenarios failed due to

1. the inability to save or update Individuals with lowestAddressLevel as null in the client, due to checks in place and additional syncCriteria mismatch (null address are ignored, if not for direct-assignment)
2. the inability to sync Individuals with lowestAddressLevel as null in the server to the client (Models has mandatory constraint for the field and there are several validations that disallow nulls there)

---

Because of these, we are forced to test this by use of FastSync replace, which can only help test already existing data anamolies and not create new ones.

[himeshr]

In-order to Validate this better, modified the filter criteria for migration180, to be all individuals, irrespective of whether the addressLevel is null or not.
With this change, tested dumpRestore for following organisations:

• apf odisha
• gvamp uat
• rwb 2023

This helped validate various type of deletions involved in the migration180, such as

• encounters
• programEnrolments
• programEncounters
• checklists
• entityApprovalStatus
• Individuals, etc..