Realm doesn't check for optional=false objects, we have to do that in our code.

[petmongrels]

• We have one to one relationship in Avni models in a few places. Without not null check we will have bad offline data with downstream failures in the app. Realm checks for primitive field works fine. We could put a check in our code in RealmProxy.create method for such relationships - while Realm fixes this bug.
• As per realm docs -
  User-defined object types are always optional except in lists and sets...
• We can do bugsnag notify to collect these errors.
• Add the check optional: false if necessary to be mentioned explicity to the necessary fields in the models

How to test:

• Make a subject type only direct assignment
• Insert data into db such that a subject outside of user's catchment is assigned to the user
• Sync the users device
• We should be able to see failure on user sync itself when the individual is fetched, instead of seeing the failure when opening the individual.
• Via playground add cases to test other fields as well or add test cases for the same.

Out of scope:

Fixing existing bad data - since sometimes as an example, say individual's address is null, it could be because individual should itself not exist within the user's device. So for fixing existing data issues we can do them as and when we face issues since the scope and severity of the same is not clear.

[petmongrels]

Individual profile failing due to missing address lineage.

https://app.bugsnag.com/samanvay-research-and-development-foundation/avni-client/errors/6529dba76307e00008a7eb28

[mahalakshme]

@petmongrels yeah the above is because the issue is not that individual doesn't have address. The issue is that, individual in itself should not be there in the user's device since the address levels of the individual are outside of user's catchment.

[mahalakshme]

@petmongrels if we do this card, for what would we apply the condition saying optional as false? Since I see for an individual both gender and address can be optional and these are the cases we have faced issues. Are there any where u think this condition will be needed on priority? There are cases like IndividualRelationGenderMapping where relation and gender cant be optional but these are more static and wont change frequently and hence change of erroring is low and the priority of this card becomes low. Let me know if u can think of any cases where doing this is important.

[petmongrels]

• Individual to SubjectType
• Program Enrolment to Program
  etc.

Also, I think in our system address is mandatory on Individual. On server it is mandatory and I think on mobile app also.

[petmongrels]

• Note all of the following are effectively optional as of today, so the change is to when we make something non-optional.
• There are some which have been made explicitly optional. The following list doesn't included that.

Full List

[ 'DashboardFilter --> dashboard',
  'ConceptAnswer --> concept',
  'LocationMapping --> parent',
  'LocationMapping --> child',
  'FormMapping --> form',
  'FormElementGroup --> form',
  'FormElement --> concept',
  'FormElement --> formElementGroup',
  'Individual --> subjectType',
  'Individual --> lowestAddressLevel',
  'ProgramEnrolment --> program',
  'ProgramEnrolment --> individual',
  'Observation --> concept',
  'ProgramEncounter --> encounterType',
  'ProgramEncounter --> programEnrolment',
  'Encounter --> encounterType',
  'Encounter --> individual',
  'Checklist --> detail',
  'Checklist --> programEnrolment',
  'ChecklistItem --> detail',
  'ChecklistItem --> checklist',
  'VisitScheduleInterval --> min',
  'VisitScheduleInterval --> max',
  'VisitScheduleConfig --> encounterType',
  'VisitScheduleConfig --> interval',
  'ProgramConfig --> program',
  'Family --> lowestAddressLevel',
  'Family --> headOfFamily',
  'IndividualRelationGenderMapping --> relation',
  'IndividualRelationGenderMapping --> gender',
  'IndividualRelationshipType --> individualAIsToBRelation',
  'IndividualRelationshipType --> individualBIsToARelation',
  'IndividualRelationship --> relationship',
  'IndividualRelationship --> individualA',
  'IndividualRelationship --> individualB',
  'ChecklistItemStatus --> from',
  'ChecklistItemStatus --> to',
  'ChecklistItemDetail --> concept',
  'ChecklistItemDetail --> checklistDetail',
  'VideoTelemetric --> video',
  'IdentifierAssignment --> identifierSource',
  'GroupPrivileges --> group',
  'GroupPrivileges --> privilege',
  'GroupRole --> groupSubjectType',
  'GroupRole --> memberSubjectType',
  'GroupSubject --> groupSubject',
  'GroupSubject --> memberSubject',
  'GroupSubject --> groupRole',
  'DashboardSectionCardMapping --> dashboardSection',
  'DashboardSectionCardMapping --> card',
  'DraftSubject --> subjectType',
  'DraftSubject --> lowestAddressLevel',
  'EntityApprovalStatus --> approvalStatus',
  'GroupDashboard --> dashboard',
  'DashboardSection --> dashboard',
  'Comment --> subject',
  'Comment --> commentThread',
  'DocumentationItem --> documentation',
  'TaskStatus --> taskType',
  'Task --> taskType',
  'Task --> taskStatus',
  'DraftEncounter --> encounterType',
  'DraftEncounter --> individual',
  'SubjectProgramEligibility --> subject',
  'SubjectProgramEligibility --> program',
'Settings --> locale'

]

Making explicitly optional after checking

'Settings --> locale'

Not sure how they are used so keeping it as it is, which is that they remain optional.

'VisitScheduleInterval --> min',
'VisitScheduleInterval --> max'
'VisitScheduleConfig --> encounterType',
'VisitScheduleConfig --> interval'
'Family --> headOfFamily',

Rest all to become mandatory.

[ashusvnath]

I've made my comments on commit 8d70e06.

[mahalakshme]

@petmongrels I dont see whatever mentioned in Full List(like lowestAddressLevel in Individual), made mandotary. By default, are the properties mandatory? If so I dont see 'VisitScheduleInterval --> min',max having optional

[petmongrels]

yes mandatory by default. cannot find any real usage of VisitScheduleInterval/VisitScheduleConfig. they are not used.

…

On Thu, Nov 2, 2023 at 6:46 PM mahalakshme ***@***.***> wrote: @petmongrels <https://github.com/petmongrels> I dont see whatever mentioned in Full List(like lowestAddressLevel in Individual), made mandotary. By default, are the properties mandatory? If so I dont see 'VisitScheduleInterval --> min',max having optional — Reply to this email directly, view it on GitHub <#1142 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAZ3C7HCGO7WJ4MDBJCBDTYCOMLNAVCNFSM6AAAAAA6B66ZBCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJQG4YDMNRRGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Vivek Singh | +91 98452 32929 | @petmongrels <https://twitter.com/petmongrels>

[ashusvnath]

Testing notes:

• Able to see sync error when address is outside catchment for a directly assigned individual
• Wrote integration tests for to assert that creation of a Subject without subjectType and lowestAddressLevel fails

[ashusvnath]

@mahalakshme : Error screenshot as per you request

[vinayvenu]

Notes

• All optional/mandatory fields look good
• This will not fix issues where Individual.gender is required (we've had situations earlier where gender was expected but empty)
• Unrelated - We don't seem to be using ProgramConfig, and therefore VisitScheduleConfig and VisitScheduleInterval. We should remove this

@mahalakshme we should add cards for 2 and 3 and fix in a later release.