Planning: Registration of Users, Organisations and Departments

[pjmonks]

Problem

The workflow for how researchers get initial user accounts for Mauro and the SDE must be determined. This needs to happen before a researcher is able to carry out any other SDE workflow requests, since all SDE requests are tied to a user account.

Also, the user journey should be for the intended new research user to submit one registration form containing all details - personal details, plus organisation/department to join (or create if they do not exist yet).

This issue will be used to gather notes/requirements and map out the design/implementation of this feature. This is a "living" document and is subject to change.

Requirements

Based on the description taken from the epic #421, the following requirements must be considered:

• A researcher needs two application accounts - one for Mauro, one for the SDE
• For the Mauro user account, the user must be assigned to the "Explorer Readers" user group to allow permissions to read the root data model.
• A researcher also needs an external system account for Single Sign-On (SSO). For OUH, this will be a Microsoft EntraID account.
• A researcher will sign-in to both Mauro and the SDE (synchronised) via SSO. All accounts will use the same email address for matching identities.
• Before a researcher has these user accounts, they must be able to access some pages in the Mauro Data Explorer anonymously - particularly to start the registration process.
• The user will view one registration form with all required and conditional fields depending on the scenario. For example, joining an organisation requires a single selected organisation record, but creating an organisation requires additional organisation information.
• A researcher may have a user account to sign-in, but they will not be able to perform any actions in the Mauro Data Explorer until their registration is completed and they are part of an organisation/department.
• A registration request must allow no user ID to be recorded against it initially. However, once the user account has been created in the SDE, the request must be updated to map the known user to the request for tracking.
• A registration request cannot be marked as "completed" until the user ID of the new user has been assigned to it.

Out of Scope

• Adding user identity accounts to Microsoft EntraID. This is considered to be a manual step for now undertaken by OUH Azure administrators.
• External notifications to users. There will be a requirement one day to send a notification (e.g. email) to users, especially before the time when the user has a sign-in account, but this can be an enhancement at a later date.
• Creation of SDE administrator accounts - this only describes how researchers get included in the system.

Summary

After extensive discussions, we have settled on a multi-request approach to handle the creation of a user account for sign-in and have that user create/join an organisation/department. The SDE already has a detailed request state workflow that covers all the approval, governance and action states that a request might need, including tracking and conversation/history.

As well as creating a user account, there are 3 scenarios identified as being part of registration:

1. User joins an existing department in an existing organisation - a JoinOrganisationRequest
2. User also creates a new organisation - a CreateOrganisationRequest
3. User joins an existing organisation but creates a new department - a CreateDepartmentRequest

So registration will be a linked pair of requests - a CreateUserAccountRequest, followed by one of the requests above. Which secondary request to action is dependent on the state of the registration form in MDE. The requests will be linked by a new Request.previousRequestId field, such that requests are chained together, e.g.

flowchart LR
    CreateUserAccountRequest --> JoinOrganisationRequest

Loading

The CreateUserAccountRequest will be submitted straight to the Await pickup state so that an SDE administrator can view it immediately. The second, joined request will be submitted to the Draft state to prevent the SDE administrator from working on that first - the order of requests being actioned is important. The second request will have it's state automatically changed to Await pickup once the CreateUserAccountRequest is changed to Complete.

The key problem in this scenario is that, to register a user, they do not yet have a UserId yet - and all SDE requests must have a UserId attached. To solve this, a special singleton ResearchUser will be created to act as the "anonymous user" (the user ID is irrelevant, so long as one exists). Initially, the pair of requests will be mapped to the "anonymous user", but during the request state transitions, they will eventually be automatically updated within the SDE to swap to the new user. The pair of requests will not be allowed to move to the Complete state until a valid, non-anonymous user ID is remapped.

Sequence of Events

Below is a flowchart of the sequence of events that can happen. Note that, although the Request Status workflow is fully supported, this diagram may only show the "happy path" of all approvals passing for brevity.

flowchart   
    subgraph explorer
        anonUser[Anonymous User]        
        registrationPage[Resgistration Page]
        createRequests[Create Requests]

        anonUser -- Visit --> registrationPage
        registrationPage -- Submit --> createRequests          

        userSignIn[User Sign-in]
        reviewRequestStatus[Review Request Status]
        userReadyForSde[Ready for SDE]
    end

    createRequests -- Submit --> userAccountAwaitPickup
    createRequests -- Submit and wait --> registrationDraft

    userSignIn -- Authenticate --> authenticateUser
    authenticateUser -- Authenticated --> reviewRequestStatus
    reviewRequestStatus --> userReadyForSde

    subgraph sde-admin
        registrationDraft[Registration - Draft]

        userAccountAwaitPickup[User Account - Await Pickup]
        userAccountAssigned[User Account - Assigned]
        userAccountPending[User Account - Pending]

        userAccountAwaitPickup -- Assigned --> userAccountAssigned
        userAccountAssigned -- Approved --> userAccountPending

        createSdeUser[Create Mauro/SDE User]

        userAccountPending --> createSdeUser

        mapUserToUserAccountRequest[Map User to User Account Request]
        mapUserToRegistrationRequest[Map User to Registration Request]
        createSdeUser --> mapUserToUserAccountRequest
        createSdeUser --> mapUserToRegistrationRequest
        mapUserToRegistrationRequest -.-> registrationDraft

        newUserReady[New User Ready]

        mapUserToUserAccountRequest --> newUserReady

        userAccountFinalise[User Account - Finalise]
        newUserReady -- Complete --> userAccountFinalise

        registrationAwaitPickup[Registration - Await Pickup]
        userAccountFinalise -- Change status --> registrationAwaitPickup
        registrationDraft -. Automated state change .-> registrationAwaitPickup

        registrationAssigned[Registration - Assigned]
        registrationReview[Registration - Review]
        registrationPending[Registration - Pending]

        registrationAwaitPickup -- Assigned --> registrationAssigned
        registrationAssigned -- Request review --> registrationReview
        registrationReview -- Record approval --> registrationPending

        createRegistrationEntities[Create Registration Entities - Organisation, Department etc]

        registrationPending --> createRegistrationEntities

        registrationFinalise[Registration - Finalise]
        createRegistrationEntities -- Complete --> registrationFinalise

    end

    newUserReady -- Notify user --> userSignIn

    subgraph azure
        addAzureAccount[Add guest account to tenant]
        authenticateUser[Authenticate User]
    end

    userAccountPending --> addAzureAccount
    addAzureAccount --> newUserReady
    registrationFinalise -- Completed --> userReadyForSde
    

Loading

This should demonstrate that:

1. A new (anonymous) user may see one registration form/page in the browser.
2. However, this creates a pair of SDE requests to be submitted and worked on.
3. Only one request can be worked on at a time - the CreateUserAccountRequest is intended to be created quicker than a full verification/background check on a user/organisation.
4. Only when the CreateUserAccountRequest has been completed can the researcher finally sign-in to the Mauro Data Explorer. They can then at least review the status of the second, linked request and submit further information or answer questions to complete the registration process.

Request Status Workflow

The benefits of re-using the SDE request management system are that:

1. All states and transitions required for executing these requests are already present, particularly around governance/review.
2. The SDE and Mauro Data Explorer already support the features for tracking requests and for the (intended) researcher and SDE administrators to converse and supply further information.

There are some drawbacks too:

1. Since the requests are initially submitted as "anonymous users", the new user will not receive any updates on their request until the CreateUserAccountRequest is created. There are possibilities to add email notifications into the SDE for a later iteration which may help. For now this drawback is accepted and the compromise for getting a user account as efficiently as possible.

Registration Form

The user will be presented with a form to fill in, with the following options:

Personal Details

• Name - first and last
• Email - their organisation email address, which will act as their username too
• Telephone
• Position - or job title of this person in this organisation/department
• Additional information - optional fields such as "ORCID", "GMC Number", "NMC Number" etc
• Confirmations - e.g. acceptance of terms and conditions, legal confirmations etc

These details will be used to create their user account in the SDE.

Organisation

Either:

• Organisation - select existing organisation from a known list

Or, if one must be created:

• Organisation name - including the legal name (which may be different)
• Organisation website
• Country of registration
• Organisation type - e.g. University, Healthcare technology company, etc
• Small-Medium Business (SMB) status

Note: these conditional fields will help determine the secondary request to follow CreateUserAccountRequest.

If creating a new Organisation, the personal details listed above will be used to add this person as the first "approver" member of this Organisation.

Department

Either:

• Department - select existing department from a known list. Conditional on the pre-selected organisation chosen above.

Or, if one must be created:

• Department name

Note: these conditional fields will help determine the secondary request to follow CreateUserAccountRequest.

If creating a new Department, the personal details listed above will be used to add this person as the first member of this Department.

Note: the user will also automatically become a "researcher" member of the parent Organisation.

[pjmonks]

Marking this as "Open PR". It does not technically have a pull request, but it signifies that these planning notes are defined and open to review if anyone else has suggestions. However, I believe this covers the planned implementation we want to achieve.

[pjmonks]

Considering this plan as complete now. The related issues have been added to GitHub to implement this plan.