Chart: Start Survey (with Channel Broker)

The following flowchart describes what's happening after someone clicks the "Start Survey" button.

More details are available below the flowchart!

flowchart TD
    START([Start Survey])
    START --> Ask.Runtime.SurveyBroker

    subgraph Ask.Runtime.SurveyBroker
        POLL([poll/0]) -- every minute --> POLL
        POLL --for each project--> RETRIES
        CHANNELS_UP{All required\nchannels up?}
        RETRIES[Retries N failed respondents]
        LOAD_BATCH[Load N active respondents]
        POLL --for each active survey--> CHANNELS_UP
        CHANNELS_UP --no--> POLL
        CHANNELS_UP --yes--> LOAD_BATCH
    end

    RETRIES --> Ask.Runtime.Session
    LOAD_BATCH --> Ask.Runtime.Session

    subgraph Ask.Runtime.Session
        SESS_CONTACT[contact_respondent/2]
    end

    SESS_CONTACT --mode=ivr--> CHBRK_SETUP
    SESS_CONTACT --mode=sms--> CHBRK_ASK

    subgraph Ask.Runtime.ChannelBroker
        CHBRK_SETUP([setup/5])
        CHBRK_ASK([ask/5])
        FREE_SLOT{channel is\nunder capacity?}
        CHBRK_SETUP-->FREE_SLOT
        CHBRK_ASK-->FREE_SLOT
        FREE_SLOT--no-->QUEUE
        FREE_SLOT--yes-->CONTACT[make contact]

        CALLBACK([callback_received/4])-->CAN_DEQUEUE{can\ndequeue?}
        QUEUE[(Queue)]-.-CAN_DEQUEUE
    
        GC([collect_garbage/2])
            --every N minutes
            -->CLEANUP[check idle contact status\nclear failed/finished contacts]
            -->CAN_DEQUEUE

        CLEANUP<-->CONTACT_STATUS[Ask.Runtime.Channel.message_inactive?/2]

        CAN_DEQUEUE--yes-->CONTACT
        CAN_DEQUEUE--no-->STOP(["stop"])

        EXPIRED{expired?}
        SCHEDULED{future\ncall?}
        EXPIRED--no-->SCHEDULED
        EXPIRED--yes-->REMOVE_CONTACT[deactivate contact]
        SCHEDULED--yes-->QUEUE
        SCHEDULED--no-->Ask.Runtime.VerboiceChannel.setup/5
        
        CONTACT--mode=ivr-->EXPIRED
        CONTACT--mode=sms-->Ask.Runtime.NuntiumChannel.ask/5
    end

Loading

Survey Broker

The survey-broker runs every minute.

1. It searches for failed respondents that must be recontacted, regardless of the project or survey (a project may be archived, a survey may be stopped: the retry will happen). For each project it fetches as many as "batch limit per minute" respondents.
2. It searches for active surveys (active state, currently in survey schedule window), and polls them in sequential order (no parallelism):
3. it activates up to "batch size" respondents in parallel at any time (10000 by default);
4. it starts up to "batch limit per minute" respondents (i.e. pushes them to the channel-broker);
5. It eventually stops surveys that expired.

Channel Broker

A channel-broker process is started on-demand for each channel that Surveda needs to interact with. Each channel-broker is responsible for communicating with the channel it manages, and owns the runtime-channel (see Ask.Runtime.Channel) so any communication with the runtime-channel service shall be served through its channel-broker process.

You can visualize the channel-broker as a dam between the survey-broker and the external. It limits how many messages can be sent at any time to the actual external services (Verboice, Nuntium, ...) which can be configured through the "channel capacity" setting. services. It activates contacts until the channel capacity is reached, then starts to enqueue contacts until the channel is under capacity (happens as we receive callbacks from the external services or through the regular GC calls), at which point it starts pushing contacts again, indefinitely.

The channel-broker also won't push expired contacts (that may be refused by the external service) and skips them, or contacts that are scheduled for later (e.g. tomorrow) and keeps them in queue until it's time to contact them. Let's note that as of 19 June 2023, this feature is only available for the IVR mode (Verboice). The survey-broker doesn't respect for SMS modes yet (see #2283).