Evolution mutualize

Goals

• don't have one general queue for all the projects => one project can't block all the other projects
• don't generate the queue directly from the web UI
• add retry possibility
• be able to have a task list for the project
• be able to have a "report" of a task
• be able to retry all the failed tiles of a task
• be able to support OGC API Process https://ogcapi.ogc.org/processes/

Structure

classDiagram
class projects
projects: string

class projects_index
projects_index: int

class task_-project-
task_-project-: str cmd
task_-project-: str created_date
task_-project-: str status
task_-project-: int queue_id

class task_-project-_index
task_-project-_index: int 

class queue_-id-
class error_-id-

projects_index --> projects: Last project run
projects --> task_-project-: Tasks of the project[n]
task_-project-_index --> task_-project-: Last queue run
task_-project- --> queue_-project-_-id-: Queue of the task
task_-project- --> error_-project-_-id-: List of tiles in error

Loading

flowchart TD
    p(projects list-str)
    pi(projects_index int)
    t(tasks_.project. list-task)
    ti(tasks_.project._index int)
    q(queue_-project-_.id.)
    e(errors_-project-_.id.)
    pi-->p
    ti-->t
    p-- n -->t
    t-- n -->q
    t-- n -->e

Loading

An list

Task

cmd: command or description of the task
created_date: Creation date of the task
status: (creating|progress|error|terminated)
quue_id: ID if the related queue

Slave

while true:
  inc projects_index  
  get value of projects_index
  get project name on project[projects_index]
  inc tasks_<project>_index
  is the queue generated:
    generate the queue
  else:
    get queue_id on tasks_<project>[tasks_<project>_index]
    get queue on queue[queue_id]
    get item on queue
    process item
    manage xclame
    put error on task_error_<queue_id>