Skip to content
This repository has been archived by the owner on Oct 8, 2022. It is now read-only.

Precompute and store visualization data object #7

Closed
moreirathomas opened this issue Jan 7, 2022 · 1 comment
Closed

Precompute and store visualization data object #7

moreirathomas opened this issue Jan 7, 2022 · 1 comment
Assignees
Labels
invalid This doesn't seem right must have

Comments

@moreirathomas
Copy link
Member

moreirathomas commented Jan 7, 2022

Once the report has been received from runner, we need to process it asap to produce vizualisation data (average, median, other stats,...). We need to parallelize this process. One possible solution is to send the work to a remote worker.

Possible infrastructure:
worker infra

This implies:

  • a nosql datastore to store the raw report waiting to be processed (redis, elasticsearch, memcached, some cgp alternative ?)
  • a cloud worker capable of retrieving the raw data, processing it and storing the result in a clean database
  • a sql database to store the processed vizualisation data
@moreirathomas moreirathomas changed the title Thinl Think about how to precompute visualization data object Jan 7, 2022
@moreirathomas moreirathomas changed the title Think about how to precompute visualization data object Precompute and store visualization data object Feb 1, 2022
@moreirathomas moreirathomas self-assigned this Feb 4, 2022
@moreirathomas
Copy link
Member Author

Computing and storing of process stats is done by the worker. It lives in its own repo.
See benchttp/worker#2 and benchttp/worker#5.

@moreirathomas moreirathomas added the invalid This doesn't seem right label Feb 4, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
invalid This doesn't seem right must have
Projects
None yet
Development

No branches or pull requests

1 participant