Skip to content
This repository has been archived by the owner on Nov 30, 2022. It is now read-only.

B/add different job managers #674

Closed
wants to merge 8 commits into from

Conversation

Alexander-Dubrawski
Copy link
Collaborator

@Alexander-Dubrawski Alexander-Dubrawski commented Jul 8, 2020

Resolves #664

Does your pull request solve a problem? Please describe:
Like in hyrise/hyrise#2180 described we can not easily load and delete tables while asking at the same time for the meta information. In my opinion, a simple solution for the moment is to stop requesting the meta-information while loading/ deleting the tables. For that reason, I split the jobs in the database object into three categories:

  • background jobs (that are executed every n second. For example getting the system data)
  • asynchronous jobs (that are executed once but the client don't need an answer. For example load/delete tables)
  • synchronous jobs (that are executed once and the client needs an answer. For example getting the plugins)

All the background jobs are executed with the Background scheduler (https://apscheduler.readthedocs.io/en/stable/index.html). All the asynchronous jobs are executed with simple python threads (https://docs.python.org/3/library/threading.html#module-threading). Every time we are loading/deleting the tables we are stopping the Background scheduler (We are getting al the meta-information with the background jobs). If the tables are loaded/deleted we are starting the Background scheduler again.

Does your pull request add a feature? Please describe:
To implement the solution and keep the coherence small I wrote three job handlers:

  • background job handler
  • asynchronous job handler
  • synchronous job handler

All the handlers are defined in their own classes. The database object has an object from every manager and delegates the requests to the right one.

image

Affected Component(s):
DatabaseManager, Database, jobs

Additional context:
Because this is a bigger change in the architecture of the database object I decided that it would be beneficial to discuss it in a draft first.


def close(self) -> None:
"""Close background scheduler."""
self._scheduler.shutdown()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we stop background jobs as well before stopping the scheduler?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default the scheduler should shutdown all its jobs (and waits for there execution to be finished).

https://apscheduler.readthedocs.io/en/stable/userguide.html#shutting-down-the-scheduler

@Alexander-Dubrawski
Copy link
Collaborator Author

Implemented in #752 #757 and #756 .

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backend bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Missing table when loading the benchmark
2 participants