Tasque (Task Queue) is a simple workload manager for single-node usage that can be used out-of-the-box. It is resource-aware (e.g. CPU, Memory, GPU, Video Memory), and can automatically schedule the submitted jobs in a sensible way. It is much more light-weight compared to cluster workload managers such as Slurm and PBS, while being much smarter than a casually rushed script using tmux, screen, or alike. Tasque is written by the author for scheduling batches of machine learning experiments (e.g. caffe, pytorch, tensorflow).
Tasque has the following characteristics:
- Submitted jobs (command lines) will be automatically scheduled to run when there is enough resource to do so.
- The daemon is resource-aware, namely it is able to plan the usage of either CPU, Memory, GPU, or Video memory. When resource is aboundant, tasks may be scheduled to run in parallel.
- The default behavior is to execute the given commands in the FIFO order.
- Tasks with high priority values will be scheduled to run prior to the rest.
- The queue is stored in an SQLite3 database, and will not be lost in case of powerloss.
- Users can assign text annotations with the tasks in database.
- Requires no configuration and can be used out-of-the-box.
This tools is available on Pypi. Just issue the following command:
pip3 install tq1
Note that some new language features may be used in the code.
There is no plan to support older versions of python3.
Hence python3 >= 3.8 is recommended.
In case you want to install it from source:
python3 setup.py install
Copyright (C) 2016-2021 Mo Zhou <[email protected]>
Released under the MIT/Expat License.