Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved the documentation regarding scheduler plugins (fixes #8719) #8729

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 33 additions & 1 deletion docs/source/plugins.rst
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,14 @@ Accessing Full Task State

If you would like to access the full :class:`distributed.scheduler.TaskState`
stored in the scheduler you can do this by passing and storing a reference to
the scheduler as so:
the scheduler. This can be done in one of two ways.

Firstly, you can use the ``dask_setup`` function and run the scheduler with
``dask scheduler --preload <filename.py>``:

.. code-block:: python

import click
from distributed.diagnostics.plugin import SchedulerPlugin

class MyPlugin(SchedulerPlugin):
Expand All @@ -83,6 +87,34 @@ the scheduler as so:
plugin = MyPlugin(scheduler)
scheduler.add_plugin(plugin)

You can also use the :meth:`distributed.diagnostics.plugin.SchedulerPlugin.start` method, which will receive a reference to the scheduler:

.. code-block:: python

from distributed.diagnostics.plugin import SchedulerPlugin

class MyPlugin(SchedulerPlugin):
def __init__(self):
self.scheduler = None

def start(self, scheduler):
self.scheduler = scheduler

def transition(self, key, start, finish, *args, **kwargs):
# Get full TaskState
ts = self.scheduler.tasks[key]

if __name__ == '__main__':
# Create and register the plugin in your main code
from distributed import Client, LocalCluster

cluster = LocalCluster()
client = Client(cluster)

plugin = MyPlugin()
client.register_plugin(plugin)


Built-In Scheduler Plugins
--------------------------

Expand Down
Loading