Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Futures executor working done and running #3929

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion doc/manuals/taskvine/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1648,7 +1648,7 @@ a Future object. The result of the task can retrieved by calling `future.result(
a = m.submit(my_sum, 3, 4)
b = m.submit(my_sum, 5, 2)
c = m.submit(my_sum, a, b) # note that the futures a and b are
# a passed as any other argument.
# passed as any other argument.

print(c.result())
```
Expand All @@ -1675,6 +1675,26 @@ can be tailored as any other task:
print(f.result())
```

Additionally, the executor the Vine Factory to submit TaskVine workers.
Specifications for the workers can be provided via the `opts` keyword argument when creating to executor.

=== "Python"
```python
import ndcctools.taskvine as vine

def my_sum(x, y):
return x + y

opts = {"memory": 8000, "disk":8000, "cores":8, "min-workers": 5}
m = vine.FuturesExecutor(manager_name='my_manager', batch_type="condor", opts=opts)

t = m.future_task(my_sum, 3, 4)
t.set_cores(1)

f = m.submit(t)

print(f.result())

Instead of tasks, the futures may also executed using [function calls](#serverless-computing) with the `future_funcall` method:

=== "Python"
Expand Down
8 changes: 4 additions & 4 deletions taskvine/src/bindings/python3/ndcctools/taskvine/futures.py
Original file line number Diff line number Diff line change
Expand Up @@ -248,15 +248,15 @@ def cancelled(self):
return False

def running(self):
state = self._task._module_manager.task_state(self._task.id)
if state == cvine.VINE_TASK_RUNNING:
state = self._task.state
if state == "RUNNING":
return True
else:
return False

def done(self):
state = self._task._module_manager.task_state(self._task.id)
if state == cvine.VINE_TASK_DONE:
state = self._task.state
if state == "DONE" or state == "RETRIEVED":
return True
else:
return False
Expand Down
9 changes: 9 additions & 0 deletions taskvine/src/bindings/python3/ndcctools/taskvine/task.py
Original file line number Diff line number Diff line change
Expand Up @@ -574,6 +574,15 @@ def category(self):
def command(self):
return cvine.vine_task_get_command(self._task)

##
# Get the state of the task.
# @code
# >>> print(t.command)
# @endcode
@property
def state(self):
return cvine.vine_task_get_state(self._task)

##
# Get the standard output of the task. Must be called only after the task
# completes execution.
Expand Down
7 changes: 7 additions & 0 deletions taskvine/src/manager/taskvine.h
Original file line number Diff line number Diff line change
Expand Up @@ -468,6 +468,13 @@ also written to that directory.

int vine_task_set_monitor_output(struct vine_task *t, const char *monitor_output);

/** Get the state line of the task.
@param t A task object.
@return a string of the task's state.
*/

const char *vine_task_get_state(struct vine_task *t);

/** Get the command line of the task.
@param t A task object.
@return The command line set by @ref vine_task_create.
Expand Down
5 changes: 5 additions & 0 deletions taskvine/src/manager/vine_task.c
Original file line number Diff line number Diff line change
Expand Up @@ -794,6 +794,11 @@ const char *vine_task_get_hostname(struct vine_task *t)
return t->hostname;
}

const char *vine_task_get_state(struct vine_task *t)
{
return vine_task_state_to_string(t->state);
}

#define METRIC(x) \
if (!strcmp(name, #x)) \
return t->x;
Expand Down
Loading