Skip to content

Commit

Permalink
Futures executor working done and running (cooperative-computing-lab#…
Browse files Browse the repository at this point in the history
…3929)

* Futures executor working done and running

* lint

* update futures doc
  • Loading branch information
BarrySlyDelgado authored and colinthomas-z80 committed Oct 15, 2024
1 parent 13cfd43 commit 6a81b9e
Show file tree
Hide file tree
Showing 5 changed files with 46 additions and 5 deletions.
22 changes: 21 additions & 1 deletion doc/manuals/taskvine/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1648,7 +1648,7 @@ a Future object. The result of the task can retrieved by calling `future.result(
a = m.submit(my_sum, 3, 4)
b = m.submit(my_sum, 5, 2)
c = m.submit(my_sum, a, b) # note that the futures a and b are
# a passed as any other argument.
# passed as any other argument.

print(c.result())
```
Expand All @@ -1675,6 +1675,26 @@ can be tailored as any other task:
print(f.result())
```

Additionally, the executor the Vine Factory to submit TaskVine workers.
Specifications for the workers can be provided via the `opts` keyword argument when creating to executor.

=== "Python"
```python
import ndcctools.taskvine as vine

def my_sum(x, y):
return x + y

opts = {"memory": 8000, "disk":8000, "cores":8, "min-workers": 5}
m = vine.FuturesExecutor(manager_name='my_manager', batch_type="condor", opts=opts)

t = m.future_task(my_sum, 3, 4)
t.set_cores(1)

f = m.submit(t)

print(f.result())

Instead of tasks, the futures may also executed using [function calls](#serverless-computing) with the `future_funcall` method:

=== "Python"
Expand Down
8 changes: 4 additions & 4 deletions taskvine/src/bindings/python3/ndcctools/taskvine/futures.py
Original file line number Diff line number Diff line change
Expand Up @@ -246,15 +246,15 @@ def cancelled(self):
return False

def running(self):
state = self._task._module_manager.task_state(self._task.id)
if state == cvine.VINE_TASK_RUNNING:
state = self._task.state
if state == "RUNNING":
return True
else:
return False

def done(self):
state = self._task._module_manager.task_state(self._task.id)
if state == cvine.VINE_TASK_DONE:
state = self._task.state
if state == "DONE" or state == "RETRIEVED":
return True
else:
return False
Expand Down
9 changes: 9 additions & 0 deletions taskvine/src/bindings/python3/ndcctools/taskvine/task.py
Original file line number Diff line number Diff line change
Expand Up @@ -574,6 +574,15 @@ def category(self):
def command(self):
return cvine.vine_task_get_command(self._task)

##
# Get the state of the task.
# @code
# >>> print(t.command)
# @endcode
@property
def state(self):
return cvine.vine_task_get_state(self._task)

##
# Get the standard output of the task. Must be called only after the task
# completes execution.
Expand Down
7 changes: 7 additions & 0 deletions taskvine/src/manager/taskvine.h
Original file line number Diff line number Diff line change
Expand Up @@ -468,6 +468,13 @@ also written to that directory.

int vine_task_set_monitor_output(struct vine_task *t, const char *monitor_output);

/** Get the state line of the task.
@param t A task object.
@return a string of the task's state.
*/

const char *vine_task_get_state(struct vine_task *t);

/** Get the command line of the task.
@param t A task object.
@return The command line set by @ref vine_task_create.
Expand Down
5 changes: 5 additions & 0 deletions taskvine/src/manager/vine_task.c
Original file line number Diff line number Diff line change
Expand Up @@ -794,6 +794,11 @@ const char *vine_task_get_hostname(struct vine_task *t)
return t->hostname;
}

const char *vine_task_get_state(struct vine_task *t)
{
return vine_task_state_to_string(t->state);
}

#define METRIC(x) \
if (!strcmp(name, #x)) \
return t->x;
Expand Down

0 comments on commit 6a81b9e

Please sign in to comment.