-
Notifications
You must be signed in to change notification settings - Fork 247
Architecture
In a glow program, datasets are linked together by computation steps. Each dataset can be partitioned into dataset shards. Each computation step can also be partitioned into computation tasks.
Glow makes full use of Golang's channel feature.
In local mode, data are fed into computation tasks by input channel(s), and output also via a channel to a new dataset.
In distributed mode, a group of tasks will run together in a server, pulling its own input data from remote dataset shards. The output are streamed to local disk, which will be pulled by downstream task groups. All these plumbing work are hidden from the computation, since the tasks only read from inputs and write to output via common go channels.
Here are the components in distributed mode. Master, Agents, Driver and TaskGroup.
Master collects resource infromation from agents. Resources are CPU, memory, allocated CPUs, allocated memory, etc.
Currently there is only a single master. But since the master only has soft states from agents, we can easily extend it to a master cluster to avoid SPOF.
An agent wears several hats:
- reports system resources and usage to master via heartbeats.
- accepts tasks from driver program.
- fetches the binary executable from driver program.
- starts the executable in task mode.
- accepts remote reads and local writes for a dataset.
Driver program is actually just the code that glow users write. If executed with "-glow" option, it will drive the distributed execution.
- creates optimized execution plan, grouping tasks into task groups.
- requests resources from master.
- allocates tasks to assigned servers.
- if a dataset has input or output channels, it writes to or receives from those datasets.
- after the flow finishes, it cleans up intermediate datasets generated during run time.
Tasks usually can be grouped together. A taskgroup program also uses the same binary executable file as the driver program, but in task mode.
- setup inputs and outputs for the tasks.
- execute the tasks.
One of the Golang's lacking feature is the capability to dynamically move execution closure across network. AFAIK it will not be possible any time soon. Given current situation, Glow is designed to just move the whole binary code, but run in different modes, i.g., task mode and driver mode.
However, the implication is that the computation flow will be static. The flow graph can not be changed. One future way to allow dynamic flow is to pre-register all the flows, and dynamically choose one or several flows to run.