DevGuide CellStates

In the typical steady-state, a Workflow converges to an ordered sequence of Cell objects in the DONE state. The Result object associated with each cell describes the effects (messages and state updates) of running the computation described by the associated Module on the state produced by running each of the preceding cells in sequential order.

Because code execution is not instantaneous (or always successful), cells reach this point thorough the following state diagram:

Clone or Thaw Cell                           Freeze Cell
         \                                        |
          v                                       V
    --- WAITING --------------------> DONE     FROZEN
   /       |                           ^
  /    ,---+            ,-> CANCELLED  |
  |   /    v           /          ^    |
  |  |  BLOCKED ---+--+--> ERROR  |    |
  |   \..,  |     /          ^    |    /
   \     V  v   ,/           |    |   /
    `--> STALE --> RUNNING --+----+--`
           ^
          /
  New Cell (or Clone cell in the ERROR state)

Cell execution and this state transition table is managed by the Scheduler. Cell states, and their Result are described as follows.

We make a distinction between pending states (WAITING, BLOCKED, STALE, RUNNING) and final states (DONE, CANCELLED, ERROR, FROZEN)

• WAITING: The state of the cell's Result is unknown. It has one, but a preceding cell is still running and we don't have enough information to guarantee that it won't invalidate this cell's Result. Accordingly resultId may or may not be valid. If this cell transitions to the DONE state without going through the BLOCKED or STALE states, resultId will remain unchanged. The cell needs to be re-executed if provenance data in resultId is stale or if resultId is None.

• BLOCKED: The cell does not have a valid Result object, either because it is freshly inserted, or because an upstream dependency of the cell was updated. Furthermore, the cell can not currently be scheduled because an upstream cell may still produce a result that this cell depends on. resultId must be ignored, and the cell always needs to be re-executed.

• STALE: The cell does not have a valid Result object, either because it is freshly inserted, or because an upstream dependency of the cell was updated. This cell will be re-evaluated by the Scheduler when the opportunity presents itself. When the cell is in this state resultId must be ignored, and the cell always needs to be re-executed.

• RUNNING: The cell has an incomplete Result object. When the cell is in this state resultId should be ignored, except for notification purposes (e.g., displaying cell outputs as they arrive).

• ERROR: The cell was in the RUNNING state at one point, but failed to evaluate completely (e.g., due to a typo in a Python cell). resultId contains a valid Result that describes the error. There is no provenance data in resultId and the cell always needs to be re-executed.

• CANCELLED: The cell was in a pending state when the workflow was aborted OR the cell was in a WAITING, BLOCKED, or STALE state when a preceding cell entered into the ERROR state. resultId, if Some(_), contains the Result of the last successful execution of the cell. This cell needs to be re-executed if resultId is None or the associated provenance data is stale.

• DONE: The computation described by the cell's Module completed successfully. resultId references the result of the execution. A cell in this state does not need to be re-executed.

• FROZEN: This cell has been temporarily removed from the workflow by user-request. Execution ignores this cell and resultId references the Result from the most recent execution. Cells in this state are never re-executed.

In summary:

State	Result	Reusable?	Pending?	Default Clone
WAITING	Tentative	Maybe [1]	Yes, Not Schedulable	WAITING
BLOCKED	Useless	No	Yes, Not Schedulable	WAITING
STALE	Useless	No	Yes, Schedulable	STALE
RUNNING	Incomplete	Maybe [2]	Yes	STALE
ERROR	Complete	No	No	STALE
CANCELLED	Tentative	Maybe [1]	No	WAITING
DONE	Complete	Yes	No	DONE
FROZEN	Complete	Maybe [3]	No	FROZEN

Notes:

1. Cells in the WAITING or STALE states may have reusable results, but this is a non-local decision. See Scheduler.updateCellState
2. Cells in the RUNNING state have invalid results until cell execution completes (whether successfully or not). The result may be reusable once the cell enters the done state.
3. Although FROZEN cells are never executed, it may be possible to re-use the cell state when the cell is thawed.

The above table describes default cell behavior when the workflow is modified. Each modification deviates from this behavior as follows:

• append: The new cell enters in the STALE state.
• insert: The new cell enters in the STALE state. All DONE cells after the insertion point move to the WAITING state.
• delete: All DONE cells after the insertion point enter the WAITING state.
• update: The updated cell moves to the STALE state. All DONE cells after the insertion point move to the WAITING state.
• freezeOne: The frozen cell moves to the FROZEN state. All DONE cells after the modification point move to the WAITING state. If the frozen cell is in a state with a non-reusable result, the result is deleted.
• thawOne: The thawed cell moves to the WAITING state. All DONE cells after the modification point move to the WAITING state.
• freezeFrom: All cells at and after the modification point move to the FROZEN state. Any cells being frozen in states with a non-reusable results have their results deleted.
• thawFrom: All cells at and after the modification point move to the WAITING state.