-
Notifications
You must be signed in to change notification settings - Fork 28
Proposed Consistent Cut Changes
Samantha Chan edited this page Jan 9, 2015
·
3 revisions
The Consistent Cut feature of Streams 3.2.2 helps support at-least-once tuple processing. Follow this link to the knowledge page center for details.
Operators in a consistent region need to handle
- drain (used to submit any pending tuples in preparation for checkpoint)
- checkpoint (used to save operator state, if any)
- reset (to recover to a consistent state) & resetToInitialState
Source operators need to make use of permits to ensure they are recoverable.
This page gives a high-level description of how the operators in this toolkit will be modified for Consistent Cut.
###InetSource
InetSource will be disallowed in a consistent region. It pulls state from an external source, and there is no way to replay that state.
HTTPGetStream will be disallowed in a consistent region.
HTTPPost cannot be placed at the start of a consistent region. It has no internal state and therefore does not implement the state handling API