spark-sql-streaming-TriggerExecutor.adoc

TriggerExecutor

TriggerExecutor is the interface for trigger executors that StreamExecution uses to execute a batch runner.

Note	Batch runner is an executable code that is executed at regular intervals. It is also called a trigger handler.

package org.apache.spark.sql.execution.streaming

trait TriggerExecutor {
  def execute(batchRunner: () => Boolean): Unit
}

Note	StreamExecution reports a IllegalStateException when TriggerExecutor is different from the two built-in implementations: OneTimeExecutor or ProcessingTimeExecutor.

Table 1. TriggerExecutor’s Available Implementations

TriggerExecutor	Description
OneTimeExecutor	Executes batchRunner exactly once.
ProcessingTimeExecutor	Executes batchRunner at regular intervals (as defined using ProcessingTime and DataStreamWriter.trigger method). ProcessingTimeExecutor(processingTime: ProcessingTime, clock: Clock = new SystemClock()) Note Processing terminates when batchRunner returns false.

notifyBatchFallingBehind Method

Caution

FIXME