Docs

Docs for tickers, graceful shutdown

README.md

@@ -65,7 +65,7 @@ Build your microservices on top of a `Connector` construct and use its simple AP
 * Client stubs
 * [Live integration tests](./docs/blocks/integration-testing.md)
 * [OpenAPI](./docs/blocks/openapi.md)
-* Graceful shutdown
+* [Graceful shutdown](./docs/blocks/graceful-shutdown.md)
 * [Distributed caching](./docs/blocks/distrib-cache.md)
 * Linked static resources
 * Recurring jobs

docs/blocks/distrib-tracing.md

@@ -1,7 +1,7 @@
 # Distributed Tracing
 
 Distributed tracing enables the visualization of the flow of function calls across microservices and processes. Without distributed tracing, it's extremely challenging to troubleshoot a distributed system.
-`Microbus` leverages [Jaeger](https://www.jaegertracing.io) and [OpenTelemetry](https://opentelemetry.io) to automatically create and collect tracing spans for executions of endpoints, tickers and callbacks of all microservices and visualize them as a single stack trace.
+`Microbus` leverages [Jaeger](https://www.jaegertracing.io) and [OpenTelemetry](https://opentelemetry.io) to automatically create and collect tracing spans for executions of endpoints, [tickers](../blocks/tickers.md) and callbacks of all microservices and visualize them as a single stack trace.
 
 ## Configuration
 

docs/blocks/graceful-shutdown.md

@@ -0,0 +1,14 @@
+# Graceful Shutdown
+
+Microservices typically run on hardware that can shut down at any time. When a microservice receives the `SIGTERM` command to shut down, it stops accepting new operations and attempts to end all pending operations gracefully.
+
+Graceful shutdown process:
+* Disable tickers so new iterations are not run
+* Stop accepting new requests
+* Wait 8 seconds for running tickers, pending requests and goroutines to end naturally
+* Cancel the lifetime `context.Context`
+* Wait 4 more seconds for running tickers and pending requests, and goroutines to end
+* Close the NATS connection
+* Exit
+
+In order for goroutines to be gracefully shut down, it is important to launch them using the `Connector`'s `Go` method rather than using the standard `go` directive. When launched with `Go`, the goroutines have the `context.Context` that gets canceled during shut down.

docs/blocks/integration-testing.md

@@ -156,7 +156,7 @@ res, err := Hello_Get(t, ctx, "")
 
 ### Testing Tickers
 
-Tickers are disabled during testing in order to avoid the unpredictability of their running schedule. Instead, tickers can be tested manually like other endpoints. Tickers don't take arguments nor return values so the only testing possible is error validation.
+[Tickers](../blocks/tickers.md) are disabled during testing in order to avoid the unpredictability of their running schedule. Instead, tickers can be tested manually like other endpoints. Tickers don't take arguments nor return values so the only testing possible is error validation.
 
 ```go
 func TestHello_TickTock(t *testing.T) {

docs/blocks/layers.md

@@ -70,9 +70,9 @@ The [`Connector`](../structure/connector.md) is the base class of all microservi
 
 The [distributed cache](../blocks/distrib-cache.md) is an in-memory cache that is shared among the replica peers of microservice.
 
-Similar to cron jobs, __tickers__ run jobs on a scheduled interval. 
+[Tickers](../blocks/tickers.md) are jobs that run on a recurring basis.
 
-Microservices are __shutdown gracefully__. All pending requests, goroutines and jobs are drained before the microservice quits.
+Microservices are [shutdown gracefully](../blocks/graceful-shutdown.md). All pending requests, goroutines and jobs are drained before the microservice quits.
 
 Images, scripts, templates and any other __static resources__ are made available to each microservice by association of a file system (FS).
 

docs/blocks/tickers.md

@@ -0,0 +1,26 @@
+# Tickers
+
+Tickers are background jobs that need to run irrespective of an external trigger. Tickers have a fixed schedule, with consecutive iterations separated by a fixed interval. If execution of a prior iteration takes longer than the duration of the interval and has not yet completed by the time the next iteration is due, the new iteration is skipped.
+
+Tickers run independently for each replica of the microservice, so a once an hour ticker will run 5 times an hour if 5 replicas are running concurrently. To avoid coordinated spikes of multiple tickers all running at the same time, iterations are aligned to the startup time of the microservice, not to clock time. 
+
+Tickers are defined using the `Connector`s `StartTicker` method which accepts a name, an interval and a callback function (handler). If the microservice is running, the ticker activates immediately. Otherwise, it will activate when the microservice starts. `StopTicker` can be used to stop a ticker identified by its name.
+
+The more common case is to define tickers in `service.yaml` and use the [code generator](../blocks/codegen.md) to generate the skeleton code.
+
+```yaml
+# Tickers
+#
+# signature - Go-style method signature (no arguments)
+# Ticker()
+# description - Documentation
+# interval - Duration between iterations (e.g. 15m)
+tickers:
+ # - signature:
+ # description:
+ # interval:
+```
+
+Tickers are disabled in the `TESTING` [deployment environment](../tech/deployments.md) in order to avoid the unpredictability of their running schedule.
+
+If a job is running at the time that the microservice shuts down, the `ctx` argument of the handler gets canceled and the job is given a grace period to end cleanly.

docs/general/milestones.md

@@ -227,3 +227,10 @@ Note: These milestones are maintained in separate branches in order to demonstra
 [Milestone 28](https://github.com/microbus-io/fabric/tree/milestone/28):
 
 * Remove dependency on Zap logger and replaced with standard slog
+
+[Milestone 29](https://github.com/microbus-io/fabric/tree/milestone/29):
+
+* Documentation
+* Fixed data race in metrics collection
+* Added `connector.StopTicker`
+* Fixed deadlock when running tickers

docs/general/mission-statement.md

@@ -118,7 +118,7 @@ Benchmarks indicate `Microbus` is capable of processing upward of 94,500 req/sec
 
 ### Reliability
 
-Reliable communication is an imperative quality of any distributed system. In `Microbus`, microservices communicate with each other over a messaging bus. Each microservice connects to the bus over a persistent bi-directional multiplexed connection that is monitored constantly and kept alive with automatic reconnects if required. Locality-aware routing, an ack of fail fast pattern and graceful shutdowns further enhance the reliability of communications.
+Reliable communication is an imperative quality of any distributed system. In `Microbus`, microservices communicate with each other over a messaging bus. Each microservice connects to the bus over a persistent bi-directional multiplexed connection that is monitored constantly and kept alive with automatic reconnects if required. Locality-aware routing, [ack of fail fast](../blocks/ack-or-fail.md) and [graceful shutdowns](../blocks/graceful-shutdown.md) further enhance the reliability of communications.
 
 It is also imperative that a distributed system remains online at all times. `Microbus` achieves that by capturing all errors and panics so that malfunctioning microservices do not crash.
 

docs/structure/connector.md

@@ -8,7 +8,7 @@ The `Connector` is the most fundamental construct of the framework, providing ke
 * HTTP-like communication over NATS, [unicast](../blocks/unicast.md) (request/response) and [multicast](../blocks/multicast.md) (pub/sub)
 * JSON logger
 * [Configuration](../blocks/configuration.md)
-* Tickers to execute jobs on a recurring basis
+* [Tickers](../blocks/tickers.md) to execute jobs on a recurring basis
 * [Distributed cache](../blocks/distrib-cache.md)
 * Bundled resource files and strings
 * [Distributed tracing](../blocks/distrib-tracing.md)

docs/tech/service-yaml.md

@@ -243,7 +243,7 @@ func (svc *Service) WebHandler(w http.ResponseWriter, r *http.Request) (err erro
 
 ## Tickers
 
-Tickers are means to invoke a function on a periodic basis. The `signature` and `interval` fields are required.
+Tickers are means to trigger a job on a periodic basis. The `signature` and `interval` fields are required.
 
 ```yaml
 # Tickers
@@ -269,6 +269,8 @@ func (svc *Service) TickerHandler(ctx context.Context) (err error) {
 }
 ```
 
+`ctx` will be canceled if when the microservice shuts down.
+
 ## Metrics
 
 The `metrics` section is used to define custom metrics, whether operational in nature (e.g. performance) or having a business purpose (e.g. usage tracking).