Global Ratelimiting

Overview

Enable Ratelimit

One time setup to bring up required sidecars

1. Configure global settings for ratelimit sidecar. Can be over-written on a per-service basis.
   Following provides a sample configuration

# values.yaml with global settings
global:
  titanSideCars:
    imageRegistry: gcr.io/cloud-tools-images
    ratelimit:
      enabled: false # set to false to disable ratelimit globaly
      imageName: ratelimit
      imageTag: v1.4.0
      redisUrl: 172.18.0.20:6379

2. Enable ratelimit side-car on a per-service basis for services that require ratelimit

# service's values.yaml
titanSideCars:
  ratelimit:
    enabled: true # set to true to enable ratelimit for this service only

3. If service intends to ratelimit on payload attributes or token claims then it must also configure and enable OPA sidecar. See here for steps

---

Config Reference

Click to expand!

Ratelimit

Ratelimit sidecar settings. Merge over-writes global.titanSideCars.ratelimit

# titanSideCars.ratelimit.

    enabled:                    bool
    imageRegistry:              string
    imageName:                  string
    imageTag:                   string
    cpu:
      request:                  string
      limit:                    string
    memory:
      request:                  string
      limit:                    string
    ephemeralStorage:
      request:                  string
      limit:                    string
    livenessFailureThreshold:   integer
    readinessFailureThreshold:  integer

    logLevel:                   string
    redisUrl:                   string 

    # TODO other redis config items

enabled

(bool, default true) Set to false to disable ratelimit sidecar

imageRegistry

(string, optional) Docker image registry path used for ratelimit sidecar. Overrides titanSideCars.imageRegistry

imageName

(string, default ratelimit)

imageTag

(string, default latest)

cpu.request

(string, default 250m)

cpu.limit

(string, default 1)

memory.request

(string, default 256Mi)

memory.limit

(string, default 1Gi)

ephemeralStorage.request

(string, default 100Mi)

ephemeralStorage.limit

(string, default 500Mi)

livenessFailureThreshold

(integer, default 50)

readinessFailureThreshold

(integer, default 100)

logLevel

(string, default INFO)

redisUrl

(string, required)

---

PerRouteRatelimit

# titanSideCars.ingress.routes[].

  match:        RouteMatch
  ratelimit:
    enabled:    bool
    action:     []RatelimitAction

match

(RouteMatch, required)

ratelimit

(optional) Ratelimit policy for requests that match corresponding route definition.
If missing, the ratelimit template parser will totally skip processing of this route entry.

ratelimit.enabled

(bool, default true) Controls enforcement of supplied ratelimit actions

ratelimit.actions

([]RatelimitAction, required) One or more ratelimit actions on request that match corresponding route definiton. Request is matched against all actions and each matching action is evaluated independently.

---

RatelimitAction

# titanSideCars.ingress.routes[].ratelimit.actions[].

    match: []RatelimitMatch
    limit: string

match

([]RatelimitMatch, optional) List of descriptors that define the attributes to ratelimit on. Corresponding route definition is an implicit descriptor. Incoming request must match all descriptors to be considered for ratelimiting. The ordering of descriptors is not relevant.

limit

(string, required) Ratelimit value in <ratelimit-per-unit>/<unit> format. Supported units are second minute hour and day.

Example: 5/second, 50/minute, 100/hour, 10/day

Instead of supplying a hard-coded limit, limit can also refer to a key from titanSideCars.ratelimit.limits key-value pairs as in example below. This pattern allows for easy configuration of limits on a per environment basis.

Click to expand!

titanSideCars:
  ratelimit:
    limits:
      small: 100/day
  ingress:
    routes:
    - match: /
      ratelimit:
        actions:
        - limit: small

---

RatelimitMatch

# titanSideCars.ingress.routes[].ratelimit.actions[].descriptors[].

    key:    string

    # comparison operators
    eq:     string          # equals
    sw:     string          # starts-with
    ew:     string          # ends-with 
    co:     string          # contains
    lk:     string          # like
    pr:     bool            # present
    neq:    string          # not equals
    nsw:    string          # not starts-with
    new:    string          # not ends-with
    nco:    string          # not contains
    nlk:    string          # not like
    npr:    bool            # not present

key

(string, required) Attribute to rate-limit on. If no comparison operator is specified, ratelimit is performed on each unique value of the attribute. Key may refer to a header, token claim or an attribute from json payload.

A header may be referenced via header. notation. Example: header.x-request-id
A token claim may be referenced via token. notation. Example: token.sub.scope
A payload attribute may be referenced via payload. notation. Example: payload.userType

eq | sw | ew | co | lk | pr | neq | nsw | new | nco | nlk | npr

(oneof optional) Comparison operator. When specified, the ratelimit is performed on result of comparision.
For binary operators, the operator value indicates the right hand operand and should be a string.

The supported operators are

• eq/neq: Exact string match
• sw/nsw: String prefix match
• ew/new: String suffix match
• co/nco: Substring match
• lk/nlk: Regex match. Regex syntax is documented here
• pr/npr: Unary operator. Indicates if key is present or not. Only true value is used. Use pr to test presence and npr to test non presence.

---

Examples

Following examples guide through various ratelimit scenarios!

Example 1

Lets start with a simple example that ratelimits a matching route

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects
        method: POST
      ratelimit:
        actions:
        - limit: 100/minute   # supported units - second, minute, hour, day

Above configuration will limit all 'POST' requests starting with '/myapp/objects' to 100 per minute.

Example 2

Lets take a slightly more complex case where we ratelimit on each unique value of a header

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects
        method: POST
      ratelimit:
        actions:
        - match:
          - key: x-product  # may also say header.x-myapp-header. 'header.' is default
          limit: 100/minute

In above configuration we again target POST requests starting with /myapp/objects. But now limit will be applied independently to each unique value of the header x-product.
Header x-product with values prod1 and prod2 each will be independently allowed 100 times per minute

Example 3 (Deprecated)

• Attention:
  • Request body is no longer support due to performance issue
  • Will provide different solution later

We can also ratelimit on an attribute from payload. Lets look at an example

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects 
        method: POST
      ratelimit:
        actions:
        - match:
          - key: payload.email  # 'payload.' notation references payload attributes
          limit: 100/minute

Above config will trigger ratelimit on each unique email in the payload of POST requests starting with /myapp/objects. No ratelimiting will be triggered if payload does not contain the specified attribute.
A sample payload like {"name": "foo", "email": "[email protected]"} will trigger above ratelimit action

Example 4

In order to ratelimit on a claim inside token, please use enrich to expose claim to a header first, see following example

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects 
        method: POST
      enrich:
        actions:
        - action: extract
          from: token.jti  # 'token.' notation references token claims
          to: x-epmp-token-jti # jit claim is exposed to the http header "x-epmp-token-jti"
      ratelimit:
        actions:
        - match:
          - key: x-epmp-token-jti    
          limit: 100/minute

With above config in place, a token with a unique jti claim value will be ratelimited to 100 requests per minute for POST requests starting with /myapp/objects

Example 5

Lets dig deeper and look at a more complex ratelimit action with multiple match rules.

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects/
        method: GET
        headers:
        - name: x-product
          eq: myprod
      enrich:
        actions:
        - action: extract
          from: token.uri
          to: x-epmp-token-uri
      ratelimit:
        actions:
        - match:
          - key: x-epmp-token-uri
          limit: 100/minute

Route definition is implicitly part of every ratelimit action.

Example 6

We now pick up a ratelimit config with multiple ratelimit actions

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects
        method: POST
      enrich:
        actions:
        - action: extract
          from: token.scope
          if_contain: system
          to: x-epmp-token-scope
          with: system
      ratelimit:
        actions:
        - match:
          - key: x-epmp-token-scope
          limit: 100/minute
        - match:
          - key: x-epmp-token-scope
            eq: system
          limit: 5/minute
        - match:
          - key: x-epmp-token-scope
            eq: customer
          limit: 10/minute

Above targets POST requests starting with /myapp/objects and has two ratelimit actions. First action sets a limit of 100 per minute on each unique scope in the token. The second action sets a limit of 5 per minute if scope in payload equals system.
Hence scope domain will be allowed 100 times per minute each, but system will be restricted to 5 times per minute and customer will be restricted to 10 times per minute

When multiple actions are configured, each action is evaluated indendendently. The most constraining action gets enforced. The order of actions is immaterial.

Example 7

Lets look at an example where a request matches multiple route definitions

titanSideCars:
  ingress:
    routes:
    - match:
        prefix: /myapp/objects
        method: POST
      ratelimit:
        actions:
        - limit: 10/minute
    - match:
        prefix: /myapp/objects
      ratelimit:
        actions:
        - limit: 100/minute      

Route matching is processed sequentially, any action (e.g. ratelimiting) will be executed at the very first match. In the above example, the POST /myapp/objects will be rate limited to 10/minute.

Example 8

Enable to easier to override different limits for different environment

titanSideCars:
  ingress:
    ratelimit:
      limits:
        small: 10/minute
        medium: 100/munite
        large: 1000/minute
    routes:
    - match:
        prefix: /myapp/objects
      ratelimit:
        actions:
        - limit: medium     
    - match:
        prefix: /myapp/objects
        method: POST
      ratelimit:
        actions:
        - limit: small

With titanSideCars.ingress.ratelimit.limits approach, you can use global override to override proper limit per environment.

---