The Things Stack supports rate limiting for all outward facing services. Access to each resource is limited by a unique identifier key, and it is possible to define rate limiting classes for fine-grained control.
For more details about the algorithm used, read here.
Example configuration
The rate limiting configuration is split into multiple profiles. For each profile, we provide a name, the maximum allowed rate per minute, and optionally a maximum burst rate. Finally, we associate the profile with a number of rate limiting classes. Refer to the Rate Limited Entities section below for a list of all available rate limiting classes and what they mean.
Enable rate limiting by adding the following configuration to your ttn-lw-stack.yml.
The values shown below are only meant as an example. Make sure to adjust them accordingly, depending on the actual traffic of your deployment.
Limit number of HTTP requests by remote IP and request URL. {{ Server }} is oauth, account, gcs, interop, metrics, health, pprof or as:io:packages:azure:iotcentral respectively. {{ AuthInfo }} is either ip:{{ RemoteIP }} or token:{{ AuthTokenID }} if the caller presents authentication information.
Classes
http:{{ Server }}:{{ URLTemplate }}</code> <code>http:{{ Server }}</code> <code>http
Key
http:{{ Server }}:{{ AuthInfo }}:url:{{ URL }}
Example
http:oauth:ip:10.10.10.10:url:/authorize
Note:
Both gRPC methods and HTTP endpoints support multiple classes. This enables overriding the generic rate limits for specific methods and endpoints.
Warning:
When The Things Stack HTTP and gRPC endpoints are served behind a reverse proxy, the X-Forwarded-For header is respected for the remote IP. The IP address of the reverse proxy must be trusted by The Things Stack for this to work, see HTTP options and gRPC Options.
Warning:
When The Things Stack MQTT and UDP endpoints are served by a reverse proxy, the remote IP address seen by The Things Stack may not be correct. In this case, rate limiting for new MQTT connections (and UDP traffic) should be handled by the reverse proxy and disabled in The Things Stack.
Rate Limiting Actions
The following table describes how The Things Stack reacts when the maximum rate limit for a resource is exceeded.
Resource
Rate Limit Action
Expected Client Action
gRPC Requests
Return an error response of type ResourceExhausted, and set rate limiting headers.
The client should wait and then retry the gRPC request. The X-Rate-Limit-Retry header from the response can help in this case.
HTTP Requests
Return an HTTP 429 (Too Many Requests) error response, and set rate limiting headers.
The client should wait and then retry the HTTP request. The X-Rate-Limit-Retry header from the response can help in this case.
New MQTT Connections
Drop (reject) new connection.
The client should reconnect with a backoff.
UDP packets
Drop packet.
The client has no way of knowing whether the sent packet was received and/or processed, which is a limitation of the Legacy UDP packet forwarder.
Application Downlinks
Reject downlink message and terminate connection.
The client should reconnect, and reduce the downlink traffic load.
Gateway Uplinks
Reject uplink message and terminate connection.
The client should reconnect, and reduce the downlink traffic load.
Rate Limiting Headers
When rate limiting is enabled, the following headers are added to all HTTP and gRPC requests:
X-Rate-Limit-Limit: The maximum number of allowed requests for the current time period.
X-Rate-Limit-Available: Number of available requests for the current time period.
X-Rate-Limit-Reset: Seconds until the rate limiter resets.
X-Rate-Limit-Retry: Seconds the client should wait before retrying the request.
Rate limiting providers
The default rate limiting provider for The Things Stack is memory. When using this provider the state of rate limiting is stored locally on each instance. The drawback of using it is inconsistent rate limiting state. The redis provider eliminates this problem by storing a shared rate limiting state for all instances.
Provider
State
memory
per instance
redis
shared
To enable redis rate limiting provider you can set it in rate limiting configuration.