Rate limiting

The Grafbase Gateway offers ways to limit the number of requests per time window either globally or per subgraph. You can define the limit in memory per gateway instance or utilize a Redis backend to share the limit state with multiple gateway instances.

You can set up global rate limiting to apply to every incoming request. The configuration involves two parameters: a limit to cap the number of requests allowed for a given duration.

[gateway.rate_limit.global]
limit = 1000
duration = "10s"

You can also define a rate limit per subgraph:

[subgraphs.products.rate_limit]
limit = 100
duration = "10s"

The default rate limiter uses the generic cell rate algorithm, a leaky bucket type scheduling algorithm. This method is very accurate and can limit sudden bursts of requests even if the current time window has not yet reached its limit. It is also the fastest method because the engine does not need to trigger any network requests per GraphQL operation. The rate limit data will be empty when restarting the gateway with the in-memory rate limiter.

Using Redis for rate limiting

To run multiple gateways and share the rate limit data with all of them, configure the gateway to use Redis as the rate limiter backend.

[gateway.rate_limit]
storage = "redis"

[gateway.rate_limit.redis]
url = "redis://localhost:6379"
key_prefix = "my_gateway"
  • storage: The storage to use for rate limiting. Either memory, the default, or redis, which uses the Redis algorithm.
  • url: The Redis endpoint URL. Use redis:// for plain text protocol, or rediss:// if connecting with TLS (default: redis://localhost:6379).
  • key_prefix: A string the rate limiter uses to prefix the counters in Redis (default: grafbase).

To connect using TLS, the Redis URL must start with rediss://. If the server CA certificate is not in the system certificates or if you want to use mTLS, define paths to these files in the TLS configuration.

[gateway.rate_limit.redis.tls]
cert = "/path/to/user.crt"
key = "/path/to/user.key"
ca = "/path/to/ca.crt"
  • cert: The path to the mTLS user certificate file.
  • key: The path to the mTLS user private key file. Must be defined together with the cert.
  • ca: The path to the server CA certificate file.

All files must be in PEM format. The cert and key are not needed if the server is not set up to use mTLS. The ca is not needed if the certificate is added to the system certificate storage. The certificates must be of version 3 and the server must use TLS version 1.3; everything else will be rejected by the TLS library.

Using TLS with Redis increases response times, and the Redis server will be called at least once for every request. Consider placing the Redis server as close as possible to the gateway instances and not using TLS for the counters.

The Redis algorithm uses an averaging fixed window rate limiting, the fastest possible algorithm, adding about 0.4 milliseconds to the requests. The algorithm uses two Redis keys:

  • {key_prefix}:{subgraph:subgraph_name || global}:{current_time_bucket}
  • {key_prefix}:{subgraph:subgraph_name || global}:{previous_time_bucket}

Both keys contain a counter value stored as an integer string. The buckets will expire soon after the time window ends and will be deleted from the database. The algorithm differs from the in-memory one. We increment the counter in the current time bucket and load the previous bucket value in a single pipelined Redis request. The algorithm calculates an averaged request count with the following formula:

  1. Load the value in the previous bucket.
  2. Count the percentage of how far we are in the current time window.
  3. Multiply the previous value with the percentage based on the percentage in the current window.
  4. Add the current counter to the averaged previous value.

The averaging prevents spikes at the window border, with an accuracy of a few percent. Set the limit slightly higher than the estimated traffic to avoid false negatives.