Skip to main content

Documentation Index

Fetch the complete documentation index at: https://engineering.unkey.com/llms.txt

Use this file to discover all available pages before exploring further.

The rate limit service lives in internal/services/ratelimit and is embedded into every regional gateway (API, Frontline, Sentinel). It enforces per-identifier limits with a sliding window over atomic local counters, an async replay path to a per-region Redis, and a denial propagation channel through MySQL that ties all regions together.

​
Local enforcement

State lives in a flat sync.Map of counter entries keyed by (workspace, namespace, identifier, duration, sequence). Each entry holds an atomic.Int64 and a sync.Once that gates the first hydration from origin. There are no mutexes in the hot path: denials are wait-free (two atomic loads plus arithmetic), allows are lock-free (one atomic add after the check). The sliding window itself contributes the full count of the current window plus a weighted portion of the previous window based on how far through the current window the request lands. This avoids the burst-at-window-start problem of fixed windows without storing per-request timestamps.

​
Origin convergence (within a region)

After a request decision the entry is buffered for async replay to the configured counter.Counter (Redis in production). Eight background workers drain the buffer, push local increments via INCRBY, and CAS-merge the global count back into the local atomic. The local view therefore lags Redis, but only briefly, and only ever by reads that arrive between increment and replay completion. Redis itself is per-region. Replay converges counts within a region, not across regions. Strict mode (described next) and cross-region propagation (described below) close the cross-region gap for the denial case.

​
Strict mode

When a request is denied, the service raises a per-identifier strict-enforcement deadline (the end of the current window). Subsequent requests on the same identifier force a synchronous origin fetch before deciding, trading latency for tighter convergence after the local counter trips. The deadline lives in a separate sync.Map so its lifecycle is decoupled from any single window’s sequence; a denial in window N can extend strict enforcement into N+1 without resurrecting an evicted counter.

​
Cross-region denial propagation

Per-region Redis means a denial in region A does not automatically deny the same identifier in region B. The propagation channel is a MySQL table, ratelimit_blocklist, used as a globally replicated gossip bus. On the cold-to-hot transition (the first denial in a window), the originating region writes one row carrying the workspace, namespace, identifier, duration, originating sequence, limit, and a sequence-derived expires_at. Every node in every region runs a periodic sync goroutine that reads the active set and inflates the local counter for each row’s exact (key, sequence). The sliding-window math then produces the same denial decision in the receiving region without needing local traffic to trigger it. Bleed into the next window is automatic: the inflated counter contributes as cur in sequence S and decays as prev in S+1. A few design choices matter for correctness: The sequence is stored in the row, not derived at receive time. If the receiver recomputed sequence from its own clock, a sync running in S+1 against a denial from S would inflate the wrong cell and over-block by a full window. Storing the originating sequence keeps the receiver math identical to the originator. expires_at is computed as (sequence + 2) * duration. That is the end of the window after the originating one, the last point at which the inflated counter still contributes to the receiver’s sliding-window result. The cleanup cron deletes rows past their expires_at. Each entry carries a blocked flag that is CAS-set on first emit. The sync goroutine pre-sets the flag on every counter it inflates from MySQL. So a region that receives a denial via sync does not re-emit it back, and a region that originated a denial does not emit it again on subsequent denials within the same window. Each (region, sequence) writes exactly once. Writes are batched (100-row buckets, one-second flush interval) and circuit-broken. A sick MySQL drops batches rather than back-pressuring the denial path; the originating denial still applies locally regardless of whether propagation succeeds. Cleanup is a Restate cron defined in svc/ctrl/worker/ratelimitblocklistcleanup that runs every minute and deletes expired rows.

​
Propagation filters

Two filters gate emission. Both are evaluated after the local denial; local strict mode applies regardless. Window duration must be at least 1 minute. The 1s flush plus 10s sync pipeline takes longer than shorter windows, so the row would arrive at receivers after the originating window has already rotated and offers no enforcement value. Sub-minute windows still enforce locally, just not cross-region. Pre-cost usage must be at or above half the limit. A denial where the user has consumed less than half their quota typically came from a single oversized request (cost approaches or exceeds the limit on a fresh user) rather than accumulated abuse. Broadcasting it would block other regions for someone who still has plenty of legitimate budget. The threshold stops at half so heavy-usage cases (e.g., consumed 8 of 10, requests cost 3) still propagate, since those users are demonstrating intent to consume well past their limit.

​
Known limitation: spread-evenly traffic

The propagation channel is denial-driven. A region only writes to MySQL when its own local counter trips the limit. With N regions and traffic split perfectly evenly across them, each region sees total/N. If the customer’s limit is L and global traffic is X·L, each region locally sees X·L/N. For X < N, no region’s local count crosses L, so no region denies, the propagation channel never fires, and the customer effectively gets up to N·L globally before any single region trips. Closing this needs a different mechanism than denial propagation: periodic per-region count reporting through the same MySQL channel, with receivers summing across regions and adding the remote contribution into the sliding-window math. That work is deferred. The current channel handles the common attack shape (burst against a single region) correctly.

​
Failure handling

The origin path is wrapped in a circuit breaker named ratelimitOrigin. After sustained Redis failures, requests bypass origin entirely and fall back to local state until the breaker recovers. The blocklist write path uses a separate breaker (ratelimit_blocklist_writes); when tripped, batches are dropped and the originating denial still applies locally. The blocklist sync path logs and continues on read errors. None of these paths can take down the gateway.