Lesson 18 · A limiter per key - Build a Rate Limiter

A limiter per key

A real service limits each client independently, not the whole world against one counter. Today you build a keyed limiter that lazily creates a fresh limiter per client id, so one client exhausting its allowance cannot starve another.

The goal

Give each key its own limiter so their allowances are independent.

Start here - the target

TO DO

Scenario: One key's exhaustion does not affect another

Givena KeyedLimiter whose factory makes a fresh TokenBucket with capacity 2 and rate 1 per key, all calls at tick 0

WhenAllow is called for key "a" three times, then for key "b" once

Thenkey "a" is allowed twice (draining its bucket) then denied on the third call

Andkey "b" is allowed, because it has its own full bucket untouched by key "a" - the two keys are fully independent

Background

Limiting every client against a single shared counter would let one noisy client use up everyone’s allowance. The fix is a keyed limiter: a map from client id to that client’s own limiter, created lazily the first time a key appears. Each key carries its own state - its own bucket, its own count - so the limits are per-client and completely independent. A factory function supplies a fresh limiter for each new key, which keeps the keyed wrapper agnostic about which algorithm it distributes (here a token bucket, but any of them would do).

The isolation is the whole point, so pin it directly: with capacity-2 buckets, key "a" is allowed twice and then denied once its own bucket empties - but key "b", seen for the first time, gets a brand-new full bucket and is allowed regardless of what "a" did. One client hitting its limit has zero effect on another. This map of per-key limiters is exactly how an API gates thousands of customers with one shared limiter object.

Make it work

type KeyedLimiter struct {
  factory  func() *TokenBucket
  limiters map[string]*TokenBucket
}
func (k *KeyedLimiter) Allow(key string, now int64) Decision {
  b, ok := k.limiters[key]
  if !ok {
    b = k.factory()        // first time we see this key
    k.limiters[key] = b
  }
  return b.Allow(now)
}

CheckpointDONE

Each key gets its own independent limiter. Commit and stop here.