Design a Feature Flag System
How to design feature flags for interviews: boolean and percentage rollouts, targeting rules, SDK polling vs streaming, kill switches, audit logs, and consistency.
Feature flags let you ship dark code and turn behaviour on per user, cohort, or percentage without redeploying. Every serious product uses them for kill switches, canaries, and experiments. The interview is less “store a bool in Redis” and more “how do millions of clients evaluate flags fast, safely, and with an audit trail.”
Scope with the framework: create/update flags, evaluate in SDKs, percentage and attribute targeting, change propagation under a few seconds, and emergency kill. Pair mentally with A/B testing and config style control planes.
Functional requirements
- CRUD flags: key, type (bool / string / number / JSON), default, targeting rules.
- Evaluate(flag_key, user_context) → variation.
- Percentage rollout sticky per user_id (same user always same bucket).
- Environments: dev / staging / prod with separate states.
- Audit log of who changed what; kill switch that forces off globally.
Non-functional requirements
- SDK evaluation in microseconds locally after rules are cached.
- Rule changes reach most clients within seconds to low tens of seconds.
- High availability of evaluation even if the control plane is briefly down (stale cache).
- Strong auditability for compliance; no silent flag flips.
Fail open or fail closed on purpose
Decide per flag: if the SDK cannot refresh, should the last known rules apply, or a hard default? Kill switches usually fail closed (off). Growth experiments often keep the last known assignment so UX does not flicker.
Architecture
- Control plane API + UI writes flag definitions to a primary store.
- Publisher pushes snapshots or diffs to a CDN / edge config channel.
- Server SDKs poll or stream updates; embed an in-process evaluator.
- Optional relay in each datacenter for air-gapped or high-QPS apps.
- Analytics sink receives evaluation events for dashboards.
Data model
| Entity | Fields |
|---|---|
| Flag | key, type, default, rules[], off_variation, version |
| Rule | clauses (attr op value), percentage, serve variation |
| Segment | named user sets reused across flags |
| Audit | actor, before/after JSON, timestamp |
Version every flag document. SDKs apply updates only if version increases. Sticky percentages hash (flag_key + user_id) into 0..9999 and compare to the rollout band - same idea as consistent bucketing in experiments.
Evaluation path
- Load cached flag for the environment.
- Walk rules in order; first match wins.
- On percentage rule, deterministic hash → bucket.
- If nothing matches, return default / off variation.
- Optionally emit an eval event asynchronously (sample if volume is huge).
Propagation
Polling every 30s is simple and fine early on. Streaming (SSE/WebSocket) from a fan-out service cuts latency. Put immutable snapshots on a CDN for mobile/browser SDKs with short TTLs. Server fleets often pull from an internal relay to avoid stampeding the control plane - related to caching stampedes.
Safety
- Require dual control for prod kill switches on critical payments paths.
- Schema-validate targeting JSON before publish.
- Guardrails: max percentage step-ups per hour.
- Store secrets out of flag payloads; flags are config, not a vault.
Worked example
- Flag new_checkout defaults off; rule: country=US AND hash% < 10% → on.
- Publish version 42; relays push to checkout pods in ~2s.
- User u123 in US hashes to 7% → sees new UI; u999 hashes to 55% → old UI.
- Error spike → kill switch forces off; pods refresh and serve default.
Interview summary
Separate control plane from local evaluation. Sticky hashing for percentages. Versioned push/poll of rules with stale-cache survival. End on kill switches and audit. That is the feature flag interview.