Design Distributed Transactions (Saga / 2PC)
How to design distributed transactions for interviews: 2PC vs Saga, choreography vs orchestration, compensations, idempotency, and when to avoid cross-service ACID.
Microservices split what used to be one DB transaction across Order, Payment, and Inventory. Interviewers ask how you keep a business action atomic enough without a global lock. Reach for the framework, then contrast two-phase commit with Sagas - and say when neither is worth it.
This shows up inside payments, checkout, and booking flows. Pair with idempotency and outbox patterns from queues.
Functional requirements
- Start a multi-step business transaction spanning services.
- Either all steps succeed (from a business view) or compensations undo partial work.
- Observe status: pending / completed / compensated / stuck.
- Retry safely; no double charge / double reserve.
Non-functional requirements
- Prefer availability - long 2PC locks hurt under partition (CAP).
- Latency of checkout measured in seconds, not minutes of blocking locks.
- Operability: dead-letter and human replay for poison steps.
Avoid distributed ACID when you can
Single-service transactions beat fancy protocols. If Order and Payment must share a store for the happy path, consider it. Reach for Saga when ownership truly splits.
Two-phase commit (sketch)
- Coordinator asks participants to Prepare.
- If all vote yes, coordinator Commits; else Rollback.
- Participants hold locks between Prepare and Commit.
Strong atomicity when the coordinator and network cooperate. Cost: blocking, coordinator SPOF (mitigated with replication), poor fit for long HTTP calls. Mention XA / 2PC as the textbook baseline you usually reject for user-facing microservices.
Saga pattern
A Saga is a sequence of local transactions. Each successful step may register a compensating action. If step k fails, run compensations for 1..k-1 in reverse (approximately). Semantic atomicity, not isolation - concurrent readers may see intermediate states.
| Style | How it runs | Pros | Cons |
|---|---|---|---|
| Choreography | Services react to events | Loose coupling | Hard to see the flow |
| Orchestration | Central saga worker drives steps | Clear status & retries | Orchestrator becomes critical |
Orchestrated checkout example
- CreateOrder (local TX) → status PENDING.
- ReserveInventory; on fail compensate CancelOrder.
- ChargePayment; on fail ReleaseInventory + CancelOrder.
- MarkOrder CONFIRMED; emit events.
Every command carries an idempotency key. Compensations must be idempotent too - ReleaseInventory twice should not go negative. Persist saga state in a store with locking or optimistic versioning so two workers do not double-drive.
Failure modes to name
- Compensation itself fails → park in DLQ, page humans.
- Out-of-order events in choreography → version vectors / ignored stale messages.
- Dirty reads during saga → UI shows “processing” not “paid” early.
- Timeout vs unknown: payment may have charged - query before compensate.
Worked example
- User buys a shoe; Order PENDING, inventory reserved, payment charge times out.
- Orchestrator queries payment: charge succeeded → continue to confirm.
- Alternate world: charge failed → ReleaseInventory, CancelOrder, user sees failure.
Interview summary
Reject naïve 2PC for long workflows. Offer Saga with orchestration, compensations, and idempotency. State visibility of intermediate states honestly. That closes distributed transactions.