System Design for SDE 2 and 3: Architecting Resilient Distributed Systems
A deep dive into distributed caching patterns, database partitioning strategies, and eliminating single points of failure in technical rounds.
Mr. Anil Kumar Sain
Founder
"In mid to senior engineering interviews at Tier 1 product companies, interviewers rarely care about reciting textbook definitions of the CAP theorem. They evaluate how you navigate architectural trade offs under high load, state corruption risks, and unpredictable network partitions."
1. The Anatomy of Distributed Caching and Cache Invalidation
Caching is often treated as a magic bullet to solve slow database queries, but in production, bad cache strategies introduce silent data divergence.
When designing a multi region system with high read throughput, you must explicitly defend whether you choose Cache Aside, Read Through, or Write Through patterns.
Cache stampede (thundering herd) occurs when a heavily queried key expires, causing thousands of concurrent requests to hit the primary database simultaneously. Production solutions require probabilistic early expiration (XFetch algorithm) or distributed mutual exclusion locks via Redis Redlock.
Production Engineering Rules
- Always implement jitter on cache TTLs to avoid simultaneous expiration cascades
- Use Redis sentinel or cluster topologies with replica reads for read heavy hotkeys
- Ensure strong idempotency tokens on write back cache updates to prevent out of order writes
2. Database Partitioning and Sharding Strategies
Vertical scaling hits physical cost and hardware limits rapidly. When single table storage crosses hundreds of gigabytes or 10,000 IOPS, database sharding is unavoidable.
Range based sharding often creates massive hotspots (e.g. time based queries overloading the latest shard). Hash based consistent hashing distributes load evenly across nodes while minimizing data rebalancing overhead when adding nodes.
Cross shard queries and distributed transactions are the primary architectural bottleneck in sharded environments. Learn to design bounded contexts where 95% of queries resolve within a single partition key.
Production Engineering Rules
- Choose natural domain keys such as tenantId or accountId as partition keys
- Replace distributed 2PC (two phase commit) with Saga orchestration patterns and compensation actions
- Implement read replicas with asynchronous streaming replication and lag monitoring
3. Eliminating Single Points of Failure and Designing for Chaos
Resilience is not the absence of failures: it is the guarantee of graceful degradation when failures inevitably strike.
Every critical path microservice must incorporate circuit breakers (Resilience4j or Envoy sidecars), exponential backoff with jitter, and dead letter queues for irrecoverable events.
During live interview defense, articulate how your system responds when third party gateways fail or regional availability zones go dark.
Production Engineering Rules
- Enforce strict timeouts on every outgoing network socket to avoid thread pool starvation
- Utilize health probes: distinguish between liveness probes and readiness probes in Kubernetes
- Design graceful degradation: fall back to cached stale data rather than returning 500 errors
Key Architectural Takeaways
- Every architecture decision is a trade off: defend why consistency was sacrificed for availability or vice versa
- Quantify your system: calculate requests per second, bandwidth requirements, and disk growth over 3 years
- Demonstrate real production empathy: talk about observability, alerting thresholds, and runbooks
Ready to master these systems in production?
Join Anima Technologies live cohorts. Learn directly through real codebases, distributed architectures, and rigorous code reviews with senior tech leads.