QuizCluster
DatabasesMid-Level to Staff Backend & Data Platform Engineer19 min read

Database System Design: SQL vs NoSQL, Sharding, Replication & Indexing

A Practitioner's Playbook for CAP Trade-offs, Partition Keys, Quorum Consistency & Storage Engine Internals

Priya Narang
Staff Data Platform Engineer & Distributed Storage Specialist
13+ Years Building Petabyte-Scale Sharded Datastores
Prep Timeline
5 to 7 Weeks
Format
System Design Deep-Dive, Data Modeling, Scalability Trade-offs
Conversion
+74% System Design Round Pass Rate
Database System Design: SQL vs NoSQL, Sharding, Replication & Indexing
Executive Summary & Key Takeaways

What You Must Master to Clear This Track

  • Choose SQL vs NoSQL based on access patterns and consistency needs, not hype — polyglot persistence is the industry norm, not the exception.
  • Pick a sharding key that distributes load evenly and matches your dominant query pattern to avoid scatter-gather reads and hot shards.
  • Understand replication topology trade-offs cold: leader-follower simplicity vs multi-leader conflict resolution vs leaderless quorum tunability.
  • Know B-Tree vs LSM-Tree internals: read-optimized in-place updates vs write-optimized append-and-compact, and their amplification trade-offs.
  • Map the CAP theorem and PACELC correctly onto real systems, and be ready to defend a consistency model choice (strong/eventual/causal) under network partitions.
Structured Preparation Timeline

Step-by-Step Study Plan

Follow this sequential roadmap designed to take you from core foundations to advanced architecture and mock interviews.

Phase 1 (Weeks 1-2)

Relational Modeling, NoSQL Data Models & the CAP Theorem

Data Modeling Foundations & SQL vs NoSQL

Normalization vs denormalization, ACID guarantees, document/key-value/wide-column/graph models, and CAP/PACELC reasoning for real systems.

Key Milestones
  • Explain the trade-offs between 3NF normalization and deliberate denormalization for read-heavy services.
  • Map five real products (banking ledger, product catalog, session store, social graph, time-series metrics) to the right data model.
  • Correctly classify at least five real databases as CP or AP under the CAP theorem, and justify it with PACELC's latency/consistency axis.
Recommended Actions
  • Whiteboard a schema for a booking system in both a normalized SQL form and a denormalized document form; compare query complexity.
  • Never say 'NoSQL scales better' without naming the specific access pattern that breaks a relational design.
Phase 2 (Weeks 3-4)

Partitioning Strategies, Consistent Hashing & Rebalancing

Horizontal Scaling: Sharding & Partitioning

Range, hash, and directory/geo sharding; hot shard mitigation; consistent hashing with virtual nodes; and online shard splitting.

Key Milestones
  • Implement a consistent-hashing shard router with virtual nodes from scratch and reason about its rebalancing cost.
  • Identify hot-shard risks in a given schema (e.g. a celebrity user, a monotonic timestamp key) and propose a salting or composite-key fix.
  • Design a zero-downtime shard-split runbook including dual-write, backfill, and cutover verification steps.
Recommended Actions
  • Always state your shard key choice out loud and justify it against the read/write ratio and query filters.
  • Practice explaining why modulo(hash, N) sharding is dangerous when the cluster grows.
Phase 3 (Weeks 5-7)

Replication Topologies, Quorum Consistency & Index Internals

Replication, Consistency Models & Storage Engines

Leader-follower, multi-leader, and leaderless/quorum replication; strong/eventual/causal consistency; B-Tree vs LSM-Tree storage engines.

Key Milestones
  • Trace the full write and read path through a leaderless (Dynamo-style) N/W/R quorum system, including read repair.
  • Explain B-Tree vs LSM-Tree internals well enough to defend a storage engine choice for a given workload (read-heavy vs write-heavy).
  • Design a consistency strategy for a multi-region app: which reads need linearizability, which can tolerate eventual consistency.
Recommended Actions
  • Run EXPLAIN ANALYZE on a real Postgres query and identify whether it hits an Index-Only Scan or a sequential scan.
  • Rehearse the failure scenario: 'the leader just died mid-write, walk me through what happens next.'
Deep-Dive Architecture & Concepts

1. SQL vs NoSQL: Data Models, CAP Theorem & PACELC

Every database system design conversation starts with a data model decision. Interviewers are not testing whether you know a list of NoSQL brand names — they are testing whether you can map access patterns and consistency requirements to the right trade-off.

Relational Model & ACID

Rows in normalized tables with foreign-key relationships. Atomicity, Consistency, Isolation, and Durability guarantee that multi-row transactions either fully commit or fully roll back, making SQL the default for financial ledgers and inventory counts.

Document, Key-Value, Wide-Column & Graph NoSQL

Document stores (MongoDB) embed nested JSON for single-entity reads; key-value stores (DynamoDB, Redis) optimize O(1) lookups at massive scale; wide-column stores (Cassandra) optimize high write throughput and time-series; graph databases (Neo4j) optimize multi-hop relationship traversal.

CAP Theorem: Partition Tolerance Is Not Optional

During a network partition, a distributed system must choose Consistency (reject/queue writes until the partition heals) or Availability (serve possibly stale reads). Partition tolerance itself cannot be dropped in any real multi-node deployment.

PACELC: Extending CAP to the Normal Case

Even when there is no partition (the common case), a system still trades Latency for Consistency. Spanner is PC/EC (consistent always, pays latency via TrueTime); Cassandra is PA/EL (available always, tunable low-latency reads).

Interviewer Insights & Pro Tips
  • Never present SQL vs NoSQL as a binary choice in an interview — describe the specific access pattern (write volume, join complexity, consistency need) that drives the decision, and mention polyglot persistence for mixed workloads.
  • When asked to justify CAP classification, name the actual failure mode: 'if the network partitions this region, does the system return an error or a stale value?'
Red Flags & Common Pitfalls
  • Claiming 'NoSQL means no ACID' — many NoSQL engines (MongoDB, DynamoDB transactions) support single-document or even multi-item ACID transactions today.
  • Forgetting that CAP only applies during an actual partition; most production incidents are about the PACELC latency/consistency trade-off, not CAP.
Deep-Dive Architecture & Concepts

2. Sharding Strategies: Range, Hash, Geo & Avoiding Hot Shards

Once a dataset outgrows a single node's disk, memory, or write throughput, it must be horizontally partitioned. The sharding key you choose determines whether your cluster scales linearly or collapses under a hot partition.

Range-Based Sharding

Contiguous key ranges (e.g. user_id 1-1M on Shard A, 1M-2M on Shard B) map naturally to range scans, but concentrate all new writes on the last shard when the key is monotonically increasing (e.g. auto-increment IDs or timestamps).

Hash-Based Sharding

Hashing the shard key (e.g. hash(user_id) % N or via a consistent-hashing ring) distributes writes uniformly, at the cost of losing efficient range scans across shards.

Directory/Geo-Based Sharding

A lookup service or explicit mapping routes each key (or region, e.g. EU users to an EU shard) to its owning shard, giving full placement control for compliance (GDPR data residency) at the cost of an extra lookup hop.

The Hot Shard Problem

A single celebrity user, viral tenant, or monotonic key can overload one shard while others sit idle. Mitigations: salting the key with a random or hashed suffix, splitting a single hot key across sub-partitions, and caching in front of the hot shard.

Sharded Write Path Through a Consistent-Hashing Router

How a single write request is routed, replicated, and acknowledged across a horizontally sharded cluster.

1
Client Write Request
Application sends an INSERT/UPDATE carrying a shard key (e.g. tenant_id) to the routing tier.
2
Hash Ring Lookup
Router hashes the shard key against a consistent-hashing ring (150 virtual nodes per shard) to resolve the owning shard and its replica set.
3
Leader Write & WAL Append
The shard's leader node appends the write to its write-ahead log before applying it to the memtable or B-Tree page.
4
Synchronous Quorum Replication
Leader streams the write to follower replicas; the client receives an ack once W replicas confirm (e.g. W=2 of 3).
5
Async Compaction & Rebalance Watch
Background compaction merges SSTables (on an LSM engine) while a cluster monitor tracks shard size to trigger a split before hot-shard thresholds are hit.
Consistent Hashing Shard Router with Virtual Nodes
typescript
// Consistent hashing ring with virtual nodes to minimize key movement
  // when a physical shard is added or removed.
  import { createHash } from "crypto";
  
  interface ShardNode {
    id: string;
    weight: number; // relative virtual-node count (bigger shard = more weight)
  }
  
  class ConsistentHashRing {
    private ring = new Map<number, string>();
    private sortedPositions: number[] = [];
  
    constructor(nodes: ShardNode[], private vnodesPerWeight = 150) {
      nodes.forEach((node) => this.addNode(node));
    }
  
    private hash(key: string): number {
      const digest = createHash("md5").update(key).digest("hex");
      return parseInt(digest.slice(0, 8), 16); // top 32 bits as ring position
    }
  
    addNode(node: ShardNode): void {
      const vnodeCount = node.weight * this.vnodesPerWeight;
      for (let i = 0; i < vnodeCount; i++) {
        this.ring.set(this.hash(`${node.id}#${i}`), node.id);
      }
      this.sortedPositions = [...this.ring.keys()].sort((a, b) => a - b);
    }
  
    removeNode(nodeId: string): void {
      for (const [pos, id] of this.ring) {
        if (id === nodeId) this.ring.delete(pos);
      }
      this.sortedPositions = [...this.ring.keys()].sort((a, b) => a - b);
    }
  
    getShardForKey(key: string): string {
      const hash = this.hash(key);
      const positions = this.sortedPositions;
      if (hash > positions[positions.length - 1]) return this.ring.get(positions[0])!;
  
      let lo = 0;
      let hi = positions.length - 1;
      while (lo < hi) {
        const mid = (lo + hi) >> 1;
        if (positions[mid] < hash) lo = mid + 1;
        else hi = mid;
      }
      return this.ring.get(positions[lo])!;
    }
  }
Why it matters: Virtual nodes (150 per shard here) ensure that removing or adding one physical shard only redistributes roughly 1/N of the keyspace instead of triggering a full cluster reshuffle, and binary search over sorted ring positions gives O(log V) routing per write.
Interviewer Insights & Pro Tips
  • Always pick the shard key to match your single most frequent WHERE clause — otherwise every query becomes a scatter-gather across all shards.
  • Prefer consistent hashing with virtual nodes over naive modulo(hash, N) sharding; modulo forces almost every key to move when N changes.
Red Flags & Common Pitfalls
  • Sharding by auto-increment primary key, which sends 100% of new writes to the highest-numbered shard.
  • Designing a schema that needs frequent cross-shard joins or transactions, discovered only after the system is already in production.
Deep-Dive Architecture & Concepts

3. Replication Topologies & Consistency Models

Sharding scales throughput; replication scales availability and read capacity. The topology you pick determines exactly what happens when a node dies, a network link flakes, or two clients write to the same key concurrently.

Leader-Follower (Single-Leader) Replication

All writes go to one leader, which streams a change log to followers. Simple to reason about and strongly consistent at the leader, but the leader is a single write bottleneck and a failover requires promoting a follower and re-pointing clients.

Multi-Leader Replication

Multiple regions each accept local writes and replicate asynchronously to each other, giving low write latency and regional write availability, at the cost of needing conflict resolution (last-write-wins, vector clocks, or CRDTs) when the same key is edited in two regions.

Leaderless Replication & Quorums

Dynamo-style systems (Cassandra, DynamoDB) send every write to N replicas and accept success once W acknowledge; reads query R replicas and merge. Choosing W + R > N guarantees at least one overlapping replica between every write and read.

Consistency Models: Strong, Eventual, Causal

Strong/linearizable consistency makes replicas behave as one node (highest coordination cost). Eventual consistency only guarantees convergence once writes stop. Causal consistency sits between them, preserving cause-and-effect ordering (a reply is never seen before the comment it replies to) without full linearizability's cost.

Leaderless Replication: Quorum Write & Read Repair

Dynamo-style N/W/R quorum tuning and how read repair heals a replica that missed a write during a network blip.

1
Coordinator Receives Write
Client's write for a key lands on any node acting as coordinator; N=3 replicas are responsible for that key.
2
Fan-Out to N Replicas
Coordinator forwards the versioned write (timestamp or vector clock attached) to all 3 replicas in parallel.
3
Write Quorum Ack (W=2)
Coordinator returns success to the client once 2 of 3 replicas persist the write; the third may be lagging due to a network blip.
4
Read From R Replicas (R=2)
A later read queries 2 replicas; since W+R=4 > N=3, at least one queried replica is guaranteed to have the latest write.
5
Read Repair
Coordinator detects the stale replica's older version and asynchronously pushes the resolved latest value back to it, healing divergence without operator action.
Interviewer Insights & Pro Tips
  • State your durability requirement before picking sync vs async replication: synchronous replication guarantees zero data loss on failover but adds write latency equal to the slowest replica.
  • When asked about multi-region writes, immediately raise conflict resolution — it's the detail that separates a senior answer from a junior one.
Red Flags & Common Pitfalls
  • Reading from a follower for a just-written value on a critical path (e.g. showing a user their own new order) without a read-your-writes guarantee, causing a confusing 'it disappeared' bug.
  • Configuring W + R <= N and assuming quorum still yields strong consistency — it does not; overlap is not guaranteed.
Deep-Dive Architecture & Concepts

4. Indexing Internals: B-Tree vs LSM-Tree Storage Engines

Every replication and sharding decision sits on top of a storage engine. Knowing whether that engine is a B-Tree or an LSM-Tree explains why a workload is fast or slow, and it is one of the most reliably asked deep-dive topics in senior database design interviews.

B-Tree Indexes (Read-Optimized)

A balanced tree of fixed-size pages (typically 4-16KB) where every lookup, insert, or delete takes O(log N) page traversals. Writes are in-place, so a single row update touches only its own leaf page, making B-Trees the default in PostgreSQL, MySQL InnoDB, and SQL Server.

LSM-Trees (Write-Optimized)

Writes go to an in-memory memtable plus a write-ahead log, and are periodically flushed as immutable sorted files (SSTables) to disk. Writes are always sequential appends, giving very high write throughput; used by Cassandra, RocksDB, LevelDB, and HBase.

Compaction & Write/Read Amplification

LSM-Trees periodically merge multiple SSTables into fewer, larger ones (compaction) to bound the number of files a read must check, which rewrites data multiple times (write amplification) in exchange for lower read amplification over time.

Covering Indexes & Index-Only Scans

A composite index that includes every column a query selects lets the engine answer entirely from the index's leaf pages, skipping the heap/table fetch entirely — the single highest-leverage indexing optimization for hot read paths.

Composite Covering Index Enabling an Index-Only Scan
sql
-- Composite index ordered to satisfy the WHERE equality predicates first,
  -- then the ORDER BY column, so no separate filesort is needed.
  CREATE INDEX idx_orders_customer_status_created
    ON orders (customer_id, status, created_at DESC)
    INCLUDE (total_amount); -- PostgreSQL covering index via INCLUDE
  
  -- This query can be satisfied entirely from the B-Tree's leaf pages:
  EXPLAIN (ANALYZE, BUFFERS)
  SELECT customer_id, status, created_at, total_amount
  FROM orders
  WHERE customer_id = 48213
    AND status = 'SHIPPED'
  ORDER BY created_at DESC
  LIMIT 20;
  
  -- Index Only Scan using idx_orders_customer_status_created
  --   Heap Fetches: 0   <- proof the query never touched the table heap
Why it matters: A composite B-Tree index whose leading columns match the equality predicates and whose trailing column matches ORDER BY lets Postgres perform an Index-Only Scan with zero heap fetches, turning what would be a sequential scan into a single bounded index descent.
Interviewer Insights & Pro Tips
  • When asked to pick a storage engine, ask about the read:write ratio first — B-Tree engines win for read-heavy OLTP, LSM-Tree engines win for write-heavy ingestion and time-series workloads.
  • Mention bloom filters when discussing LSM-Tree reads: they let the engine skip SSTables that provably do not contain a key, cutting read amplification.
Red Flags & Common Pitfalls
  • Adding a secondary index for every filterable column without considering that each additional index slows down every write, since the engine must maintain it transactionally.
  • Choosing an LSM-Tree engine for a point-lookup-heavy workload without tuning compaction strategy or bloom filter false-positive rate, leading to read amplification worse than a B-Tree would have given.
Real-World Example

Splitting a Monolithic Postgres Table Before It Became a Scaling Wall

A B2B SaaS analytics company ran all customer event data in a single 900GB PostgreSQL 'events' table shared across every tenant. During Q4 onboarding pushes, write latency p99 spiked past 2 seconds and autovacuum stalls began blocking dashboard reads for every customer simultaneously.

  • 1Profiled query and write patterns and found over 95% of reads and writes filtered by tenant_id, making it the natural shard key instead of the existing auto-increment event_id.
  • 2Stood up a consistent-hashing router in front of 8 new PostgreSQL shards (128 virtual nodes per shard) and migrated historical data via a dual-write-plus-backfill process.
  • 3Switched each shard from a single instance to one leader with two asynchronous follower replicas, routing analytics dashboard queries to followers to isolate them from OLTP write load.
  • 4Added a composite covering index on (tenant_id, event_type, created_at) per shard, eliminating the sequential scans that had been driving CPU above 90% during ingestion spikes.
  • 5Ran a two-week shadow-read verification period comparing results from the old monolith and the new sharded cluster before cutting traffic over behind a feature flag.
Outcome: p99 write latency dropped from 2.1 seconds to 140 milliseconds, and the team was able to onboard four times as many tenants without provisioning another read replica.
Real-World Interview Questions

Top Must-Know Interview Questions & Model Answers

SQL vs NoSQLMust-Know

Q1: When would you choose NoSQL over a relational database for a new service, and what are you giving up?

Executive Answer:Choose NoSQL when you need horizontal write scale, a flexible/evolving schema, or simple key-based access patterns; you typically give up multi-row ACID transactions, ad-hoc joins, and mature relational tooling.
Deep Dive Analysis:
  • NoSQL engines shine when the access pattern is dominated by single-entity reads/writes keyed by a partition key (e.g. a user profile document or a session blob).
  • You lose the ability to run arbitrary JOINs across entities efficiently; instead you must denormalize or perform joins in application code, increasing complexity.
  • Most teams end up with polyglot persistence: relational for billing/ledger data, NoSQL for high-volume event or session data.
Interviewer Takeaway: Pick the data store after mapping out concrete access patterns, not before — the access pattern determines the model, not the other way around.
SQL vs NoSQLHard

Q2: What is polyglot persistence and when does it add more operational risk than it removes?

Executive Answer:Polyglot persistence means using different databases for different subdomains based on their access patterns; it adds risk when it multiplies operational surface area (backups, monitoring, on-call runbooks) without a proportional benefit.
Deep Dive Analysis:
  • Using Postgres for orders, Redis for sessions, and Elasticsearch for search is a classic, well-justified polyglot split because each engine solves a problem the others solve poorly.
  • The risk appears when teams adopt a new engine for a marginal performance gain, then must maintain expertise, backups, and monitoring for yet another storage system indefinitely.
Interviewer Takeaway: Justify every additional data store by a concrete access pattern it uniquely solves, not by a marginal benchmark win.
SQL vs NoSQLMedium

Q3: Explain the differences between document, key-value, wide-column, and graph NoSQL models with one canonical use case each.

Executive Answer:Document stores hold nested JSON per entity (product catalogs), key-value stores optimize O(1) lookups (session caching), wide-column stores optimize high write throughput over sparse columns (time-series/IoT), and graph databases optimize multi-hop traversal (social/recommendation graphs).
Deep Dive Analysis:
  • Document (MongoDB): one document per catalog item with embedded variants/attributes, read in a single fetch.
  • Key-Value (Redis, DynamoDB): session tokens or shopping-cart state keyed by user ID, no query flexibility needed.
  • Wide-Column (Cassandra): sensor readings keyed by (device_id, timestamp), optimized for sequential writes.
  • Graph (Neo4j): 'friends of friends' or fraud-ring detection queries that would require many recursive SQL joins.
Interviewer Takeaway: Match the NoSQL sub-category to the shape of the query, not just to 'NoSQL' as a monolithic category.
SQL vs NoSQLMedium

Q4: What is NewSQL and how do systems like Google Spanner or CockroachDB provide ACID transactions at global scale?

Executive Answer:NewSQL systems combine relational ACID semantics with horizontal scalability by using distributed consensus (Paxos/Raft) per shard and a globally synchronized clock or hybrid logical clock to order transactions.
Deep Dive Analysis:
  • Spanner uses TrueTime, a globally synchronized clock with bounded uncertainty, to assign commit timestamps that guarantee external consistency across data centers.
  • CockroachDB uses hybrid logical clocks and Raft consensus per range to achieve similar serializable guarantees without specialized atomic clock hardware.
Interviewer Takeaway: NewSQL trades some write latency for the ability to say 'ACID transactions' and 'horizontally scalable' in the same sentence.
Data ModelingHard

Q5: Why can normalizing a schema hurt performance at scale, and when should you deliberately denormalize?

Executive Answer:Normalization minimizes redundancy but requires joins across many tables; at high read volume those joins become the bottleneck, so denormalizing hot read paths (or using materialized views) trades storage and write complexity for read speed.
Deep Dive Analysis:
  • A fully normalized order system might require joining orders, order_items, products, and customers just to render one order confirmation page.
  • Denormalizing by embedding a snapshot of product name/price into order_items avoids the join but requires explicit handling when the source product record changes.
Interviewer Takeaway: Normalize for correctness by default; denormalize deliberately and only on the specific read paths that are measurably hot.
ShardingMust-Know

Q6: How do you choose a sharding key, and what happens if you pick a low-cardinality or monotonically increasing key?

Executive Answer:A good sharding key has high cardinality, distributes load evenly, and matches your dominant query filter; a low-cardinality or monotonically increasing key (like a boolean flag or auto-increment ID) concentrates all new writes onto a single shard.
Deep Dive Analysis:
  • Low cardinality (e.g. a status flag with 3 values) means only 3 possible shard destinations regardless of cluster size, capping horizontal scale.
  • A monotonically increasing key like an auto-increment ID or a raw timestamp sends 100% of new inserts to whichever shard owns the current highest range, creating a permanent hot shard.
Interviewer Takeaway: Test a candidate shard key against three questions: is it high-cardinality, is it evenly distributed, and does it match your most frequent query filter?
ShardingHard

Q7: Explain consistent hashing and why it minimizes data movement compared to modulo-based sharding when nodes are added or removed.

Executive Answer:Consistent hashing places both keys and nodes on a hash ring so that adding or removing a node only remaps the keys between it and its neighbor, roughly 1/N of the keyspace, versus modulo(hash, N) which remaps nearly all keys whenever N changes.
Deep Dive Analysis:
  • With modulo sharding, changing N from 4 to 5 changes the result of hash(key) % N for almost every key, forcing a near-total data reshuffle.
  • Consistent hashing with virtual nodes (100-200 per physical shard) also smooths load distribution, since a single physical node maps to many ring positions instead of one.
Interviewer Takeaway: Consistent hashing is the standard answer whenever an interviewer asks how your sharded system rebalances after scaling the cluster.
ShardingMedium

Q8: What is a 'hot shard' and name three mitigation strategies.

Executive Answer:A hot shard receives disproportionate traffic relative to its peers, typically from a viral key, a monotonic write key, or a skewed access distribution; mitigations include key salting, caching in front of the shard, and splitting the hot key into sub-partitions.
Deep Dive Analysis:
  • Salting appends a random or hashed suffix to a hot key (e.g. celebrity_id#0-9) so writes spread across 10 sub-keys, merged at read time.
  • A read-through cache (Redis) in front of the hot shard absorbs read traffic without touching the underlying shard.
  • Splitting a single celebrity tenant into its own dedicated shard removes it from the shared pool entirely.
Interviewer Takeaway: Detect hot shards via per-shard request-rate monitoring, not just aggregate cluster metrics, since aggregates hide skew.
ShardingHard

Q9: How do you handle cross-shard joins or transactions in a sharded relational database?

Executive Answer:Avoid them where possible by co-locating related data under the same shard key; where unavoidable, use application-side scatter-gather joins, a distributed transaction protocol (2PC or a Saga), or route analytics to a separately CDC-fed OLAP store.
Deep Dive Analysis:
  • Co-location: choosing tenant_id as the shard key for every table in a multi-tenant app ensures a tenant's orders and order_items always live on the same shard.
  • When cross-shard reads are unavoidable (e.g. a global leaderboard), stream changes via CDC (Debezium) into a single analytical store rather than querying shards live.
Interviewer Takeaway: The best fix for cross-shard joins is choosing a shard key that makes them unnecessary in the first place.
ShardingMedium

Q10: Compare range-based, hash-based, and directory/geo-based sharding strategies.

Executive Answer:Range sharding preserves ordered scans but risks hot shards on monotonic keys; hash sharding distributes load evenly but loses range-scan locality; directory/geo sharding gives explicit placement control (e.g. for data residency) at the cost of an extra lookup hop.
Deep Dive Analysis:
  • Range sharding is ideal for time-series queries like 'get all events between two timestamps' if the key isn't purely monotonic.
  • Hash sharding is the default choice when write distribution matters more than range queries.
  • Directory-based sharding is required when compliance (GDPR) mandates that EU user data physically resides in EU-based shards.
Interviewer Takeaway: Range for ordered scans, hash for even distribution, directory for explicit compliance/placement control.
ShardingMust-Know

Q11: How would you design an online shard-splitting/rebalancing operation with zero downtime?

Executive Answer:Dual-write to both the old and new shard layout, backfill historical data in the background, verify consistency with a shadow-read comparison, then atomically flip the router's ring mapping once verified.
Deep Dive Analysis:
  • Start dual-writing new data to both the source shard and the destination shard(s) so no writes are lost during migration.
  • Backfill historical rows in batches with rate limiting to avoid overloading the source shard's I/O.
  • Run a shadow-read comparison period where reads are served from the old shard but silently validated against the new one before cutover.
Interviewer Takeaway: Every zero-downtime data migration follows the same shape: dual-write, backfill, verify, then cut over behind a feature flag.
ReplicationMust-Know

Q12: Compare leader-follower, multi-leader, and leaderless replication topologies.

Executive Answer:Leader-follower is simple and strongly consistent at the leader but has a single write bottleneck; multi-leader allows regional write availability at the cost of conflict resolution; leaderless (quorum-based) trades strict consistency for tunable availability via N/W/R.
Deep Dive Analysis:
  • Leader-follower (Postgres streaming replication, MySQL): best when writes are naturally centralized and simplicity matters more than multi-region write latency.
  • Multi-leader: best for multi-region apps needing local write latency, at the cost of needing conflict resolution logic (LWW, CRDTs, vector clocks).
  • Leaderless (Cassandra, DynamoDB): best for very high availability requirements where tunable per-request consistency (via N/W/R) is acceptable.
Interviewer Takeaway: Pick the topology based on where writes originate geographically and how much conflict-resolution complexity you're willing to own.
ReplicationHard

Q13: What is replication lag, and what problems does it cause for a read-after-write user experience?

Executive Answer:Replication lag is the delay between a write committing on the leader and that write becoming visible on a follower; if a user's next read hits a lagging follower, they may not see their own just-made change.
Deep Dive Analysis:
  • A classic symptom: a user submits a comment, gets redirected to a page that reads from a follower, and briefly sees the comment missing.
  • Fixes include read-your-writes session guarantees (route a user's reads to the leader for a short window after their write), sticky sessions, or version-stamping reads to require a minimum replica freshness.
Interviewer Takeaway: Replication lag is invisible until you design the specific read-after-write scenario an interviewer will ask you to defend.
ReplicationMedium

Q14: How does synchronous vs asynchronous replication affect durability and latency trade-offs?

Executive Answer:Synchronous replication waits for follower acknowledgment before confirming a write, guaranteeing zero data loss on failover but adding latency equal to the slowest required replica; asynchronous replication acknowledges immediately and replicates in the background, risking data loss if the leader fails before replicating.
Deep Dive Analysis:
  • Fully synchronous replication to all followers is rare in practice because one slow or unreachable follower would block every write.
  • A common middle ground is semi-synchronous replication: wait for at least one follower to ack before confirming, balancing durability and latency.
Interviewer Takeaway: State your durability requirement (can you tolerate losing the last few seconds of writes on failover?) before picking sync vs async replication.
ReplicationHard

Q15: In multi-leader replication, how are write conflicts detected and resolved (LWW, vector clocks, CRDTs)?

Executive Answer:Conflicts are detected when two leaders accept concurrent writes to the same key, then resolved via last-write-wins (highest timestamp survives), vector clocks (track causal history to detect true concurrency), or CRDTs (data structures that merge concurrent updates deterministically without loss).
Deep Dive Analysis:
  • Last-write-wins is simple but can silently discard a legitimate concurrent update, which is unacceptable for financial data.
  • Vector clocks let the system detect when two writes were truly concurrent (neither caused the other) versus sequential, surfacing genuine conflicts for app-level resolution.
  • CRDTs (e.g. a G-Counter for view counts, an OR-Set for shopping cart items) are designed so concurrent merges always converge to the same correct result without a conflict-resolution step.
Interviewer Takeaway: LWW is the easy default that silently loses data; CRDTs are the principled fix when the data type supports a mergeable structure.
ReplicationMedium

Q16: Explain quorum reads/writes (N, W, R) in leaderless replication systems like Cassandra or DynamoDB.

Executive Answer:N is the number of replicas holding a key, W is how many must acknowledge a write to succeed, and R is how many must respond to a read; setting W + R > N guarantees every read overlaps with the most recent write.
Deep Dive Analysis:
  • A common configuration is N=3, W=2, R=2: any two writes and any two reads are guaranteed to share at least one common replica.
  • Lowering W or R increases availability and lowers latency but weakens consistency guarantees — W=1 accepts a write even if only one replica is reachable.
Interviewer Takeaway: Quorum consistency is tunable per request in systems like Cassandra — know how to justify a specific N/W/R choice for a given workload.
Replication / ConsensusHard

Q17: Walk through what happens during a leader failover in a Raft or Paxos-based replicated database.

Executive Answer:When followers stop receiving heartbeats from the leader, one triggers a leader election, requiring a majority (quorum) vote; the new leader is promoted only if it has the most up-to-date committed log, and clients are re-pointed to it.
Deep Dive Analysis:
  • A follower whose election timeout expires without a heartbeat increments its term and requests votes from peers; it becomes leader only with a majority.
  • Any uncommitted entries the old leader had replicated to a minority of followers before failing may be discarded to preserve consistency.
Interviewer Takeaway: Consensus-based failover guarantees safety (no split-brain with two leaders) by requiring a strict majority quorum for every leadership change.
IndexingMust-Know

Q18: Explain how a B-Tree index works and why it's efficient for read-heavy OLTP workloads.

Executive Answer:A B-Tree is a balanced, sorted tree of fixed-size pages where every search, insert, or delete takes O(log N) page traversals, and in-place updates touch only the affected leaf page, making it ideal for random point lookups and range scans in OLTP systems.
Deep Dive Analysis:
  • Each internal node holds sorted keys and pointers to child pages, keeping the tree shallow (a 3-4 level B-Tree can index billions of rows).
  • Because pages are updated in place, a B-Tree naturally stays 'read-optimized': a lookup always does the same bounded number of page reads regardless of write history.
Interviewer Takeaway: B-Trees are the default choice whenever a workload is dominated by point lookups and range scans rather than pure write throughput.
Storage EnginesMust-Know

Q19: Explain how an LSM-Tree (memtable + SSTables + compaction) works and why it's optimized for write-heavy workloads.

Executive Answer:An LSM-Tree buffers writes in an in-memory memtable (backed by a write-ahead log for durability), flushes full memtables as immutable sorted SSTables on disk, and periodically compacts SSTables together — since every write is a sequential append, throughput is far higher than in-place B-Tree writes.
Deep Dive Analysis:
  • Writes never require a random disk seek: they append to the WAL and insert into an in-memory sorted structure (e.g. a skip list).
  • Reads may need to check the memtable plus multiple SSTables (using bloom filters to skip ones that can't contain the key), which is why LSM-Trees trade write speed for higher read cost unless tuned.
Interviewer Takeaway: LSM-Trees convert random writes into sequential writes, which is the single biggest reason engines like Cassandra and RocksDB outperform B-Trees on write-heavy ingestion.
Storage EnginesHard

Q20: Compare write amplification and read amplification between B-Tree and LSM-Tree storage engines.

Executive Answer:LSM-Trees have higher write amplification (each row may be rewritten multiple times across compaction levels) but tunable read amplification; B-Trees have low write amplification (in-place updates) but each read is a fixed, low-cost page traversal.
Deep Dive Analysis:
  • Write amplification in LSM-Trees comes from compaction: a row written once may be rewritten at L0, then merged into L1, L2, etc.
  • Read amplification in LSM-Trees comes from having to check multiple SSTable levels before finding the latest value, mitigated by bloom filters and leveled compaction.
  • B-Trees keep both figures modest and predictable, which is why they remain the default for general-purpose OLTP databases.
Interviewer Takeaway: There is no free lunch: LSM-Trees buy write throughput by spending it back later as compaction I/O and read complexity.
IndexingMedium

Q21: What is a covering index, and how does it let a query be satisfied by an Index-Only Scan?

Executive Answer:A covering index includes every column a query selects (via indexed or INCLUDE columns), so the database can answer the query entirely from the index's leaf pages without a separate fetch to the underlying table heap.
Deep Dive Analysis:
  • In PostgreSQL, INCLUDE columns are stored in the index but not used for ordering/searching, letting a query select extra columns without expanding the search key.
  • The performance win shows up directly in EXPLAIN output as 'Index Only Scan' with 'Heap Fetches: 0', versus a regular Index Scan that still touches the table.
Interviewer Takeaway: A covering index is the single highest-leverage optimization for a hot, frequently-run read query.
IndexingMedium

Q22: Why can having too many secondary indexes hurt write throughput?

Executive Answer:Every index must be updated transactionally alongside the base table on every insert, update, or delete, so each additional index adds write amplification and lock contention proportional to the number of indexes.
Deep Dive Analysis:
  • A table with 8 secondary indexes effectively performs up to 9 writes (1 heap + 8 index updates) for every single logical row write.
  • Wide indexes also increase buffer-pool memory pressure, pushing hot pages out of cache and increasing disk I/O for both reads and writes.
Interviewer Takeaway: Audit indexes periodically and drop unused ones — every index is a permanent tax on every future write to that table.
Storage EnginesHard

Q23: Explain bloom filters and their role in reducing unnecessary SSTable reads in an LSM-Tree engine.

Executive Answer:A bloom filter is a compact probabilistic structure that can say a key is 'definitely not present' or 'possibly present' in an SSTable, letting the read path skip SSTables that provably don't contain the key and dramatically cutting unnecessary disk reads.
Deep Dive Analysis:
  • Each SSTable gets its own bloom filter built at flush/compaction time; a read checks the filter before touching disk for that SSTable.
  • False positives are possible (the filter says 'maybe' but the key isn't actually there) but false negatives are impossible, preserving correctness while still saving most unnecessary I/O.
Interviewer Takeaway: Bloom filters are the standard mitigation for LSM-Tree read amplification — mention them whenever discussing LSM-Tree read performance.
Consistency ModelsMust-Know

Q24: Explain the CAP theorem and why 'consistency, availability, and partition tolerance: pick two' is a common misconception.

Executive Answer:In any real distributed system, network partitions will happen, so partition tolerance is mandatory, not optional; the actual choice during a partition is between consistency (reject/delay requests) and availability (serve possibly stale data).
Deep Dive Analysis:
  • The 'pick two' phrasing wrongly implies you can build a CA system that ignores partitions; in a multi-node system, a partition will eventually occur regardless of your choice.
  • CAP classification only describes behavior during an active partition — most of a system's life is spent partition-free, which is where PACELC's latency/consistency trade-off matters more.
Interviewer Takeaway: Reframe CAP as 'given a partition, do you choose C or A' rather than a three-way pick, and pair it with PACELC for the non-partitioned case.
Consistency ModelsHard

Q25: What is PACELC and how does it extend CAP theorem reasoning to the normal (non-partitioned) case?

Executive Answer:PACELC states: if Partitioned, choose Availability or Consistency; Else (normal operation), choose Latency or Consistency — acknowledging that even without a partition, stronger consistency requires more coordination and therefore higher latency.
Deep Dive Analysis:
  • Spanner is PC/EC: it stays consistent during partitions and pays a latency cost (via TrueTime commit-wait) even in the normal case to guarantee external consistency.
  • Cassandra is PA/EL: it stays available during partitions and, in the normal case, favors low latency via tunable (often relaxed) consistency levels.
Interviewer Takeaway: PACELC is the more practically useful framework than CAP alone because most production time is spent in the non-partitioned 'else' branch.
Consistency ModelsMedium

Q26: Differentiate strong consistency, eventual consistency, and causal consistency with concrete examples.

Executive Answer:Strong consistency guarantees every read reflects the most recent write across all replicas; eventual consistency only guarantees replicas converge once writes stop; causal consistency guarantees operations that are causally related are seen in the same order everywhere, without requiring full global ordering.
Deep Dive Analysis:
  • Strong: a bank balance check after a transfer must reflect the transfer immediately, everywhere — requires coordination like Raft/Paxos.
  • Eventual: a 'likes' counter on a social post may briefly show different values on different replicas before converging.
  • Causal: a comment reply must never be visible before the comment it replies to, even if the two are stored on different replicas.
Interviewer Takeaway: Causal consistency is often the sweet spot: it prevents confusing 'effect before cause' bugs without paying for full linearizability.
Consistency ModelsMedium

Q27: What are session guarantees like read-your-writes and monotonic reads, and why do client applications need them?

Executive Answer:Session guarantees are client-scoped consistency promises layered on top of eventual consistency: read-your-writes ensures a client always sees its own prior writes, and monotonic reads ensure a client never sees data go 'backward in time' across successive reads.
Deep Dive Analysis:
  • Without read-your-writes, a user who just updated their profile picture might refresh and briefly see the old one if routed to a lagging replica.
  • Without monotonic reads, a client could read a newer value from one replica, then an older value from a different replica on the next request, appearing to go back in time.
Interviewer Takeaway: Session guarantees are usually implemented by routing a client's reads to the same replica (sticky sessions) or by passing a version/timestamp token the client requires reads to meet.
Consistency ModelsHard

Q28: How does a distributed database achieve linearizability, and what's the performance cost?

Executive Answer:Linearizability requires every operation to appear to take effect instantaneously at some point between its start and end, which in practice requires consensus (Raft/Paxos) or a single authoritative leader for each piece of data, costing extra round-trips and reduced availability during network issues.
Deep Dive Analysis:
  • Achieving linearizable reads often means routing reads through the leader or using a quorum read combined with a lease mechanism to avoid stale data.
  • The cost shows up as higher tail latency (every operation waits for consensus round-trips) and reduced availability during partitions, since a minority partition cannot make progress.
Interviewer Takeaway: Linearizability is the strongest, most expensive consistency guarantee — reserve it for the specific fields (account balances, inventory counts) that truly require it, not the entire dataset.
Common Mistakes

Mistakes That Sink Otherwise Strong Candidates

Sharding by an auto-increment ID or raw timestamp.

Why it happens: It's the easiest key to reach for since it already exists as the primary key and requires no extra design work.

The fix: Hash the key or choose a composite/business key (e.g. tenant_id, user_id) with high cardinality so new writes spread across all shards instead of piling onto the newest one.

Choosing NoSQL purely because 'it scales better,' without validating query patterns first.

Why it happens: Cargo-culting the stack used by a well-known high-scale company without checking whether the same access patterns actually apply.

The fix: Model the dominant read/write access patterns first (single-table design exercise), then choose the storage engine that best serves those patterns.

Assuming eventual consistency means 'eventually, within milliseconds' with no explicit bound.

Why it happens: Default asynchronous replication configurations hide the actual lag until a traffic spike or network issue widens it.

The fix: Add read-your-writes guarantees (route a user's own reads to the leader shortly after their write) or use causal consistency instead of assuming the lag window is negligible.

Reading from a follower replica for a critical, just-written value.

Why it happens: Read replicas are added purely to reduce leader load, without considering which specific reads must see the latest write.

The fix: Route session-critical reads (e.g. confirming a user's own new order) to the leader, or implement sticky-session / version-token read-your-writes guarantees.

Over-normalizing a schema in a system that needs very high write throughput.

Why it happens: Relational best practices (3NF) are taught as a default without weighing them against the actual read/write ratio of the system.

The fix: Selectively denormalize hot write/read paths, or introduce materialized views/CQRS read models for the specific queries that suffer from excessive joins.

Configuring quorum reads/writes with W + R less than or equal to N.

Why it happens: Teams tune N, W, and R independently for latency without checking the overlap guarantee that makes quorum consistency meaningful.

The fix: Always verify W + R > N when strict quorum consistency is required, and understand that sloppy quorums during a partition can still return stale data.

Picking an LSM-Tree storage engine for a heavy point-lookup workload without tuning it.

Why it happens: LSM-Trees are chosen reflexively for their write throughput reputation without checking the workload's actual read/write ratio.

The fix: Tune bloom filter false-positive rates and choose an appropriate compaction strategy (leveled vs size-tiered), or use a B-Tree engine if the workload is genuinely read-heavy.

Rebalancing a sharded cluster with a naive modulo(hash, N) scheme.

Why it happens: Modulo sharding is the simplest formula to implement and works fine until the first time a node is added or removed.

The fix: Use consistent hashing with virtual nodes so scaling the cluster only remaps a small fraction of the keyspace instead of nearly all of it.

Not planning for cross-shard queries until a reporting requirement surfaces in production.

Why it happens: Initial sharding design optimizes purely for OLTP write throughput and ignores analytics/reporting access patterns.

The fix: Design the shard key around the dominant OLTP query pattern, and stream changes via CDC into a separate OLAP store for cross-shard analytical queries.

Cheat Sheet

Quick-Reference Cheat Sheet

SQL vs NoSQL Decision Matrix
Structured data with relational joinsSQL (PostgreSQL/MySQL)
Flexible, evolving schema per entityNoSQL Document (MongoDB)
Simple key lookups at massive scaleNoSQL Key-Value (DynamoDB/Redis)
High write throughput / time-seriesWide-Column (Cassandra/HBase)
Deep multi-hop relationship traversalGraph DB (Neo4j)
ACID transactions at global scaleNewSQL (Spanner/CockroachDB)
CAP / PACELC at a Glance
CP systemSacrifices availability during a partition (HBase, MongoDB majority reads)
AP systemSacrifices consistency during a partition (Cassandra, DynamoDB)
PACELC 'else' branchEven with no partition, trade Latency vs Consistency
PC/EC exampleSpanner: always consistent, pays latency via TrueTime
PA/EL exampleCassandra: always available, tunable low-latency reads
Sharding Key Selection Criteria
High cardinalityAvoid keys with few distinct values (e.g. boolean flags)
Even distributionHash-based keys spread load; avoid raw monotonic IDs/timestamps
Query localityKey should match the most frequent WHERE clause to avoid scatter-gather
Hot key mitigationSalt or split celebrity/tenant keys that draw disproportionate traffic
Rebalance costConsistent hashing minimizes key movement when adding/removing shards
Replication Topology Trade-offs
Leader-FollowerSimple, strongly consistent at leader; single write bottleneck
Multi-LeaderRegional write availability; needs conflict resolution (CRDTs/LWW)
Leaderless (Quorum)Tunable via N/W/R; require W + R > N for strict quorum overlap
Synchronous replicationZero data loss on failover; adds write latency
Asynchronous replicationLow write latency; risk of data loss on failover
B-Tree vs LSM-Tree Storage Engines
Write patternB-Tree: in-place random writes; LSM: sequential append to memtable/WAL
Read patternB-Tree: single O(log N) descent; LSM: checks multiple SSTables + bloom filters
Write amplificationLSM higher (compaction rewrites); B-Tree lower
Read amplificationLSM higher (multiple levels); B-Tree lower (one index)
Best forB-Tree: read-heavy OLTP (Postgres/MySQL); LSM: write-heavy (Cassandra/RocksDB)
Consistency Model Cheat Sheet
Strong / LinearizableEvery read sees the latest write; needs coordination (Spanner, Raft/etcd)
EventualReplicas converge once writes stop; no ordering guarantee in the interim
CausalPreserves cause-effect order (a reply is never seen before its comment)
Read-Your-WritesA client always sees the effect of its own prior writes
Monotonic ReadsA client's successive reads never appear to go backward in time
Assessment Integration

Recommended Practice Quizzes on QuizCluster

Test your retention and prepare for timed live coding and MCQ technical screening rounds:

Frequently Asked Questions

Is it better to over-prepare SQL or NoSQL for a database system design interview?

Prepare both, but frame your answer around trade-offs rather than allegiance to one camp. Interviewers reward candidates who can justify a specific choice for a specific access pattern over those who default to one technology.

Do I need to memorize exact internals of every database (Postgres, Cassandra, DynamoDB, MongoDB)?

No — focus on the underlying mechanisms (B-Tree vs LSM-Tree, leader-follower vs leaderless replication, CAP/PACELC positioning) that generalize across products. Naming one or two real systems as examples is enough to show applied knowledge.

How deep should I go into consistency models for a mid-level interview versus a staff-level interview?

Mid-level candidates should clearly explain strong vs eventual consistency and when each is acceptable. Staff-level candidates are expected to also discuss causal consistency, session guarantees, and the operational cost of achieving linearizability.

What is the single most common mistake candidates make in this topic area?

Picking a sharding key or replication topology without stating the trade-off out loud. Interviewers are evaluating your reasoning process, not just whether you land on a 'correct' final answer.

Explore Other Preparation Guides

Software Engineering
How to Prepare for SDE Interview: Complete 2026 Roadmap
16 min readRead →
Java Ecosystem
How to Prepare for Java Developer Interview: Core to Spring Boot & JVM
18 min readRead →
Microservices & Distributed Systems
How to Prepare for Microservices Developer Interview: Distributed Architecture & Cloud
17 min readRead →
System Design
System Design Interview Guide: Complete 2026 Roadmap
21 min readRead →
Databases
SQL Interview Questions & Preparation Guide: Beginner to Advanced
17 min readRead →
Programming Languages
Python Interview Preparation: Complete Guide for 2026
17 min readRead →
Frontend Engineering
React Interview Preparation: React 19 & Next.js Guide
17 min readRead →
Cloud & DevOps
Kubernetes Interview Guide: Architecture, Pods, Networking & Troubleshooting
17 min readRead →
Cloud & DevOps
AWS Solutions Architect Interview Guide: Real Architecture Scenarios
17 min readRead →
Microservices & Distributed Systems
Kafka Interview Guide: Architecture, Consumers, Partitions & Exactly-Once Semantics
17 min readRead →
Backend Engineering
REST API Design Interview Guide: Authentication, Pagination, Versioning & Rate Limiting
15 min readRead →
Cloud & DevOps
Docker Interview Guide: Images, Containers, Networking & Production Debugging
15 min readRead →
Programming Languages
JavaScript & TypeScript Interview Guide: From Closures to the Event Loop
17 min readRead →
Backend Engineering
Node.js Backend Interview Guide: Event Loop, Streams, APIs & Scaling
17 min readRead →
Databases
Redis System Design Guide: Caching, Eviction, Persistence & Distributed Locks
17 min readRead →
Software Engineering
Concurrency Interview Guide: Threads, Locks, Race Conditions & Deadlocks
17 min readRead →
Software Engineering
Dynamic Programming Patterns: How to Recognize and Solve DP Problems
17 min readRead →