Database System Design: SQL vs NoSQL, Sharding, Replication & Indexing
A Practitioner's Playbook for CAP Trade-offs, Partition Keys, Quorum Consistency & Storage Engine Internals

What You Must Master to Clear This Track
- Choose SQL vs NoSQL based on access patterns and consistency needs, not hype — polyglot persistence is the industry norm, not the exception.
- Pick a sharding key that distributes load evenly and matches your dominant query pattern to avoid scatter-gather reads and hot shards.
- Understand replication topology trade-offs cold: leader-follower simplicity vs multi-leader conflict resolution vs leaderless quorum tunability.
- Know B-Tree vs LSM-Tree internals: read-optimized in-place updates vs write-optimized append-and-compact, and their amplification trade-offs.
- Map the CAP theorem and PACELC correctly onto real systems, and be ready to defend a consistency model choice (strong/eventual/causal) under network partitions.
Step-by-Step Study Plan
Follow this sequential roadmap designed to take you from core foundations to advanced architecture and mock interviews.
Relational Modeling, NoSQL Data Models & the CAP Theorem
Normalization vs denormalization, ACID guarantees, document/key-value/wide-column/graph models, and CAP/PACELC reasoning for real systems.
- •Explain the trade-offs between 3NF normalization and deliberate denormalization for read-heavy services.
- •Map five real products (banking ledger, product catalog, session store, social graph, time-series metrics) to the right data model.
- •Correctly classify at least five real databases as CP or AP under the CAP theorem, and justify it with PACELC's latency/consistency axis.
- •Whiteboard a schema for a booking system in both a normalized SQL form and a denormalized document form; compare query complexity.
- •Never say 'NoSQL scales better' without naming the specific access pattern that breaks a relational design.
Partitioning Strategies, Consistent Hashing & Rebalancing
Range, hash, and directory/geo sharding; hot shard mitigation; consistent hashing with virtual nodes; and online shard splitting.
- •Implement a consistent-hashing shard router with virtual nodes from scratch and reason about its rebalancing cost.
- •Identify hot-shard risks in a given schema (e.g. a celebrity user, a monotonic timestamp key) and propose a salting or composite-key fix.
- •Design a zero-downtime shard-split runbook including dual-write, backfill, and cutover verification steps.
- •Always state your shard key choice out loud and justify it against the read/write ratio and query filters.
- •Practice explaining why modulo(hash, N) sharding is dangerous when the cluster grows.
Replication Topologies, Quorum Consistency & Index Internals
Leader-follower, multi-leader, and leaderless/quorum replication; strong/eventual/causal consistency; B-Tree vs LSM-Tree storage engines.
- •Trace the full write and read path through a leaderless (Dynamo-style) N/W/R quorum system, including read repair.
- •Explain B-Tree vs LSM-Tree internals well enough to defend a storage engine choice for a given workload (read-heavy vs write-heavy).
- •Design a consistency strategy for a multi-region app: which reads need linearizability, which can tolerate eventual consistency.
- •Run EXPLAIN ANALYZE on a real Postgres query and identify whether it hits an Index-Only Scan or a sequential scan.
- •Rehearse the failure scenario: 'the leader just died mid-write, walk me through what happens next.'
1. SQL vs NoSQL: Data Models, CAP Theorem & PACELC
Every database system design conversation starts with a data model decision. Interviewers are not testing whether you know a list of NoSQL brand names — they are testing whether you can map access patterns and consistency requirements to the right trade-off.
Rows in normalized tables with foreign-key relationships. Atomicity, Consistency, Isolation, and Durability guarantee that multi-row transactions either fully commit or fully roll back, making SQL the default for financial ledgers and inventory counts.
Document stores (MongoDB) embed nested JSON for single-entity reads; key-value stores (DynamoDB, Redis) optimize O(1) lookups at massive scale; wide-column stores (Cassandra) optimize high write throughput and time-series; graph databases (Neo4j) optimize multi-hop relationship traversal.
During a network partition, a distributed system must choose Consistency (reject/queue writes until the partition heals) or Availability (serve possibly stale reads). Partition tolerance itself cannot be dropped in any real multi-node deployment.
Even when there is no partition (the common case), a system still trades Latency for Consistency. Spanner is PC/EC (consistent always, pays latency via TrueTime); Cassandra is PA/EL (available always, tunable low-latency reads).
- Never present SQL vs NoSQL as a binary choice in an interview — describe the specific access pattern (write volume, join complexity, consistency need) that drives the decision, and mention polyglot persistence for mixed workloads.
- When asked to justify CAP classification, name the actual failure mode: 'if the network partitions this region, does the system return an error or a stale value?'
- Claiming 'NoSQL means no ACID' — many NoSQL engines (MongoDB, DynamoDB transactions) support single-document or even multi-item ACID transactions today.
- Forgetting that CAP only applies during an actual partition; most production incidents are about the PACELC latency/consistency trade-off, not CAP.
3. Replication Topologies & Consistency Models
Sharding scales throughput; replication scales availability and read capacity. The topology you pick determines exactly what happens when a node dies, a network link flakes, or two clients write to the same key concurrently.
All writes go to one leader, which streams a change log to followers. Simple to reason about and strongly consistent at the leader, but the leader is a single write bottleneck and a failover requires promoting a follower and re-pointing clients.
Multiple regions each accept local writes and replicate asynchronously to each other, giving low write latency and regional write availability, at the cost of needing conflict resolution (last-write-wins, vector clocks, or CRDTs) when the same key is edited in two regions.
Dynamo-style systems (Cassandra, DynamoDB) send every write to N replicas and accept success once W acknowledge; reads query R replicas and merge. Choosing W + R > N guarantees at least one overlapping replica between every write and read.
Strong/linearizable consistency makes replicas behave as one node (highest coordination cost). Eventual consistency only guarantees convergence once writes stop. Causal consistency sits between them, preserving cause-and-effect ordering (a reply is never seen before the comment it replies to) without full linearizability's cost.
Dynamo-style N/W/R quorum tuning and how read repair heals a replica that missed a write during a network blip.
- State your durability requirement before picking sync vs async replication: synchronous replication guarantees zero data loss on failover but adds write latency equal to the slowest replica.
- When asked about multi-region writes, immediately raise conflict resolution — it's the detail that separates a senior answer from a junior one.
- Reading from a follower for a just-written value on a critical path (e.g. showing a user their own new order) without a read-your-writes guarantee, causing a confusing 'it disappeared' bug.
- Configuring W + R <= N and assuming quorum still yields strong consistency — it does not; overlap is not guaranteed.
4. Indexing Internals: B-Tree vs LSM-Tree Storage Engines
Every replication and sharding decision sits on top of a storage engine. Knowing whether that engine is a B-Tree or an LSM-Tree explains why a workload is fast or slow, and it is one of the most reliably asked deep-dive topics in senior database design interviews.
A balanced tree of fixed-size pages (typically 4-16KB) where every lookup, insert, or delete takes O(log N) page traversals. Writes are in-place, so a single row update touches only its own leaf page, making B-Trees the default in PostgreSQL, MySQL InnoDB, and SQL Server.
Writes go to an in-memory memtable plus a write-ahead log, and are periodically flushed as immutable sorted files (SSTables) to disk. Writes are always sequential appends, giving very high write throughput; used by Cassandra, RocksDB, LevelDB, and HBase.
LSM-Trees periodically merge multiple SSTables into fewer, larger ones (compaction) to bound the number of files a read must check, which rewrites data multiple times (write amplification) in exchange for lower read amplification over time.
A composite index that includes every column a query selects lets the engine answer entirely from the index's leaf pages, skipping the heap/table fetch entirely — the single highest-leverage indexing optimization for hot read paths.
-- Composite index ordered to satisfy the WHERE equality predicates first,
-- then the ORDER BY column, so no separate filesort is needed.
CREATE INDEX idx_orders_customer_status_created
ON orders (customer_id, status, created_at DESC)
INCLUDE (total_amount); -- PostgreSQL covering index via INCLUDE
-- This query can be satisfied entirely from the B-Tree's leaf pages:
EXPLAIN (ANALYZE, BUFFERS)
SELECT customer_id, status, created_at, total_amount
FROM orders
WHERE customer_id = 48213
AND status = 'SHIPPED'
ORDER BY created_at DESC
LIMIT 20;
-- Index Only Scan using idx_orders_customer_status_created
-- Heap Fetches: 0 <- proof the query never touched the table heap- When asked to pick a storage engine, ask about the read:write ratio first — B-Tree engines win for read-heavy OLTP, LSM-Tree engines win for write-heavy ingestion and time-series workloads.
- Mention bloom filters when discussing LSM-Tree reads: they let the engine skip SSTables that provably do not contain a key, cutting read amplification.
- Adding a secondary index for every filterable column without considering that each additional index slows down every write, since the engine must maintain it transactionally.
- Choosing an LSM-Tree engine for a point-lookup-heavy workload without tuning compaction strategy or bloom filter false-positive rate, leading to read amplification worse than a B-Tree would have given.
Splitting a Monolithic Postgres Table Before It Became a Scaling Wall
A B2B SaaS analytics company ran all customer event data in a single 900GB PostgreSQL 'events' table shared across every tenant. During Q4 onboarding pushes, write latency p99 spiked past 2 seconds and autovacuum stalls began blocking dashboard reads for every customer simultaneously.
- 1Profiled query and write patterns and found over 95% of reads and writes filtered by tenant_id, making it the natural shard key instead of the existing auto-increment event_id.
- 2Stood up a consistent-hashing router in front of 8 new PostgreSQL shards (128 virtual nodes per shard) and migrated historical data via a dual-write-plus-backfill process.
- 3Switched each shard from a single instance to one leader with two asynchronous follower replicas, routing analytics dashboard queries to followers to isolate them from OLTP write load.
- 4Added a composite covering index on (tenant_id, event_type, created_at) per shard, eliminating the sequential scans that had been driving CPU above 90% during ingestion spikes.
- 5Ran a two-week shadow-read verification period comparing results from the old monolith and the new sharded cluster before cutting traffic over behind a feature flag.
Top Must-Know Interview Questions & Model Answers
Q1: When would you choose NoSQL over a relational database for a new service, and what are you giving up?
- •NoSQL engines shine when the access pattern is dominated by single-entity reads/writes keyed by a partition key (e.g. a user profile document or a session blob).
- •You lose the ability to run arbitrary JOINs across entities efficiently; instead you must denormalize or perform joins in application code, increasing complexity.
- •Most teams end up with polyglot persistence: relational for billing/ledger data, NoSQL for high-volume event or session data.
Q2: What is polyglot persistence and when does it add more operational risk than it removes?
- •Using Postgres for orders, Redis for sessions, and Elasticsearch for search is a classic, well-justified polyglot split because each engine solves a problem the others solve poorly.
- •The risk appears when teams adopt a new engine for a marginal performance gain, then must maintain expertise, backups, and monitoring for yet another storage system indefinitely.
Q3: Explain the differences between document, key-value, wide-column, and graph NoSQL models with one canonical use case each.
- •Document (MongoDB): one document per catalog item with embedded variants/attributes, read in a single fetch.
- •Key-Value (Redis, DynamoDB): session tokens or shopping-cart state keyed by user ID, no query flexibility needed.
- •Wide-Column (Cassandra): sensor readings keyed by (device_id, timestamp), optimized for sequential writes.
- •Graph (Neo4j): 'friends of friends' or fraud-ring detection queries that would require many recursive SQL joins.
Q4: What is NewSQL and how do systems like Google Spanner or CockroachDB provide ACID transactions at global scale?
- •Spanner uses TrueTime, a globally synchronized clock with bounded uncertainty, to assign commit timestamps that guarantee external consistency across data centers.
- •CockroachDB uses hybrid logical clocks and Raft consensus per range to achieve similar serializable guarantees without specialized atomic clock hardware.
Q5: Why can normalizing a schema hurt performance at scale, and when should you deliberately denormalize?
- •A fully normalized order system might require joining orders, order_items, products, and customers just to render one order confirmation page.
- •Denormalizing by embedding a snapshot of product name/price into order_items avoids the join but requires explicit handling when the source product record changes.
Q6: How do you choose a sharding key, and what happens if you pick a low-cardinality or monotonically increasing key?
- •Low cardinality (e.g. a status flag with 3 values) means only 3 possible shard destinations regardless of cluster size, capping horizontal scale.
- •A monotonically increasing key like an auto-increment ID or a raw timestamp sends 100% of new inserts to whichever shard owns the current highest range, creating a permanent hot shard.
Q7: Explain consistent hashing and why it minimizes data movement compared to modulo-based sharding when nodes are added or removed.
- •With modulo sharding, changing N from 4 to 5 changes the result of hash(key) % N for almost every key, forcing a near-total data reshuffle.
- •Consistent hashing with virtual nodes (100-200 per physical shard) also smooths load distribution, since a single physical node maps to many ring positions instead of one.
Q8: What is a 'hot shard' and name three mitigation strategies.
- •Salting appends a random or hashed suffix to a hot key (e.g. celebrity_id#0-9) so writes spread across 10 sub-keys, merged at read time.
- •A read-through cache (Redis) in front of the hot shard absorbs read traffic without touching the underlying shard.
- •Splitting a single celebrity tenant into its own dedicated shard removes it from the shared pool entirely.
Q9: How do you handle cross-shard joins or transactions in a sharded relational database?
- •Co-location: choosing tenant_id as the shard key for every table in a multi-tenant app ensures a tenant's orders and order_items always live on the same shard.
- •When cross-shard reads are unavoidable (e.g. a global leaderboard), stream changes via CDC (Debezium) into a single analytical store rather than querying shards live.
Q10: Compare range-based, hash-based, and directory/geo-based sharding strategies.
- •Range sharding is ideal for time-series queries like 'get all events between two timestamps' if the key isn't purely monotonic.
- •Hash sharding is the default choice when write distribution matters more than range queries.
- •Directory-based sharding is required when compliance (GDPR) mandates that EU user data physically resides in EU-based shards.
Q11: How would you design an online shard-splitting/rebalancing operation with zero downtime?
- •Start dual-writing new data to both the source shard and the destination shard(s) so no writes are lost during migration.
- •Backfill historical rows in batches with rate limiting to avoid overloading the source shard's I/O.
- •Run a shadow-read comparison period where reads are served from the old shard but silently validated against the new one before cutover.
Q12: Compare leader-follower, multi-leader, and leaderless replication topologies.
- •Leader-follower (Postgres streaming replication, MySQL): best when writes are naturally centralized and simplicity matters more than multi-region write latency.
- •Multi-leader: best for multi-region apps needing local write latency, at the cost of needing conflict resolution logic (LWW, CRDTs, vector clocks).
- •Leaderless (Cassandra, DynamoDB): best for very high availability requirements where tunable per-request consistency (via N/W/R) is acceptable.
Q13: What is replication lag, and what problems does it cause for a read-after-write user experience?
- •A classic symptom: a user submits a comment, gets redirected to a page that reads from a follower, and briefly sees the comment missing.
- •Fixes include read-your-writes session guarantees (route a user's reads to the leader for a short window after their write), sticky sessions, or version-stamping reads to require a minimum replica freshness.
Q14: How does synchronous vs asynchronous replication affect durability and latency trade-offs?
- •Fully synchronous replication to all followers is rare in practice because one slow or unreachable follower would block every write.
- •A common middle ground is semi-synchronous replication: wait for at least one follower to ack before confirming, balancing durability and latency.
Q15: In multi-leader replication, how are write conflicts detected and resolved (LWW, vector clocks, CRDTs)?
- •Last-write-wins is simple but can silently discard a legitimate concurrent update, which is unacceptable for financial data.
- •Vector clocks let the system detect when two writes were truly concurrent (neither caused the other) versus sequential, surfacing genuine conflicts for app-level resolution.
- •CRDTs (e.g. a G-Counter for view counts, an OR-Set for shopping cart items) are designed so concurrent merges always converge to the same correct result without a conflict-resolution step.
Q16: Explain quorum reads/writes (N, W, R) in leaderless replication systems like Cassandra or DynamoDB.
- •A common configuration is N=3, W=2, R=2: any two writes and any two reads are guaranteed to share at least one common replica.
- •Lowering W or R increases availability and lowers latency but weakens consistency guarantees — W=1 accepts a write even if only one replica is reachable.
Q17: Walk through what happens during a leader failover in a Raft or Paxos-based replicated database.
- •A follower whose election timeout expires without a heartbeat increments its term and requests votes from peers; it becomes leader only with a majority.
- •Any uncommitted entries the old leader had replicated to a minority of followers before failing may be discarded to preserve consistency.
Q18: Explain how a B-Tree index works and why it's efficient for read-heavy OLTP workloads.
- •Each internal node holds sorted keys and pointers to child pages, keeping the tree shallow (a 3-4 level B-Tree can index billions of rows).
- •Because pages are updated in place, a B-Tree naturally stays 'read-optimized': a lookup always does the same bounded number of page reads regardless of write history.
Q19: Explain how an LSM-Tree (memtable + SSTables + compaction) works and why it's optimized for write-heavy workloads.
- •Writes never require a random disk seek: they append to the WAL and insert into an in-memory sorted structure (e.g. a skip list).
- •Reads may need to check the memtable plus multiple SSTables (using bloom filters to skip ones that can't contain the key), which is why LSM-Trees trade write speed for higher read cost unless tuned.
Q20: Compare write amplification and read amplification between B-Tree and LSM-Tree storage engines.
- •Write amplification in LSM-Trees comes from compaction: a row written once may be rewritten at L0, then merged into L1, L2, etc.
- •Read amplification in LSM-Trees comes from having to check multiple SSTable levels before finding the latest value, mitigated by bloom filters and leveled compaction.
- •B-Trees keep both figures modest and predictable, which is why they remain the default for general-purpose OLTP databases.
Q21: What is a covering index, and how does it let a query be satisfied by an Index-Only Scan?
- •In PostgreSQL, INCLUDE columns are stored in the index but not used for ordering/searching, letting a query select extra columns without expanding the search key.
- •The performance win shows up directly in EXPLAIN output as 'Index Only Scan' with 'Heap Fetches: 0', versus a regular Index Scan that still touches the table.
Q22: Why can having too many secondary indexes hurt write throughput?
- •A table with 8 secondary indexes effectively performs up to 9 writes (1 heap + 8 index updates) for every single logical row write.
- •Wide indexes also increase buffer-pool memory pressure, pushing hot pages out of cache and increasing disk I/O for both reads and writes.
Q23: Explain bloom filters and their role in reducing unnecessary SSTable reads in an LSM-Tree engine.
- •Each SSTable gets its own bloom filter built at flush/compaction time; a read checks the filter before touching disk for that SSTable.
- •False positives are possible (the filter says 'maybe' but the key isn't actually there) but false negatives are impossible, preserving correctness while still saving most unnecessary I/O.
Q24: Explain the CAP theorem and why 'consistency, availability, and partition tolerance: pick two' is a common misconception.
- •The 'pick two' phrasing wrongly implies you can build a CA system that ignores partitions; in a multi-node system, a partition will eventually occur regardless of your choice.
- •CAP classification only describes behavior during an active partition — most of a system's life is spent partition-free, which is where PACELC's latency/consistency trade-off matters more.
Q25: What is PACELC and how does it extend CAP theorem reasoning to the normal (non-partitioned) case?
- •Spanner is PC/EC: it stays consistent during partitions and pays a latency cost (via TrueTime commit-wait) even in the normal case to guarantee external consistency.
- •Cassandra is PA/EL: it stays available during partitions and, in the normal case, favors low latency via tunable (often relaxed) consistency levels.
Q26: Differentiate strong consistency, eventual consistency, and causal consistency with concrete examples.
- •Strong: a bank balance check after a transfer must reflect the transfer immediately, everywhere — requires coordination like Raft/Paxos.
- •Eventual: a 'likes' counter on a social post may briefly show different values on different replicas before converging.
- •Causal: a comment reply must never be visible before the comment it replies to, even if the two are stored on different replicas.
Q27: What are session guarantees like read-your-writes and monotonic reads, and why do client applications need them?
- •Without read-your-writes, a user who just updated their profile picture might refresh and briefly see the old one if routed to a lagging replica.
- •Without monotonic reads, a client could read a newer value from one replica, then an older value from a different replica on the next request, appearing to go back in time.
Q28: How does a distributed database achieve linearizability, and what's the performance cost?
- •Achieving linearizable reads often means routing reads through the leader or using a quorum read combined with a lease mechanism to avoid stale data.
- •The cost shows up as higher tail latency (every operation waits for consensus round-trips) and reduced availability during partitions, since a minority partition cannot make progress.
Mistakes That Sink Otherwise Strong Candidates
Why it happens: It's the easiest key to reach for since it already exists as the primary key and requires no extra design work.
The fix: Hash the key or choose a composite/business key (e.g. tenant_id, user_id) with high cardinality so new writes spread across all shards instead of piling onto the newest one.
Why it happens: Cargo-culting the stack used by a well-known high-scale company without checking whether the same access patterns actually apply.
The fix: Model the dominant read/write access patterns first (single-table design exercise), then choose the storage engine that best serves those patterns.
Why it happens: Default asynchronous replication configurations hide the actual lag until a traffic spike or network issue widens it.
The fix: Add read-your-writes guarantees (route a user's own reads to the leader shortly after their write) or use causal consistency instead of assuming the lag window is negligible.
Why it happens: Read replicas are added purely to reduce leader load, without considering which specific reads must see the latest write.
The fix: Route session-critical reads (e.g. confirming a user's own new order) to the leader, or implement sticky-session / version-token read-your-writes guarantees.
Why it happens: Relational best practices (3NF) are taught as a default without weighing them against the actual read/write ratio of the system.
The fix: Selectively denormalize hot write/read paths, or introduce materialized views/CQRS read models for the specific queries that suffer from excessive joins.
Why it happens: Teams tune N, W, and R independently for latency without checking the overlap guarantee that makes quorum consistency meaningful.
The fix: Always verify W + R > N when strict quorum consistency is required, and understand that sloppy quorums during a partition can still return stale data.
Why it happens: LSM-Trees are chosen reflexively for their write throughput reputation without checking the workload's actual read/write ratio.
The fix: Tune bloom filter false-positive rates and choose an appropriate compaction strategy (leveled vs size-tiered), or use a B-Tree engine if the workload is genuinely read-heavy.
Why it happens: Modulo sharding is the simplest formula to implement and works fine until the first time a node is added or removed.
The fix: Use consistent hashing with virtual nodes so scaling the cluster only remaps a small fraction of the keyspace instead of nearly all of it.
Why it happens: Initial sharding design optimizes purely for OLTP write throughput and ignores analytics/reporting access patterns.
The fix: Design the shard key around the dominant OLTP query pattern, and stream changes via CDC into a separate OLAP store for cross-shard analytical queries.
Quick-Reference Cheat Sheet
Recommended Practice Quizzes on QuizCluster
Test your retention and prepare for timed live coding and MCQ technical screening rounds:
SQL & NoSQL Engines
Drill relational schema design, NoSQL data modeling, indexing, and transaction isolation across major database engines.
High-Level System Design (HLD)
Practice sharding, replication, caching, and CAP theorem trade-off scenarios in full system design contexts.
Frequently Asked Questions
Is it better to over-prepare SQL or NoSQL for a database system design interview?
Prepare both, but frame your answer around trade-offs rather than allegiance to one camp. Interviewers reward candidates who can justify a specific choice for a specific access pattern over those who default to one technology.
Do I need to memorize exact internals of every database (Postgres, Cassandra, DynamoDB, MongoDB)?
No — focus on the underlying mechanisms (B-Tree vs LSM-Tree, leader-follower vs leaderless replication, CAP/PACELC positioning) that generalize across products. Naming one or two real systems as examples is enough to show applied knowledge.
How deep should I go into consistency models for a mid-level interview versus a staff-level interview?
Mid-level candidates should clearly explain strong vs eventual consistency and when each is acceptable. Staff-level candidates are expected to also discuss causal consistency, session guarantees, and the operational cost of achieving linearizability.
What is the single most common mistake candidates make in this topic area?
Picking a sharding key or replication topology without stating the trade-off out loud. Interviewers are evaluating your reasoning process, not just whether you land on a 'correct' final answer.