Interview Practice

Interview Guides

Grokking the Advanced System Design Interview

How real distributed systems work: Dynamo, Cassandra, Kafka, Chubby, GFS, HDFS and BigTable, followed by 20 reusable system design patterns.

Start reading Original document

Introduction

  1. 1What Is This Course About?Here is an overview of what to expect from this course.

Dynamo

  1. 2Dynamo: IntroductionLet’s explore Dynamo and its use cases.
  2. 3High-level ArchitectureThis lesson gives a brief overview of Dynamo’s architecture.
  3. 4Data PartitioningLet’s learn how Dynamo distributes its data across a set of nodes.
  4. 5ReplicationLet’s learn how Dynamo replicates its data and handles temporary failures through replication.
  5. 6Vector Clocks and Conflicting DataLet’s learn how Dynamo uses vector clocks to keep track of data history and reconcile divergent histories at read time.
  6. 7The Life of Dynamo’s put() & get() OperationsLet’s learn how Dynamo handles get() and put() requests.
  7. 8Anti-entropy Through Merkle TreesLet's understand how Dynamo uses Merkle trees for anti-entropy operations.
  8. 9Gossip ProtocolLet's explore how Dynamo uses gossip protocol to keep track of the cluster state.
  9. 10Dynamo Characteristics and CriticismLet's explore the characteristics and the criticism of Dynamo's architecture.
  10. 11Summary: DynamoHere is a quick summary of Dynamo for you!

Cassandra

  1. 12Cassandra: IntroductionLet’s explore Cassandra and its use cases.
  2. 13High-level ArchitectureThis lesson gives a brief overview of Cassandra’s architecture.
  3. 14ReplicationLet's explore Cassandra's replication strategy.
  4. 15Cassandra Consistency LevelsLet's explore how Cassandra manages data consistency.
  5. 16GossiperLet's explore how Cassandra uses gossip protocol to keep track of the state of the system.
  6. 17Anatomy of Cassandra's Write OperationLet’s dig deeper into the components involved in Cassandra’s write path.
  7. 18Anatomy of Cassandra's Read OperationLet’s explore Cassandra’s read path.
  8. 19CompactionLet's explore how Cassandra handles compaction.
  9. 20TombstonesLet's explore how Tombstones work in Cassandra.
  10. 21Summary: CassandraHere is a quick summary of Cassandra for you!

Kafka

  1. 22Messaging Systems: IntroductionThis lesson gives a brief overview of messaging systems.
  2. 23Kafka: IntroductionThis lesson presents a brief introduction and common use cases of Kafka.
  3. 24High-level ArchitectureThis lesson gives a brief overview of Kafka’s architecture.
  4. 25Kafka: Deep DiveAs of now, we have discussed the core concepts of Kafka. Let us now throw some light on the workflow of Kafka.
  5. 26Consumer GroupsLet's explore the role of consumer groups in Kafka.
  6. 27Kafka WorkflowLet's explore different messaging workflows that Kafka offers.
  7. 28Role of ZooKeeperLet's delve into how Kafka interacts with ZooKeeper.
  8. 29Controller BrokerThis lesson will explain the role of the controller broker in Kafka.
  9. 30Kafka Delivery SemanticsLet's explore what message delivery guarantees Kafka provides to its clients.
  10. 31Kafka CharacteristicsLet's explore different characteristics of Kafka.
  11. 32Summary: KafkaHere is a quick summary of Kafka for you!

Chubby

  1. 33Chubby: IntroductionThis lesson will introduce Chubby and its use cases.
  2. 34High-level ArchitectureThis lesson gives a brief overview of Chubby’s architecture.
  3. 35Design RationaleLet's explore the rationale behind Chubby's architecture.
  4. 36How Chubby WorksLet’s learn how Chubby works.
  5. 37File, Directories, and HandlesLet's explore how Chubby works with files, directories, and handles.
  6. 38Locks, Sequencers, and Lock-delaysLet's explore how Chubby implements Locks and Sequencers.
  7. 39Sessions and EventsThis lesson will explain Chubby sessions and what different Chubby events are.
  8. 40Master Election and Chubby EventsThis lesson will explain what actions a newly elected master performs. Additionally, we will look into different Chubby events.
  9. 41CachingLet's learn how Chubby implements its cache.
  10. 42DatabaseLet's learn how Chubby uses a database for storage.
  11. 43Scaling ChubbyThis lesson will explain different techniques that Chubby uses for scaling.
  12. 44Summary: ChubbyHere is a quick summary of Chubby for you!

Google File System (GFS)

  1. 45Google File System: IntroductionLet’s explore Google File System and its use cases.
  2. 46High-level ArchitectureThis lesson gives a brief overview of GFS’s architecture.
  3. 47Single Master and Large Chunk SizeThis lesson will explain why Chubby has a single master and a large chunk size.
  4. 48MetadataLet's explore how GFS manages the filesystem metadata.
  5. 49Master OperationsLet's learn the different operations performed by the master.
  6. 50Anatomy of a Read OperationLet’s learn how GFS handles a read operation.
  7. 51Anatomy of a Write OperationLet’s learn how GFS handles a write operation.
  8. 52Anatomy of an Append OperationLet’s learn how GFS handles an append operation.
  9. 53GFS Consistency Model and SnapshottingThis lesson will explain how GFS handles the consistency of its operations and data. Additionally, we will look into how GFS implements a snapshotting operation.
  10. 54Fault Tolerance, High Availability, and Data IntegrityLet's learn how GFS implements fault tolerance, high availability, and data integrity.
  11. 55Garbage CollectionLet's learn how GFS implements garbage collection.
  12. 56Criticism on GFSHere is the summary of criticism on GFS's architecture.
  13. 57Summary: GFSHere is a quick summary of Google File System for you!

Hadoop Distributed File System (HDFS)

  1. 58Hadoop Distributed File System: IntroductionThis lesson gives a brief introduction to the Hadoop Distributed File System.
  2. 59High-level ArchitectureThis lesson gives a brief overview of HDFS’s architecture.
  3. 60Deep DiveLet's explore some of HDFS's design components.
  4. 61Anatomy of a Read Operation
  5. 62Anatomy of a Write OperationThis lesson explains how HDFS handles a write operation.
  6. 63Data Integrity & CachingLet's explore how HDFS ensures data integrity and implements caching.
  7. 64Fault ToleranceLet's explore what techniques HDFS uses for fault tolerance.
  8. 65HDFS High Availability (HA)Let's learn how HDFS achieves high availability.
  9. 66HDFS CharacteristicsThis lesson will explore some important aspects of HDFS architecture.
  10. 67Summary: HDFSHere is a quick summary of HDFS for you!

BigTable

  1. 68BigTable: IntroductionLet’s explore Bigtable and its use cases.
  2. 69BigTable Data ModelThis lesson explains how BigTable models its data.
  3. 70System APIsLet's explore BigTable APIs.
  4. 71Partitioning and High-level ArchitectureThis lesson gives a brief overview of BigTable's architecture and its data partitioning scheme.
  5. 72SSTableLet's learn how Tablets are stored in SSTables.
  6. 73GFS and ChubbyLet's explore how BigTable interacts with GFS and Chubby.
  7. 74Bigtable ComponentsLet's explore the components that constitute BigTable.
  8. 75Working with Tablets
  9. 76The Life of BigTable's Read & Write OperationsLet's explore how BigTable handles its read and write operations.
  10. 77Fault Tolerance and CompactionLet's learn how BigTable handles fault tolerance and data compaction.
  11. 78BigTable RefinementsThis lesson will explore different refinements that BigTable implemented.
  12. 79BigTable CharacteristicsThis lesson will explore some miscellaneous characteristics of BigTable.
  13. 80Summary: BigTableHere is a quick summary of BigTable for you!

System Design Patterns

  1. 81Introduction: System Design PatternsThis lesson gives a brief overview of the system design patterns that we will be discussing in the following lessons.
  2. 821. Bloom FiltersLet's learn about Bloom filters and how to use them.
  3. 832. Consistent HashingLet's learn about Consistent Hashing and its usage.
  4. 843. QuorumLet's learn about Quorum and its usage.
  5. 854. Leader and FollowerLet's learn about leader and followers patterns and its usage in distributed systems.
  6. 865. Write-ahead LogLet's learn about write-ahead logging and its usage.
  7. 876. Segmented LogLet's learn about segmented log and its usage.
  8. 887. High-Water MarkLet's learn about high-water mark and its usage.
  9. 898. LeaseLet's learn about lease and its usage.
  10. 909. HeartbeatLet's learn about heartbeat and its usage.
  11. 9110. Gossip ProtocolLet's learn about gossip protocol and its usage.
  12. 9211. Phi Accrual Failure DetectionLet's learn about Phi Accrual Failure Detection algorithm and its usage.
  13. 9312. Split BrainLet's learn about split-brain and how to handle it.
  14. 9413. FencingLet's learn about fencing and its usage.
  15. 9514. ChecksumLet's learn about checksum and its usage.
  16. 9615. Vector ClocksLet's learn about vector clocks and their usage.
  17. 9716. CAP TheoremLet's learn about the CAP theorem and its usage in distributed systems.
  18. 9817. PACELC TheoremLet's learn about PACELC theorem and its usage.
  19. 9918. Hinted HandoffLet's learn about hinted handoff and its usage.
  20. 10019. Read RepairLet's learn about read repair and its usage.
  21. 10120. Merkle TreesLet's learn about Merkle trees and their usage.