For Candidates Preparing for Interviews

The LLM systems interview rewards candidates who derive answers, not candidates who recall them. Every technique in this section is aimed at building that derivation habit before you sit down with an interviewer.

What the Interview Actually Tests

A senior-level LLM systems interview has three observable signals interviewers are collecting simultaneously.

First: vocabulary precision. “The KV cache grows with sequence length” is not an answer. “At 4,096 tokens, a single sequence in a 7B GQA model with 8 KV heads and head dimension 128 costs approximately 0.17 GB per sequence in bf16” is. The arithmetic formulas in Appendix A are the vocabulary; the chapters explain where each term comes from.

Second: constraint-driven reasoning. Every design question has a binding constraint-usually KV cache memory pressure for serving, usually communication overhead for training. Strong candidates name the binding constraint before proposing any optimization. Candidates who list optimizations without identifying the bottleneck signal that they have read blog posts, not reasoned from hardware physics.

Third: adaptability under constraint changes. The most reliable signal in a live interview is what happens when the interviewer says “what if context length doubles?” or “what if you now need to support five model variants?” A candidate who derived their original answer updates the arithmetic immediately. A candidate who memorized an architecture has to restart. This is why every technical chapter in this book teaches derivation, not recall.

A Four-Week Preparation Plan

This plan assumes roughly two hours of active study per day. Adjust the pacing to your timeline; the sequence of topics is fixed, but the time budget per topic is flexible. Week 1 - Foundation Read Chapters 1-5 and Appendix A fully. After each chapter, close the book and re-derive the key formulas from memory on paper. Target: be able to estimate model memory footprint (inference and training), forward-pass FLOPs, and KV cache size for any model configuration given to you verbally. Week 2 - Hardware and Distributed Systems Read Chapters 8-12. Work through every numerical example with a calculator. Practice explaining the roofline model out loud to an imaginary interviewer. Spend extra time on Chapter 12 (parallelism strategies): draw the TP/PP/DP combination diagram from memory until it takes less than two minutes. Week 3 - Inference and Scaling Read Chapters 13-19. Focus on Chapter 15 (the inference workload) and Chapter 18 (speculative decoding and serving). For each optimization technique in Chapters 16-19, write one sentence explaining which constraint it addresses and what it trades away. Read Appendix C (checklists) and internalize the ten-minute system design checklist. Week 4 - Integration and Drills Read Chapters 20-28 at pace (one per day). Spend the last three days on Part XI (Chapters 29-31), working each drill as a timed mock interview: 45 minutes, whiteboard or

paper, no references. After each drill, compare your answer against the chapter’s annotated response and identify which constraints you named late or which numbers you could not derive.

How to Drill a Chapter

Passive reading does not build the derivation habit. Use this sequence for every chapter in the arithmetic and implementation tiers.

1. Read the chapter fully once, following along with any derivations.

2. Cover the page. Write down the key formula from memory.

3. Plug in a different model configuration (change $d$, change $L$, change $N_{\text{kv}}$) and re-derive the result.

4. Answer the “How This Shows Up” questions from the chapter out loud, targeting 90 seconds per answer before expanding to the full 5-minute version.

5. Read the Key Takeaways bullet list and verify you can explain each point without the book.

Timing Your Answers

A 45-minute design interview has roughly the following pacing for a strong candidate:

The requirement clarification phase is the most commonly skipped and the most damaging to skip. Interviewers at staff and principal level watch for it explicitly.

Common Failure Modes

Cargo-culting numbers without derivation Stating “Chinchilla says 20 tokens per parameter” without being able to derive where the 20 comes from, or what changes when inference cost is amortized over more queries. Chapter 14 derives this ratio and explains when it does not apply. Conflating latency and throughput Describing TTFT and tokens-per-second as if they optimize together. They do not; Chapter 15 derives why and what each responds to. Listing optimizations before naming the bottleneck Proposing speculative decoding, FlashAttention, and quantization in the same breath before establishing whether the system is KV-cache-constrained, compute-constrained, or communication-constrained. Chapter 2 provides the framework for identifying the constraint first. Treating batch size as a free variable Failing to recognize that batch size is bounded by VRAM and that both data parallelism and pipeline parallelism consume it as a resource. Appendix A and Chapter 12 derive the precise constraints. Ignoring inference economics during training design Designing a training run without considering the serving cost of the resulting model. Chapter 13 establishes why inference cost is a first-class training constraint.

Using the Appendices

Appendix A (formulas) is the most important reference during active preparation. Print it. Put it next to your desk. After two weeks of preparation you should not need to look at it; if you still do, spend another session re-deriving each formula from scratch.

Appendix B (question bank) is organized by topic. Use it to simulate a 45-minute interview: pick one design question and one deep-dive question from the same topic area, set a timer, and answer both without references.

Appendix C (checklists) provides a ten-minute LLM system design checklist, a pre-training run readiness checklist, and an inference deployment readiness checklist. Internalize the design checklist until you can reproduce it verbally in under two minutes.

Appendix D (reading list) is for candidates preparing at the staff or principal level, where interviewers expect familiarity with the primary literature. Papers are annotated with the specific claims you are expected to reproduce, not just name.