How to Use This Book

The book supports three distinct uses: interview preparation, on-the-job reference, and systematic study of LLM systems. Read it linearly if you are building up the subject from scratch. Follow a targeted path if you are preparing for a specific role or plugging specific gaps.

Linear Reading Order

The parts are ordered by dependency. Each part assumes the vocabulary and arithmetic developed in all prior parts.

Part	Topic	Prerequisite
I	Interview landscape and framework	None
II	Transformer architecture and training	None
III	Mixture of Experts	Part II
IV	GPU architecture and custom kernels	Part II
V	Distributed training and parallelism	Parts II, IV
VI	Scaling laws and training economics	Parts II, V
VII	Inference systems	Parts II, IV, VI
VIII	Data pipelines	Part II
IX	Evaluation	Parts II, VIII
X	Alignment and post-training	Parts II, VIII, IX
XI	End-to-end design drills	All prior parts

Appendix A (parameter counts, FLOPs, KV cache formulas) and Appendix C (checklists) are designed to be printed and kept at your desk during active preparation. Appendix B is a curated question bank organized by topic. Appendix D is an annotated reading list of papers and engineering reports.

Targeted Reading Paths

Path 1 - Candidate Preparing for an Inference / Serving Interview

Focus: memory arithmetic, KV cache, latency-throughput trade-offs, speculative decoding, quantization.

Chapter 2 (framework) - read fully, internalize the four-axis requirement template.
Chapter 3, Sections 1-2 (transformer essentials, parameter count).
Appendix A (reference formulas) - commit to memory.
Chapter 8 (GPU architecture, roofline model, arithmetic intensity).
Chapter 15 (the inference workload: prefill vs. decode, latency vs. throughput).
Chapter 16 (KV cache reduction: GQA, MQA, MLA).
Chapter 18 (speculative decoding and continuous batching).
Chapter 19 (quantization and compression).
Chapter 29 (full end-to-end serving design drill).

Path 2 - Candidate Preparing for a Training / Infrastructure Interview

Focus: distributed training, parallelism strategies, memory budgets, scaling laws, resilience.

Chapter 2 (framework).
Chapter 3 (architecture) and Chapter 4 (hyperparameters).
Chapter 5 (training stability).
Chapter 8 (GPU architecture).
Chapter 11 (multi-GPU fundamentals: collectives, bandwidth).
Chapter 12 (parallelism strategies: TP, PP, DP, ZeRO/FSDP).
Chapter 13 (scaling laws and napkin math).
Chapter 14 (Chinchilla and compute-optimal training).
Appendix A (FLOPs and memory formulas). 10. Chapter 30 (end-to-end pre-training design drill).

Path 3 - Candidate Preparing for an Applied / Product Engineering Interview

Focus: serving economics, fine-tuning, alignment, evaluation, end-to-end system design.

Chapter 1 (interview landscape, role definitions).
Chapter 2 (framework).
Chapter 15 (inference workload fundamentals).
Chapter 19 (compression and deployment).
Chapter 20-22 (data pipelines and data quality).
Chapter 23-25 (evaluation design, benchmarks, contamination).
Chapter 26 (supervised fine-tuning).
Chapter 27 (RLHF and preference alignment).
Chapter 31 (fine-tuning and alignment pipeline drill).

Chapter Difficulty

Chapters are written at three levels. The level is not labeled explicitly because the same content serves both preparation and reference; the depth at which you engage with it is the variable. Conceptual (most of Parts I, II, VIII, IX, X) Develops vocabulary, mental models, and interview framing.

Suitable as first-pass reading and as review before an interview day. Arithmetic (most of Parts IV, VI, VII; Appendix A) Derives quantitative estimates from first principles. Re-

quires pencil and paper. Work through the examples actively rather than reading passively. Implementation (most of Parts III, V; Chapter 10) Discusses kernel-level, compiler-level, or systems-level

mechanics. Assumed background: familiarity with PyTorch, CUDA concepts, and distributed training frameworks.

Conventions in Each Chapter

Every chapter follows the same internal structure:

The Take - the single most important insight for an interview, stated in one paragraph.
Technical content - derivations, diagrams, or system descriptions.
How This Shows Up - two or three representative interview questions with annotated strong answers.
Key Takeaways - a short bullet list of the points an interviewer will probe. Questions in How This Shows Up appear in italics. Strong-answer annotations focus on structure-what to say first, what to derive, what follow-up to anticipate-rather than providing a script. Interviewers at frontier labs probe depth by changing constraints; the annotations show which numbers to re-derive when that happens.