Source Index

Source: llm-system-interview.pdf (527 PDF pages).

Front Matter

Section	PDF pages
Title Page	1
Contents	2–6
Preface	7
How to Use This Book	8–10
For Candidates Preparing for Interviews	11–13
For Engineers Building LLM Systems	14–16
Notation and Symbols	17–21
Acknowledgments	22
About the Author	23

Chapters and Appendices

Section	PDF pages	Printed pages
The LLM Systems Interview	25–30	2–7
How to Approach an LLM System Design Question	31–38	8–15
Transformer Architecture Interview Essentials	40–62	17–39
Hyperparameters You Will Be Asked to Justify	63–85	40–62
Stability Tricks in Large-Scale Training	86–98	63–75
Why MoE Is Now Standard in Frontier Systems	100–111	77–88
MoE Training and Systems Considerations	112–129	89–106
GPU Architecture for LLM Engineers	131–143	108–120
Making a Single GPU Go Fast	144–161	121–138
Writing and Benchmarking Custom Kernels	162–181	139–158
Multi-GPU and Multi-Node Fundamentals	183–200	160–177
Parallelism Strategies	201–225	178–202
Predictable Scaling for Interview Answers	227–238	204–215
Chinchilla and Beyond	239–256	216–233
The Inference Workload	258–269	235–246
Reducing the KV Cache	270–280	247–257
Going Beyond the Transformer for Inference	281–292	258–269
Speculative Decoding and Serving Optimizations	293–305	270–282
Compression Techniques for Deployment	306–316	283–293
Pre-Training Data Pipelines	318–334	295–311
Data Filtering and Deduplication Algorithms	335–352	312–329
Mid-Training and Post-Training Data	353–371	330–348
Designing an Evaluation	373–384	350–361
Benchmarks You Must Know	385–402	362–379
Validity, Contamination, and Real-World Use	403–413	380–390
Supervised Fine-Tuning	415–427	392–404
Preference-Based Alignment (RLHF)	428–445	405–422
Reinforcement Learning from Verifiable Rewards	446–463	423–440
Building a Production LLM Serving Stack	465–476	442–453
Designing a Pre-Training Run from Scratch	477–488	454–465
Designing a Fine-Tuning and Alignment Pipeline	489–500	466–477
Napkin Math for LLM Interviews	501–505	478–482
Common Interview Questions	506–512	483–489
Checklists for System Design Answers	513–517	490–494
Further Reading	518–527	495–504

Part Dividers

Part-divider titles are retained in the index.

Part	PDF page
Part I — Overview and Interview Landscape	24
Part II — Foundations: Transformer Architecture and Training Signals	39
Part III — Mixture of Experts and Sparse Architectures	99
Part IV — GPUs, Kernels, and Single-Device Performance	130
Part V — Distributed Training: Parallelism at Scale	182
Part VI — Scaling Laws and Training Economics	226
Part VII — Inference Systems	257
Part VIII — Data: The Real Differentiator	317
Part IX — Evaluation	372
Part X — Alignment and Post-Training	414
Part XI — End-to-End System Design Drills	464