Preface
This book exists because the interview and the job have converged in a way they did not five years ago.
Until recently, most software engineering interviews tested general systems-design intuition: databases, caches, load balancers, message queues. The underlying assumption was that specialized domain knowledge could be learned on the job once a strong generalist was hired. That assumption has broken down at frontier AI companies. A candidate who cannot reason about KV cache memory pressure, arithmetic intensity on a GPU, or the difference between compute-bound prefill and memory-bandwidth-bound decoding will not pass a senior-level systems interview at the teams building today’s most important models. The domain has become so operationally specific that generalist intuition is no longer sufficient, and no single paper, blog post, or course covers it end to end.
The gap this book fills. Academic courses teach the theory of transformers; they rarely explain why GQA with reduces KV cache memory by and how that changes the feasible batch size on a single H100. Research papers present results; they do not explain how to answer “walk me through how you would size the hardware for a 70B model serving 5,000 QPS” in the 45 minutes of a live interview. Engineering blogs cover individual components in depth; they do not provide the unified arithmetic framework that lets you move fluidly from model parameters to memory budgets to parallelism strategies to cost estimates. This book does.
What this book is. It is a technical reference and an interview preparation guide written for engineers who already know how to code and who already understand basic ML. Every chapter develops the arithmetic behind a topic from first principles, shows how that arithmetic connects to real hardware constraints, and then frames the result as an interview question with a worked answer. The appendices collect the most important formulas for fast reference during active preparation.
What this book is not. It is not an introduction to deep learning. It does not teach backpropagation or explain what a transformer is from scratch. It does not survey the research literature comprehensively. Readers who need that foundation first should work through a course on neural networks before returning here. It is also not a prescriptive guide to any specific company’s interview process-the frameworks and derivations are generic because the physics of hardware and the economics of inference are generic.
Coverage. The book follows a deliberate progression. Part I maps the interview landscape and introduces the four-question framework that structures every design answer. Parts II through IV build the technical foundation: transformer architecture, training hyperparameters, GPU hardware, and kernel-level performance. Parts V and VI scale that foundation across distributed training and scaling laws. Part VII covers the inference stack in depth, from the prefill-decode split through KV cache optimizations, speculative decoding, and compression. Parts VIII through X address the three domains that interviewers probe as differentiators-data pipelines, evaluation design, and alignment methods. Part XI synthesizes everything into three full end-to-end design drills.
Using derivations, not memorization. Every number in this book is derived, not asserted. The estimate that a 7B model in bf16 requires approximately 13 GB of inference memory follows from bytes, with . The estimate that a single H100 decode step takes roughly 8 ms at batch size 1 follows from dividing by the HBM bandwidth. These derivations are the point. An interviewer who changes one constraint- “now make it a 13B model”-expects a candidate who can update the arithmetic in real time, not one who has memorized a table. First principles produce that flexibility; memorization does not.