AI Engineering by Chip Huyen: Comprehensive Guide

AI Engineering: Building Applications with Foundation Models

Author: Chip Huyen (O'Reilly Media, 2025)

A comprehensive, production-grade guide for engineers transitioning from traditional machine learning and data engineering to building, evaluating, optimizing, and operating reliable foundation model applications.

Key Focus Areas:

• Modern Foundation Model Architecture & Token Economics
• Systematic Evaluation & LLM-as-a-Judge Methodologies
• Production Prompt Engineering & Defensive Guardrails
• Hybrid RAG & Autonomous Multi-Step Agentic Loops
• PEFT, LoRA/QLoRA & Direct Preference Optimization (DPO)
• Inference Latency, PagedAttention & Speculative Decoding

+---------------------------------------------------------------------------------+
|          "AI Engineering: Building Applications with Foundation Models"         |
|                          By Chip Huyen (O'Reilly, 2025)                         |
+---------------------------------------------------------------------------------+
|  A comprehensive synthesis for engineers transitioning from traditional ML/DE   |
|  to building, evaluating, and operating reliable foundation model applications. |
+---------------------------------------------------------------------------------+

About This Book & Study Track

Chip Huyen (former lecturer at Stanford, co-founder of Claypot AI, and author of Designing Machine Learning Systems) published the definitive field guide for the emerging discipline of AI Engineering.

While traditional software engineering treats systems as deterministic code, and traditional Machine Learning focuses on training models from scratch on tabular/image datasets, AI Engineering focuses on building reliable, steerable, and cost-effective production applications on top of existing foundation models.

The 10 Core Chapters

graph TD
    subgraph Fundamentals
        C1[Ch 1: Introduction to AI Apps] --> C2[Ch 2: Understanding Foundation Models]
    end
    subgraph Evaluation
        C2 --> C3[Ch 3: Evaluation Methodology]
        C3 --> C4[Ch 4: Evaluate AI Systems]
    end
    subgraph Application_Building
        C4 --> C5[Ch 5: Prompt Engineering]
        C5 --> C6[Ch 6: RAG & Agents]
    end
    subgraph Adaptation_&_Data
        C6 --> C7[Ch 7: Finetuning]
        C7 --> C8[Ch 8: Dataset Engineering]
    end
    subgraph Production_Serving
        C8 --> C9[Ch 9: Inference Optimization]
        C9 --> C10[Ch 10: Architecture & User Feedback]
    end

The Three Shifts from Traditional ML to AI Engineering

1. From Model Training to Model Steering: Engineers spend less time optimizing backward propagation loss and more time curating prompts, building retrieval pipelines, and tuning steering adapters.
2. From Deterministic Evaluation to Probabilistic Graders: Replacing static test sets with continuous evaluation pipelines, LLM-as-a-judge rubrics, and online user feedback.
3. From Compute-Bound Training to Memory-Bound Inference: The major cost and latency bottleneck shifts from training cluster FLOPs to serving KV cache memory bandwidth and token generation economics.