Andrew Ng's AI Engineering Skills Map describes the work involved in building and operating AI applications, beyond calling a model API.
In an essay shared by Andrew Ng (@AndrewYNg), he presented an AI Engineering Skills Map informed by more than 10,000 job postings and dozens of structured interviews.
Ng on iterative development: "The key difference between AI applications and non-AI software is that the former’s output is less predictable... Skilled AI engineers repeatedly build a piece of software, examine it, and decide what to try next, taking a sequence of steps that are highly influenced by intermediate results. Being able to skillfully decide what to do next allows you to create reliable software systems based on unreliable AI components."
The map's four top-level areas
Ng organizes the map into four broad areas:
LLM foundations, data grounding, agentic loops, evaluations, and production operations.
Clean code, modular API architecture, distributed systems, concurrency, and robust error handling.
Using coding agents such as Claude Code, Cursor, and Codex as part of the development workflow.
Product intuition, scoping feasibility, identifying high-leverage bottlenecks, and system design tradeoffs.
My notes on the application-building part of the map
Everything in this section is my compression and interpretation of Ng's map. The examples and implementation choices are mine unless explicitly quoted; they are not a list of tools endorsed by Ng.
Model behavior
Tokenization and autoregressive generation help explain some model constraints, but they do not make individual failures predictable.
- Context Window Tradeoffs: Managing KV cache hits, knowledge cutoffs, and cost-per-token budgets.
- Sampling & Reasoning Parameters: Calibrating temperature, top-p, reasoning effort levels, and tool-calling schemas.
- Deployment Strategy: Knowing when to fine-tune, when to self-host (e.g. via FreeToken / vLLM), and when to use multimodal models.
Grounding with data
The map treats data grounding as broader than vector search, including multimodal document processing and hybrid retrieval:
- Prompt Context vs. Tool Retrieval: Deciding what data lives statically in the system prompt versus what the LLM queries on-demand via tools.
- Retrieval choices: Selecting retrieval methods that fit the structure of the available data.
- Data preparation: Parsing documents, tables, web pages, and images into material the application can retrieve.
Tools and control loops
My interpretation is that agentic systems require explicit control over loops, tools, and failure handling (as explored in my agent-harness note):
- Loop Control & Fallbacks: Determining when to execute deterministic code vs. triggering LLM reasoning steps.
- Tool execution: Giving a model constrained access to the tools required for the task.
- Orchestration and security: Choosing a simple loop before adding more agents, while limiting prompt-injection and data-exposure risks.
Evaluation and error analysis
Ng gives particular weight to repeated error analysis: build a version, inspect its failures, and use those results to choose the next change.
- Trace analysis: Inspecting execution traces and error patterns to locate failures.
- Evaluation methods: Combining deterministic checks, model-based review, and human review where each is appropriate.
- Test maintenance: Rechecking evaluation sets as the application changes.
Production operation
Managing live applications under the constraints of latency, token cost, and non-deterministic behavior:
- Regression testing: Setting test thresholds according to the cost of errors.
- Observability & Drift Detection: Monitoring token usage, latency percentiles, and catching silent output drift.
- Cost and latency: Selecting models and workflow steps that meet the application's budget and response-time limits.
Machine-learning basics
Machine-learning concepts provide useful tools for diagnosing probabilistic systems:
- Core Mental Models: Bias vs. variance tradeoffs, training loss dynamics, and reinforcement learning principles.
- Data Engineering: Structuring datasets, synthetic data curation, and evaluation benchmark curation.
What the map emphasizes
The map puts evaluation and iteration alongside model knowledge, software engineering, and production operations.
Reliable AI applications require more than prompt techniques. They need explicit harnesses, evaluation pipelines, monitoring, and repeated review of model failures.