Reference · AI Hero
Matt Pocock's vocabulary, in his wording. Once a term is here, every lesson uses it this way.
| Term | Meaning | Source |
|---|---|---|
| AI engineer | “A software developer who builds applications powered by AI.” Builds applications that use AI models via APIs. | what-is-an-ai-engineer |
| ML engineer | Builds the AI APIs themselves. Model training and optimisation. The other side of the API boundary. | what-is-an-ai-engineer |
| Agent | “Doesn't use predetermined steps. It calls an LLM and gives it a bunch of tools.” The LLM decides when to stop. | what-is-an-agent |
| Workflow | Pre-defined code paths. “The code itself decides when to stop the program, when to call the next LLM.” | what-is-an-agent |
| Plan mode | “A restricted mode”: read, explore and analyse, but no editing, commands or tests. | plan-mode-introduction |
| Context window | Everything the agent can see this turn. “An agent can only work with what’s in its context window.” | plan-mode-introduction |
| Feedback loop | Any automatic pass-or-fail signal “so the AI can verify its own work”. A typecheck, a test run, a blocked commit. | essential-ai-coding-feedback-loops |
| Skill | “A teachable capability bundled as a unit — kept out of the context window until a context pointer pulls it in for the task at hand.” | aihero.dev/skills |
| Eval | “Evals are the AI engineer's unit tests. They are how you wrangle predictability from a probabilistic system.” Produces a score, not a pass or fail. | what-are-evals |
| Deterministic eval | Returns “a pass/fail result”. Seeks “to extract determinism from a probabilistic system”. | three-types-of-evals |
| LLM-as-a-judge | An LLM scoring another LLM's output, including factuality against a ground truth. A first screen, not a final test. | three-types-of-evals |
| Human eval | “Some evals can only be usefully evaluated by humans.” Long-form generation. “Human oversight is needed for any type of LLM app.” | three-types-of-evals |
| Staircase Of Complexity Hell | 18 techniques ordered by cost. Simple at the top. “Start at the top of the staircase, and work your way down only when you've exhausted all the simpler options.” | how-to-improve-your-llm-powered-app |
| Deep module | “Lots of implementation controlled by a simple interface.” | how-to-make-codebases-ai-agents-love |
| Grey box architecture | “You own the interface. AI owns the implementation. Tests keep it honest.” | how-to-make-codebases-ai-agents-love |
| Progressive disclosure | Structuring files so the agent can read types and understand services “without digging into the implementation”. Also the rule for AGENTS.md. | a-complete-guide-to-agents-md |
| Hook | “User-defined shell commands” that Claude Code runs at points in its own lifecycle, “which gives you deterministic control”. PostToolUse fires “After a tool call succeeds”; Stop fires “When Claude finishes responding”. Not a git hook. |
code.claude.com/docs/en/hooks-guide |
| Instruction budget | Quoting Kyle at Humanlayer: “Frontier thinking LLMs can follow ~150-200 instructions with reasonable consistency.” Not Matt's own measurement. | a-complete-guide-to-agents-md |
| Vertical slice | A ticket that cuts through every integration layer, rather than a horizontal architectural slice. What /to-tickets produces. |
5-agent-skills-i-use-every-day |
| Tool | A way “of getting LLMs to produce certain types of messages which you can then intercept and execute on your machine”. Defined by name, description and parameters. | what-are-tools |
| Structured output | Passing a schema “directly to the generateObject function from the AI SDK” and reading back a typed, validated object. |
structured-outputs-with-vercel-ai-sdk |
| Evalite | “It's based on Vitest, and it's called Evalite.” A local-first, TypeScript-native tool for testing LLM-powered apps. No API key needed. | evalite-an-early-preview |
| TTFT / TPOT | Time to first token, and time per output token. The two latency numbers when choosing a model. | how-to-choose-an-llm |