AI glossary: my cheat sheet

Models and tokens

LLM (large language model): a model trained on huge amounts of text to predict the next token. What is an LLM
Token: the unit a model reads and writes, usually a word or part of a word. Tokens and the context window
Tokenizer: splits text into tokens; it differs between model families. Tokens and the context window
Context window: everything the model can see in one request, including its answer. Tokens and the context window
Context rot: accuracy and recall degrade as the context grows. Tokens and the context window
Transformer / attention: the architecture behind LLMs; attention lets each token draw on earlier tokens. How a language model is built
Pretraining: the first training stage, on huge amounts of text; produces a base model. How a language model is built
SFT, RLHF, DPO: supervised fine-tuning on example conversations, then training on human or AI preferences. How a language model is built
Knowledge cutoff: the date after which the model knows nothing. How a language model is built
Open-weight model: a model whose weights are published, so you can run it yourself. Running models locally

Logits / softmax: raw scores per token, turned into probabilities. Temperature, top-p and hallucinations
Temperature, top-p, top-k: knobs that control how the next token is picked. Temperature, top-p and hallucinations
Hallucination: a confident answer that is wrong. Temperature, top-p and hallucinations
System prompt: instructions that apply to the whole conversation. Prompting
Few-shot: steering the model with a few examples in the prompt. Prompting
Prefill: starting the assistant’s answer yourself; rejected by current Claude models. Prompting
Structured outputs: output constrained to a JSON schema. Structured outputs and streaming
Streaming: receiving the answer piece by piece as it is written. Structured outputs and streaming
Thinking / reasoning model: a model that works through a problem before answering. Reasoning models
Effort: a setting that trades thoroughness for speed and cost. Reasoning models
Multimodal: input beyond text, such as images and PDFs. Multimodal
stop_reason: why the model stopped: end_turn, max_tokens, tool_use, refusal and more. What is an LLM

Embedding: a vector that represents the meaning of a text. Embeddings and semantic search
Cosine similarity: a measure of how close two vectors point. Embeddings and semantic search
Vector database: stores vectors and finds the nearest ones fast. Embeddings and semantic search
RAG (retrieval-augmented generation): retrieve relevant text and put it in the prompt. RAG
Chunking: splitting documents into small pieces before indexing. RAG
BM25: the classic keyword-ranking algorithm. Better retrieval
Hybrid search / RRF: combining keyword and vector search, often with reciprocal rank fusion. Better retrieval
Reranker: a model that scores query and candidate together, to reorder results. Better retrieval
Contextual retrieval: adding a short context to each chunk before indexing it. Better retrieval
Agentic RAG: the model searches as a tool, as often as it needs. Agentic RAG and GraphRAG
Knowledge graph / GraphRAG: facts as connected entities; GraphRAG summarizes graph communities to answer global questions. Agentic RAG and GraphRAG

Tool / function calling: the model asks your code to run a function. Tools and function calling
tool_use / tool_result: the model’s request, and the result your code sends back. Tools and function calling
MCP (Model Context Protocol): an open standard for packaging tools, resources and prompts as servers. MCP
Agent: a model that chooses its own steps, using tools in a loop. Agents
Workflow: steps fixed in code, with the model filling in each one. Five workflow patterns
Prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer: the five workflow patterns. Five workflow patterns
Context engineering: deciding what goes into the context on every turn. Memory and context management
Compaction: replacing old turns with a summary. Memory and context management
Memory tool: files the model reads and writes to remember across sessions. Memory and context management
Subagent / multi-agent: a lead agent delegates focused work to agents with their own context. Multi-agent systems
Computer use: a model that operates a desktop through screenshots, mouse and keyboard. Computer use and Agent Skills
Agent Skills: folders of instructions and scripts loaded only when needed. Computer use and Agent Skills
WebMCP: a proposed browser API that lets a web page register tools for agents with document.modelContext. WebMCP

Fine-tuning: further training on your own examples, which changes the weights. Fine-tuning
LoRA / adapter: training a small add-on instead of the whole model. Fine-tuning
QLoRA: LoRA on top of a 4-bit base model. Fine-tuning
Distillation: training a small student model to imitate a large teacher. Distillation
Soft targets: the teacher’s full probability distribution, used to train the student. Distillation
Quantization: storing weights with fewer bits to save memory. Running models locally
Ollama / GGUF: a tool for running open models locally, and a common file format for them. Running models locally

Eval: test cases plus a way to grade the output. Evals
LLM-as-judge: a model that grades output against a rubric. Evals
Trace / span: all the steps of one request, and each step within it. Observability
Prompt caching: reusing a repeated prompt prefix, cheaper and faster. Cost and latency
Batch API: asynchronous requests at a discount. Cost and latency
Prompt injection: untrusted text that tries to act as an instruction; direct or indirect. Security
Excessive agency: a model with more tools or permissions than the task needs. Security
Guardrails: checks around the model on input, output and actions. Guardrails
Red-teaming: attacking your own system on purpose before attackers do. Security