Agentic RAG and GraphRAG
Two steps beyond a fixed RAG pipeline: letting the model search as a tool, as many times as it needs, and building a knowledge graph for questions that span many documents.
Tag: AI
31 entries
Agents: a model with tools in a loop
What makes a system an agent, how it differs from a workflow, when an agent is worth it and when it is not, a minimal agent in Python, and the Claude Agent SDK.
Computer use and Agent Skills
Agents that operate a desktop through screenshots, mouse and keyboard, and skills: folders of instructions and scripts that give an agent expertise only when it needs it.
Cost and latency: prompt caching, batches and model choice
Where the money and time go in an LLM system, and the levers that cut them: the right model and effort, shorter outputs, prompt caching, the Batch API and streaming.
Decision map: prompt, RAG, tools, agent or fine-tune?
The whole series in one place: a decision map for choosing the right technique, what each one changes and costs, a worked example, and a checklist before going to production.
Distillation: a small model that learns from a large one
What knowledge distillation is, why a teacher's soft probabilities teach more than the right answer, and how distillation of LLMs works in practice: teacher data, filtering, fine-tuning a student.
Embeddings and semantic search
What an embedding is, how cosine similarity finds texts that mean the same thing, how to create embeddings with Voyage AI, and where to store them.
Putting it together: a support assistant from start to finish
One small app built with the whole series: a prompt, RAG over help articles, a tool for live order data, input screening, evals, caching and logging. Each step links to the post that explains it.
Evals: measuring an LLM system
Why every LLM system needs an evaluation set, how to build one, code-based checks, LLM-as-judge with a rubric, and how to use evals to compare prompts, models and settings.
Fine-tuning: when it helps and how LoRA works
When fine-tuning beats prompting and RAG, and when it does not, what training data looks like, how LoRA and QLoRA train a model cheaply, and how to check the result.
AI glossary: my cheat sheet
The AI terms from the series in one place, grouped by topic, each with a one-line definition and a link to the post that explains it: tokens, RAG, tools, MCP, agents, LoRA, evals and more.
Guardrails: checks around the model
Layered guardrails for LLM applications: screening input with a small model, validating and redacting output, checking actions, limits, and what to show the user when a check fails.
How a language model is built
Transformers, attention, pretraining, fine-tuning and preference training (RLHF, DPO): the stages that turn a pile of text into a chat assistant.
Running models locally: Ollama and quantization
Why run an open-weight model on your own machine, how much memory it needs, what quantization does, and how to call a local model with Ollama, including from the Anthropic SDK.
MCP: the Model Context Protocol
What MCP is, why it exists, hosts, clients and servers, tools, resources and prompts, the two transports, and a small MCP server in Python.
Memory and context management
How agents keep working when a task outgrows the context window: context engineering, compaction, clearing tool results, notes and the memory tool, with safe file handling.
Multi-agent systems and subagents
When several agents beat one: the orchestrator-subagent design, what Anthropic measured in its research system, the token cost, and subagents in the Claude Agent SDK.
Multimodal: images and PDFs
Sending images and PDFs to a model, how image tokens are counted, what vision is still bad at, and what to do with audio.
Observability: seeing what an LLM system does
What to log for every model call and agent step, traces and spans, a tiny tracer in Python, the metrics worth watching, privacy, and turning production failures into evals.
Temperature, top-p and hallucinations
How the next token is picked, what temperature and top-p do, why current Claude models no longer accept them, and why models hallucinate.
Prompting: how to ask a model
The system prompt, clear instructions with a reason, few-shot examples, XML tags and long documents: the prompting techniques the official guide recommends.
Better retrieval: hybrid search, reranking and contextual chunks
Three techniques that fix most RAG retrieval failures: combining keyword and vector search, reranking the candidates, and adding context to every chunk.
RAG: retrieval-augmented generation
The full RAG pipeline: when you need it, splitting documents into chunks, retrieving the closest ones, and building a prompt that answers only from them.
Reasoning models: thinking before answering
What thinking is, how adaptive thinking and the effort parameter control it, how to read thinking blocks, and what it costs.
Security: prompt injection and the OWASP LLM Top 10
The main risks of LLM apps, from prompt injection to excessive agency, and the defenses that work: keeping data apart from instructions, least privilege and screening.
Structured outputs and streaming
Get JSON that always matches your schema with structured outputs and Pydantic, and show the answer while it is being written with streaming.
Tokens and the context window
What a token is, how to count tokens before you send a request, what fills the context window, and what happens when it runs out.
Tools and function calling
How a model calls your functions: defining a tool, the tool use loop, tool results and errors, the raw JSON on the wire, the SDK's tool runner, and how to write good tools.
WebMCP: web pages as tools for AI agents
What WebMCP is and how it relates to MCP: a page registers tools with document.modelContext, forms become tools with two attributes, annotations, iframes, testing in Chrome, and the security risks.
Five workflow patterns
Prompt chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer: the patterns that solve most LLM tasks without a full agent, with short Python for each.
What is an LLM
A large language model predicts the next token, over and over. What that means, what it gets you, and your first API call to one in Python.