Fine-tuning: when it helps and how LoRA works
When fine-tuning beats prompting and RAG, and when it does not, what training data looks like, how LoRA and QLoRA train a model cheaply, and how to check the result.
AI: adapting a model
AI series 5 of 5. Fine-tuning, distillation and running models locally. When it is worth the effort and when it is not.
3 parts
- Part 1
- Part 2
Distillation: a small model that learns from a large one
What knowledge distillation is, why a teacher's soft probabilities teach more than the right answer, and how distillation of LLMs works in practice: teacher data, filtering, fine-tuning a student.
- Part 3
Running models locally: Ollama and quantization
Why run an open-weight model on your own machine, how much memory it needs, what quantization does, and how to call a local model with Ollama, including from the Anthropic SDK.