Fine-Tuning Large Language Models
LoRA, QLoRA and instruction tuning on open models.
Taught by Rohan Maharjan · ML Engineer
What you'll learn
- Decide between prompting, retrieval and fine-tuning for a given task using cost and evidence
- Build, clean and format an instruction dataset in chat-template form with a proper split
- Explain how LoRA and QLoRA reduce trainable parameters and memory, and choose their hyperparameters
- Run supervised fine-tuning with Transformers, PEFT and TRL on a single GPU
- Evaluate a fine-tuned model against a baseline with automatic metrics and model-graded review
- Merge, quantise and serve the model with vLLM or llama.cpp
About this course
Fine-Tuning Large Language Models is an advanced, hands-on course for engineers who want to adapt open models such as Llama, Qwen and Mistral to their own tasks. You will learn when fine-tuning beats prompting and retrieval, how to build and clean an instruction dataset in chat format, how LoRA and QLoRA make training affordable on a single GPU, and how to run supervised fine-tuning with Hugging Face Transformers, PEFT and TRL. You will then evaluate the result against a held-out set and a baseline, merge and quantise the adapter for inference, and serve it with vLLM or llama.cpp. The course assumes solid Python, familiarity with PyTorch tensors and training loops, and access to a GPU with at least 16 GB of memory or a cloud equivalent. By the end you will have a repeatable pipeline from raw examples to a deployed, measured, task-specific model.
Course content
7 lessons- 1When to Fine-Tune, Prompt or Use RAGA decision framework comparing the three approaches on cost, data, latency, knowledge freshness and behaviour change.Preview
- 2Preparing an Instruction DatasetCollect, clean, format and split chat-style examples, and audit quality before a single training step.
- 3LoRA and PEFT ExplainedWhy full fine-tuning is expensive, how low-rank adapters work, and how to choose rank, alpha and target modules.
- 4QLoRA: Fine-Tuning on a Single GPULoad the base model in 4-bit with bitsandbytes, train LoRA adapters on top, and manage memory on consumer hardware.
- 5Training with TRL's SFTTrainerConfigure and run supervised fine-tuning with TRL, monitor loss and validation, and save adapters correctly.
- 6Evaluating a Fine-Tuned ModelCompare the adapter against the baseline with held-out metrics, task-specific checks, model-graded review and regression tests.
- 7Merging, Quantising and ServingMerge the adapter, export GGUF for llama.cpp or serve with vLLM, and choose quantisation for your latency and quality needs.