My Skill Lab
Machine LearningIntermediate

Deep Learning Foundations

Neural networks from neuron to training loop.

Taught by Bibek Sharma · Data Scientist

What you'll learn

  • Explain how a perceptron turns weighted inputs into a decision, and why a single linear unit cannot solve XOR.
  • Choose the right activation function (ReLU, sigmoid, tanh, softmax) for a given layer and task, and avoid dead/saturated neurons.
  • Trace a complete forward pass through a multi-layer network by hand, tracking shapes at every step.
  • Describe how backpropagation uses the chain rule to assign blame, and how gradient descent uses those gradients to update weights.
  • Diagnose overfitting from training/validation curves and apply L2 regularisation and dropout to close the gap.
  • Write and read a complete PyTorch-style training loop, including zero_grad, backward, and optimizer.step in the correct order.

About this course

Build a real mental model of how neural networks learn, starting from a single perceptron and ending with a working training loop you can read line by line. You will understand every piece — activations, layers, the forward pass, loss, backpropagation, and gradient descent — not as magic, but as a small set of ideas that compose. By the end you will be able to reason about why a network overfits and how dropout and regularisation fight back.

Course content

9 lessons
  1. 1The Perceptron: One Neuron That DecidesHow a single artificial neuron weighs inputs, adds a bias, and fires a decision.Preview
  2. 2Activation Functions: Adding the BendWhy networks need non-linearity, and how ReLU, sigmoid, tanh, and softmax differ.Preview
  3. 3Layers and the Forward PassHow neurons combine into layers and how data flows shape-by-shape to a prediction.
  4. 4Loss Functions: Measuring How WrongTurning predictions and truth into a single number the network can minimise.
  5. 5Backpropagation Intuition: Assigning BlameHow the chain rule sends the error backward to tell each weight how to change.
  6. 6Gradient Descent: Learning by Stepping DownhillUsing gradients to update weights, and how the learning rate controls the journey.
  7. 7Overfitting and the Bias-Variance Trade-offWhy a model that aces training data can fail on new data, and how to spot it.
  8. 8Regularisation and Dropout: Fighting OverfittingConcrete techniques — L2 weight decay and dropout — that force a network to generalise.
  9. 9A Complete PyTorch-Style Training LoopAssemble every concept into a readable, correct loop that actually trains a network.