Feature Engineering and Model Tuning
The work that actually wins: features, validation and hyperparameters.
Taught by Bibek Sharma · Data Scientist
What you'll learn
- Encode categorical features with one-hot, ordinal and target encoding and know when each is appropriate
- Scale, transform and impute numeric features inside a scikit-learn Pipeline
- Recognise and eliminate data leakage using proper cross-validation and scikit-learn Pipelines
- Tune hyperparameters efficiently with random search and Optuna
- Train and regularise gradient boosting models with XGBoost and LightGBM
- Build voting and stacking ensembles and judge whether they beat the best single model
About this course
This feature engineering and model tuning course is for analysts and junior data scientists who already know how to fit a scikit-learn model and want to know why theirs underperform. You will learn how to encode categorical variables, scale and transform numeric columns, handle missing values, and avoid the data leakage that makes offline scores lie. You will then build honest cross-validation, run hyperparameter search with grid search, random search and Optuna, train gradient boosting models with XGBoost and LightGBM, and combine models into ensembles. Everything is done in pipelines so the same code runs in evaluation and production. You should be comfortable with Python and pandas and have trained at least one model before. By the end you will have a repeatable workflow that turns a weak baseline into a tuned, trustworthy model.
Course content
7 lessons- 1Why Features Beat AlgorithmsWhat feature engineering is, why it matters more than model choice, and the workflow this course follows.Preview
- 2Encoding Categorical VariablesOne-hot, ordinal and target encoding, and how to pick the right one for cardinality and model type.
- 3Scaling, Transforms and Missing ValuesStandardisation, log and quantile transforms, and imputation strategies that preserve signal.
- 4Validation Without LeakageCross-validation designs that match reality and the leakage traps that inflate offline scores.
- 5Hyperparameter Search: Grid, Random and OptunaSearch strategies from exhaustive grids to Bayesian optimisation, with early stopping to save compute.
- 6Gradient Boosting With XGBoost and LightGBMHow boosting works, the parameters that matter, and early stopping to train fast without overfitting.
- 7Ensembles and StackingCombine diverse models with voting, averaging and stacking to squeeze out the last few points.