What's changed: In-scope coverage: data prep/ingestion/storage/stores/extraction services
1.1Data Preparation and Feature Engineering for ML
Understand data preparation for ML (cleansing, handling missing/outliers, encoding, scaling) and feature engineering basics. The starting point for "Data Preparation for ML" in MLA-C01.
ML success largely depends on data quality. The first step is data preparation and feature engineering—shaping raw data into a form a model can learn from.
1.1.1The data-preparation flow
- Cleansing: impute/drop missing values, handle outliers, remove duplicates.
- Encoding: turn categorical variables into numbers (e.g., one-hot). Scaling: normalize/standardize numeric values.
- Feature engineering: craft predictive features using domain knowledge (aggregations, ratios, time features).
- Imbalanced data: address with oversampling of the minority class, class weighting, etc.
Common on MLA: categorical = encoding (one-hot, etc.), differently scaled numerics = normalize/standardize, imbalanced data = resampling/weighting, handle outliers/missing. Data prep strongly affects performance.
Data prep proceeds "clean → numericize → engineer features." Cleansing imputes missing values (mean/median/mode) or drops them, and handles outliers and duplicates. Encoding turns categoricals into numbers: one-hot for unordered, label/ordinal for ordered, and target/frequency encoding for high cardinality. Scaling includes standardization (z-score) and normalization (Min-Max), important for gradient/distance-based models (tree models need it less); fix skew with log transforms. Feature engineering crafts predictive features with domain knowledge (aggregations, ratios, time features, interactions, binning); use TF-IDF/embeddings for text and preprocessing/augmentation for images. Imbalanced data is handled with oversampling like SMOTE, undersampling, or class weighting, evaluated with F1/PR-AUC rather than accuracy. The biggest pitfall is data leakage: fit scalers/encoders only on the post-split training data and apply the same transform to validation/test.
| Data type/issue | Technique |
|---|---|
| Unordered categorical | One-hot encoding |
| Differently scaled numerics | Standardization / normalization |
| Class imbalance | SMOTE/weighting + F1/PR-AUC |
| Text | TF-IDF / embeddings |
Scenario: a fraud model where positives (fraud) are only 0.5%. One-hot the categoricals; log-transform and standardize amounts. Handle imbalance with SMOTE or class weights, and evaluate with PR-AUC/F1, not accuracy. Fit the scaler on train, apply to val/test to prevent leakage. Add features (recent transaction frequency, ratio to average amount) to lift accuracy.
Q. Categorical variables? Encoding (one-hot, etc.). Q. Differently scaled numerics? Standardize/normalize. Q. Imbalanced data? SMOTE/weighting + F1/PR-AUC. Q. Prevent leakage? Fit only on post-split train. Q. Scaling for tree models? Generally unnecessary.
Watch the mix-ups: (1) Fitting on all data before splitting leaks—always fit on train only after splitting. (2) Accuracy alone on imbalanced data looks high even for "all negative" (use F1/PR-AUC). (3) One-hot explodes columns at high cardinality (consider target/frequency encoding). (4) Don’t compute imputation values from test info (leakage).
Fit scalers/encoders on training data only and apply the same transform to validation/test, so test information does not leak (prevent data leakage).
1.1.2Section summary
- Cleansing / encoding / scaling / feature engineering
- Imbalance via resampling/weighting; fit transforms on training data
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. What commonly makes categorical variables (e.g., color = red/blue/green) usable by an ML model?
Q2. What is a common way to handle a highly imbalanced classification dataset?
Q3. On which data should you fit scalers/encoders?
Keep track of your progress
The full study guide is free to read. Sign up free to practice with the question bank, track what you have read, review your mistakes, and highlight passages.