What's changed: In-scope coverage: data prep/ingestion/storage/stores/extraction services
1.1Data Preparation and Feature Engineering for ML
Understand data preparation for ML (cleansing, handling missing/outliers, encoding, scaling) and feature engineering basics. The starting point for "Data Preparation for ML" in MLA-C01.
ML success largely depends on data quality. The first step is data preparation and feature engineering—shaping raw data into a form a model can learn from.
1.1.1The data-preparation flow
- Cleansing: impute/drop missing values, handle outliers, remove duplicates.
- Encoding: turn categorical variables into numbers (e.g., one-hot). Scaling: normalize/standardize numeric values.
- Feature engineering: craft predictive features using domain knowledge (aggregations, ratios, time features).
- Imbalanced data: address with oversampling of the minority class, class weighting, etc.
Common on MLA: categorical = encoding (one-hot, etc.), differently scaled numerics = normalize/standardize, imbalanced data = resampling/weighting, handle outliers/missing. Data prep strongly affects performance.
Data prep proceeds "clean → numericize → engineer features." Cleansing imputes missing values (mean/median/mode) or drops them, and handles outliers and duplicates. Encoding turns categoricals into numbers: one-hot for unordered, label/ordinal for ordered, and target/frequency encoding for high cardinality. Scaling includes standardization (z-score) and normalization (Min-Max), important for gradient/distance-based models (tree models need it less); fix skew with log transforms. Feature engineering crafts predictive features with domain knowledge (aggregations, ratios, time features, interactions, binning); use TF-IDF/embeddings for text and preprocessing/augmentation for images. Imbalanced data is handled with oversampling like SMOTE, undersampling, or class weighting, evaluated with F1/PR-AUC rather than accuracy. The biggest pitfall is data leakage: fit scalers/encoders only on the post-split training data and apply the same transform to validation/test.
| Data type/issue | Technique |
|---|---|
| Unordered categorical | One-hot encoding |
| Differently scaled numerics | Standardization / normalization |
| Class imbalance | SMOTE/weighting + F1/PR-AUC |
| Text | TF-IDF / embeddings |
Scenario: a fraud model where positives (fraud) are only 0.5%. One-hot the categoricals; log-transform and standardize amounts. Handle imbalance with SMOTE or class weights, and evaluate with PR-AUC/F1, not accuracy. Fit the scaler on train, apply to val/test to prevent leakage. Add features (recent transaction frequency, ratio to average amount) to lift accuracy.
Q. Categorical variables? Encoding (one-hot, etc.). Q. Differently scaled numerics? Standardize/normalize. Q. Imbalanced data? SMOTE/weighting + F1/PR-AUC. Q. Prevent leakage? Fit only on post-split train. Q. Scaling for tree models? Generally unnecessary.
Watch the mix-ups: (1) Fitting on all data before splitting leaks—always fit on train only after splitting. (2) Accuracy alone on imbalanced data looks high even for "all negative" (use F1/PR-AUC). (3) One-hot explodes columns at high cardinality (consider target/frequency encoding). (4) Don’t compute imputation values from test info (leakage).
Fit scalers/encoders on training data only and apply the same transform to validation/test, so test information does not leak (prevent data leakage).
1.1.2Section summary
- Cleansing / encoding / scaling / feature engineering
- Imbalance via resampling/weighting; fit transforms on training data
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. What commonly makes categorical variables (e.g., color = red/blue/green) usable by an ML model?
Q2. What is a common way to handle a highly imbalanced classification dataset?
Q3. On which data should you fit scalers/encoders?