Feature Engineering in Scikit-Learn: A KDnuggets Cheat Sheet — Practical Rules for Pipelines

# Why pipeline-first preprocessing matters

# Core components to drop into a Pipeline These are the concrete tools the cheat sheet collects and why they matter.

ColumnTransformer and make_column_selector

• ColumnTransformer lets you apply parallel branches of preprocessing to different column groups without manually splitting the DataFrame.
• make_column_selector lets you pick columns by dtype (for example, numeric vs. object) so adding a new column won't force pipeline edits.

Imputation with SimpleImputer(add_indicator=True)

• Use SimpleImputer to fill missing values and set add_indicator=True to create a binary indicator of where values were missing. The pattern of missingness can itself be predictive.

• OneHotEncoder(handle_unknown="ignore") prevents prediction-time failures when unseen categories appear. It's a pragmatic default where cardinality is moderate.
• For high-cardinality categorical features, prefer a TargetEncoder instead of one-hot encoding to avoid exploding the feature space.

Tuning preprocessing with GridSearchCV

• Once preprocessing is inside an estimator (Pipeline), decisions like imputation strategy become hyperparameters. GridSearchCV can tune these alongside model hyperparameters in one search, instead of juggling separate preprocessing experiments.

# Practical pattern to adopt

1. 1Build a Pipeline that contains: column selection (make_column_selector), per-column transformers (SimpleImputer, OneHotEncoder, TargetEncoder, etc.), any feature builders (e.g., PolynomialFeatures), and the estimator.
2. 2Use set_output(transform="pandas") during development to inspect transformed columns with get_feature_names_out().
3. 3Treat preprocessing choices (imputer strategy, encoder type, polynomial degree) as tunable parameters in GridSearchCV.

# Short checklist for day-to-day use

• Put all preprocessing inside Pipeline before fitting.
• Use ColumnTransformer and make_column_selector to keep code robust to schema changes.
• Keep SimpleImputer(add_indicator=True) as a default habit.
• Use OneHotEncoder(handle_unknown="ignore") to avoid runtime crashes.
• Switch to TargetEncoder when categorical cardinality gets large.
• Inspect pipeline output with set_output and get_feature_names_out.
• Include preprocessing choices in GridSearchCV so you can optimize them with model hyperparameters.