Real datasets arrive messy, and most of a practitioner's time goes into preparing them. Data cleaning is the process of detecting and correcting errors such as missing values, duplicates, and outliers. Missing data — values recorded as NaN or NULL — can be addressed by dropping affected rows or columns, by imputation using the mean, median, or mode, by more sophisticated model-based imputation, or by choosing algorithms that are robust to missingness. Outliers are points that differ sharply from the rest of the data, and they can be flagged through z-scores, the interquartile range (IQR), boxplots, or model-based methods such as isolation forests. The choice depends on whether an outlier represents an error to remove or a genuine extreme that the model should learn from.
Before modeling, exploratory data analysis (EDA) is the initial investigation that uncovers distributions, anomalies, and early hypotheses. EDA guides what cleaning and engineering steps are actually needed. Two common numerical preparations are normalization, which scales features to a common range such as \([0,1]\), and standardization, which centers features to mean 0 and standard deviation 1. These steps matter because many algorithms are sensitive to the scale of their inputs, and distance-based or gradient-based methods in particular behave poorly when one feature dominates others by virtue of its units.
Categorical variables require their own transformations. One-hot encoding converts a category into a set of binary columns, one per level; label encoding maps each category to an integer; and target encoding replaces a category with the mean of the target variable within that category, which can capture predictive signal but risks data leakage if applied carelessly. Beyond encoding, feature engineering creates new features from raw data — ratios, time deltas, interaction terms, text statistics — to better expose the underlying signal. Feature selection, by contrast, prunes the set of features down to the most relevant ones, reducing noise, speeding training, and sometimes improving generalization. In very high-dimensional settings, practitioners face the curse of dimensionality: distances become uninformative and data becomes sparse, motivating dimensionality reduction methods discussed in a later chapter.
Class imbalance, where one class has far more examples than another, can cripple a naive classifier. It is typically handled by resampling — oversampling the minority, undersampling the majority — or by using synthetic methods like SMOTE, which creates new minority examples by interpolating between existing ones. Class weighting and reframing the problem as anomaly detection are alternative strategies. Together, these preparation choices often determine whether a model is useful at all.