Before feeding data to machine-learning models, raw columns usually need to be transformed into more informative features. The simplest step is creating new columns through vectorized arithmetic, such as df['total'] = df['price'] * df['quantity'], or through the element-wise application of a custom function with df['col'].apply(lambda x: x ** 2). These operations are fast because they run on underlying NumPy arrays rather than row-by-row Python loops.
Categorical variables typically need to be converted into numerical form. One-hot encoding creates a separate binary column for each category, and in Pandas this is done with pd.get_dummies(df, columns=['col']). Continuous variables, on the other hand, are often discretized into bins. pd.cut(df['age'], bins=[0, 18, 35, 60, 100], labels=['child', 'young', 'middle', 'senior']) assigns each value to a predefined interval, while pd.qcut(df['value'], q=4, labels=['Q1', 'Q2', 'Q3', 'Q4']) creates equal-frequency bins such as quartiles, so each bin contains roughly the same number of observations.
Feature scaling is essential for algorithms that are sensitive to the magnitude of features, such as K-nearest neighbors or support vector machines. The two most common approaches are min-max scaling, which rescales values to the range \([0, 1]\) using \(x' = \frac{x - x_{\min}}{x_{\max} - x_{\min}}\), and standardization, which centers values around zero with unit variance using \(z = \frac{x - \mu}{\sigma}\). Both are easily applied through scikit-learn: instantiate a MinMaxScaler or StandardScaler, call fit_transform on a column reshaped as df[['col']], and assign the result back. Choosing the right scaler depends on the model: min-max is common for neural networks and image data, while standardization is generally preferred for models that assume approximately Gaussian inputs.