Data Preprocessing for Machine Learning

Models don’t accept raw data. Scikit-learn requires numbers only, no gaps — so before any algorithm runs, every dataset has to be cleaned, encoded and split. This step routinely takes longer than the modelling itself, and getting it wrong silently ruins results.

This page works through the whole pipeline on a genuinely messy dataset — missing values and text categories in the same table.

The dataset

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

dataset = pd.read_csv('Data1.csv')
print(dataset)

   Country   Age   Salary Purchased
0   France  44.0  72000.0        No
1    Spain  27.0  48000.0       Yes
2  Germany  30.0  54000.0        No
3    Spain  38.0  61000.0        No
4  Germany  40.0      NaN       Yes
5      NaN   NaN      NaN       NaN
6    Spain   NaN      NaN        No
7   France  48.0  79000.0       Yes
8  Germany  50.0  83000.0        No
9   France  37.0  67000.0       Yes

Two problems a model cannot handle: NaN gaps (rows 4–6) and text columns (Country, Purchased).

Step 1 — Split features (X) from the target (y)

By convention X holds the input features and y the thing you’re predicting. Here we predict Purchased from the other three columns.

x = dataset.iloc[:, :-1].values   # all rows, all columns EXCEPT the last
y = dataset.iloc[:, -1].values    # all rows, ONLY the last column

.iloc selects by position, -1 means the last column, and .values converts the DataFrame into a NumPy array (what scikit-learn works with).

print(x)

[['France' 44.0 72000.0]
 ['Spain' 27.0 48000.0]
 ['Germany' 30.0 54000.0]
 ['Spain' 38.0 61000.0]
 ['Germany' 40.0 nan]
 [nan nan nan]
 ['Spain' nan nan]
 ['France' 48.0 79000.0]
 ['Germany' 50.0 83000.0]
 ['France' 37.0 67000.0]]

X is a 2-D array (a matrix) — rows are samples, columns are features.

print(y)
# ['No' 'Yes' 'No' 'No' 'Yes' nan 'No' 'Yes' 'No' 'Yes']

y is a 1-D array (a vector) — one label per row. This shape difference matters: scikit-learn always expects 2-D X and 1-D y.

Step 2 — Handle missing data

Three options: drop the rows, drop the column, or impute (fill with a substitute). Dropping loses data — with only 10 rows, deleting three is brutal — so we impute.

SimpleImputer fills gaps with a column statistic:

from sklearn.impute import SimpleImputer

imputer = SimpleImputer(missing_values=np.nan, strategy='mean')
imputer.fit(x[:, 1:3])                  # learn the means of Age & Salary
x[:, 1:3] = imputer.transform(x[:, 1:3])  # fill the gaps with them

Note x[:, 1:3] — columns 1 and 2 (Age, Salary) only. You cannot take the mean of a text column, so Country is excluded.

print(x)

[['France' 44.0 72000.0]
 ['Spain' 27.0 48000.0]
 ['Germany' 30.0 54000.0]
 ['Spain' 38.0 61000.0]
 ['Germany' 40.0 66285.71428571429]   <- filled
 [nan 39.25 66285.71428571429]        <- filled
 ['Spain' 39.25 66285.71428571429]    <- filled
 ['France' 48.0 79000.0]
 ['Germany' 50.0 83000.0]
 ['France' 37.0 67000.0]]

The gaps became 39.25 (mean age) and 66285.71 (mean salary).

fit vs transform — the distinction that matters

Method	Does
.fit()	Learns the parameters (here: the column means)
.transform()	Applies them to the data
.fit_transform()	Both at once

This split exists for a reason. Later you must impute the test set using the training set’s means — call .fit() on training data only, then .transform() on both. Fitting on the full dataset leaks information from the test set into training, which inflates your scores. (Strictly, the notebook above imputes before splitting — fine for a teaching example, but in real work impute after the split, or inside a Pipeline.)

Choosing a strategy

strategy=	Fills with	Use when
'mean'	column average	roughly symmetric numeric data
'median'	middle value	numeric data with outliers
'most_frequent'	mode	categorical columns
'constant'	a value you supply	”unknown” is meaningful

The mean-vs-median choice is the same trade-off covered in the statistics notes: outliers drag the mean, so skewed columns should use the median.

Step 3 — Encode categorical data

Models do arithmetic, so text has to become numbers. How you convert it depends on whether the categories have an order.

Independent variables → One-Hot Encoding

The naive approach — France=0, Germany=1, Spain=2 — creates a fake ordering. The model would infer Spain > Germany > France, and that Germany is the “average” of France and Spain. Nonsense.

One-hot encoding avoids this by giving each category its own 0/1 column:

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder

ct = ColumnTransformer(
    transformers=[('encoder', OneHotEncoder(), [0])],  # encode column 0 only
    remainder='passthrough'                            # keep the rest as-is
)
x = np.array(ct.fit_transform(x))
print(x)

[[1.0 0.0 0.0 0.0 44.0 72000.0]
 [0.0 0.0 1.0 0.0 27.0 48000.0]
 [0.0 1.0 0.0 0.0 30.0 54000.0]
 [0.0 0.0 1.0 0.0 38.0 61000.0]
 [0.0 1.0 0.0 0.0 40.0 66285.71428571429]
 [0.0 0.0 0.0 1.0 39.25 66285.71428571429]
 [0.0 0.0 1.0 0.0 39.25 66285.71428571429]
 [1.0 0.0 0.0 0.0 48.0 79000.0]
 [0.0 1.0 0.0 0.0 50.0 83000.0]
 [1.0 0.0 0.0 0.0 37.0 67000.0]]

The single Country column became four columns — France, Germany, Spain and one for the nan, each row flagging its category with a 1. remainder='passthrough' is what keeps Age and Salary in the output; without it they’d be dropped.

Watch the widening. One-hot encoding a column with many distinct values (say 500 postcodes) explodes it into 500 columns. For high-cardinality categories, group rare values into “Other” or use target/frequency encoding instead.

Dependent variable → Label Encoding

The target is different: it’s a single column of class labels, and a model just needs distinct integers for them. No fake ordering problem, because y isn’t used in arithmetic the same way.

from sklearn.preprocessing import LabelEncoder

le = LabelEncoder()
y = le.fit_transform(y)
print(y)
# [0 1 0 0 1 2 0 1 0 1]

No → 0, Yes → 1. But notice the 2 at index 5 — that’s the NaN in the original target getting treated as a third class. It’s a good illustration of a real trap: LabelEncoder will silently encode missing values as their own category. Rows with a missing target should be dropped before encoding, since there’s nothing to learn from them:

dataset = dataset.dropna(subset=['Purchased'])   # do this first

Which encoder to use

Encoder	Use for	Because
OneHotEncoder	nominal features (no order) — Country, Colour	avoids inventing an order
LabelEncoder	the target column y	just needs distinct integers
OrdinalEncoder	ordered features — Small/Medium/Large	the order is real and useful

Step 4 — Split into training and test sets

You must judge a model on data it has never seen. Otherwise you’re testing memorisation, not learning.

from sklearn.model_selection import train_test_split

x_train, x_test, y_train, y_test = train_test_split(
    x, y, test_size=0.2, random_state=1
)

• test_size=0.2 — hold back 20% for testing, train on 80%. (0.2–0.3 is the usual range.)
• random_state=1 — fixes the shuffle so the split is reproducible. Without it you get a different split every run and your scores wobble for no reason.

print(x_train)   # 8 rows
print(x_test)    # 2 rows
print(y_train)   # [0 1 0 0 1 1 0 2]
print(y_test)    # [0 1]

For classification with imbalanced classes, add stratify=y so both sets keep the same class proportions:

train_test_split(x, y, test_size=0.2, random_state=1, stratify=y)

Step 5 — Feature scaling

Age runs 27–50; Salary runs 48,000–83,000. Any algorithm that measures distance (KNN, SVM) or uses gradient descent will let Salary dominate purely because its numbers are bigger.

from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
x_train[:, 4:] = sc.fit_transform(x_train[:, 4:])   # fit on TRAIN only
x_test[:, 4:] = sc.transform(x_test[:, 4:])         # apply the same scaling

Fit on training data, transform both. Fitting the scaler on the test set leaks its distribution into training.

Scaler	Produces	Use when
StandardScaler	mean 0, SD 1	the default; data roughly normal
MinMaxScaler	range 0–1	you need bounded values
RobustScaler	uses median & IQR	data has outliers

Which models need scaling?

Needs scaling	Doesn’t care
KNN, SVM / SVR	Decision Trees
Neural networks	Random Forest
Ridge / Lasso, PCA	Gradient Boosting / XGBoost
K-Means	Plain Linear Regression*

* Linear regression’s predictions are unaffected, but scaling makes the coefficients comparable to each other.

Note we scale only the numeric columns (4:) — one-hot columns are already 0/1 and scaling them destroys their meaning.

The complete pipeline

import numpy as np
import pandas as pd
from sklearn.impute import SimpleImputer
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, LabelEncoder, StandardScaler
from sklearn.model_selection import train_test_split

# 1. Load, drop rows with no target
dataset = pd.read_csv('Data1.csv').dropna(subset=['Purchased'])
x = dataset.iloc[:, :-1].values
y = dataset.iloc[:, -1].values

# 2. Impute missing numbers
imputer = SimpleImputer(missing_values=np.nan, strategy='mean')
x[:, 1:3] = imputer.fit_transform(x[:, 1:3])

# 3. Encode
ct = ColumnTransformer([('encoder', OneHotEncoder(), [0])], remainder='passthrough')
x = np.array(ct.fit_transform(x))
y = LabelEncoder().fit_transform(y)

# 4. Split
x_train, x_test, y_train, y_test = train_test_split(
    x, y, test_size=0.2, random_state=1
)

# 5. Scale the numeric columns
sc = StandardScaler()
x_train[:, -2:] = sc.fit_transform(x_train[:, -2:])
x_test[:, -2:] = sc.transform(x_test[:, -2:])

# Ready for model.fit(x_train, y_train)

Scikit-learn’s Pipeline wraps all of this so the fit/transform discipline is enforced automatically — the professional way to do it once you’re past learning the steps.

Takeaways

• X is 2-D, y is 1-D — iloc[:, :-1] and iloc[:, -1] is the standard split.
• Impute, don’t delete, when data is scarce — mean for symmetric, median for skewed, most-frequent for categorical.
• fit learns, transform applies. Fit on training data only; otherwise you leak test information and overstate your accuracy.
• One-hot encode features (no fake ordering), label encode the target — and drop rows whose target is missing, or LabelEncoder turns NaN into a phantom class.
• Always hold out a test set, with random_state for reproducibility and stratify for imbalanced classes.
• Scale for distance- and gradient-based models; trees don’t need it.