Exploratory Data Analysis (EDA) & Plotting in Python

Exploratory Data Analysis (EDA) is the step where you explore, summarize and visualize data to understand its structure — before applying any machine learning or statistical model. You detect patterns, spot anomalies, test assumptions and check relationships between variables.

Skipping EDA is how people end up training a model on a column that’s 40% empty, or “discovering” a relationship that’s really one outlier. It’s the cheapest insurance in data science.

Why EDA matters

• Gives a clear understanding of the dataset — number of features, data types, and how values are distributed.
• Reveals patterns and relationships between variables.
• Identifies errors and outliers that would otherwise distort your analysis.
• Highlights the most important features for building a model.
• Tests the assumptions your later statistics depend on.

The EDA workflow

Step 1: the numeric summary

Before any chart, get the numbers. These come from Pandas:

import pandas as pd

df = pd.read_csv('data.csv')

df.shape         # => (169, 4)     how big is it?
df.info()        # dtypes + non-null counts (where are the gaps?)
df.describe()    # count, mean, std, min, 25%, 50%, 75%, max
df.isnull().sum()   # exact count of missing values per column
df.nunique()     # how many distinct values per column
df.corr()        # relationships between numeric columns

describe() is the workhorse — in one call you see centre (mean/median), spread (std), and range (min/max) for every numeric column.

How to actually read describe()

Don’t just glance at it. Each row answers a question, and disagreements between rows are where the problems hide:

Row	What it tells you	Red flag
count	how many non-null values	lower than df.shape[0] -> missing data
mean vs 50% (median)	centre of the data	far apart -> skewed, or outliers present
std	typical distance from the mean	0 -> the column is constant and useless; huge -> wild spread
min / max	the extremes	impossible values (negative age, 999 for “unknown”)
25% / 75%	where the middle half sits	a max far beyond 75% -> a long tail

The single most useful check is comparing mean against median. If a column’s mean is 207 and its median is 35, the data is heavily skewed and every average you compute later will be misleading — that’s your cue to investigate outliers before going further.

Step 2: plotting with Pandas + Matplotlib

Pandas has plotting built in; it uses Matplotlib underneath, so you import both.

# pip install matplotlib

import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv('data.csv')

df.plot()        # plots every numeric column as a line
plt.show()       # renders the window / image

plt.show() is required to actually display the figure when running a script.

Scatter plot — the relationship between two columns

Use kind='scatter' and name the two axes:

import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv('data.csv')

df.plot(kind='scatter', x='Duration', y='Calories')
plt.show()

Each dot is one row. If the dots trend upward together, the columns are positively correlated — this is the visual version of df.corr().

Histogram — how one column is distributed

A histogram needs only one column. It shows the frequency of each interval, e.g. “how many workouts lasted between 50 and 60 minutes?”

df["Duration"].plot(kind='hist')
plt.show()

df["Duration"].plot(kind='hist', bins=20)   # more bins = finer detail

The plot kinds you’ll actually use

kind=	Shows	Use it for
'line' (default)	values over a sequence	trends over time
'scatter'	two columns against each other	relationships
'hist'	frequency of value ranges	distribution of one column
'box'	median, quartiles, outliers	spread + outlier detection
'bar'	one bar per category	comparing categories
'pie'	proportions of a whole	parts of a total

Labelling a chart properly

An unlabelled chart is unreadable to anyone else:

df.plot(kind='scatter', x='Duration', y='Calories')
plt.title('Calories burned vs workout duration')
plt.xlabel('Duration (minutes)')
plt.ylabel('Calories')
plt.grid(True)
plt.show()

plt.savefig('chart.png')     # save it to a file instead of displaying

Step 3: the three levels of analysis

EDA is usually organized by how many variables you look at once. This isn’t academic tidiness — it’s a deliberate order of operations. You look at each variable alone first, because a column with 40% missing values or an impossible maximum will poison every relationship you try to measure with it. Only once you trust the individual columns do you start comparing pairs, and only then do you look at everything together.

1. Univariate analysis — one variable

Studies one variable at a time to understand its characteristics and distribution.

Technique	What it reveals
Histogram	how the values are distributed
Box plot	outliers and the spread
Bar chart	counts for a categorical variable

df["Duration"].plot(kind='hist')        # distribution
df["Duration"].plot(kind='box')         # spread + outliers
df["Category"].value_counts().plot(kind='bar')   # category counts

df["Duration"].mean()      # centre
df["Duration"].std()       # spread

What you’re looking for in a histogram: the shape. A single hump near the middle (roughly normal) means means and standard deviations are meaningful. A long tail on one side means the data is skewed and the median is the honest summary. Two separate humps usually means you’ve got two different populations mixed together — often the most valuable discovery in an EDA, because it implies a missing category column. A tall bar at exactly one value can reveal a default or placeholder that was never real data.

What a box plot shows: the box spans the middle 50% of the data (25th to 75th percentile), the line inside is the median, and the whiskers reach out to roughly the normal range. Anything drawn beyond the whiskers is flagged as an outlier — which is exactly why box plots are the fastest outlier detector you have.

2. Bivariate analysis — two variables

Examines the relationship between two variables — how they interact or influence each other.

Technique	What it reveals
Scatter plot	relationship between two numeric variables
Correlation coefficient	strength of that relationship (-1 to 1)
Cross-tabulation	how two categorical variables relate

df.plot(kind='scatter', x='Duration', y='Calories')   # visual
df['Duration'].corr(df['Calories'])                   # => 0.92  (strong!)
pd.crosstab(df['Gender'], df['Category'])             # categorical vs categorical
df.groupby('Category')['Calories'].mean()             # numeric by category

Always plot the scatter alongside the number. A correlation coefficient compresses a whole relationship into one digit, and it only measures how well a straight line fits. It can mislead badly in two directions:

• A curved relationship (strong, but not linear) can score near 0 — the correlation says “nothing here” while the scatter plot shows an obvious arc.
• A single outlier can manufacture a high correlation out of a formless cloud, or destroy a real one.

The scatter plot shows you which of these you’re dealing with in one glance; the number never will. This is the practical lesson of Anscombe’s quartet — four data sets with identical means, variances and correlations that look completely different when plotted.

3. Multivariate analysis — three or more variables

Studies three or more variables together to understand complex relationships.

Technique	What it reveals
Pair plots	relationships between many variables at once
Correlation heatmap	the whole correlation matrix, colour-coded
PCA	reduces dimensions while keeping the important information

import seaborn as sns          # pip install seaborn

sns.pairplot(df)               # scatter plot for every pair of columns
sns.heatmap(df.corr(), annot=True, cmap='coolwarm')   # correlation heatmap
plt.show()

A heatmap of df.corr() is the single most useful multivariate chart — you see every relationship in the data set at a glance.

A complete worked EDA

Putting the whole workflow together on one data set:

import pandas as pd
import matplotlib.pyplot as plt

# 1. LOAD
df = pd.read_csv('data.csv')

# 2. FIRST LOOK
print(df.shape)          # how much data?
print(df.head())         # what does a row look like?
print(df.info())         # types and missing values
print(df.describe())     # numeric summary

# 3. CLEAN
print(df.isnull().sum())                       # where are the gaps?
df.fillna({"Calories": df["Calories"].mean()}, inplace=True)
df.drop_duplicates(inplace=True)

# 4. UNIVARIATE - one variable at a time
df["Duration"].plot(kind='hist', title='Workout duration')
plt.show()

# 5. BIVARIATE - pairs
df.plot(kind='scatter', x='Duration', y='Calories',
        title='Calories vs Duration')
plt.show()
print(df['Duration'].corr(df['Calories']))     # => 0.92

# 6. MULTIVARIATE - everything together
print(df.corr())

# 7. INSIGHT
# Duration correlates strongly (0.92) with Calories, while Pulse barely
# matters (0.03) -> Duration is the feature worth modelling on.

Practical tips

• Always plot before you conclude. Datasets with identical means and correlations can look wildly different when charted (the classic Anscombe’s quartet).
• Investigate outliers, don’t just delete them. An outlier can be a typo — or the most interesting row in your data.
• Check missing data patterns. If values are missing systematically rather than randomly, filling them with the mean will bias your results.
• Correlation is not causation. A strong corr() value means the columns move together, not that one causes the other.

Takeaways

• EDA = explore, summarize, visualize — before modelling, always.
• Start numeric: shape, head(), info(), describe(), isnull().sum().
• Then visual: df.plot() with kind='scatter' for relationships and kind='hist' for distributions; plt.show() to render.
• Work up the three levels — univariate (one variable), bivariate (a pair), multivariate (many at once, via pair plots and heatmaps).
• The goal isn’t pretty charts; it’s knowing your data well enough to trust whatever you build on it.