Detect and Remove the Outliers using Python

Last Updated : 7 Apr, 2026

Outliers are data points that are very different from most other values in a dataset. They can occur due to measurement errors, unusual events or natural variation in the data. If not handled properly, they can affect analysis results and reduce machine learning model performance.

👁 Outliers

Outliers

Detecting and Removing Outliers

There are several ways to detect and handle outliers in Python. We can use visualization methods or statistical techniques depending on the type of data. In this section, we will use Pandas and Matplotlib on the Diabetes dataset, which is available in the Scikit-learn library.

1. Visualizing and Removing Outliers Using Box Plots

A box plot helps visualize how data is distributed using quartiles. Any points outside the whiskers of the box plot are considered outliers. It is a simple way to see where most data values lie and identify unusual values.

Output:

In a box plot, outliers appear as points outside the whiskers. These values are much higher or lower than most of the data. For example, BMI values above 0.12 may be considered outliers

👁 bmi11

Box Plot

Removing Outliers

To remove outliers, we can define a threshold value and filter the data.

Output:

👁 download

Box Plot

2. Visualizing and Removing Outliers Using Scatter Plots

Scatter plots help show the relationship between two variables. They are useful when working with paired numerical data. In a scatter plot, outliers appear as points that are far away from the main group of data points.

Output:

From the graph, most data points are grouped in the bottom left corner. However, a few points appear in the opposite area, the top right corner of the graph. These points can be considered outliers because they are far from the main cluster of data.

👁 bm133

Visualizing outliers Using Scatter Plots

Removing Outliers

np.where(): Used to find the positions (indices) in the DataFrame where a condition is true.
(df_diabetics['bmi'] > 0.12) and (df_diabetics['bp'] < 0.8): This condition identifies outliers where bmi is greater than 0.12 and bp is less than 0.8.

Output:

This removes rows where BMI > 0.12 and BP < 0.8 conditions derived from visual inspection.

👁 bmi44

Removing Outliers Using Scatter Plots

3. Z-Score Method for Outlier Detection

The Z-score, also called the standard score, shows how far a data point is from the mean in terms of standard deviations. If the Z-score is greater than a chosen threshold , the value is considered an outlier.

Z-scores are calculated for the 'age' column in df_diabetics.
The zscore() function from SciPy stats is used.
The result z shows how far each value is from the mean in standard deviations.

Output:

An outlier threshold is usually set to 3.0. This is because about 99.7% of data points in a normal (Gaussian) distribution lie within standard deviations from the mean.

👁 bmi55

Z scores

Removing Outliers: Trimming and Capping

After identifying outliers using the Z-score method, we can handle them in two common ways: trimming or capping.

Trimming removes the rows that contain outliers from the dataset.
Capping keeps the rows but replaces extreme values with a predefined limit.

Trimming Outliers

Output:

Original DataFrame Shape: (442, 10)
DataFrame Shape after Removing Outliers: (426, 10)

Capping Outliers

Output:

Original DataFrame Shape: (442, 10)
DataFrame Shape after Capping Outliers: (442, 10)

4. Interquartile Range (IQR) Method

The IQR (Interquartile Range) method is a common and reliable technique for detecting outliers. It works well even when the data is skewed and identifies extreme values using quartiles. The IQR is calculated as the difference between the third quartile (Q3) and the first quartile (Q1):

Here, we calculate the IQR for the 'bmi' column in df_diabetics. First Q1 and Q3 are found, then IQR = Q3 − Q1, which shows the spread of the middle 50% of the data.

Output: