Pandas DataFrame describe() Method

The describe() method in Pandas generates descriptive statistics of DataFrame columns which provides key metrics like mean, standard deviation, percentiles and more. It works with numeric data by default but can also handle categorical data which offers insights like the most frequent value and the number of unique entries. In this article, we'll see how to use describe() for both numeric and categorical data.

Syntax:

DataFrame.describe(percentiles=None, include=None, exclude=None)

Parameters:

• percentiles: A list of numbers between 0 and 1 specifying which percentiles to return. The default is None which returns the 25th, 50th and 75th percentiles.
• include: A list of data types to include in the summary like int, float, object for strings. Default is None means all numeric types are included.
• exclude: A list of data types to exclude from the summary. Default is None means no types are excluded.

The describe() method returns a statistical summary of the DataFrame or Series which helps to understand the key characteristics of our data quickly. Lets see some examples for its better understanding.

Here we will be using NBA dataset which you can download it from here.

1. Using describe() on a DataFrame

Here we will see how the describe() method generates a statistical summary for numeric columns such as age and salary. This is a basic use case of describe() to give us an overview of key statistical metrics across the dataset.

Output:

Descriptive Statistics for Numerical Columns generated using .describe() Method

1. count: Total number of non-null entries in each column.
2. mean: Average (mean) of the values in the column.
3. std: Standard deviation showing how spread out the values are.
4. min: Minimum value in the column.
5. 25%: The 25th percentile (Q1) which means 25% of the data points are less than this value.
6. 50%: Median value (50th percentile) where half the data points are below it.
7. 75%: The 75th percentile (Q3) means 75% of the data points are below this value.
8. max: Maximum value in the column.

This summary provides us a quick overview of the numeric columns in the dataset which helps us understand the distribution of key variables like age and salary.

2. Customizing describe() with Percentiles

We can customize the describe() method by specifying custom percentiles. By passing a list of percentiles we can obtain more detailed insights into our data’s distribution beyond the default 25th, 50th and 75th percentiles.

Output:

This customization is helpful when we need more insights into the distribution of our data such as understanding how values fall within certain ranges or percentiles.

3. Describing String (Object) Data

The describe() method also works with string data i.e object data type. When used on string data, it provides different statistics such as the count of unique values, most frequent values etc. This example shows how to apply describe() to a column containing categorical (string) data.

Output:

For string (object) data, describe() provides:

1. count: Total number of non-null values.
2. unique: Number of unique values in the column.
3. top: Most frequent value in the column.
4. freq: Frequency of the most common value.

This is useful for quickly understanding the distribution of categorical data or identifying the most frequent values.

4. Describing Specific Columns with describe()

We may sometimes want to generate a summary for a specific column in our DataFrame. For example we may be interested in analyzing just the "Salary" column without summarizing the other columns.

Output:

5. Describing Data with include='all'

By using the include='all' parameter we can generate a summary for all columns in the DataFrame regardless of data type. This is helpful when we want to analyze both numeric and categorical data at the same time.

Output:

This is helpful for getting a complete overview of all data in the DataFrame in a single summary.