Pandas - Multi-index and Groupby Tutorial

Multi-index and Groupby are very important concepts of data manipulation. Multi-index allows you to represent data with multi-levels of indexing, creating a hierarchy in rows and columns.

Groupby lets you create groups of similar data and apply aggregate functions (e.g., mean, sum, count, standard deviation) to each group, condensing large datasets into meaningful summaries.

Using both these tools together allows you to analyze data from a different aspect.

In this article, we will discuss Multi-index for Pandas Dataframe and Groupby operations.

Multi-index in Python Pandas

Multi-index allows you to select more than one row and column in your index.

It is a multi-level or hierarchical object for Pandas object.

We can use various methods of multi-index such as MultiIndex.from_arrays(),MultiIndex.from_tuples(),MultiIndex.from_product(),MultiIndex.from_frame, etc., which helps us to create multiple indexes from arrays, tuples, DataFrame, etc.

Syntax

pandas.MultiIndex(levels=None, codes=None, sortorder=None, names=None, dtype=None, copy=False, name=None, verify_integrity=True)

Parameters

• levels: It is a sequence of arrays that shows the unique labels for each level.
• codes: It is also a sequence of arrays where integers at each level help us to designate the labels in that location.
• sortorder: optional int. It helps us to sort the levels lexicographically.
• dtype:data-type(size of the data which can be of 32 bits or 64 bits)
• copy: It is a boolean type parameter with a default value of False. It helps us to copy the metadata.
• verify_integrity: It is a boolean type parameter with a default value of True. It checks the integrity of the levels and codes i.t if they are valid.

Let us see some examples to understand the concept better.

Example 1:Creating multi-index from arrays

After importing all the important Python libraries, we are creating an array of names along with arrays of marks and age respectively.

Now with the help of MultiIndex.from_arrays, we are combining all three arrays such that elements from all three arrays form multiple indexes together. After that, we show the above result.

Output:

👁 multi index array

Example 2: Creating multi-index from DataFrame using Pandas. 

In this example, we are doing the same thing as the previous example. We created a DataFrame using pd.DataFrame and after that, we created multi-index from that DataFrame using multi-index.from_frame() along with the names. 

Output:

👁 created dataframe

Now using MultiIndex.from_frame(), we are creating multiple indexes with this DataFrame.

Output:

👁 multi index dataframe

Example 3:Using DataFrame.set_index([col1,col2,..])

After importing the Pandas library, we created data and then converted it into tabular form with the help of pandas.DataFrame.

After that using Dataframe.set_index we are setting some columns as the index columns(Multi-Index).

The drop parameter is kept as false which will not drop the columns mentioned as index columns and thereafter append parameter is used for appending passed columns to the already existing index columns. 

Output:

👁 created dataframe

Now, we are printing the index of DataFrame in the form of a multi-index.

Output:

👁 multi index dataframe

GroupBy in Python Pandas

A groupby operation in Pandas helps us to split the object by applying a function and there-after combine the results.

After grouping the columns according to our choice, we can perform various operations which can eventually help us in the analysis of the data. 

Syntax

DataFrame.groupby(by=None, axis=0, level=None, as_index=True, sort=True, group_keys=True, squeeze=<object object>, observed=False, dropna=True)

Parameter

• by: It helps us to group by specific or multiple columns in the DataFrame.
• axis: It has a default value of 0 where 0 stands for index and 1 stands for columns.
• level: Let us consider that the DataFrame we are working with has hierarchical indexing. In that case, level helps us to determine the level of the index we are working with.
• as_index: It is a boolean data type with a default value of true. It returns an object with group labels as the index.
• sort: It helps us to sort the key values. It is preferable to keep it as false for better performance.
• group_keys: It is also a boolean value with a default value of true. It adds group keys to indexes to identify pieces
• dropna: It helps to drop the 'NA' values in a dataset 

Example 1: Using Groupby with DataFrame

First, let's create a DataFrame on which we will perform the groupby operation.

Output:

👁 created dataframe

Now let us group them according to some features:

As we can see, we have grouped them according to 'Status' and 'Temperature and Status'. Let us perform some functions now:

Example: Finding the mean of a Group

This will create the mean of the numerical values according to the 'status'. 

👁 mean of dataframe group

Conclusion

We have covered the concept of Multi index and groupby in Pandas Python in this tutorial. Both these concepts are very crucial in data manipulation while doing data analysis.

Multi-index allows you to create a hierarchal structure in your data structure, while groupby allows you to group similar data to perform analysis on it.

Using both these techniques together will help in better data presentation and provide you with some unseen insights, increasing the quality of your data analysis project.