Python | Convert list of nested dictionary into Pandas dataframe

Given a list of the nested dictionary, write a Python program to create a Pandas dataframe using it. We can convert list of nested dictionary into Pandas DataFrame. Let's understand the stepwise procedure to create a Pandas Dataframe using the list of nested dictionary.

Convert Nested List of Dictionary into Pandas Dataframe

Below are the methods that will be used

• Using from_dict(orient='index')
• Native Method

Nested Dictionary to Pandas DataFrame Using orient='index'

In this example, the Panda's library is used to create a DataFrame named df from a dictionary (countries) where each key represents a unique identifier for a country, and the values contain information about the country's name, capital, and population. The resulting DataFrame is printed, displaying the modified country details with columns for "Country," "Capital," and "Population.

Output:

 Country Capital Population
1 New Country 1 New Capital 1 123,456,789
2 New Country 2 New Capital 2 987,654,321
3 New Country 3 New Capital 3 111,222,333

Nested Dictionary to Pandas DataFrame using Native Method

Below are the steps by which we can convert list of nested dictionary into Pandas DataFrame:

Step 1: Create a List of Nested Dictionary

In this step, a list of dictionaries represents students with associated exam scores and grades. The data is structured such that each dictionary includes a "Student" key containing a list of exam details, and a "Name" key indicating the student's name.

Output:

[{'Student': [{'Exam': 90, 'Grade': 'a'}, {'Exam': 99, 'Grade': 'b'}, {'Exam': 97, 'Grade': 'c'}],
 'Name': 'Paras Jain'},
 {'Student': [{'Exam': 89, 'Grade': 'a'}, {'Exam': 80, 'Grade': 'b'}], 
 'Name': 'Chunky Pandey'}]

Step 2: Adding Dict Values to Rows

In this step, a new list named "rows" is initialized to store flattened data. The code iterates through the original list of dictionaries, extracting each student's exam details along with their name and appends the flattened rows to create a Pandas DataFrame, which is then displayed.

Output:

 Exam Grade Name
0 90 a Paras Jain
1 99 b Paras Jain
2 97 c Paras Jain
3 89 a Chunky Pandey
4 80 b Chunky Pandey

Step 3: Pivoting DataFrame and Assigning Column Names

In this step, the Pandas pivot_table function is utilized to reshape the DataFrame. The "Name" column becomes the index, "Grade" values are transformed into separate columns ('Maths', 'Physics', 'Chemistry'), and corresponding 'Exam' values fill the table. The resulting DataFrame is then displayed with the updated column names.

Output:

 Name Maths Physics Chemistry
0 Chunky Pandey 89.0 80.0 NaN
1 Paras Jain 90.0 99.0 97.0

Below is the complete code:

Output:

 Name Maths Physics Chemistry
0 Chunky Pandey 89 80 NaN
1 Paras Jain 90 99 97