Break a List into Chunks of Size N in Python

Given a list of elements and a number n, the task is to split the list into smaller sublists (chunks), where each sublist contains at most n elements. This helps in processing large data in smaller parts or batches.

For Example:

a = [1, 2, 3, 4, 5, 6, 7, 8]
n = 3
Result: [[1, 2, 3], [4, 5, 6], [7, 8]]

Let’s explore different methods one by one.

Using List Comprehension

List comprehension is an efficient method for chunking a list into smaller sublists. This method creates a list of lists, where each inner list represents a chunk of the original list of a given size.

[[1, 2, 3], [4, 5, 6], [7, 8]]

Explanation:

• range(0, len(a), n) generates starting indices for each chunk, i.e., 0, 3, 6, etc.
• a[i:i + n] slices the list a starting at index i and ending at i + n (the chunk size).
• For each index in the range, the list is sliced into chunks of size n.

Using itertools.islice

For very large lists, itertools.islice can be a memory-efficient way to create chunks without loading the entire list into memory. It allows you to process the list incrementally.

[[1, 2, 3], [4, 5, 6], [7, 8]]

Explanation:

• iter(a) converts list a into an iterator it for sequential access to elements.
• islice(it, n) fetches n elements at a time from the iterator it using islice.
• for _ in range((len(a) + n - 1) // n) iterates the required number of times to generate chunks, ensuring even an incomplete final chunk is included.

Using zip_longest() from itertools

The zip_longest() function from the itertools module can be used to break a list into evenly sized chunks by grouping elements together in tuples of size n.

[[1, 2, 3], [4, 5, 6], [7, 8]]

Explanation:

• zip_longest() groups elements into tuples of size n using a single iterator.
• filter(None, group) removes None values added for padding the last incomplete chunk.

Using Slicing

This is a straightforward approach using a for loop to iterate over the list and slice it manually into chunks of size n. It’s simple to understand and works well for smaller datasets.

[[1, 2, 3], [4, 5, 6], [7, 8]]

Explanation:

• For loop increments by n, ensuring each slice covers a chunk of size n.
• Each sliced chunk is appended to the chunks list.