Tips to reduce Python object size

Python objects are significantly heavier than C/C++ structures due to dynamic typing and memory overhead. For memory-intensive applications, standard Python containers can cause performance bottlenecks or OOM (Out of Memory) errors.

Let's look at some ways in which we can use this memory effectively and reduce the size of objects.

Using built-in Dictionaries (High Overhead)

Dictionaries prioritize lookup speed (hashing) over memory efficiency. A standard dictionary incurs significant overhead for the hash table structure.

Size: 184 bytes

Notice that one instance of the data type dictionary takes 184 bytes, which makes it inefficient for storing millions of object instances.

Tuples (Immutable Efficiency)

Tuples store data in fixed-size arrays without hash table overhead. They are the most efficient built-in structure for immutable data.

Size: 64 bytes

It provides excellent memory footprint, but lacks mutability and readability (indices vs. attribute names).

Classes with __slots__ (Optimal)

Standard Python classes store attributes in a private dictionary (__dict__). This creates a "double overhead": the object instance plus the internal dictionary. To overcome this, we can use "define __slots__". This instructs Python to allocate a fixed amount of memory for specific attributes, preventing the creation of __dict__.

Size: 56 bytes

It retains class usability (attribute names) with tuple-like memory efficiency.

Note: "sys.getsizeof()" function is non-recursive (shallow). It measures the container size, not referenced objects. However, replacing a Dictionary container (~232 bytes) with a Slotted container (48 bytes) yields massive savings at scale.

DataClasses (Python 3.10+)

Python 3.10 introduced the slots=True parameter to the dataclasses module. This automates the __slots__ creation, removing boilerplate.

Size: 56 bytes

Third-Party Libraries (recordclass)

For scenarios requiring mutable, tuple-like objects without writing classes, recordclass (C-extension) is a viable alternative. It's an external library so it needs to be installed first.

Install the "recordclass" library using the following command in terminal:

pip install recordclass

Output

Size: 48 bytes

Summary of Memory Consumption