Reserved THP Feature Proposed For Linux To Combine The Best Of HugeTLB & THP
Linux kernel developer and Bytedance engineer Qi Zheng sent out a request for comments (RFC) patch series on a new feature called Reserved THP to combine the best of HugeTLB and THP kernel functionality.
Bytedance has been working on Reserved THP as a potential stepping stone or goal of unifying HugeTLB and Transparent Huge Page (THP) support within the Linux kernel for kernel huge pages. HugeTLB brings nice elements like reservations and guaranteed allocation within reserved pools, but not supporting swap. Meanwhile, THP doesn't support reservations and allocations aren't guaranteed, but is more tightly integrated with the core memory management code and can handle swap.
Qi Zheng explained of their experience in coming up with the idea of Reserved THP:
In prior years there have been kernel discussions over unifying THP and HugeTLB albeit still separate to date.
Those wanting to learn more about this Reserved THP proposal can do so via this LKML patch series.
Bytedance has been working on Reserved THP as a potential stepping stone or goal of unifying HugeTLB and Transparent Huge Page (THP) support within the Linux kernel for kernel huge pages. HugeTLB brings nice elements like reservations and guaranteed allocation within reserved pools, but not supporting swap. Meanwhile, THP doesn't support reservations and allocations aren't guaranteed, but is more tightly integrated with the core memory management code and can handle swap.
Qi Zheng explained of their experience in coming up with the idea of Reserved THP:
"In our internal scenario, a user process needs to reserve double the amount of Hugetlb memory due to hot-upgrade requirements. For example, if the process needs 16GB of Hugetlb, an additional 16GB is required during the hot-upgrade to satisfy memory allocations. After the upgrade, the old process exits and releases the 16GB of HugeTLB. Therefore, in most cases, the extra 16GB of HugeTLB is wasted.
A straightforward idea is to use the Hugetlb CMA feature, reserving a total of 32GB of hugetlb_cma. During normal operation, 16GB is consumed, and the remaining 16GB can be used by other processes. During hot-upgrade, we could try to migrate the memory used by other processes to allocate the required extra 16GB of Hugetlb. This might work, but it still requires reserving 32GB of memory.
We also found that during the hot upgrade, about 10GB of the old process's hugetlb is actually cold memory, which could theoretically be reclaimed. In extreme cases, we could reserve only 22GB of memory and reclaim the remaining 10GB during the hot upgrade. But unfortunately, hugetlb currently does not support swap, and supporting it seems quite difficult.
Therefore, we are wondering if we can introduce "reserved THP", which is THP that can be reserved. It can be consumed through methods like madvise(), while normal memory allocation cannot consume it. This can achieve an effect similar to hugetlb. And because it is THP, it can relatively easily support swap features, which perfectly solves the above problem."
In prior years there have been kernel discussions over unifying THP and HugeTLB albeit still separate to date.
Those wanting to learn more about this Reserved THP proposal can do so via this LKML patch series.