Follow below link to my last article to understand about transparent huge page in detail
What is transparent hugepage (THP) and how to check THP usage per process in Linux (Explained)
Types of HugePages:
There can be two types of HugePages in the system
- Explicit Huge Pages which are allocated explicitly by vm.nr_hugepages sysctl parameter i.e. the pages that are used as huge pages are reserved inside the kernel and cannot be used for other purposes.
- Transparent Huge Pages which are allocated automatically by the kernel.
How to disable Transparent Hugepages on runtime?
Running the below commands will stop only creation and usage of the new THP.
The THP which were created and used at the moment the above commands were run would not be disassembled into the regular memory pages.
To get rid of THP completely the system should be rebooted with THP disabled at boot time.
# echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
# echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
"always" means that an application requesting THP will stall on allocation failure and directly reclaim pages and compact memory in an effort to allocate a THP immediately.
"defer" means that an application will wake kswapd in the background to reclaim pages and wake kcompactd to compact memory so that THP is available in the near future. It's the responsibility of khugepaged to then install the THP pages later.
"madvise" will enter direct reclaim like "always" but only for regions that are have used madvise(MADV_HUGEPAGE). This is the default behaviour.
"never" means do not allocate THP at all
By default kernel tries to use huge zero page on read page fault to anonymous mapping. It's possible to disable huge zero page by writing 0 or enable it back by writing 1
echo 0 >/sys/kernel/mm/transparent_hugepage/use_zero_page
echo 1 >/sys/kernel/mm/transparent_hugepage/use_zero_page
How to disable Transparent Hugepages permanently using GRUB2?
You can change the sysfs boot time defaults of Transparent Hugepage support by passing the parameter "transparent_hugepage=always" or "transparent_hugepage=madvise" or "transparent_hugepage=never" to the kernel command line.
To disable THP use "transparent_hugepage=never" to the kernel command line in the grub configuration file.
For my setup I am using GRUB2 (/etc/sysconfig/grub)
GRUB_CMDLINE_LINUX="novga console=ttyS0,115200 panic=1 numa=off elevator=cfq rd.md.uuid=d265dd3d:9ee4d53a:597b8c08:8201b9af rd.lvm.lv=os/root rd.md.uuid=5398452a:ab1b8e91:4307b53b:5c3cccbd rd.md.uuid=131bc1e7:7c9087c3:03f3ad4a:7cde170c noht rhgb quiet transparent_hugepage=never"
Next rebuild your grub configuration file
# grub2-mkconfig -o /boot/grub2/grub.cfg
Reboot the node for the changes to take affect. Once the node is up make sure the newly added entry exists in your loaded grub configuration
# grep transparent_hugepage /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-862.6.3.el7.x86_64 root=/dev/mapper/os-root ro novga console=ttyS0,115200 panic=1 numa=off elevator=cfq rd.md.uuid=d265dd3d:9ee4d53a:597b8c08:8201b9af rd.lvm.lv=os/root rd.md.uuid=5398452a:ab1b8e91:4307b53b:5c3cccbd rd.md.uuid=131bc1e7:7c9087c3:03f3ad4a:7cde170c noht biosdevname=0 net.ifnames=0 rhgb quiet transparent_hugepage=never
So the configuration is correctly loaded.
ALSO READ:
How to check if CPU supports HugePages and change default hugepage size in RHEL/CentOS 7
How to check Transparent HugePage usage per process in Linux with examples
How to disable Transparent Hugepages permanently using tuned profile?
You can either use your existing tuned profile or you can create a new profile.
Use your existing tuned profile.
My node is configured with throughput-performance profile, you can check your loaded profile using below command
# tuned-adm active
Current active profile: throughput-performance
Next edit your tuned profile configuration file and append below value under '[vm]'
# vim /usr/lib/tuned/throughput-performance/tuned.conf
[vm]
transparent_hugepage=never
Next activate your profile
# tuned-adm profile throughput-performance
Validate your changes
# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
IMPORTANT NOTE: This does not requires reboot and the changes will get activated immediately
Create a new customised tuned profile
With this we will create a customized version of the currently running profile. The customized version will only disable THP.
Find out which profile is active, create a copy. In the following example we currently use the throughput-performance profile:
# tuned-adm active
Current active profile: throughput-performance
To create customized profile, create a new directory in /etc/tuned directory with desired profile name.
# mkdir /etc/tuned/myprofile-disablethp
Then create a new tuned.conf file for myprofile-disablethp, and insert the new tuning info:
# cat /etc/tuned/myprofile-disablethp/tuned.conf
[main]
include= throughput-performance
[vm]
transparent_hugepages=never
Make the script executable
# chmod +x /etc/tuned/myprofile-disablethp/tuned.conf
Enable myprofile like
# tuned-adm profile myprofile-disablethp
Validate your changes
# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
**IMPORTANT NOTE:**This does not requires reboot and the changes will get activated immediately
How to check if HugePages is disabled?
In the above steps we disabled the THP to make sure kernel doesnot allocates or reserves any hugepage.
If the value of HugePages_Total is "0" it means HugePages is disabled on the system.
# grep -i HugePages_Total /proc/meminfo
HugePages_Total: 0
Check below file
# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
This must show that [never] is selected for hugepage as above
How to check if HugePages is enabled?
If the value of HugePages_Total is greater than "0", it means HugePages is enabled on the system
# grep -i HugePages_Total /proc/meminfo
HugePages_Total: 23090
Check below file
# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
This must "not" show highlighted option as [never].
How to disable Explicit Transparent Hugepages on runtime?
nr_hugepages indicates the current number of "persistent" huge pages in the kernel's huge page pool. "Persistent" huge pages will be returned to the huge page pool when freed by a task.
When multiple huge page sizes are supported, /proc/sys/vm/nr_hugepages indicates the current number of pre-allocated huge pages of the default size.
Thus, one can use the following command to dynamically allocate/deallocate default sized persistent huge pages. If the value in /proc/sys/vm/nr_hugepages file or vm.nr_hugepages sysctl parameter is "0" it means HugePages is disabled on the system
# echo 0 > /proc/sys/vm/nr_hugepages
# cat /proc/sys/vm/nr_hugepages
0
# sysctl vm.nr_hugepages
vm.nr_hugepages = 0
How to disable Explicit Transparent Hugepages permanently?
To disable the explicit hugepage permanently add below entry in "/etc/sysctl.conf"
# grep -i nr_hugepages /etc/sysctl.conf
vm.nr_hugepages=0
Refresh the values
# sysctl -p
Below command will show the updated value
# sysctl -a | grep nr_hugepages
vm.nr_hugepages = 0
NOTE: A reboot is required to activate the changes.
How to enable and assign explicit hugepages permanently?
One allocate persistent huge pages on the kernel boot command line by specifying the "hugepages=N" parameter, where 'N' = the number of huge pages requested. This is the most reliable method of allocating huge pages as memory has not yet become fragmented.
Append "hugepages=N" to "/etc/sysconfig/grub" to GRUB_CMDLINE_LINUX as shown below
GRUB_CMDLINE_LINUX="novga console=ttyS0,115200 panic=1 numa=off elevator=cfq rd.md.uuid=c0dd5b9a:ff7440ae:3bfc1616:defc0291 rd.lvm.lv=os/root rd.md.uuid=05e8d321:d3191ad9:48a0aba2:efdd0b6c rd.md.uuid=2826bc19:9266a29e:e0989473:987348ad rhgb quiet console=tty0 hugepages=2048"
Rebuild the initramfs
# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-693.21.1.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-693.21.1.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-fae7c244c3134771a10cc7c3ace3edcb
Found initrd image: /boot/initramfs-0-rescue-fae7c244c3134771a10cc7c3ace3edcb.img
done
Reboot the node for the changes to take affect
Validate your changes post reboot
# sysctl -a | grep hugepages
vm.nr_hugepages = 2048
You can also reserve hugepages using sysctl.conf
# echo "vm.nr_hugepages = 2048" >> /etc/sysctl.conf
Refresh the values
# sysctl -p
Reboot the node for the changes to take affect.
I hope the article was useful.