ksm over-commit
1 Intro
1.1 KSM
KSM (Kernel Samepage Merging) is running in the Linux kernel and scans the memory of all the virtual machines running on a single host, looking for duplication and consolidating. With KSM we’re able to improve virtual machine density by as much as 300% without impacting performance. One of the great benefits of using Linux as the hypervisor means KSM is not limited to KVM and virtual machines, but can also reduce memory pressure with normal Linux applications.
It was integrated in PVE since version 1.5, and is implemented with the package “ksm-control-daemon” (check your version with the cli command “pveversion -v”). [1]
1.2 KSM in action
Just install several KVM virtual machines with the same OS (using at least 80% of your physical memory on the host) and wait a few minutes. You will notice higher CPU activities on the host (ksm daemon) and the used memory on the host will be lowered significantly (see start page showing the overall memory usage).
Howto verify that KSM is working (how many pages are being shared between your KVM guests):
watch cat /sys/kernel/mm/ksm/pages_sharing
Note: a page is 4096 bytes
The file /etc/ksmtuned.conf allows for some customization of its behaviour. [1]
2 Tuning KSM in PVE
2.1 Tuning
The KSM configuration file is located at /etc/ksmtuned.conf on PVE
To edit/modify/tune the configuration for KSM on PVE we can use following command
nano /etc/ksmtuned.conf
We will see following contents
# Configuration file for ksmtuned.
# How long ksmtuned should sleep between tuning adjustments
# KSM_MONITOR_INTERVAL=60
# Millisecond sleep between ksm scans for 16Gb server.
# Smaller servers sleep more, bigger sleep less.
# KSM_SLEEP_MSEC=100
# KSM_NPAGES_BOOST=300
# KSM_NPAGES_DECAY=-50
# KSM_NPAGES_MIN=64
# KSM_NPAGES_MAX=1250
# KSM_THRES_COEF=20
# KSM_THRES_CONST=2048
# uncomment the following if you want ksmtuned debug info
# LOGFILE=/var/log/ksmtuned
# DEBUG=1KSM_THRES_COEF=20 表示当只有20%内存的时候开始使用ksm压缩
KSM_THRES_COEF=20 represents that when there is only 20% free memory left, KSM will kick in (In other words, when memory utilization is 80% / a threshold to activate ksm)
We can change this number to fit our needs, in most cases, the default value is good enough and stable.
Note: ksmd is activated if either of the following occurs
The amount of free memory drops below the threshold, set in KSM_THRES_CONST.
The amount of committed memory plus the threshold, KSM_THRES_CONST, exceeds the total amount of memory.
2.2 Other values explained
# Configuration file for ksmtuned.
# How long ksmtuned should sleep between tuning adjustments
# KSM_MONITOR_INTERVAL=60
# Millisecond sleep between ksm scans for 16Gb server.
# Smaller servers sleep more, bigger sleep less.
# KSM_SLEEP_MSEC=10
# KSM_NPAGES_BOOST - is added to the `npages` value, when `free memory` is less than `thres`.
# KSM_NPAGES_BOOST=300
# KSM_NPAGES_DECAY - is the value given is subtracted to the `npages` value, when `free memory` is greater than `thres`.
# KSM_NPAGES_DECAY=-50
# KSM_NPAGES_MIN - is the lower limit for the `npages` value.
# KSM_NPAGES_MIN=64
# KSM_NPAGES_MAX - is the upper limit for the `npages` value.
# KSM_NPAGES_MAX=1250
# KSM_THRES_COEF - is the RAM percentage to be calculated in parameter `thres`.
# KSM_THRES_COEF=20
# KSM_THRES_CONST - If this is a low memory system, and the `thres` value is less than `KSM_THRES_CONST`, then reset `thres` value to `KSM_THRES_CONST` value.
# KSM_THRES_CONST=2048
# uncomment the following to enable ksmtuned debug information
# LOGFILE=/var/log/ksmtuned
# DEBUG=13 Monitoring KSM status
Kernel same-page merging (KSM) stores monitoring data in the /sys/kernel/mm/ksm/ directory. Files in this directory are updated by the kernel and are an accurate record of KSM usage and statistics.
The variables in the list below are also configurable variables in the /etc/ksmtuned.conf file, as noted above.
Files in /sys/kernel/mm/ksm/:
Name Value
full_scans Full scans run
merge_across_nodes Whether pages from different NUMA nodes can be merged
pages_shared Total pages shared
pages_sharing Pages currently shared
pages_to_scan Pages not scanned
pages_unshared Pages no longer shared
pages_volatile Number of volatile pages
run Whether the KSM process is running
sleep_millisecs Sleep milliseconds[2]
4 Disable KSM
systemctl disable ksmtuned
reboot5 Enable KSM
systemctl enable ksmtuned
reboot6 References
[1] “Dynamic Memory Management – Proxmox VE”, Pve.proxmox.com, 2020. [Online]. Available: https://pve.proxmox.com/wiki/Dynamic_Memory_Management#KSM
[2] “8.3. Kernel Same-page Merging (KSM) Red Hat Enterprise Linux 7 | Red Hat Customer Portal”, Red Hat Customer Portal, 2020. [Online]. Available: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_tuning_and_optimization_guide/chap-ksm
[3]https://zhuanlan.zhihu.com/p/62829713
[4]https://blog.csdn.net/liukuan73/article/details/90315886