maxgarrick.com » Understanding OpenVZ resource limits
OpenVZ resource limits can be a little tricky, even after reading through the documentation on the OpenVZ website. With this post, I hope to explain the /proc/user_beancounters file as well as provide an illustration of memory guarantees/limits in an example 256mb container.
Before attempting to understand OpenVZ’s memory limits, let’s recap Linux’s memory setup. This background information is important to firmly grasp OpenVZ’s behavior. You can safely skip this section if you already understand Linux’s memory overcommit accounting.
Linux Overcommit Accounting
Memory in Linux refers to the combined RAM + swap values. Memory is allocated from this pool using one of the *alloc() functions. If malloc() is used to allocate memory, the pages are not zeroed out. malloc() is unique in one regard: the kernel can malloc() more memory than what is available in RAM + swap. This behavior of allocating more than the total available memory is called overcommitting memory.
Seems like a recipe for disaster, huh? Imagine… processes are humming along allocating memory and BAM! some process wants to use the n’th + 1 byte of memory and an out of memory (OOM) condition provokes the kernel to start killing processes.
Why does the kernel allow memory to be overcommitted? Because most applications do not actually consume all of the malloc()’d memory. If an app doesn’t use a memory location, it does not really need to subtract from your total available memory.
Consider a typical Apache process. A full 20-30% of an Apache processes malloc()’d memory may not be consumed. With memory overcommit, this slice of memory can be used by other applications. Java applications also allocate much more than they use, as do many scientific applications.
At the system-wide level, these “malloc’d(), but not consumed” slices of memory begin to add up. By overcommitting memory, these slices don’t take up physical memory locations — resulting in more efficient use of memory resources.
OpenVZ and memory overcommit
OpenVZ also has the ability to overcommit memory. When creating an OpenVZ container, we specify the vmguarpages and oomguarpages resource guarantees. vmguarpages represents the maximum guaranteed amount of malloc()’d memory a container may have, while oomguarpages represents the maximum consumed memory. These are called guarantees because (If your server has enough RAM + swap) container processes will never be killed due to OOM if they are within these limits.
What if your container exceeds the vmguarpages or oomguarpages value? This is where a new limit comes in: privvmpages. privvmpages represents the absolute upper limit on container memory. When you run free or `cat /proc/meminfo` in a container, you see this privvmpages value take form as “Total memory”.
The memory gap between privvmpages and the two resource guarantees (vmguarpages and oomguarpages) is not safe to use in an ongoing basis if the sum of all container privvmpages exceeds RAM + swap of the hardware node. This is because if the hardware node runs out of memory, it begins looking for containers using more than their guaranteed memory. The kernel kills a process from the container who’s physpages exceeds oomguarpages the most.
I’ve created a diagram that hopefully illustrates the relationship between these three key memory limits/guarantees. Also included in the diagram is kmemsize, which is used for non-swappable kernel memory–not very important as long as it scales with privvmpages.
OpenVZ memory constraints