Or: How a thoughtless configuration can consume the memory of an entire worker node.
Java-based applications in containers
If a Java-based application is executed in a container in a runtime environment such as Kubernetes or OpenShift, many relevant parameters can be configured. These include the Java distribution and version to be used, as well as the available CPU and RAM resources. With regard to the RAM, further aspects can be influenced for Java-based applications via various JVM flags. This article focuses on Java-based microservices that are executed in a container, how the memory for these microservices can be configured and allocated and what needs to be taken care of.
Memory, heap and non-heap in containers
The memory that is made available to a container as a resource cannot be fully used by the microservice running in the container. The available memory is divided into the heap and non-heap areas. The Java objects and threads of the application are created and managed in the heap memory. This memory must be sufficiently large so that the application can be executed reliably and to prevent out-of-memory exceptions. The non-heap memory is intended for static objects (stack), metadata (metaspace) and the JVM itself. The size ratio of heap and non-heap memory depends on the specific Java application. For microservices, in my experience, it makes sense to divide the available memory into 75% for the heap memory and 25% for the non-heap memory.
Since version 10 of Java, the JVM can recognize whether it is being executed in a container. In such cases, it can also recognize the available memory of the container. This is also easy to check yourself. With a local container runtime environment, such as Podman, the following command can be executed.
podman run --memory 1g --rm ghcr.io/graalvm/jdk-community:21 cat /sys/fs/cgroup/memory.max
The result of the cat command is 1073741824. The value has the unit bytes. If you convert this value to GB, you get 1 GB. This matches the --memory 1g parameter used. For older images, the value may have to be read from the file /sys/fs/cgroup/memory/memory.limit_in_bytes.
Special characteristic 1: Only 25% for the heap in the standard version
In the next step, you can now look at how the available memory is divided between the heap and non-heap memory. The following command can be executed for this purpose:
podman run --memory 1g --rm ghcr.io/graalvm/jdk-community:21 java -XX:+PrintFlagsFinal -version | grep 'MaxHeapSize\|MaxRAMPercentage'
The command provides the following output:
size_t MaxHeapSize = 268435456
double MaxRAMPercentage = 25.000000
size_t SoftMaxHeapSize = 268435456
Here you can see that the maximum size of the heap memory (MaxHeapSize) is limited to 256 MB. This corresponds to 25 % of the available memory and represents the default setting of the JVM. The microservice that is executed in the container therefore only has access to a fraction of the available memory.
JVM flag MaxRAMPercentage
The JVM flag MaxRAMPercentage can be used to configure the maximum available memory for the JVM. When a Java application is started, the JVM flag is set with -XX:MaxRAMPercentage=<VALUE>. The previously executed command can therefore be adapted as follows:
podman run --memory 1g --rm ghcr.io/graalvm/jdk-community:21 java -XX:+PrintFlagsFinal -XX:MaxRAMPercentage=75.0 -version | grep 'MaxHeapSize\|MaxRAMPercentage'
The adapted command now provides the following output:
size_t MaxHeapSize = 805306368
double MaxRAMPercentage = 75.000000
size_t SoftMaxHeapSize = 805306368
It can be seen that 75 % or 768 MB of the memory is now available for the heap memory.
JVM flag MinRAMPercentage
The supposed counterpart to MaxRAMPercentage could be MinRAMPercentage. The name suggests that this flag can be used to define the minimum percentage of available memory to be allocated for the heap area. It is worth having a look at the default setting of the JVM here too. To do this, execute the following command:
podman run --memory 1g --rm ghcr.io/graalvm/jdk-community:21 java -XX:+PrintFlagsFinal -version | grep 'MinHeapSize\|MinRAMPercentage\|MaxHeapSize\|MaxRAMPercentage'
The command provides the following output:
size_t MaxHeapSize = 268435456
double MaxRAMPercentage = 25.000000
size_t MinHeapSize = 8388608
double MinRAMPercentage = 50.000000
size_t SoftMaxHeapSize = 268435456
The output is somewhat irritating. MinHeapSize has the value 8 MB. The value of 50 % for MinRAMPercentage is ignored. Why is that?
Special characteristic 2: Meaning of MinRAMPercentage
MinRAMPercentage sets the maximum(!) size of the heap memory if the available memory is approximately 200 to 250 MB or less. If the container has more memory available, MinRAMPercentage is ignored. Very irritating! There are also discussions about this in the Java community.
The following commands illustrate the effects of MinRAMPercentage in connection with the available memory.
Low memory and default values
Command:
podman run --memory 128m --rm ghcr.io/graalvm/jdk-community:21 java -XX:+PrintFlagsFinal -version | grep 'MinHeapSize\|MinRAMPercentage\|MaxHeapSize\|MaxRAMPercentage'
Output:
size_t MaxHeapSize = 67108864
double MaxRAMPercentage = 25.000000
size_t MinHeapSize = 8388608
double MinRAMPercentage = 50.000000
size_t SoftMaxHeapSize = 67108864
MaxHeapSize is 64 MB, although MaxRamPercentage has the value 25%. MinHeapSize is still 8 MB, although MinRAMPercentage has the value 50%. MinRAMPercentage has thus determined the value of MaxHeapSize.
Low memory and individual values
Command:
podman run --memory 128m --rm ghcr.io/graalvm/jdk-community:21 java -XX:+PrintFlagsFinal -XX:MinRAMPercentage=25.0 -XX:MaxRAMPercentage=75.0 -version | grep 'MinHeapSize\|MinRAMPercentage\|MaxHeapSize\|MaxRAMPercentage'
Output:
size_t MaxHeapSize = 33554432
double MaxRAMPercentage = 75.000000
size_t MinHeapSize = 8388608
double MinRAMPercentage = 25.000000
size_t SoftMaxHeapSize = 33554432
Although MaxRAMPercentage was explicitly specified as 75 %, MaxHeapSize is nevertheless set to 32 MB. This corresponds to the 25 % that was set for MinRAMPercentage.
High memory and default values
Command:
podman run --memory 1g --rm ghcr.io/graalvm/jdk-community:21 java -XX:+PrintFlagsFinal -version | grep 'MinHeapSize\|MinRAMPercentage\|MaxHeapSize\|MaxRAMPercentage'
Output:
size_t MaxHeapSize = 268435456
double MaxRAMPercentage = 25.000000
size_t MinHeapSize = 8388608
double MinRAMPercentage = 50.000000
size_t SoftMaxHeapSize = 268435456
MaxHeapSize has the value 256 MB and thus corresponds to the 25 % of MaxRAMPercentage. The value of MinRAMPercentage is ignored, and the minimum size of the heap area is still 8 MB.
High memory and individual values
Command:
podman run --memory 1g --rm ghcr.io/graalvm/jdk-community:21 java -XX:+PrintFlagsFinal -XX:MinRAMPercentage=25.0 -XX:MaxRAMPercentage=75.0 -version | grep 'MinHeapSize\|MinRAMPercentage\|MaxHeapSize\|MaxRAMPercentage'
Output:
size_t MaxHeapSize = 805306368
double MaxRAMPercentage = 75.000000
size_t MinHeapSize = 8388608
double MinRAMPercentage = 25.000000
size_t SoftMaxHeapSize = 805306368
MaxHeapSize has the value 768 MB and therefore corresponds to the 75 % of MaxRAMPercentage. The value of MinRAMPercentage is ignored, and the minimum size of the heap area is still 8 MB.
The examples clearly show that the use of MinRAMPercentage should be avoided.
JVM flag InitialRAMPercentage
An alternative to MinRAMPercentage for containers with more memory is InitialRAMPercentage. This flag can be used to determine how large the heap area should be when the microservice is started. In the following example, an initial 50% and a maximum of 75% of the memory should be allocated for the heap:
podman run --memory 1g --rm ghcr.io/graalvm/jdk-community:21 java -XX:+PrintFlagsFinal -XX:InitialRAMPercentage=50.0 -XX:MaxRAMPercentage=75.0 -version | grep 'InitialHeapSize\|InitialRAMPercentage\|MinHeapSize\|MinRAMPercentage\|MaxHeapSize\|MaxRAMPercentage'
The command provides the following output:
size_t InitialHeapSize = 536870912
double InitialRAMPercentage = 50.000000
size_t MaxHeapSize = 805306368
double MaxRAMPercentage = 75.000000
size_t MinHeapSize = 8388608
double MinRAMPercentage = 50.000000
size_t SoftMaxHeapSize = 805306368
Here you can see that the initial size of the heap of 512 MB actually corresponds to the set 50 %. Without the specification of -XX:InitialRAMPercentage=<VALUE>, the default value is approx. 1.5 %, so that InitialHeapSize would then have the value 16 MB.
Special characteristic 3: Container without request limit
Horizontal scaling, e.g. via a horizontal pod autoscaler, is very common for stateless microservices that are executed in containers. For this purpose, a request limit for CPU and RAM is defined in the deployment. Together with a threshold, it is then defined at what level of utilization of a running container another container is started, which can then also process incoming requests. However, it may also be the case that horizontal scaling cannot be implemented for every type of microservice. In these cases, vertical scaling of the container may be considered. This could lead to the idea of not setting a request limit for CPU and/or RAM. This would allow the container or the JVM of the microservice to allocate more memory for the heap area if required and release it again later when it is no longer needed. But how do the JVM flags shown above behave if no request limit is defined for the memory? You can execute the previous command without the --memory 1g parameter:
podman run --rm ghcr.io/graalvm/jdk-community:21 java -XX:+PrintFlagsFinal -XX:InitialRAMPercentage=50.0 -XX:MaxRAMPercentage=75.0 -version | grep 'InitialHeapSize\|InitialRAMPercentage\|MinHeapSize\|MinRAMPercentage\|MaxHeapSize\|MaxRAMPercentage'
The command provides the following output:
size_t InitialHeapSize = 1031798784
double InitialRAMPercentage = 50.000000
size_t MaxHeapSize = 1545601024
double MaxRAMPercentage = 75.000000
size_t MinHeapSize = 8388608
double MinRAMPercentage = 50.000000
size_t SoftMaxHeapSize = 1545601024
In this specific example (Podman Desktop under macOS), MaxHeapSize now has the value 1474 MB and InitialHeapSize has the value 984 MB. The base for calculating the heap size is again the file /sys/fs/cgroup/memory.max (or /sys/fs/cgroup/memory/memory.limit_in_bytes). If the container is started without a request limit for the memory, the file contains the value max. This means that the entire available or configured memory of the host or worker node is used as the base for calculating the initial and maximum heap size.
If a container is started without a request limit for the memory and the microservice is started with the JVM flag -XX:InitialRAMPercentage=50.0, the JVM attempts to allocate 50% of the available memory of the host or worker node for itself, regardless of whether the microservice actually needs the memory or not. This behavior is probably not desirable in most cases.
Summary
The JVM flags shown above come with a few surprises. Therefore, before using them, you must carefully consider the requirements of your own microservice and the goals you are trying to achieve. It is better not to use MinRAMPercentage. With MaxRAMPercentage you can reliably define the upper limit of the heap memory. The default value of 25 % is usually set too low. Last, but not least, InitialRAMPercentage offers the possibility of determining the initial size of the heap. However, this should only be considered if the container has a request limit for the memory.
Sources:
Photo credits: pixabay