This topic describes how to implement Garbage-First (G1) garbage collection (GC). G1 is a server-style garbage collector for multi-processor machines with large memories.
To implement G1 garbage collection and heap dump flags when Dremio executors are not on Yarn,
add the following property to the dremio-env file on all Dremio coordinator and executor nodes.
DREMIO_JAVA_SERVER_EXTRA_OPTS="-XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:MaxGCPauseMillis=500 -XX:InitiatingHeapOccupancyPercent=25"
By default, the GC logs and heap dumps are stored under the Dremio log folder.
The following is an example:
501 3479 1 0 Mon01PM ttys001 36:38.94 /usr/bin/java -Djava.util.logging.config.class=org.slf4j.bridge.SLF4JBridgeHandler -Djava.library.path=/Users/dremio/Downloads/dremio-enterprise-3.2.0-201905102005330382-0598733/lib -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/Users/dremio/Downloads/dremio-enterprise-3.2.0-201905102005330382-0598733/log/server.gc -Ddremio.log.path=/Users/dremio/Downloads/dremio-enterprise-3.2.0-201905102005330382-0598733/log -Xmx4096m -XX:MaxDirectMemorySize=8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/Users/dremio/Downloads/dremio-enterprise-3.2.0-201905102005330382-0598733/log -Dio.netty.maxDirectMemory=0 -DMAPR_IMPALA_RA_THROTTLE -DMAPR_MAX_RA_STREAMS=400 -XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:MaxGCPauseMillis=500 -XX:InitiatingHeapOccupancyPercent=25 -cp /Users/dremio/Downloads/dremio-enterprise-3.2.0-201905102005330382-0598733/conf:/Users/dremio/Downloads/dremio-enterprise-3.2.0-201905102005330382-0598733/jars/*:/Users/dremio/Downloads/dremio-enterprise-3.2.0-201905102005330382-0598733/jars/ext/*:/Users/dremio/Downloads/dremio-enterprise-3.2.0-201905102005330382-0598733/jars/3rdparty/* com.dremio.dac.daemon.DremioDaemon
To implement G1 garbage collection and heap dump flags when Dremio executors are deployed on Yarn,
add the following properties to the data source via the Dremio UI > Advanced Options > Connection Properties.
| Property Type | Name | Value |
|---|---|---|
| JAVA | provisioning.yarn.heapsize | 8192 |
| SYSTEM | -Xloggc:/<path-to-gc-logs>/gc.log-date +'%Y%m%d%H%M' |
|
| SYSTEM | -XX:+UseG1GC | |
| SYSTEM | -XX:+HeapDumpOnOutOfMemoryError | |
| SYSTEM | -XX:HeapDumpPath | <path> |
| SYSTEM | -XX:+PrintGCDetails | |
| SYSTEM | -XX:+PrintGCTimeStamps | |
| SYSTEM | -XX:+PrintGCDateStamps | |
| SYSTEM | -XX:ErrorFile | <path>/hs_err_pid%p.log |
| SYSTEM | -XX:G1HeapRegionSize | 32M |
| SYSTEM | -XX:MaxGCPauseMillis | 500 |
| SYSTEM | -XX:InitiatingHeapOccupancyPercent | 25 |
For Kubernetes-based deployments, garbage collection is done via the Helm charts.