Configuring Memory

Memory is configured via the /etc/dremio/dremio-env file. When this configuration file is changed from the default, it must be updated on all nodes of interest.

[info] Note: In addition to memory, this configuration file is used to set Java options and log directories.

Maximum memory size (recommended) allows Dremio to automatically determine the best allocation between HEAP and DIRECT memory depending on the node type. To change the default maximum memory, modify the following line:

DREMIO_MAX_MEMORY_SIZE_MB=16384

Alternatively, you can set HEAP and DIRECT memory instead of maximum memory. The maximum HEAP and DIRECT memory can be set separately.

  • If both are specified, then the maximum memory option is ignored.
  • If only one is configured, Dremio automatically determines the other, based on leftover memory.

Heap memory

Heap memory is used for running Dremio server. To change the default maximum heap memory, modify the following line:

DREMIO_MAX_HEAP_MEMORY_SIZE_MB=8192

Direct memory

Direct memory is used for query execution. To change the default maximum direct memory, modify the following line:

DREMIO_MAX_DIRECT_MEMORY_SIZE_MB=16384

If you see queries failing on an instance that's running out of memory, increasing the amount of memory that Dremio is able to consume may solve your problem.

[warning] Warning: For the DREMIO_MAX_DIRECT_MEMORY_SIZE_MB allocation, be sure to leave at least 1-2 GB of memory for the OS.


results matching ""

    No results matching ""