Advanced Configuration

Configure Engines

You can configure query engines for a Dremio Kubernetes deployment. Dremio 26.0 introduced Managed Engines. Classic engines are still supported but will be deprecated in Dremio 27 — migrate to Managed Engines before upgrading.

note

For most deployments, engine configuration changes here are not necessary. Managed Engines are designed to work out of the box — create and size engines through the Dremio console rather than through Helm values.

Configure the Default Resource Offset

Dremio's default resource offset is reserve-2-8, where the first value represents 2 vCPUs and the second represents 8 GB of RAM. To change this default, add the following to your values-overrides.yaml and set defaultOffset to one of the available values.

Available offsets:

• reserve-0-0
• reserve-2-4
• reserve-2-8
• reserve-2-16

Default resource offset configuration

engine:
  options:
    resourceAllocationOffsets:
      defaultOffset: reserve-2-8
  ...

Configure Extra Java Start Options

engine.options.javaOptions lets you declare curated JVM flags that Managed Engines are permitted to use. Each entry defines a named option with a pattern, an optional validation regex, and a default value. The declared options are rendered into engine-options.yaml and applied when engines start.

Each entry has the following fields:

Field	Required	Description
name	Yes	Identifier used to reference this option in engine API calls.
pattern	Yes	JVM flag template. Use %s as the placeholder for the value. For example, "-Dmy.property=%s".
valueMatcher	No	Java regex that the supplied value must match before the flag is applied. Always set this — omitting it means the raw value is passed directly to the JVM with no validation.
defaultValue	No	Value used when applyByDefault is true and no per-engine override is set.
applyByDefault	No	When true, applies the option to all engines using defaultValue. When false, the option must be set explicitly per engine via the engine API.

Extra Java start options for engines

engine:
  options:
    javaOptions:
    - name: "NumberOfGCLogFiles"
      pattern: "-XX:NumberOfGCLogFiles=%s"
      valueMatcher: "^[1-9][0-9]*$"
      defaultValue: "5"
      applyByDefault: true
    - name: "GCLogFileSize"
      pattern: "-XX:GCLogFileSize=%s"
      valueMatcher: "^[1-9][0-9]*[kKmMgG]$"
      defaultValue: "4000k"
      applyByDefault: true

note

javaOptions requires Dremio 26.1.0 or later. Classic engines do not support this feature — use executor.extraStartParams instead.

Configure Classic Engines

warning

Classic engines will be deprecated in Dremio 27. Migrate to Managed Engines before upgrading. Classic engines and Managed Engines cannot be used side by side, and classic engines do not support auto-start and auto-stop.

You can configure classic engines by adding the following to your values-overrides.yaml. Adjust the values to fit your environment.

Classic engine configuration

executor:
  resources:
    requests:
      cpu: "16"
      memory: "120Gi"
    limits:
      memory: "120Gi"
  engines: ["default"]
  count: 3
  volumeSize: 128Gi
  cloudCache:
    enabled: true
    volumes:
      - size: 128Gi
  ...

Pass Extra JVM Flags to Classic Engines

Use executor.extraStartParams to pass additional JVM flags to classic engine executors. Flags are appended to DREMIO_JAVA_SERVER_EXTRA_OPTS at startup.

Extra JVM flags for classic engine executors

executor:
  extraStartParams: >-
    -Dmy.property=value
    -XX:+UseG1GC
  ...

For Managed Engines, use engine.options.javaOptions instead — see Configure Extra Java Start Options.

Configure Engine Overrides

Engine overrides modify the configuration of one or more named engines. The values set under executor act as the default for all engines — overrides only need to specify what differs.

Engine overrides for an engine named 'small'

engineOverride:
 small:
   cpu: "8"
   memory: "60Gi"
   count: 2
   cloudCache:
     enabled: false

Engine overrides are also supported on Managed Engines. One common use is to disable the Cloud Columnar Cache (C3), which is enabled by default on all Managed Engines.

Pass Environment Variables to Engines

You can pass extra environment variables to coordinators or executors by adding the following to your values-overrides.yaml.

Extra environment variables for the coordinator

coordinator:
  extraEnvs:
    - name: <your-variable-name>
      value: "<your-variable-value>"
  ...

Extra environment variables for executors

executor:
  extraEnvs:
    - name: <your-variable-name>
      value: "<your-variable-value>"
  ...

Environment variables defined under executor apply to executors of both classic engines and Managed Engines.

Configure Immutable Admins

You can configure which users are immutable admins. For an overview, see Immutable Admins. This defaults to no immutable admins.

Immutable admins configuration

coordinator:
  security:
    immutable_admins:
      - "[email protected]"
      - "[email protected]"
  ...

Configure Pod Scheduling

You can control how Dremio pods are scheduled across your Kubernetes cluster by setting labels, annotations, tolerations, and node selectors. These settings can be applied globally or to individual StatefulSets.

warning

Define these values with caution. Any misconfiguration may prevent Kubernetes from scheduling your pods.

To understand the full metadata structure available for each StatefulSet, download Dremio's Helm charts and inspect values.yaml. For instructions, see Download Dremio's Helm Charts.

Labels

Global label

labels:
  foo: bar

Label for the Open Catalog StatefulSet

catalog:
  labels:
    foo: bar
  ...

For more information, see Labels and Selectors in the Kubernetes documentation.

Annotations

Global annotation

annotations:
  foo: bar

Annotation for the MongoDB StatefulSet

mongodb:
  annotations:
    foo: bar
  ...

For more information, see Annotations in the Kubernetes documentation.

Tolerations

Global toleration

tolerations:
- key: "key1"
  operator: "Equal"
  value: "value1"
  effect: "NoSchedule"

Toleration for the Open Catalog StatefulSet

catalog:
  tolerations:
  - key: "key1"
    operator: "Equal"
    value: "value1"
    effect: "NoSchedule"
  ...

For more information, see Taints and Tolerations in the Kubernetes documentation.

Node Selectors

Global node selector

nodeSelector:
  nodetype: coordinator

Node selector for the coordinator StatefulSet

coordinator:
  nodeSelector:
    nodetype: coordinator
  ...

Configure Load Balancer and DNS

You can configure the Dremio load balancer and cluster DNS settings to match your network environment.

Configure the Load Balancer

Dremio creates a public load balancer by default. The dremio-client service exposes an external IP for connecting to Dremio. For information on connecting after deployment, see Deploy Dremio on Kubernetes.

Use a Private Load Balancer

For private Kubernetes clusters with no public endpoint, set internalLoadBalancer: true.

Internal load balancer configuration

service:
  type: LoadBalancer
  internalLoadBalancer: true
  ...

Set a Static IP

To assign a static IP to your load balancer, set loadBalancerIP. If not set, an available IP is assigned when the load balancer is created.

Static IP configuration for the load balancer

service:
  type: LoadBalancer
  loadBalancerIP: <your-desired-ip>
  ...

tip

A static IP is useful when DNS is configured to expect Dremio at a specific address.

Configure for Amazon EKS in Auto Mode

When deploying to Amazon EKS in Auto Mode, add service annotations for the load balancer to start. For more information, see Use Service Annotations to configure Network Load Balancers in the AWS documentation.

Service annotations for Amazon EKS in Auto Mode

service:
  type: LoadBalancer
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
  ...

Configure a Custom Cluster DNS Domain

If your Kubernetes cluster uses a custom DNS domain, configure it in Dremio. By default, Dremio uses cluster.local.

Custom cluster DNS domain configuration

cluster:
  domain: custom.domain

For the given DNS pattern, the example custom domain results in the following: [pod].[service].[namespace].svc.custom.domain.

Extend Pod Specs

You can extend the pod spec of any Dremio component without modifying chart templates using the following keys. All values are merged into the rendered pod spec at deploy time.

Key	Type	Description
extraInitContainers	string (YAML)	Init containers to run before the main container starts. Value is a YAML string so Helm can template it (e.g. reference $.Values.dremio.image).
extraVolumes	list	Additional volumes to attach to the pod.
extraVolumeMounts	list	Additional volume mounts for the main container.
extraEnvs	list	Additional environment variables for the main container.
extraStartParams	string	Additional JVM flags appended to DREMIO_JAVA_SERVER_EXTRA_OPTS at startup.

These keys are supported on coordinator, executor, catalog, and catalogservices. For Managed Engines, the keys are split across value scopes — see below.

Managed Engine Key Placement

Managed Engine executor pods split the four pieces across three value scopes. Using the wrong keys results in settings being silently ignored.

	Key
Init containers	engine.executor.extraInitContainers
Volumes	engine.extraVolumes
Volume mounts (main container)	executor.extraVolumeMounts
Environment variables	executor.extraEnvs

engine.executor.extraEnvs and engine.executor.extraVolumeMounts are silently ignored — environment variables and volume mounts for the executor main container come from the executor.* coalesce chain instead.

For an example using these keys to distribute a CA certificate to coordinator and executor pods, see Configure TLS with S3-Compatible Storage.

Pass Extra JVM Flags to the Coordinator

Use coordinator.extraStartParams to pass additional JVM flags to the coordinator. Flags are appended to DREMIO_JAVA_SERVER_EXTRA_OPTS at startup.

Extra JVM flags for the coordinator

coordinator:
  extraStartParams: >-
    -Dmy.property=value
    -XX:+UseG1GC
  ...

Embed Configuration Files

You can embed additional configuration files and variables directly in your values-overrides.yaml file. During deployment, these are combined to create a Kubernetes ConfigMap that Dremio uses as the source of truth for various settings.

Add Text Configuration Files

Use configFiles to add text-based configuration files (TXT, XML, or JSON) to your Dremio deployment. Each entry is a key-value pair where the key is the filename and the value is the file content. Files are mounted in /opt/dremio/conf/.

Example configuration files including a Hashicorp Vault configuration

dremio:
  configFiles:
    vault_config.json: |
      {
        "vaultUrl": "https://your-vault.com",
        "namespace": "optional/dremio/global/vault/namespace",
        "auth": {
          "kubernetes": {
            "vaultRole": "dremio-vault-role",
            "serviceAccountJwt": "file:///optional/custom/path/to/serviceAccount/jwt",
            "loginMountPath": "optional/custom/kubernetes/login/path"
          }
        }
      }
    another_config.json: |
      {
        "key-in-this-file": "content-of-this-key"
      }
  ...

Add Binary Configuration Files

Use configBinaries to provide binary configuration files. Content must be base64-encoded. Files are mounted in /opt/dremio/conf/.

Binary configuration file

dremio:
  configBinaries:
    custom-binary.conf: "base64EncodedBinaryContent"
  ...

Add dremio.conf Settings

Use dremioConfExtraOptions to add or override settings in dremio.conf.

Example dremio.conf settings to enable fabric TLS

dremio:
  dremioConfExtraOptions:
    "services.fabric.ssl.enabled": true
    "services.fabric.ssl.auto-certificate.enabled": true
  ...

Add a Java Truststore

Use trustStore under advancedConfigs to provide a Java truststore file. The content must be base64-encoded. To extract the encoded content, run cat truststore.jks | base64.

Java truststore configuration

dremio:
  advancedConfigs:
    trustStore:
      enabled: true
      password: "<your-truststore-password>"
      binaryData: "base64EncodedBinaryContent"

Configure Hive

Use hive2ConfigFiles and hive3ConfigFiles to provide Hive configuration files.

Hive 2 configuration

dremio:
  hive2ConfigFiles:
    hive-site.xml: |
      <?xml version="1.0" encoding="UTF-8"?>
      <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
      <configuration>
        <property>
          <n>hive.metastore.uris</n>
          <value>thrift://hive-metastore:9083</value>
        </property>
      </configuration>
  ...

Hive 3 configuration

dremio:
  hive3ConfigFiles:
    hive-site.xml: |
      <?xml version="1.0" encoding="UTF-8"?>
      <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
      <configuration>
        <property>
          <n>hive.metastore.uris</n>
          <value>thrift://hive3-metastore:9083</value>
        </property>
      </configuration>
  ...

Configure Telemetry

Telemetry egress is enabled by default. These metrics provide visibility into various components and services. To disable egress, add the following to your values-overrides.yaml:

Disable telemetry

telemetry:
  enabled: false
  ...

For more information on what telemetry collects and how to configure it, see Service Telemetry on Kubernetes.

Configure Logging

Dremio enables logging by default with a predefined volume size. To override the default configuration, add the following to your values-overrides.yaml:

Logging configuration

dremio:
  log:
    enabled: true
    volume:
      size: 10Gi
      storageClass: ""
  ...

To check the default volume size, download Dremio's Helm chart and inspect values.yaml. For instructions, see Download Dremio's Helm Charts.

Disable Semantic Search

To disable Semantic Search, add the following to your values-overrides.yaml:

Disable Semantic Search

opensearch:
  enabled: false
  replicas: 0