Skip to main content
Version: current [26.x]

Upgrade Dremio on Kubernetes

This guide covers upgrading Dremio on Kubernetes for two different scenarios:

Choose the appropriate section below based on your current Dremio version.

Upgrade Dremio 26.0+​

Prerequisites​

Before upgrading Dremio, make sure that all engines are stopped. For more information, see Stopping All Engines.

Upgrade Procedure​

To upgrade the Dremio platform, update the Helm chart version to the most recent, which tags the version of Dremio you want to upgrade to. The Dremio release notes will provide the corresponding Helm chart version. There will be Dremio Helm chart releases that do not upgrade Dremio but update some other component. However, with every Dremio release, there will be a Helm chart release where the image tags for the various services are updated. Enterprise customers can view a list of all Helm charts and image tags on Quay.io.

During the upgrade process, existing pods are terminated, and new pods are created with the new images. After all the newly created pods are restarted and running, your Dremio cluster is upgraded.

tip

If you do not know your Helm chart release name, use helm list to list the Helm deployments in a selected namespace.

To upgrade Dremio:

  1. Ensure that your Dremio is backed up. For more information, see Backup.
  2. Ensure that no queries are running on the cluster, as any running queries will fail when services start terminating.
  3. Construct the appropriate helm upgrade command, for example: 
    helm upgrade <chart-release-name> oci://quay.io/dremio/dremio-helm -f <your-local-path>/values-overrides.yaml --version <new-desired-version>
  4. Execute the helm upgrade command.
  5. Pods will begin restarting with the new images and, once finished, Dremio will be accessible.
note

The job results cleanup optimization uses a secondary index to optimize the results cleanup, which implies a one-time reindexing of the jobs table during the upgrade.

The reindexing duration depends on the total number of jobs stored in the KV store. In environments with a large volume of jobs, this can increase the overall upgrade time.

Upgrade from Dremio 24/25 to 26.0+​

For Enterprise customers, version Dremio 26 brought the v3 Helm charts with it. The former v2 Helm charts, distributed via the dremio-cloud-tool GitHub, used for Dremio versions 24 and 25, are not compatible with version 26.

It is possible to upgrade an existing deployment. However, Enterprise customers need to migrate from the v2 Helm charts to the v3 Helm charts before any upgrade can take place. The v3 Helm charts are distributed via our image repository Quay.io.

Customers must move the relevant content from their existing value.yaml (and any other deployment-specific configurations like Identity Provider authentication) into the new values-overrides.yaml configuration file, as detailed in Configure your Values.

Some configurations can be left behind. For example, the new UI experience has superseded the executor configuration in the charts. For more information, see Managing Engines in Kubernetes.

warning

Skip the next paragraph if you did not use the Executor HPA and node life cycle policy.

Before upgrading to Dremio 26.0+, if you intend to continue to use Classic Engines, the no longer supported node life cycle policy should be disabled. To check for this option, look at the executor section in your old Helm Charts values.yaml, and see if node_lifecycle_service_enabled: true is set. If it's set to true, change it to false and redeploy Dremio. If it's not present, that is the same as false. Despite this, if post upgrade you note the Executors of a Classic Engine marked as paused on the node activity panel, you can resolve this with a call to Dremio's Blacklist API (see Allowing all Nodes).

Once the new values-overrides.yaml configuration file for 26+ and other deployment configurations are prepared, you can proceed with the upgrade.

For help with this process, please reach out to Dremio Support and your Account Executive. More detailed guides and help from Dremio's professional services team can be provided.

To upgrade Dremio:

  1. Ensure you have created a new values-overrides.yaml configuration file with relevant values from your existing deployment ported over per Configure your Values

  2. Ensure that your Dremio is backed up. For more information, see Backup.

  3. Ensure that no queries are running on the cluster, as any running queries will fail when services start terminating.

  4. Uninstall your existing Dremio deployment:

    helm uninstall <chart-release-name>

    This will delete existing pods and remove other elements of the existing Dremio deployment. Crucially, it will not delete the dremio-master-0 volume, which contains the KV store and Dremio's state.

    note

    The term master is a legacy label. We now refer to this as the main coordinator pod.

  5. Confirm the dremio-master-0 volume still exists in the namespace you want to reinstall Dremio. This can be confirmed with:

    kubectl get pvc --namespace <dremio-install-namespace>

    Each of your executors should have left behind two volumes, but the main executor should have left only one.

  6. We're now ready to install Dremio 26.0+. Follow the instructions in Deploying Dremio to Kubernetes to complete the installation.

warning

Dremio must be deployed to the same location as the previous version to mount the dremio-master-0 volume. It's the content of this volume that is being upgraded.

note

The job results cleanup optimization uses a secondary index to optimize the results cleanup, which implies a one-time reindexing of the jobs table during the upgrade.

The reindexing duration depends on the total number of jobs stored in the KV store. In environments with a large volume of jobs, this can increase the overall upgrade time.