Skip to main content
Version: current [26.x Preview]

Deploying Dremio to Kubernetes

You can follow these instructions to deploy Dremio in Kubernetes provisioned through a cloud provider or running in an on-premises environment.

important

To access the 26.x preview, please complete the Get Started form and select Deploy on Kubernetes to begin your 30-day trial. For more information, see Dremio Free Trial. Then, follow the instructions for the Free Trial below.

Prerequisites​

Before deploying Dremio to Kubernetes, you must make sure you have the following:

  • A hosted Kubernetes environment to deploy and manage the Dremio cluster.
    Each Dremio release is tested against Amazon EKS, Azure AKS, and GCP GKE to ensure compatibility. Dremio also has successful customers deployed on OpenShift and Rancher. If you have a containerization platform built on top of Kubernetes but not listed here, please contact your provider and Dremio Account Team regarding compatibility.
  • Helm 3 set up on a local machine to run Helm commands. For installation instructions, refer to Installing Helm.
  • A local kubectl configured to access your Kubernetes cluster. For installation instructions, refer to Installing kubectl.
  • Object Storage: AWS S3 (including S3-Compatible, e.g., Minio), Azure Storage, and GCP Cloud Storage.

Additional Prerequisites for the Free Trial​

For the Free Trial, you will need to complete the steps in How to Obtain a Free Trial License. Once done, you will receive an email with a link to download the values-overrides.yaml configuration file, already with two key configurations for you to deploy your Free Trial without further due:

  • The free trial license.
  • Access to Dremio image repository in Quay.io. Unlike the Enterprise Edition, you won't need to create a RedHat account, log in to Quay.io, and get access to Dremio's Helm charts and images.

Additional Prerequisites for the Enterprise Edition​

FREE TRIAL

Ignore the prerequisites in this section if deploying a Free Trial.

For the Enterprise Edition you must:

  • Create an account on Quay.io to access Dremio's OCI repository, which stores Dremio's Helm charts and images.
    To get access, contact your Dremio account executive or Dremio Support.
    note

    If your internet access doesn't allow reaching Dremio's OCI repository in Quay.io, consider using a private mirror to fetch Dremio's Helm chart images.

  • Get a valid license key issued by Dremio to put in the Helm chart. To obtain the license, refer to Licensing.

Step 1: Deploying Dremio to Kubernetes​

To deploy the Dremio cluster in Kubernetes, do the following:

  1. Configure your values to deploy Dremio to Kubernetes in the file values-overrides.yaml. To configure it, see Configuring Your Values to Deploy Dremio to Kubernetes.

  2. On your terminal, start the deployment by installing Dremio's Helm chart:

    helm install <your-dremio-install-release> oci://quay.io/dremio/dremio-helm -f <your-local-path>/values-overrides.yaml --version <helm-chart-version> --set-file coordinator.web.auth.ssoFile=<your-local-path>/identity-provider-config.json

    Where:

    • <your-dremio-install-release> - The name that identifies your Dremio installation. For example, dremio-1-0.
    • <your-local-path>/values-overrides.yaml - The path to reach your values-overrides.yaml configuration file.
    • (Optional) --version <helm-chart-version> - The version of Dremio's Helm chart to be used. For example, 1.0. You can check the available versions using the helm list command. For more information, refer to Helm List.
    • (Optional) coordinator.web.auth.ssoFile=<your-local-path>/identity-provider-config.json - If you have configured Dremio to use an identity provider while configuring your values in the values-overrides.yaml, provide the required configuration file here.

  3. Monitor the deployment using the following command:

    $ kubectl get pods

    When all of the pods are in the Ready state, the deployment is complete.

    Troubleshooting

    • If a pod remains in Pending state for more than a few minutes, run the following command to view its status to check for issues, such as insufficient resources for scheduling:

      $ kubectl describe pods <pod-name>

    • If the events at the bottom of the output mention insufficient CPU or memory, do one of the following:

      • Adjust the values in the values-overrides.yaml configuration file and redeploy.
      • Add more resources to your Kubernetes cluster.

Step 2: Connecting to Dremio​

Now that you've installed the Helm chart and deployed Dremio on Kubernetes, the next step is connecting to Dremio, where you have the following options:

Connecting to the Dremio Console​

Run the following command to use the services dremio-client in Kubernetes to find the host for the Dremio Console:

$ kubectl get services dremio-client
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                          AGE
...             ...            ...             ...               ...                              ...

  • If the value in the TYPE column of the output is LoadBalancer, access the Dremio UI through the address in the EXTERNAL_IP column and port 9047.
    For example, in the output below, the value under the EXTERNAL-IP column is 8.8.8.8. Therefore, access the Dremio UI through http://8.8.8.8:9047.

    $ kubectl get services dremio-client
    NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                          AGE
    dremio-client   LoadBalancer   10.99.227.180   8.8.8.8           31010:32260/TCP,9047:30620/TCP   2d

    If you want to change the exposed port on the load balancer, change the value of the setting coordinator.web.port in the file values-overrides.yaml.

  • If the value in the TYPE column of the output is NodePort, access the Dremio UI through http://localhost:30670.

Connecting to Dremio from BI Tools via ODBC/JDBC​

Run the following command to use the services dremio-client in Kubernetes to find the host for ODBC/JDBC connections by using the following command:

$ kubectl get services dremio-client
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                          AGE
...             ...            ...             ...               ...                              ...

  • If the value in the TYPE column of the output is LoadBalancer, access Dremio using ODBC/JDBC through the address in the EXTERNAL_IP column and port 31010.
    For example, in the output below, the value under the EXTERNAL-IP column is 8.8.8.8. Therefore, access Dremio using ODBC/JDBC in port 31010 through http://8.8.8.8:31010.

    $ kubectl get services dremio-client
    NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                          AGE
    dremio-client   LoadBalancer   10.99.227.180   8.8.8.8           31010:32260/TCP,9047:30620/TCP   2d

    If you want to change the exposed port on the load balancer, change the value of the setting coordinator.client.port in the file values-overrides.yaml.

  • If the value in the TYPE column of the output is NodePort, access Dremio using ODBC/JDBC through http://localhost:32390.

Connecting to Dremio from BI Tools via Apache Arrow Flight​

Run the following command to use the services dremio-client in Kubernetes to find the host for Apache Arrow Flight connections by using the following command:

$ kubectl get services dremio-client
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                          AGE
...             ...            ...             ...               ...                              ...

  • If the value in the TYPE column of the output is LoadBalancer, access Dremio using Apache Arrow Flight through the address in the EXTERNAL_IP column and port 32010.
    For example, in the output below, the value under the EXTERNAL-IP column is 8.8.8.8. Therefore, access Dremio using Apache Arrow Flight through http://8.8.8.8:32010.

    $ kubectl get services dremio-client
    NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                          AGE
    dremio-client   LoadBalancer   10.99.227.180   8.8.8.8           31010:32260/TCP,9047:30620/TCP   2d

    If you want to change the exposed port on the load balancer, change the value of the setting coordinator.flight.port in the file values-overrides.yaml.

  • If the value in the TYPE column of the output is NodePort, access Dremio using Apache Arrow Flight through http://localhost:31357.