Skip to main content
Version: current [26.x Preview]

Deploying Dremio to Kubernetes

You can follow these instructions to deploy Dremio in Kubernetes provisioned through a cloud provider or running in an on-premises environment.

important

To access the 26.x Preview please follow the Start for Free Flow at the top of the page and select Kubernetes 30 day Trial. For more information, see Dremio Free Trial.

Prerequisites​

Before deploying Dremio to Kubernetes, you must ensure you have the following:

  • A hosted Kubernetes environment to deploy and manage the Dremio cluster. For more information, refer to Kubernetes Options.

  • A valid license key issued by Dremio to put in the Helm chart. To obtain one, refer to Licensing.

  • Image access:

    1. Enterprise customers must create an account on Quay.io to access Dremio's OCI repository, which stores Dremio's Helm charts and images. To get access, contact your Dremio account executive or Dremio Support.
    2. For free trial users, limited access to charts and images is provided automatically. Learn more in Dremio Free Trial.
    note

    If your internet access does not allow reaching Dremio's OCI repository in Quay.io, consider using a private mirror to fetch Dremio's Helm chart images.

  • Helm 3 set up on a local machine to run Helm commands. For installation instructions, refer to Installing Helm.

  • A local kubectl configured to access your Kubernetes cluster. For installation instructions, refer to Installing kubectl.

  • Object Storage: AWS S3 (including S3-Compatible, e.g., Minio), Azure Storage, and GCP Cloud Storage.

Step 1: Deploying Dremio to Kubernetes​

To deploy the Dremio cluster in Kubernetes, do the following:

  1. Configure your values to deploy Dremio to Kubernetes in the file values-overrides.yaml. To obtain the file and configure it, see Configuring Your Values to Deploy Dremio to Kubernetes.

  2. On your terminal, start the deployment by installing Dremio's Helm chart:

    helm install <your-dremio-install-release> oci://quay.io/dremio/dremio-helm -f <your-local-path>/values-overrides.yaml --version <helm-chart-version> --set-file coordinator.web.auth.ssoFile=<your-local-path>/identity-provider-config.json

    Where:

    • <your-dremio-install-release> - The name that identifies your Dremio installation. For example, dremio-1-0.
    • <your-local-path>/values-overrides.yaml - The path to reach your values-overrides.yaml configuration file.
    • (Optional) --version <helm-chart-version> - The version of Dremio's Helm chart to be used. For example, 1.0. You can check the available versions using the helm list command. For more information, refer to Helm List.
    • (Optional) coordinator.web.auth.ssoFile=<your-local-path>/identity-provider-config.json - If you have configured Dremio to use an Identity Provider while configuring your values in the values-overrides.yaml, provide the required config file here.

  3. Monitor the deployment using the following command:

    $ kubectl get pods

    When all of the pods are in the Ready state, the deployment is complete.

    Troubleshooting

    • If a pod remains in Pending state for more than a few minutes, run the following command to view its status to check for issues, such as insufficient resources for scheduling:

      $ kubectl describe pods <pod-name>

    • If the events at the bottom of the output mention insufficient CPU or memory, do one of the following:

      • Adjust the values in the values-overrides.yaml configuration file and redeploy.
      • Add more resources to your Kubernetes cluster.

Step 2: Connecting to Dremio​

Now that you've installed the Helm chart and deployed Dremio on Kubernetes, the next step is connecting to Dremio, where you have the following options:

Connecting to the Dremio Console​

Run the following command to use the services dremio-client in Kubernetes to find the host for the Dremio Console:

$ kubectl get services dremio-client
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                          AGE
...             ...            ...             ...               ...                              ...

  • If the value in the TYPE column of the output is LoadBalancer, access the Dremio UI through the address in the EXTERNAL_IP column and port 9047.
    For example, in the output below, the value under the EXTERNAL-IP column is 8.8.8.8. Therefore, access the Dremio UI through http://8.8.8.8:9047.

    $ kubectl get services dremio-client
    NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                          AGE
    dremio-client   LoadBalancer   10.99.227.180   8.8.8.8           31010:32260/TCP,9047:30620/TCP   2d

    If you want to change the exposed port on the load balancer, change the value of the setting coordinator.web.port in the file values-overrides.yaml.

  • If the value in the TYPE column of the output is NodePort, access the Dremio UI through http://localhost:30670.

Connecting to Dremio from BI Tools via ODBC/JDBC​

Run the following command to use the services dremio-client in Kubernetes to find the host for ODBC/JDBC connections by using the following command:

$ kubectl get services dremio-client
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                          AGE
...             ...            ...             ...               ...                              ...

  • If the value in the TYPE column of the output is LoadBalancer, access Dremio using ODBC/JDBC through the address in the EXTERNAL_IP column and port 31010.
    For example, in the output below, the value under the EXTERNAL-IP column is 8.8.8.8. Therefore, access Dremio using ODBC/JDBC in port 31010 through http://8.8.8.8:31010.

    $ kubectl get services dremio-client
    NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                          AGE
    dremio-client   LoadBalancer   10.99.227.180   8.8.8.8           31010:32260/TCP,9047:30620/TCP   2d

    If you want to change the exposed port on the load balancer, change the value of the setting coordinator.client.port in the file values-overrides.yaml.

  • If the value in the TYPE column of the output is NodePort, access Dremio using ODBC/JDBC through http://localhost:32390.

Connecting to Dremio from BI Tools via Apache Arrow Flight​

Run the following command to use the services dremio-client in Kubernetes to find the host for Apache Arrow Flight connections by using the following command:

$ kubectl get services dremio-client
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                          AGE
...             ...            ...             ...               ...                              ...

  • If the value in the TYPE column of the output is LoadBalancer, access Dremio using Apache Arrow Flight through the address in the EXTERNAL_IP column and port 32010.
    For example, in the output below, the value under the EXTERNAL-IP column is 8.8.8.8. Therefore, access Dremio using Apache Arrow Flight through http://8.8.8.8:32010.

    $ kubectl get services dremio-client
    NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                          AGE
    dremio-client   LoadBalancer   10.99.227.180   8.8.8.8           31010:32260/TCP,9047:30620/TCP   2d

    If you want to change the exposed port on the load balancer, change the value of the setting coordinator.flight.port in the file values-overrides.yaml.

  • If the value in the TYPE column of the output is NodePort, access Dremio using Apache Arrow Flight through http://localhost:31357.