Skip to main content
Version: 24.3.x

Azure AKS

This topic describes the deployment architecture of Dremio on Azure Kubernetes Service (AKS).

Architecture​

Azure AKS Diagram

Requirements​

  • Azure AKS version 1.12.7 or later

  • Kubernetes cluster on Azure AKS (see Setting up an AKS Cluster for instructions)

  • Worker node instance type (minimum): E16s_v3 (16 core, 128 GiB memory, 400 GiB SSD, and 32 GB disk size)

Setting up an AKS Cluster​

To set up a Kubernetes cluster on Azure Kubernetes Service (AKS), use the Azure portal or Azure CLI. The following steps are for Azure Portal:

  1. Sign in to the Azure portal at https://portal.azure.com.

  2. Create an AKS cluster (see Quickstart: Deploy an Azure Kubernetes Service (AKS) cluster using the Azure portal for more information).

  • Make sure to create a node group with an instance type that has 16CPU and 128GB of memory (E16s_v3 is recommended).

  • The number of allocated nodes in the AKS cluster must be equivalent to the number of Dremio executors plus one (1) for the Dremio master-coordinator.

  1. Connect to the cluster (see Quickstart: Deploy an Azure Kubernetes Service (AKS) cluster using the Azure portal for more information).

  2. Install Helm (see Install Apps with Helm in AKS for more information).

note

If you plan on using a local shell for access, install Helm.

  1. Begin Deploying Dremio.

Deploying Dremio​

To deploy Dremio on AKS, follow the steps in Installing Dremio on Kubernetes in the dremio-cloud-tools repository on GitHub.

High Availability​

High availability is dependent on the Kubernetes infrastructure. If any of the Kubernetes pods go down for any reason, Kubernetes brings up another pod to replace the pod that is out of commission.

  • The Dremio master-coordinator and secondary-coordinator pods are each StatefulSet. If the master-coordinator pod goes down, it recovers with the associated persistent volume and Dremio metadata preserved.

  • The Dremio executor pods are a StatefulSet with an associated persistent volume. secondary-coordinator pods do not have a persistent volume. If an executor pod goes down, it recovers with the associated persistent volume and data preserved.

Load Balancing​

Load balancing distributes the workload from Dremio's web client (UI and REST) and ODBC/JDBC clients. All web and ODBC/JDBC clients connect to a single endpoint (load balancer) rather than directly to an individual pod. These connections are then distributed across available coordinator (master-coordinator and secondary-coordinator) pods.

For More Information​