Customizing EKS Deployment
This topic discusses additional topics associated with customizing or modifying the default configuration for deploying Dremio in an Amazon EKS environment.
Dremio Enterprise Edition
If you are deploying with Dremio Enterprise Edition, do the following:
- Contact Dremio to obtain read-access to the Dremio EE image (a dockerhub ID is needed). The Dremio EE image is not public.
- Create a Dremio-specific Kubernetes secret.
Follow the instructions in Specifying ImagePullSecrets on a Pod.
For example, to create a secret in Kubernetes, specify the following parameters:
Note: Pods can only reference image pull secrets in their own namespace, so this process needs to be done on the namespace where Dremio is being deployed.kubectl create secret docker-registry dremio-docker-secret --docker-username=your_username --docker-password=your_password_for_username --docker-email=DOCKER_EMAIL - Modify the following properties in values.yaml file:
image: Dremio EE image name.imagePullSecrets: Dremio-specific Kubernetes secret.
Cluster Configuration
Dremio provides a default cluster configuration that is deployed via the provided helm charts. To change the cluster configuration, modify the properties in the values.yaml file.
In the following default Dremio cluster configuration, each Dremio master-coordinator, slave-coordinator, and executor are a pod.
| Pod Type | Pod Count | Memory/CPU | Persistent Volume Size/Usage | Description |
|---|---|---|---|---|
| master-coordinator | 1 | 16GB 8 cores |
100GB: Persistent volume with metadata. | The master-coordinator pod manages the metadata. |
| Dremio admin | 1 | n/a | 0: No persistent volume. | The admin pod is used to run offline Dremio admin commands. To accomplish this, it uses the master-coordinator's persistent volume and metadata. Typically, this pod is not running. |
| slave-coordinator | 0 | 16GB 8 cores |
0: No persistent volume. | The slave-coordinator pod manages query planning, Dremio UI/REST requests, and ODBC/JDBC client requests. |
| executor | 3 | 16GB 4 cores |
100GB: Persistent volume with reflections, spilling, results, and upload data. | The executor pod is responsible for query execution. |
| zookeeper | 3 | 1024KB 0.5 cores |
10Gi | The zookeeper pod manages coordination between the Dremio pods. |
Load Balancer
Load balancing distributes the workload from Dremio's web (UI and REST) client and ODBC/JDBC clients. All web and ODBC/JDBC clients connect to a single endpoint (load balancer) rather than directly to an individual pod. These connections are then distributed across available coordinator (master-coordinator and slave-coordinator) pods.
[info] Recommendation
Load balancing is recommended when more than one coordinator pod (master-coordinator and slave-coordinator) is implemented.
If your LoadBalancer supports session affinity, it is recommended that you enable the sessionAffinity property.
Specifically, if you plan to configure two or more (2+) coodinator pods (master and slave-coordinator),
you must have the Load Balancer service and session affinity property enabled.
See the values.yaml section for more information.
The following table shows which configurations require a load balancer configured and sessionAffinity enabled:
| Master-coordinator Pod |
Slave-coordinator Pod |
Load Balancer Session Affinity |
|---|---|---|
| 1 | 0 | No |
| 1 | 1+ | Yes |
[info] As of Amazon EKS v1.11, session affinity is not available.
values.yaml
The values.yaml file is used to set the Dremio image, Dremio pods, and load balancing properties.
| Property | Description | Default |
|---|---|---|
| image | Specifies the image containing the Dremio software. If your image is Dremio Enterprise Edition, you must provide the Kubernetes secret for Dremio with the imagePullSecretes property. |
dremio/dremio-oss (Community Edition). |
| coordinator | Specifies the configuration (memory, cpu, count, and volume size) for coordinator pods. | See the values.yaml file for current values. |
| executor | Specifies the configuration (memory, cpu, count, and volume size) for Dremio's executor pods. | See the values.yaml file for current values. |
| zookeeper | Specifies the configuration (memory, cpu, count, and volume size) for Dremio Zookeeper pods. | See the values.yaml file for current values. |
| serviceType | Specifies the load balancer service. | LoadBalancer |
| sessionAffinity | Specifies your load balancer session affinity. If you have configured more than one (1+) slave-coodinators, this property must be enabled. | ClientIP |
| storageClass | Specifies a custom storage class. If you want to use a value other than the default, uncomment this value and provide the custom storage class. | The default persistent storage supported by the Kubernetes platform. |
| imagePullSecrets | Specifies the authorization required for Kubernetes to pull Dremio Enterprise Edition images. This property is used in conjunction with the image property. | n/a |
Config Directory
The files in the config directory are used for the following:
- Dremio configuration and environment properties
- Dremio Logs
- Deployment-specific files
Each file in the config directory provides defaults that can be used out-of-the-box, however, if you want to customize Dremio, do the following:
- Review and modify the files.
- Add deployment-specific files (for example, core-site.xml) by copying your file(s) to this directory.
Any customizations to your Dremio environment are propagated to all the pods when installing or upgrading the deployment.
| File | Description |
|---|---|
| dremio.conf | Used to specify various options related to node roles, metadata storage, distributed cache storage and more. See Configuration Overview if you want to customize your Dremio environment. |
| dremio-env | Used for setting Java options and log directories. See Configuration Overview if you want to customize your Dremio environment. |
| logback-access.xml | Used to control the log access. |
| logback.xml | Used to control the log levels. |