Configuring Dremio Services
This topic describes how to configure Dremio services depending on your deployment. Dremio services are configured in the dremio.conf file. In addition, each node in the Dremio cluster must be configure accordingly.
The Dremio services
property determines whether the node is enabled with the
master coordinator, secondary coordinator, or executor role.
Coordinator Node
To configure a coordinator node with the master-coordinator role, use the following Dremio services configuration.
services: {
coordinator.enabled: true,
coordinator.master.enabled: true,
executor.enabled: false
}
Secondary Coordinator Nodes
Configure multiple secondary coordinator nodes to improve concurrency and distribute query planning for ODBC and JDBC client requests to your deployment. To distribute queries across your deployment, Dremio recommends that you also configure ZooKeeper.
Secondary coordinator nodes run ODBC and JDBC queries only from clients that connect to the secondary coordinator either directly or with a load balancer.
Run metadata commands only on the master coordinator, not on secondary coordinators, unless one or both of these conditions are true:
- You are working with Iceberg tables.
- Unlimited splits are enabled on the cluster.
To enable this feature for YARN-based and standalone deployments, add the following Dremio services configuration to the dremio.conf file on each secondary coordinator node in your cluster and restart your deployment:
services: {
coordinator.enabled: true,
coordinator.master.enabled: false,
coordinator.web.enabled: false,
executor.enabled: false
}
To enable secondary coordinator nodes for Azure AKE and AWS EKS deployments, update coordinator.count
in values.yaml
of the helm chart and restart your deployment.
Tip: Dremio recommends a maximum of five secondary coordinator nodes for a deployment.
Dremio deployments on AWS Edition and Azure ARM do not support secondary coordinator nodes.
Executor Node
To configure a node with the executor role, use the following Dremio services configuration.
services: {
coordinator.enabled: false,
coordinator.master.enabled: false,
executor.enabled: true
}