This topic describes how to configure Zookeeper.
To configure Zookeeper, the
zookeeper property with the hostname and port must be added to the dremio.conf file
on all nodes in the Dremio cluster. This is particularly important when Zookeeper is on an external node.
Default port: 2181
|Single-Node cluster||The Zookeeper property is not added to the dremio.conf file with a single node cluster.|
|Multi-node cluster||The Zookeeper property must be added to the dremio.conf file on each coordinator and executor node in the Dremio cluster.|
[info] Zookeeper Node List
Note that there are no spaces between the comma-separated list of Zookeeper nodes. If there are extra spaces, you may receive an error message similar to the following:
Error: java.net.UnknownHostException: 220.127.116.11: Name or service not known
zookeeperhost is the hostname (or ID address) where Zookeeper is located. See for more information.
- If Zookeeper is an embedded Zookeeper on the master node, then the Zookeeper hostname is the hostname of the master node.
- If Zookeeper is on an external node(s), then it is the hostname of the node(s) where it is located.
Embedded Zookeeper Examples
The following examples show how to configure an embedded Zookeeper depending on different deployment variables.
If you have one coordinator node (masterA) with an embedded Zookeeper and multiple executor nodes, then you specify that master node with the Zookeeper property in the dremio.conf file on every node in the Dremio cluster.
json zookeeper: "masterA:2181"it sat
If you are configuring for HA and you have an embedded Zookeeper on each coordinator node, then the Zookeeper property specifies those master nodes.
For example, if masterA is the primary master node and masterB is the secondary master node, then your zookeeper setting is:
If you want to change the default port used by the embedded ZooKeeper, then you add the following property:
If you are configuring for HA, you have an embedded Zookeeper on each master node, and you are using a different port number, then you specify the following settings on each Dremio coordinator and executor node in the cluster.
For example, if masterA is the primary master node, masterB is the secondary master node, and the zookeeper port is 1234, you specify the following:
services.coordinator.master.embedded-zookeeper.port: 1234 zookeeper: "masterA:1234,masterB:1234"
External Zookeeper Examples
To use your own external ZooKeeper cluster rather than Dremio's embedded ZooKeeper,
zookeeper property in the dremio.conf file.
If no ZooKeeper path is specified, Dremio defaults to
[info] The dremio.conf must be updated on every Dremio coordinator and executor node in the cluster.
The following example shows the default property and syntax:
services.coordinator.master.embedded-zookeeper.enabled: false zookeeper: "<host1>:2181,<host2>:2181"
If you are configuring for HA, then you set the Zookeeper property for every node that Zookeeper is on.
For example, if zooA is the primary Zookeeper node and zooB is the secondary Zookeeper node, then your Zookeeper setting is
If you are configuring multiple Dremio clusters to run using the same Zookeeper quorum, then you set the property to point to a different root.
The following example shows sample zookeeper entries in dremio.conf:
/dremioat the end of the path is the actual clusterID and is mandatory.