Configuring ZooKeeper

note

Dremio recommends ZooKeeper 3.6 or later. ZooKeeper 3.4 has reached end-of-life and is no longer supported.

To configure ZooKeeper, the zookeeper property with the hostname and port must be added to the dremio.conf file on all nodes in the Dremio cluster. This is particularly important when ZooKeeper is on an external node. The default port is 2181.

Deployment	ZooKeeper property
Single-Node cluster	The ZooKeeper property is not added to the dremio.conf file with a single node cluster.
Multi-node cluster	The ZooKeeper property must be added to the dremio.conf file on each coordinator and executor node in the Dremio cluster.

note

There are no spaces between the comma-separated list of ZooKeeper nodes. If there are extra spaces, you may receive an error message similar to the following: Error: java.net.UnknownHostException: 11.22.33.142: Name or service not known.

Property Syntax

Syntax for ZooKeeper property

zookeeper: "<host1>:2181,<host2>:2181"

note

The zookeeper host is the hostname (or IP address) where ZooKeeper is located.

• If ZooKeeper is an embedded ZooKeeper on the master node, then the ZooKeeper hostname is the hostname of the master node.
  zookeeper: "<master-coordinator-host1>:2181,<master-coordinator-host2>:2181"
• If ZooKeeper is on an external node, then the ZooKeeper hostname is the hostname of the node where it is located.
  zookeeper: "<zookeeper-host1>:2181,<zookeeper-host2>:2181"

Embedded ZooKeeper Examples

The following examples illustrate how to configure an embedded ZooKeeper according to different deployment variables.

• If you have one coordinator node (masterA) with an embedded ZooKeeper and multiple executor nodes, then you specify that master node with the ZooKeeper property in the dremio.conf file on every node in the Dremio cluster.

  One coordinator node with embedded ZooKeeper and multiple executor nodes

  zookeeper: "masterA:2181"

• If you are configuring for HA and you have an embedded ZooKeeper on each coordinator node, then the ZooKeeper property specifies those master nodes. For example, if masterA is the primary master node and masterB is the secondary master node, then your ZooKeeper setting is:

  Embedded ZooKeeper on each coordinator node

  zookeeper: "masterA:2181,masterB:2181"

• If you want to change the default port used by the embedded ZooKeeper, add the following property:

  Property to change default port used by embedded ZooKeeper

  services.coordinator.master.embedded-zookeeper.port: <port>

• If you are configuring for HA, you have an embedded ZooKeeper on each master node, and you are using a different port number, then you specify the following settings on each Dremio coordinator and executor node in the cluster. For example, if masterA is the primary master node, masterB is the secondary master node, and the ZooKeeper port is 1234, you specify the following:

  Embedded ZooKeeper on each master node and different port number

  services.coordinator.master.embedded-zookeeper.port: 1234
  zookeeper: "masterA:1234,masterB:1234" 

External ZooKeeper Examples

To use your own external ZooKeeper cluster rather than Dremio's embedded ZooKeeper, modify the zookeeper property in the dremio.conf file. If no ZooKeeper path is specified, Dremio defaults to /dremio.

note

The dremio.conf file must be updated on every Dremio coordinator and executor node in the cluster.

The following example shows the default property and syntax:

Default property and syntax for external ZooKeeper

services.coordinator.master.embedded-zookeeper.enabled: false
zookeeper: "<host1>:2181,<host2>:2181"

• If you are configuring for HA, then you set the ZooKeeper property for every node that ZooKeeper is on.

  For example, if zooA is the primary ZooKeeper node and zooB is the secondary ZooKeeper node, then your ZooKeeper setting is zookeeper: "zooA:2181,zooB:2181".

• If you are configuring multiple Dremio clusters to run using the same ZooKeeper quorum, then you set the property to point to a different root.

  The following illustrate syntax and example ZooKeeper entries in dremio.conf (note that /dremio needs to be placed at the end of the config entry and is mandatory):

  ZooKeeper Syntax in dremio.conf

  zookeeper: "<host1>:2181,<host2>:2181,<host3>:2181/<path to dremio>/dremio"
  Example of a Valid ZooKeeper Entry in dremio.conf

  zookeeper: "<host1>:2181,<host2>:2181,<host3>:2181/dremioprod/dremio"
  Example of Invalid ZooKeeper Entries in dremio.conf

  zookeeper: "<host1>:2181,<host2>:2181,<host3>:2181/dremioprod"
  zookeeper: "<host1>:2181/dremioprod,<host2>:2181/dremioprod,<host3>:2181/dremioprod"

ZooKeeper Versions Before 3.6

If you use an external ZooKeeper version earlier than 3.6, add the system property -Dcurator-cache-bridge-force-tree-cache=true to dremio-env as shown in the following example:

Example for ZooKeeper versions earlier than 3.6

export DREMIO_JAVA_SERVER_EXTRA_OPTS="-<otherProperties> -Dcurator-cache-bridge-force-tree-cache=true"

For More Information

For more information about ZooKeeper, see ZooKeeper.