Configuring ZooKeeper

note

Dremio recommends ZooKeeper 3.6 or later. ZooKeeper 3.4 has reached end-of-life and is no longer supported.

To configure ZooKeeper, the zookeeper property with the hostname and port must be added to the dremio.conf file on all nodes in the Dremio cluster. This is particularly important when ZooKeeper is on an external node. The default port is 2181.

Deployment	ZooKeeper property
Single-Node cluster	The ZooKeeper property is not added to the dremio.conf file with a single node cluster.
Multi-node cluster	The ZooKeeper property must be added to the dremio.conf file on each coordinator and executor node in the Dremio cluster.

note

There are no spaces between the comma-separated list of ZooKeeper nodes. If there are extra spaces, you may receive an error message similar to the following: Error: java.net.UnknownHostException: 11.22.33.142: Name or service not known.

Property Syntax

Syntax for ZooKeeper property

zookeeper: "<host1>:2181,<host2>:2181"

The zookeeper host is the hostname (or IP address) where ZooKeeper is located.

• If ZooKeeper is an embedded ZooKeeper on the main coordinator node, then the ZooKeeper hostname is the hostname of the main coordinator node.
  zookeeper: "<main-coordinator-host1>:2181,<main-coordinator-host2>:2181"
• If ZooKeeper is on an external node, then the ZooKeeper hostname is the hostname of the node where it is located.
  zookeeper: "<zookeeper-host1>:2181,<zookeeper-host2>:2181"

Embedded ZooKeeper Examples

The following examples illustrate how to configure an embedded ZooKeeper according to different deployment variables.

• If you have one coordinator node (mainA) with an embedded ZooKeeper and multiple executor nodes, then you specify that main coordinator node with the ZooKeeper property in the dremio.conf file on every node in the Dremio cluster.

  One coordinator node with embedded ZooKeeper and multiple executor nodes

  zookeeper: "mainA:2181"

• If you want to change the default port used by the embedded ZooKeeper, add the following property:

  Property to change default port used by embedded ZooKeeper

  services.coordinator.master.embedded-zookeeper.port: <port>
  note

  The term master is a legacy label used in this property. We now refer to this as the main coordinator.

External ZooKeeper Examples

To use your own external ZooKeeper cluster rather than Dremio's embedded ZooKeeper, modify the zookeeper property in the dremio.conf file. If no ZooKeeper path is specified, Dremio defaults to /dremio.

note

The dremio.conf file must be updated on every Dremio coordinator and executor node in the cluster.

The following example shows the default property and syntax:

Default property and syntax for external ZooKeeper

services.coordinator.master.embedded-zookeeper.enabled: false
zookeeper: "<host1>:2181,<host2>:2181"

note

The term master is a legacy label used in this property. We now refer to this as the main coordinator.

• If you are configuring for HA, then you set the ZooKeeper property for every node that ZooKeeper is on.

  For example, if zooA is the primary ZooKeeper node and zooB is the secondary ZooKeeper node, then your ZooKeeper setting is zookeeper: "zooA:2181,zooB:2181".

• If you are configuring multiple Dremio clusters to run using the same ZooKeeper quorum, then you set the property to point to a different root.

  The following illustrate syntax and example ZooKeeper entries in dremio.conf (note that /dremio needs to be placed at the end of the config entry and is mandatory):

  Example ZooKeeper Syntax

  zookeeper: "<host1>:2181,<host2>:2181,<host3>:2181/<path to dremio>/dremio"
  Example of a Valid ZooKeeper Entry

  zookeeper: "<host1>:2181,<host2>:2181,<host3>:2181/dremioprod/dremio"
  Example of Invalid ZooKeeper Entries

  zookeeper: "<host1>:2181,<host2>:2181,<host3>:2181/dremioprod"
  zookeeper: "<host1>:2181/dremioprod,<host2>:2181/dremioprod,<host3>:2181/dremioprod"

ZooKeeper 3.5 Support

If you use an external ZooKeeper 3.5 cluster, add the system property -Dcurator-cache-bridge-force-tree-cache=true to dremio-env as shown in the following example:

Example for ZooKeeper 3.5 clusters

export DREMIO_JAVA_SERVER_EXTRA_OPTS="-<otherProperties> -Dcurator-cache-bridge-force-tree-cache=true"