Skip to main content
Version: current [25.0.x]

ZooKeeper

Dremio utilizes Apache ZooKeeper behind the scenes for cluster coordination.

Dremio automatically runs an embedded ZooKeeper instance on every coordinator node. Alternatively, you can configure Dremio to use an external ZooKeeper cluster. The default port 2181.

DeploymentZooKeeper Property
Single node clusterThe ZooKeeper property is not added to the dremio.conf file with a single node cluster.
Multi-node clusterThe ZooKeeper property must be added to the dremio.conf file on each coordinator and executor node in the Dremio cluster.

Embedded ZooKeeper​

By default, Dremio runs an embedded ZooKeeper on port 2181. In a Dremio cluster, other than a standalone node, you must add the ZooKeeper property to the dremio.conf file on each coordinator and executor node in the Dremio cluster.

External ZooKeeper​

To use your own external ZooKeeper cluster rather than Dremio's embedded ZooKeeper, modify the zookeeper property in the dremio.conf file. If no ZooKeeper path is specified, Dremio defaults to /dremio.

note

  • Dremio recommends ZooKeeper 3.6 or later. ZooKeeper 3.4 has reached end-of-life and is no longer supported.

  • The dremio.conf must be updated on every Dremio coordinator and executor node in the cluster.

The following example shows the default property and syntax:

Default property and syntax
services.coordinator.master.embedded-zookeeper.enabled: false
zookeeper: "<host1>:2181,<host2>:2181"

  • If you are configuring for HA, then each dremio.conf file on all nodes in the Dremio cluster must specify the ZooKeeper property with every ZooKeeper location.

    For example, if zooA is the primary ZooKeeper node and zooB is the secondary ZooKeeper node, then your ZooKeeper setting is zookeeper: "zooA:2181,zooB:2181".

  • If you are configuring multiple Dremio clusters to run using the same ZooKeeper quorum, then you set the property to point to a different root.

ZooKeeper 3.5 Support (non-MapR)​

If you use an external ZooKeeper 3.5 cluster, add the system property -Dcurator-cache-bridge-force-tree-cache=true to dremio-env as shown in the following example:

Example for ZooKeeper 3.5 (non-MapR) clusters
export DREMIO_JAVA_SERVER_EXTRA_OPTS="-<otherProperties> -Dcurator-cache-bridge-force-tree-cache=true"

For More Information​

For more information about configuring ZooKeeper, see Configuring Zookeeper.