Architecture

This section provides high-level conceptual information related to cluster deployments.

The following diagram shows the basic Dremio cluster architecture that is generally applicable to all deployments whereas:

• Queries: Access can be granted via the Dremio web application, REST API, or Dremio ODBC/JDBC drivers.
• Coordinator node: One (1) or more nodes can be configured with the master-coordinator role. A single coordinator node is recommended for a Dremio cluster.
• Executor nodes: One (1) or more nodes can be configured with executor role. The number of executor nodes depends on your load. For example, you would increase the number of executor nodes when you have a large queries (large amount of data or heavy computation) and/or a large number of queries.
• Metadata storage: Typically locally located on the coordinator node. Default: no configuration required. If configuring for high availability, metadata storage must be configured as an external store.
• Zookeeper: (not shown) Can be either embedded on the coordinator node or external. Default: embedded. If configuring for high availability, an external zookeeper quorum is required.
• Distributed Store: (not shown) Mounted locally and configured on all the Dremio nodes.

Deployment Models

For specific architectural deployment models, see:

• Azure AKS

• Amazon EKS

• Azure Template

• Amazon Template

• Google GKE

• Hadoop using YARN

• MapR using YARN

• Standalone Cluster

For More Information

In addition to architecture deployment information, this topic covers:

• Dremio Services

• Metadata Storage

• Distributed Storage

• ZooKeeper

• High Availability

• Wire Encryption

• Authentication

• Limits