This section provides high-level conceptual information related to cluster deployments.
The following diagram shows the basic Dremio cluster architecture that is generally applicable to all deployments whereas:
- Queries: Access can be implement via Dremio REST/UI or Dremio OJBC/JDBC drivers.
- Coordinator node: One (1) or more nodes can be configured with the master-coordinator role.
A single coordinator node is recommended for Dremio cluster.
Multiple coordinator nodes are not supported except for HA. HA requires two (2) or more coordinator nodes configured with the master-coordinator role.
- Executor nodes: One (1) or more nodes can be configured with executor role. The number of executor nodes depends on your load. For example, you would increase the number of executor nodes when you have a large queries (large amount of data or heavy computation) and/or a large number of queries.
- Metadata storage: Typically locally located on the coordinator node. Default: no configuration required. If configuring for high availability, metadata storage must be configured as an external store.
- Zookeeper: (not shown) Can be either embedded on the coordinator node or external. Default: embedded. If configuring for high availability, an external zookeeper quorum is required.
- Distributed Store: (not shown) Mounted locally and configured on all the Dremio nodes.
For specific architectural deployment models, see:
- Azure AKS
- Amazon EKS
- Azure Template
- Amazon Template
- Hadoop using YARN
- MapR using YARN
- Standalone Cluster
For More Information
In addition to architecture deployment information, this topic covers:
- Dremio Services
- Metadata Storage
- Distributed Storage
- High Availability
- Wire Encryption
- Multiple AWS Clusters