Dremio's Well-Architected Framework
Dremio’s Well-Architected Framework is provided to anyone who is designing and/or operating solutions with Dremio. It provides insight from the lessons learned over time helping hundreds of customers be successful. The framework is composed of pillars that provide design principles and concrete best practices that should be used with Dremio based on those principles. The best practices are generally applicable across on-premise, Dremio Cloud and public cloud deployments. There are links to more detailed how-to articles that are specific to a particular type of deployment.
Dremio’s Well-Architected Framework follows five common pillars from cloud providers AWS, Microsoft, and Google, and includes a sixth pillar, the self-service semantic layer, that is specific to Dremio:
- Performance Efficiency
- Cost Optimization
- Operational Excellence
- Self-service Semantic Layer
Each pillar is a grouping of principles, best practices, and how-to articles around that specific pillar.
Dremio is deployed in production as a cluster. The diagram below outlines the key components in the Dremio architecture.
Access to Dremio can be granted via the Dremio console (web application), REST API, Arrow Flight, or Dremio ODBC/JDBC drivers. One or more coordinator nodes can be configured with the master-coordinator role. Zero or more scale-out coordinators can be added to help with planning queries. One or more executor nodes can be configured -- the number of executor nodes depends on your workload. Executor nodes are organized into logical engines to provide workload isolation. The Distributed Store is mounted locally and configured on all the Dremio nodes.