Skip to main content

Dremio Sonar

Dremio's control plane hosts Sonar projects and the objects in them, and the execution plane runs queries against data sources that are connected to Sonar projects.

How Queries Run Across the Two Types of Planes​

This diagram gives a simplified account of how the two planes interact when a user logs into Sonar and runs a query:

  1. A user authenticates to Sonar through a BI client application.
  2. The SQL proxy passes the credentials to the authentication manager, which validates the credentials and approves the authentication request.
  3. The user issues a query to Sonar.
  4. The SQL proxy forwards the query to the query planner.
  5. The query planner notifies the engine manager of the request.
  6. The engine manager finds or starts up a compute engine that has the resources to run the query. The compute engine runs within a subnet of your tenant, which might have multiple subnets available, each with resources for additional compute engines. Within one of the subnets runs the preview engine, the compute engine that Sonar uses to return previews (or subsets) of data to its SQL runner when a user runs a query there.
  7. The query planner passes the plan for the query to the compute engine that the engine manager has designated.
  8. The compute engine passes the results of the query back to the query planner, which passes them to the SQL proxy, which then passes them to the BI client application.

Objects in Sonar​

When you work with Sonar, you work in or with the objects that are depicted in this diagram:

Sonar Projects​

A Sonar project isolates compute, data and other resources needed by a team for data analysis. An organization may contain multiple projects. A project must be linked to a single cloud account. Your first project was created and linked to your cloud account as part of the sign-up process. For more information, see Managing Sonar Projects.

This diagram shows the clouds and projects in an organization, and how they are related. A cloud can be associated with more than one Sonar project, but a project can be associated with only one cloud:

Engines​

An engine process jobs that run queries issued by users (either through a client application or through the user interface) or by Sonar (as, for example, when Sonar creates a reflection that a user has defined). Compute resources for an engine are allocated in the cloud associated with the project. All engines in a project are associated with the same cloud.

Engines are made up of one or more VM instances and are automatically started and stopped by the Sonar control plane. Engines can be configured to have multiple replicas, which allow for scaling up. For more information, see Managing Engines.

Data Sources​

A data source can be a data lake, such as Amazon S3 and Azure Storage, or a relational database (referred to as an external source). For more information, see Connecting to Your Data.

Tables and Views​

A table contains the data from your source, formatted as rows and columns. A table cannot be modified by Sonar.

A view is a virtual table, created by running SQL statements or functions on a table or another view.

To learn more, see Datasets.

Reflections​

A reflection is an optimized materialization of source data or a query, similar to a materialized view, that is derived from an existing table or view. To learn more, see Accelerating Queries with Reflections.