Managing Your Data

note

An Arctic catalog, created during the Sonar project creation process, provides data management capabilities for your project.

Data Modeling

With Sonar, administrators can organize data using folders, tables, and views, and create a single source of truth for analysts.

Folders, Tables, and Views

You can use folders to organize your data into common themes, such as data quality (e.g., raw, enrichment, and presentation layers), business units (e.g., Marketing, Sales, and Finance), or geographic regions. Folders can contain tables, views, and even other folders.

A view is a virtual table based on the result set of a query. Views do not contain any data. You can create views from data that resides in any data source, folder, table, or view that you have access to. Views leverage the Iceberg view specification.

You can also create tables in your Arctic catalog. Dremio automates maintenance processes for these tables, including compaction and garbage collection. Tables in Arctic catalogs always use the Iceberg table format.

Data Discovery

Each dataset has associated metadata, such as when the dataset was created and updated. To further enhance the dataset and make it easy for your users to understand and use the dataset, you can add wikis and labels.

Wikis and Labels

You can add helpful context to folders and datasets for teammates using wikis and labels.

You can use wikis to provide detailed descriptions and context for your datasets, just like a README for your data. Wikis support Markdown for formatting. For example, you can add a description of what the dataset contains, the sources it was created from, and provide example queries.

You can use labels to easily categorize datasets for users. For example, you can add a PII label to a dataset to indicate that it contains personally identifiable information.

note

Currently, wikis and labels can only be managed and viewed in an Arctic catalog's main branch.

Datasets Page

When you log into your Sonar project, you are first taken to the Datasets page. The Datasets page allows you to easily navigate and manage your data:

Location	Description
1	By default, an Arctic catalog, created during the Sonar project creation process, provides data management capabilities for your project. You can create folders, tables, and views in this catalog.
2	A folder is a directory which contains tables, views, and even other folders. Folders provide a way to group datasets by common themes such as a project, purpose, department, or geographic region. Folders can be further organized by creating a hierarchy of folders.
3	A source is a data lake or external source (such as a relational database) that you can connect to Dremio Sonar.
4	The title indicates the source or folder you are currently exploring. In this case, the user is viewing the contents of the Raw folder in Gnarly Catalog.