Connecting Dremio Sonar to Arctic preview
To use Dremio Sonar with Arctic, you need to connect a Nessie source in the Sonar project.
Before you can connect to a Nessie source, ensure you have the following resources available:
- An Arctic-enabled project, which hosts the Nessie repository. If you do not have an Arctic-enabled project, see Adding an Arctic-enabled Project.
- A Sonar project, which will connect to the Nessie repository. If you want to create a new Sonar project, see Adding a Project.
- A personal access token. If you have not created a personal access token, see Personal Access Tokens for information about how Dremio uses these tokens and how to generate one. You will need to enter this token ID when you connect to a Nessie source.
You will also need admin access to an Amazon S3 bucket when creating tables (CREATE TABLE). Sonar will use its Project Store by default, but a user can also provide a LOCATION in the CREATE TABLE statement. The Arctic data optimization service writes metadata and data to the table’s existing LOCATION.
Connecting a Nessie Source to Sonar
To use Dremio Sonar with Arctic, you need to connect a Sonar project to a Nessie source.
To add a Nessie source:
Select a Sonar project in which you want to connect to a Nessie source.
From the Datasets pane, Nessie Repositories section, click the Add Nessie (plus) icon.
On the Add Nessie dialog box, click Nessie (Preview).
On the New Nessie (Preview) Source dialog box > General tab:
Enter a name for the Nessie source.
Under Nessie Repository, either select Custom Repository to create a new source or, if available, select an existing one.
Under Nessie Endpoint, paste the endpoint that you recorded earlier.
Under Nessie Access Token, enter the personal access token that you generated earlier.
Next, select the Storage tab.
On the Storage page:
Under S3 Authentication, you can choose Project Data Credentials or enter your own Data Source Credentials. For information about setting up the Authentication Method, see Amazon S3 Authentication Options for Nessie.
Under Default Storage Location, enter the root path to your S3 bucket.
When creating new tables in Arctic, this is the default location where tables will be created when you do not specify a
LOCATIONattribute in a
CREATE TABLEstatement. Example root path:
Amazon S3 Authentication for Nessie
For Authentication, select one of the following options.
|Authentication Option||Description||Configuration Steps|
|Project Data Credentials||Default credentials used to access all sources in your project. Added as part of your signup and project creation.||Project Data Credentials with Access Key/IAM Role|
|Data Source Credentials||Credentials to access a a specific source. Created as part of the source configuration.||Data Source Credentials with Access Key|
|Data Source Credentials||Credentials to access a specific source. Created as part of the source configuration. Project role/key assumes this role to access the source.||Data Source Credentials with IAM Role|