On this page

    Connecting Apache Flink to Arctic preview

    To connect Apache Flink to Arctic, use the configuration instructions provided in Project Nessie for Flink via Iceberg and edit the following information:

    • The full url of the Nessie API Endpoint to Nessie: Replace the example URL in the configuration with the Nessie API Endpoint.
    • The authentication type: Set the Nessie authentication type to BEARER.
    • The authentication token: Provide the personal access token you generated in your Dremio Cloud organization. If you have not created a personal access token, see Personal Access Tokens for information about how Dremio Cloud uses these tokens and how to generate one.

    Replace the following properties in the Project Nessie configuration example:

    • Replace <catalog_name> with a unique value that is valid in Apache Flink (for example, nessie).

    note:

    Project Nessie uses the example catalog name of nessie.

    • Replace <project-id> with the Project ID of the Arctic project.
      For example: 06eg3b55-cc99-4c96-9acb-6f0ce010a81e

    To obtain the Arctic project ID:

    1. Sign in to Dremio Cloud.

    2. From the side navigation bar, select an Arctic project.

    3. Select the Settings (gear) icon.

    4. From the General Information tab, copy the Project ID and replace 〈project-id〉.

    • Replace <personal-access-token> with your personal access token. See Creating a Token to learn how to generate one.
      For example: RDViJJHrS/u+JAwrzQVV2+kAuLxiNkbTgdWQKQhAUS72o2BMKuRWDnjuPEjACw==

      CREATE CATALOG <catalog_name> WITH (
         'type'='iceberg',
         'catalog-impl'='org.apache.iceberg.nessie.NessieCatalog',
         'uri'='https://nessie.dremio.cloud/v1/projects/<project-id>',
         'ref'='main',
         'warehouse' = '/path/to/flink/warehouse',
         'authentication.type' = 'BEARER',
         'authentication.token' = '<personal-access-token>'
          )
      

    To create tables in Apache Flink that are managed by Nessie and Apache Iceberg, see Create tables in the Project Nessie website.

    To read tables in Apache Flink, see Reading tables in the Project Nessie website.

    To learn more about the available Data Definition Language (DDL) commands in Apache Flink and the SQL queries you can run, see DDL commands in the Apache Iceberg documentation.