Skip to main content
Version: current [25.0.x]

Using Azure Key Vault for Secrets Management

Use Dremio's Azure Key Vault integration to reference a secret rather than providing a password or other sensitive information when you connect to a data source or configure a Dremio configuration file.

note

Dremio must be deployed on Azure to use the Azure Key Vault integration for secrets management.

After you complete the prerequisites, you can create secrets in Azure Key Vault and use them in Dremio. Dremio uses the secret references you provide to retrieve secret values at runtime and authenticate to data sources and other services.

Supported Data Sources

Dremio supports Azure Key Vault secrets for all data source types except Hadoop Distributed File System (HDFS), Hive 2.x and 3.x, MapR File System, and network-attached storage (NAS).

Supported Configuration Files and Properties

You can use an Azure Key Vault secret as the value for any of the configuration file properties listed in Encrypt Credentials.

Prerequisites

Dremio uses managed identities to connect to Azure Key Vault. To enable Dremio to connect to Azure Key Vault using managed identities, complete the following steps:

  1. Configure a managed identity in Azure.

  2. Assign the managed identity to the Dremio coordinator and executor virtual machines (VMs).

    note

    To prevent connection issues, make sure that the VM has only one managed identity.

    If you want to route all secret lookups to the coordinator VM, assign the managed identity to the Dremio coordinator VM and set the services.credentials.exec.remote_lookup.enabled support key to true in Dremio.

  3. Set the access policy in Azure Key Vault to allow access to the managed identity.

Retrieving the Secret Reference from Azure Key Vault

The secret reference for an Azure Key Vault secret is the secret identifier URL, without the secret version number. The secret identifier is available on the secret's details page in the Azure Key Vault console:

Location of the secret identifier for an Azure Key Vault secret

In this example, the Azure Key Vault secret value to use in Dremio is https://sourcecreds.vault.azure.net/secrets/azurestorage.

Using the Secret Reference when Connecting to Data Sources

When you configure a new data source or edit the settings for an existing data source, enter the partial secret identifier URL for the Azure Key Vault secret directly into the corresponding password or secret key field in the Dremio console. If you configure a data source using the Dremio API, provide the partial secret identifier as the value for the corresponding parameter in your request body.

Using the Secret Reference in Dremio Configuration Files

In Dremio configuration files, use the partial secret identifier URL for the Azure Key Vault secret in place of a plaintext secret. You can use the partial secret identifier URL as the value for any of the configuration file properties listed in Encrypt Credentials.

note

You must add dremio+ as a prefix to secret references before using them in core-site.xml files. For example:

dremio+https://sourcecreds.vault.azure.net/secrets/azurestorage