Source Configuration

This page details the config payload to create each source type.

Amazon Redshift

Dremio source type: REDSHIFT. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Redshift, see Adding a Redshift Cluster as a Data Source for additional information.

Config attributes for connecting to an Amazon Redshift source

"config": {
  "connectionString": "jdbc:redshift://redshift.example.com:5439/database",
  "authenticationType": [
    "ANONYMOUS",
    "MASTER",
    "AWS_PROFILE"
  ],
  "username": "myUsername",
  "password": "myPassword",
  "secretResourceUrl": arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy,
  "awsProfile": "profileName",
  "dbUser": "userName",
  "fetchSize": 200,
  "maxIdleConns": 8,
  "idleTimeSec": 60,
  "queryTimeoutSec": 0,
  "propertyList": [
    {
      "name": "ApplicationName", 
      "value": "MarketingDashboard"
    }
  ]
}

Parameters

connectionString Body String

The JDBC connection string. Read JDBC for more information.

Example: jdbc:redshift://redshift.example.com:5439/database

authenticationType Body String

Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed, MASTER when either credentials from a master database user or a secret resource URL is required, or AWS_PROFILE for authentication to an AWS source.

Enum: ANONYMOUS, AWS_PROFILE, MASTER

username Body String

The Amazon Redshift user name.

Example: myUsername

password Body String

The Amazon Redshift password.

Example: myPassword

secretResourceUrl Body String

For secret-based authentication, the Amazon Resource Name (ARN) for the secret. To keep the secret secure, Dremio returns the secretResourceUrl value as $DREMIO_EXISTING_VALUE$ in API responses.

Example: arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy

awsProfile Body String

The AWS profile name. If this is left blank, the default profile will be used.

Example: profileName

dbUser Body String

The name of the Amazon Redshift DbUser that will be used for authentication. If this is left blank, the default user name for your AWS IAM role will be used.

Example: userName

fetchSize Body Integer

The number of records to fetch per request. Set to 0 to enable Dremio to automatically set the size.

Example: 200

maxIdleConns Body Integer

Sets the maximum number of idle connections that you want to keep.

Example: 8

idleTimeSec Body Integer

Sets the idle time, in seconds, before a connection is evaluated for closure.

Example: 60

queryTimeoutSec Body Integer

Sets the timeout, in seconds, for a query execution before it is canceled. if you do not want a timeout, enter 0.

Example: 0

propertyList Body Array of String

An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.", "example":

Example: ["name": "ApplicationName", "value": "MarketingDashboard"]

Examples of Different Configurations for authenticationType

No authentication required

{
  "authenticationType": "ANONYMOUS"
}

Master credentials

{
  "authenticationType": "MASTER",
  "username": "myUsername",
  "password": "myPassword"
}

Secret resource URL

{
  "authenticationType": "MASTER",
  "username": "myUsername",
  "secretResourceUrl": "arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy"
}

AWS Profile

{
  "authenticationType": "AWS_PROFILE",
  "awsProfile": "profileName",
  "dbUser": "userName"
}

Amazon S3

Dremio source type: S3. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for S3, see Adding an S3 Source for additional information.

Config attributes for connecting to an Amazon S3 source

"config": {
  "credentialType": [
    "PROJECT_DATA_CRED",
    "ACCESS_KEY"
  ]
  "assumedRoleARN": "arn:aws:iam::123456789012:root",
  "awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
  "awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
  "externalBucketList": /bucket-name/,
  "secure": false,
  "enableAsync": true,
  "compatibilityMode": false,
  "requesterPays": false,
  "enableFileStatusCheck": true,
  "isPartitionInferenceEnabled": false,
  "rootPath": "/",
  "kmsKeyARN": "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab",
  "defaultCtasFormat": "ICEBERG",
  "propertyList": [
    {
      "name": "fs.s3a.proxy.host", 
      "value": "proxyHost.example.com"
    },
  ],
  "whitelistedBuckets": [
    "sales"
  ],
  "isCachingEnabled": true,
  "maxCacheSpacePct": 100
}

Parameters

credentialType Body String

Specifies the type of credential to enable Dremio to connect to the source. Must be either ACCESS_KEY or PROJECT_DATA_CRED.

Enum: ACCESS_KEY, PROJECT_DATA_CRED

assumedRoleARN Body String Optional

The ARN of the IAM role to assume.

Example: arn:aws:iam::123456789012:root

awsAccessKey Body String

The AWS access key.

Example: AKIAIOSFODNN7EXAMPLE

awsAccessSecret Body String

The AWS access secret.

Example: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

externalBucketList Body Array of String

Connects Dremio to one or more Amazon S3 buckets. Read Accessing a bucket using S3 for more information.

Example: ["bucket-name"]

secure Body Boolean

Option to enable a secure connection via SSL encryption between the S3 buckets and Dremio. Set to true to enable SSL encryption.

enableAsync Body Boolean

Option to enable asynchronous access, which allows cloud caching for the source to support simultaneous actions such as adding and editing. Set to true to enable asynchronous access. Read Amazon S3 > Advanced Options for more information.

compatibilityMode Body Boolean

Option to enable compatibility mode for an Amazon S3 source, which allows you to use S3-compatible storage. Set to true to enable compatibility mode.

requesterPays Body Boolean

Option to apply S3 requests to the requestor. Set to true enable this option.

enableFileStatusCheck Body Boolean

Option to enable file status check. Set to true to enable file status check.

isPartitionInferenceEnabled Body Boolean

Option to enable partition column inference. Set to true to enable partition column inference.

rootPath Body String

The root path of the S3 bucket.

Example: /

kmsKeyARN Body String

Enables you to add a server side encryption key ARN that was created in AWS Key Management Service (KMS).

Example: arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab

defaultCtasFormat Body String

Enables you to choose the default format for tables you create in Dremio, either ICEBERG or PARQUET.

Enum: ICEBERG, PARQUET

propertyList Body Array of Object

An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.

Example: {"name": "fs.s3a.proxy.host", "value": "proxyHost.example.com"}

whitelistedBuckets Body Array of String

Add one or more approved S3 buckets.

Example: ["sales"]

isCachingEnabled Body Boolean

Option to enable local caching. When local caching is possible, set this parameter to true to enable it. Read Amazon S3 > Advanced Options for more information.

maxCacheSpacePct Body Integer

Sets the maximum percent of total available cache space to use when possible. The value, in percent, must be between 1 and 100.

Example: 100

Examples of Different Configurations for credentialType

Project data credentials

{
  "credentialType": "PROJECT_DATA_CRED"
}

Data source credentials (IAM role)

{
  "credentialType": "PROJECT_DATA_CRED",
  "assumedRoleARN": "arn:aws:iam::123456789012:root"
}

Data source credentials (access key)

{
  "credentialType": "ACCESS_KEY",
  "assumedRoleARN": "arn:aws:iam::123456789012:root",
  "awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
  "awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}

Arctic

Dremio source type: ARCTIC. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Arctic, see Dremio Sonar for additional information.

Config attributes for configuring an Arctic source with AWS storage

"config": {
  "awsRootPath": "/my.s3bucket.com/path",
  "propertyList": [
    {
      "name": "fs.s3a.proxy.host",
      "value": "proxyHost.example.com"
    }
  ],
  "asyncEnabled": true,
  "isCachingEnabled": true,
  "maxCacheSpacePct": 100,
  "defaultCtasFormat": "ICEBERG",
  "storageProvider": "AWS",
  "credentialType": "PROJECT_DATA_CRED",
  "assumedRoleARN": "arn:aws:iam::123456789012:root",
  "arcticCatalogId": "12c7598e-8055-4326-afde-873f3ad942d0"
}

Config attributes for configuring an Arctic source with Azure storage

"config": {
  "propertyList": [
    {
      "name": "propertyName",
      "value": "propertyValue"
    }
  ],
  "asyncEnabled": true,
  "isCachingEnabled": true,
  "maxCacheSpacePct": 100,
  "defaultCtasFormat": "ICEBERG",
  "storageProvider": "AZURE",
  "azureStorageAccount": "teamStorage",
  "azureRootPath": "newCatalogStorage/folder",
  "azureAuthenticationType": "ACCESS_KEY",
  "azureAccessKey": "8iWvLxAFwBkFQuzkKEU8iPEdX%Zrm&ChyZfXUBMQQ*r$gCnidpRsAwPe^AjcieN!V9sJu/cefsSrGfEXAMPLE==$",
  "arcticCatalogId": "5bc2f423-d46c-4b1f-a029-ca8148bc196e"
}

Parameters

awsAccessKey Body String

The AWS access key for the Amazon S3 bucket to use for storage. Include only if you specify AWS as the storageProvider value and ACCESS_KEY as the credentialType value.

Example: AKIAIOSFODNN7EXAMPLE

awsRootPath Body String

The root path of the Amazon S3 bucket to use for storage. Include only if you specify AWS as the storageProvider.

Example: /your-s3-bucket-name

propertyList Body Array of Object Optional

Connection properties for the source. Each object in propertyList includes the name of the property and the corresponding value to use for the property.

Example: [{"name": "fs.s3a.proxy.host","value": "proxyHost.example.com"}]

asyncEnabled Body Boolean

Option to enable asynchronous access, when this functionality is available in your Arctic source. This option enables cloud caching for the source to support simultaneous actions such as adding and editing. Set to true to enable asynchronous access.

isCachingEnabled Body Boolean

Option to enable local caching, when this functionality is available in your Arctic source. Set to true to enable local caching.

maxCacheSpacePct Body Integer

Sets the maximum percent of total available cache space to use when possible. The value, in percent, must be between 1 and 100.

Example: 100

defaultCtasFormat Body String

Enables you to choose the default format for tables you create in Dremio, either ICEBERG or PARQUET.

Enum: ICEBERG, PARQUET

storageProvider Body String

Specifies the storage provider to use for the Arctic catalog source.

Enum: AWS, AZURE

azureStorageAccount Body String

The name of the Azure Storage account to use for the Arctic catalog source. Include only if you specify AZURE as the storageProvider value.

Example: teamStorage

azureRootPath Body String

The root path inside your Azure Storage account. The root path is the name of the Azure Storage container, including any folders. Include only if you specify AZURE as the storageProvider value.

Example: newCatalogStorage/folder

azureAuthenticationType Body String

The method to use for authenticating to your Azure Storage account. Include only if you specify AZURE as the storageProvider value.

Enum: ACCESS_KEY, AZURE_ACTIVE_DIRECTORY

azureAccessKey Body String

The shared access key for your Azure Storage account. Include only if you specify AZURE as the storageProvider value and ACCESS_KEY as the azureAuthenticationType value.

Example: 8iWvLxAFwBkFQuzkKEU8iPEdX%Zrm&ChyZfXUBMQQ*r$gCnidpRsAwPe^AjcieN!V9sJu/cefsSrGfEXAMPLE==

azureApplicationId Body String

The application (client) ID for your Azure Storage account. Include only if you specify AZURE as the storageProvider value and AZURE_ACTIVE_DIRECTORY as the azureAuthenticationType value.

Example: ac2e0fbf-2507-4a32-983e-9550eb5b3191

azureClientSecret Body String

The secret generated for the application in your Azure Storage account. Include only if you specify AZURE as the storageProvider value and AZURE_ACTIVE_DIRECTORY as the azureAuthenticationType value.

Example: cb^tB3dDor.YtxywDuSvxgdRHEXAMPLE

azureOAuthTokenEndpoint Body String

The OAuth 2.0 token endpoint (v1.0), including the tenant ID, that the application in your Azure Storage account uses to get an access token or a refresh token. Include only if you specify AZURE as the storageProvider value and AZURE_ACTIVE_DIRECTORY as the azureAuthenticationType value.

Example: https://login.microsoftonline.com/01ca1052-ab53-4fb6-8b16-0b6f3dc73cc6/oauth2/token

credentialType Body String

Specifies the type of credential to use to enable Dremio to connect to the Arctic catalog source. Include only if you specify AWS as the storageProvider value.

Enum: ACCESS_KEY, PROJECT_DATA_CRED

assumedRoleARN Body String Optional

The Amazon Resource Name (ARN) of the IAM role to assume for the Arctic catalog source. Include only if you specify AWS as the storageProvider value.

Example: arn:aws:iam::123456789012:root

arcticCatalogId Body String

The unique ID of the Arctic catalog you are accessing.

Example: ab12cdef-a123-b456-ab1234-cdef5678ghij

Examples of Different Configurations for credentialType and azureAuthenticationType

Authentication parameters to include for credentialType `PROJECT_DATA_CRED` and storageProvider `AWS`

{
  "credentialType": "PROJECT_DATA_CRED",
  "assumedRoleARN": "arn:aws:iam::123456789012:root"
}

Authentication parameters to include for credentialType `ACCESS_KEY` and storageProvider `AWS`

{
  "credentialType": "ACCESS_KEY",
  "assumedRoleARN": "arn:aws:iam::123456789012:root",
  "awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
  "awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}

Authentication parameters to include for credentialType `ACCESS_KEY` and storageProvider `AZURE`

{
  "azureAuthenticationType": "ACCESS_KEY",
  "azureAccessKey": "8iWvLxAFwBkFQuzkKEU8iPEdX%Zrm&ChyZfXUBMQQ*r$gCnidpRsAwPe^AjcieN!V9sJu/cefsSrGfEXAMPLE=="
}

Authentication parameters to include for credentialType `ACCESS_KEY` and storageProvider `AZURE`

{
  "azureAuthenticationType": "AZURE_ACTIVE_DIRECTORY",
  "azureApplicationId": "ac2e0fbf-2507-4a32-983e-9550eb5b3191",
  "azureClientSecret": "cb^tB3dDor.YtxywDuSvxgdRHEXAMPLE",
  "azureOAuthTokenEndpoint": "https://login.microsoftonline.com/01ca1052-ab53-4fb6-8b16-0b6f3dc73cc6/oauth2/token"
}

AWS Glue Data Catalog

Dremio source type: AWSGLUE. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for AWS Glue, see Adding an AWS Glue Data Catalog for additional information.

Config attributes for connecting to an AWS Glue Data Catalog

"config": {
  "credentialType": [
    "PROJECT_DATA_CRED",
    "ACCESS_KEY"
  ]
  "assumedRoleARN": "arn:aws:iam::123456789012:root",
  "awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
  "awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
  "regionNameSelection": "us-east-1",
  "secure": false,
  "enableAsync": true,
  "propertyList": [
   {
      "name": "hive.metastore.warehouse.dir", 
      "value": "s3://bucket-name/folder"
    },
  ],
  "allowedDatabases": ["sales","finance"],
  "isCachingEnabled": true,
  "maxCacheSpacePct": 100
}

Parameters

credentialType Body String

Specifies the type of credential to enable Dremio to connect to the source. Must be either ACCESS_KEY or PROJECT_DATA_CRED.

Enum: ACCESS_KEY, PROJECT_DATA_CRED

assumedRoleARN Body String Optional

The ARN of the IAM role to assume.

Example: arn:aws:iam::123456789012:root

awsAccessKey Body String

The AWS access key.

Example: AKIAIOSFODNN7EXAMPLE

awsAccessSecret Body String

The AWS access secret.

Example: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

regionNameSelection Body String

Sets the AWS Region where the Glue Data Catalog is located.

Example: us-east-1

secure Body Boolean

Option to enable a secure connection via SSL encryption between the Glue Data Catalog and Dremio. Set to true to enable SSL encryption.

enableAsync Body Boolean

propertyList Body Array of Object

An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.

Example: [{"name": "hive.metastore.warehouse.dir", "value": "s3://bucket-name/folder"}]

allowedDatabases Body Array of String

A list of the databases that users of Dremio are allowed to access. Present the names in a comma-separated list.

If one or more databases are added or removed from the list, the change takes effect immediately.

Example: ["sales","finance”]

isCachingEnabled Body Boolean

Option to enable local caching. When local caching is possible, set this parameter to true to enable it. Read AWS Glue Data Catalog > Advanced Options for more information.

maxCacheSpacePct Body Integer

Sets the maximum percent of total available cache space to use when possible. The value, in percent, must be between 1 and 100.

Example: 100

Examples of Different Configurations for credentialType

Project data credentials

{
  "credentialType": "PROJECT_DATA_CRED"
}

Data source credentials (IAM role)

{
  "credentialType": "PROJECT_DATA_CRED",
  "assumedRoleARN": "arn:aws:iam::123456789012:root"
}

Data source credentials (access key)

{
  "credentialType": "ACCESS_KEY",
  "assumedRoleARN": "arn:aws:iam::123456789012:root",
  "awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
  "awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}

Azure Storage

Dremio source type: AZURE_STORAGE. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Azure Storage, see Adding an Azure Storage Source for additional information.

Config attributes for connecting to an Azure Storage source

"config": {
  "accountKind": "STORAGE_V2",
  "accountName": "testazure",
  "accessKey": "l3kertpoifkegoivns93c/9AEUa+hie8XEF039/oiWrgbx2g7e+T3bk9H/4R7vua7ljA2pwDKscPZTEXAMPLE==",
  "rootPath": "/",
  "propertyList": [
   {
      "name": "dremio.azure.mode", 
      "value": "STORAGE_V2"
    },
  ],
  "enableSSL": true,
  "enableAsync": true,
  "credentialsType": "ACCESS_KEY",
  "isCachingEnabled": true,
  "maxCacheSpacePct": 100,
  "defaultCtasFormat": "ICEBERG",
  "isPartitionInferenceEnabled": false
}

Parameters

accountKind Body String

Type of the Azure storage account.

Enum: STORAGE_V1, STORAGE_V2

Example: STORAGE_V2

accountName Body String

Name of the Azure storage account.

Example: testazure

credentialsType Body String

Specifies the type of authentication to use to enable Dremio to connect to the source.

Enum: ACCESS_KEY, AZURE_ACTIVE_DIRECTORY

accessKey Body String

If using the ACCESS_KEY credentialsType, the Azure access key for the Azure Storage account. To keep the access key secure, Dremio returns the accessKey value as $DREMIO_EXISTING_VALUE$ in API responses.

Example: l3kertpoifkegoivns93c/9AEUa+hie8XEF039/oiWrgbx2g7e+T3bk9H/4R7vua7ljA2pwDKscPZTEXAMPLE==

clientId Body String

If using the AZURE_ACTIVE_DIRECTORY credentialsType, the client ID of the registered app in Azure.

Example: ba72f3e5-edc6-4aec-ad0a-c1147e81dc8d

tokenEndpoint Body String

If using the AZURE_ACTIVE_DIRECTORY credentialsType, the OAuth 2.0 token endpoint for the registered app in Azure.

Example: https://login.microsoftonline.com/2dfe8f9a-c80b-40c9-81de-7713ca393904/oauth2/token

clientSecret Body String

If using the AZURE_ACTIVE_DIRECTORY credentialsType, the client secret for the registered app in Azure. To keep the client secret secure, Dremio returns the accessKey value as $DREMIO_EXISTING_VALUE$ in API responses.

Example: pbt+a]xWAk2E*2.WmjU0wtfiLEXAMPLE

rootPath Body String Optional

Root path of the Azure Storage source. Default is /

Example: /

propertyList Body Array of Object Optional

An array of name/value pairs defining optional connection properties. Use a comma-separated list to specify multiple name/value pairs.

Example: [{"name": "dremio.azure.mode", "value": "STORAGE_V2"}]

enableSSL Body Boolean Optional

To enable a secure connection with SSL encryption between the Azure Storage source and Dremio, set to true (default). Otherwise, set to false.

Example: true

enableAsync Body Boolean Optional

To enable asynchronous access for the source and allow cloud caching to support simultaneous actions such as adding and editing, set to true (default). Otherwise, set to false.

Example: true

isCachingEnabled Body Boolean Optional

To enable local caching, set to true (default). Otherwise, set to false.

Example: true

maxCacheSpacePct Body Integer Optional

Maximum percentage of the total available cache space to use on any single executor node. Used only when isCachingEnabled is set to true. Minimum is 1. Maximum is 100 (default).

Example: 100

defaultCtasFormat Body String Optional

Default format for the tables you create in Dremio. Default is ICEBERG.

Enum: ICEBERG,PARQUET

Example: ICEBERG

isPartitionInferenceEnabled Body Boolean Optional

To enable partition column inference, set to true. Otherwise, set to false (default).

Example: false

Examples of Different Configurations for credentialsType

Azure Active Directory

{
  "credentialsType": "AZURE_ACTIVE_DIRECTORY",
  "clientId": "ba72f3e5-edc6-4aec-ad0a-c1147e81dc8d",
  "tokenEndpoint": "https://login.microsoftonline.com/2dfe8f9a-c80b-40c9-81de-7713ca393904/oauth2/token",
  "clientSecret": "pbt+a]xWAk2E*2.WmjU0wtfiLEXAMPLE"
}

Microsoft SQL Server

Dremio source type: MSSQL. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for SQL Server, see Configuring SQL Server as a Source for additional information.

Config attributes for connecting to a Microsoft SQL Server source

"config": {
  "authenticationType": [
    "ANONYMOUS",
    "MASTER"
  ]
  "username": "myUsername",
  "password": "myPassword".
  "hostname": "localhost",
  "port": "1433",
  "database": "my_database",
  "useSsl": true,
  "showOnlyConnectionDatabase": false,
  "fetchSize": 200,
  "maxIdleConns": 8,
  "idleTimeSec": 60,
  "enableServerVerification": true,
  "hostnameOverride": "localhost",
  "propertyList": [
    {
      "name": "applicationIntent", 
      "value": "ReadWrite"
    }
  ]
}

Parameters

authenticationType Body String

Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed and MASTER when credentials from a master database user is required.

Enum: ANONYMOUS, MASTER

username Body String

The SQL Server user name.

Example: myUsername

password Body String

The SQL Server password.

Example: myPassword

hostname Body String

The SQL Server host name.

Example: localhost

port Body String

The SQL Server port number. The coomonly used default port numbers for SQL Server are 1433 and 1434.

Example: 1433

database Body String Optional

The database name.

Example: my_database

useSsl Body Boolean

Option to enable SSL encryption. Set to true to enable this option.

showOnlyConnectionDatabase Body Boolean

Option to show only the initial database used for connecting. Set to true to enable this option.

fetchSize Body String

The number of records to fetch per request. Set to 0 to enable Dremio to automatically set the size.

Example: 200

maxIdleConns Body Integer

Sets the maximum number of idle connections that you want to keep.

Example: 8

idleTimeSec Body Integer

Sets the idle time, in seconds, before a connection is evaluated for closure.

Example: 60

enableServerVerification Body Boolean

Option to verify the server certificate. Set to true to enable this option.

hostnameOverride Body String

This parameter can be used when the SSL/TLS server certificate contains a distinguished name. You can leave this blank if the hostname in the certificate matches the server name.

Example: localhost

propertyList Body Array of Object

An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.

Example: [{"name": "applicationIntent", "value": "ReadWrite"}]

Examples of Different Configurations for authenticationType

No authentication required

{
  "authenticationType": "ANONYMOUS"
}

Master credentials

{
  "authenticationType": "MASTER",
  "username": "myUsername",
  "password": "myPassword"
}

MySQL

Dremio source type: MYSQL. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for MySQL, see Configuring MySQL as a Source for additional information.

Config attributes for connecting to a MySQL source

"config": {
  "authenticationType": [
    "ANONYMOUS",
    "MASTER",
  ]
  "username": "myUsername",
  "password": "myPassword"
  "hostname": "localhost",
  "port": "3306",
  "netWriteTimeout": 60,
  "fetchSize": 200,
  "maxIdleConns": 8,
  "idleTimeSec": 60,
  "propertyList": [
    {
      "name": "DataSource", 
      "value": "sales2023"
    }
  ]
}

Parameters

authenticationType Body String

Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed and MASTER when credentials from a master database user is required.

Enum: ANONYMOUS, MASTER

username Body String

The MySQL user name.

Example: myUsername

password Body String

The MySQL password.

Example: myPassword

hostname Body String

The MySQL host name.

Example: localhost

port Body String

The MySQL port number. The coomonly used default port number for MySQL is 3306.

Example: 3306

netWriteTimeout Body Integer

The amount of time, in seconds, to wait for data from the server before aborting the connection.

Example: 60

fetchSize Body Integer

The number of records to fetch per request. Set to 0 to enable Dremio to automatically set the size.

Example: 200

maxIdleConns Body Integer

Sets the maximum number of idle connections that you want to keep.

Example: 8

idleTimeSec Body Integer

Sets the idle time, in seconds, before a connection is evaluated for closure.

Example: 60

propertyList Body Array of Object

An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.

Example: [{"name": "DataSource", "value": "sales2023"}]

Examples of Different Configurations for authenticationType

No authentication required

{
  "authenticationType": "ANONYMOUS"
}

Master credentials

{
  "authenticationType": "MASTER",
  "username": "myUsername",
  "password": "myPassword"
}

Oracle

Dremio source type: ORACLE. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Oracle, see Configuring Oracle as a Source for additional information.

Config attributes for connecting to an Oracle source

"config": {
  "authenticationType": [
    "ANONYMOUS",
    "MASTER"
  ]
  "username": "myUsername",
  "password": "myPassword",
  "hostname": "localhost",
  "port": "1521",
  "instance": "sales",
  "useSsl": true,
  "nativeEncryption": "ACCEPTED", 
  "useTimezoneAsRegion": true,
  "includeSynonyms": false,
  "mapDateToTimestamp": true,
  "fetchSize": 200,
  "maxIdleConns": 8,
  "idleTimeSec": 60,
  "useLdap": false,
  "bindDN": String,
  "sslServerCertDN": String,
  "propertyList": [
    {
      "name": "connect_timeout", 
      "value": "0"
    }
  ]
}

Parameters

authenticationType Body String

Enum: ANONYMOUS, MASTER, KERBEROS

username Body String

The Oracle user name.

Example: myUsername

password Body String

The Oracle password.

Example: myPassword

secretResourceUrl Body String

For secret-based authentication, the Amazon Resource Name (ARN) for the secret. To keep the secret secure, Dremio returns the secretResourceUrl value as $DREMIO_EXISTING_VALUE$ in API responses.

Example: arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy

useKerberos Body Boolean

Set to true if you are using Kerberos.

hostname Body String

The Oracle host name.

Example: localhost

port Body String

The Oracle port number. The commonly used default port numbers for Oracle are 1521 and 1830.

Example: 1521

instance Body String Optional

The Oracle service name.

Example: sales

useSsl Body Boolean

Option to enable SSL encryption. Set to true to enable this option.

nativeEncryption Body Boolean

Sets the Oracle native encryption. This parameter can only be used if useSsl is set to false.

Enum: REJECTED, ACCEPTED, REQUESTED, REQUIRED

useTimezoneAsRegion Body Boolean

Option to use the timezone to set the connection region. Set to true to enable this option.

includeSynonyms Body Boolean

Option to include synonyms as datasets. Set to true to enable this option.

mapDateToTimestamp Body Boolean

Option to display Oracle DATE columns as a TIMESTAMP. Set to true to enable this option.

fetchSize Body Integer

The number of records to fetch per request. Set to 0 to enable Dremio to automatically set the size.

Example: 200

maxIdleConns Body Integer

Sets the maximum number of idle connections that you want to keep.

Example: 8

idleTimeSec Body Integer

Sets the idle time, in seconds, before a connection is evaluated for closure.

Example: 60

useLdap Body Boolean

Option to use LDAP naming services. Set to true to enable this option.

bindDN Body String

Sets the LDAP hostname resolution when the useLdap parameter is set to true. You can omit this parameter if useLdap is set to false.

sslServerCertDN Body String

Use when you have a SSL/TLS server certificate distinguished name. Otherwise, you can leave this parameter blank to disable DN match.

propertyList Body Array of Object

An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.

Example: [{"name": "connect_timeout", "value": "0"}]

Examples of Different Configurations for authenticationType

No authentication required

{
  "authenticationType": "ANONYMOUS"
}

Master credentials

{
  "authenticationType": "MASTER",
  "username": "myUsername",
  "password": "myPassword"
}

Secret resource URL

{
  "authenticationType": "MASTER",
  "username": "myUsername",
  "secretResourceUrl": "arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy"
}

Using Kerberos

{
  "credentialType": "KERBEROS",
  "username": null,
  "password": null,
  "useKerberos": true
}

PostgreSQL

Dremio source type: POSTGRES. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for PostgreSQL, see Configuring PostgreSQL as a Source for additional information.

Config attributes for connecting to a PostgreSQL source

"config": {
  "authenticationType": [
    "ANONYMOUS",
    "MASTER"
  ]
  "username": "myUsername",
  "password": "myPassword",
  "hostname": "localhost",
  "port": "5432",
  "databaseName": "sales_2023",
  "useSsl": false,
  "fetchSize": 200,
  "maxIdleConns": 8,
  "idleTimeSec": 60,
  "encryptionValidationMode": "CERTIFICATE_AND_HOSTNAME_VALIDATION",
  "propertyList": [
    {
      "name": "connect_timeout", 
      "value": "10&application_name=myapp"
    }
  ]
}

Parameters

authenticationType Body String

Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed and MASTER when either credentials from a master database user or a secret resource URL is required.

Enum: ANONYMOUS, MASTER

username Body String

The PostgreSQL user name.

Example: myUsername

password Body String

The PostgreSQL password.

Example: myPassword

secretResourceUrl Body String

For secret-based authentication, the Amazon Resource Name (ARN) for the secret. To keep the secret secure, Dremio returns the secretResourceUrl value as $DREMIO_EXISTING_VALUE$ in API responses.

Example: arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy

hostname Body String

The PostgreSQL host name.

Example: localhost

port Body String

The PostgreSQL port number. The coomonly used default port number for PostgreSQL is 5432.

Example: 5432

databaseName Body String Optional

The database name.

Example: sales_2023

useSsl Body Boolean

Option to enable SSL encryption. Set to true to enable this option.

fetchSize Body Integer

The number of records to fetch per request. Set to 0 to enable Dremio to automatically set the size.

Example: 200

maxIdleConns Body Integer

Sets the maximum number of idle connections that you want to keep.

Example: 8

idleTimeSec Body Integer

Sets the idle time, in seconds, before a connection is evaluated for closure.

Example: 60

encryptionValidationMode Body String

When encryption is enabled, you can set the validation mode to one of the available presets.

Enum: CERTIFICATE_AND_HOSTNAME_VALIDATION, CERTIFICATE_ONLY_VALIDATION, NO_VALIDATION

propertyList Body Array of Object

An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.

Example: [{"name": "connect_timeout", "value": "10&application_name=myapp"}]

Examples of Different Configurations for authenticationType

No authentication required

{
"authenticationType": "ANONYMOUS"
}

Master credentials

{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}

Snowflake

Dremio source type: SNOWFLAKE. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Snowflake, see Configuring Snowflake as a Source for additional information.

Config attributes for connecting to a Snowflake source

"config": {
  "hostname": "https://locator_id.snowflakecomputing.com",
  "username": "myUsername",
  "password": "myPassword",
  [ "database": "myDatabase" ],
  [ "role": "PUBLIC" ],
  [ "schema": "mySchema" ],
  [ "warehouse": "myWarehouse" ]  
}

Parameters

hostname Body String

The Snowflake host name.

Example: https://locator_id.snowflakecomputing.com

username Body String

The Snowflake user name.

Example: myUsername

password Body String

The Snowflake password.

Example: myPassword

database Body String Optional

Specifies the default database to use.

Example: myDatabase

role Body String Optional

Specifies the default access-control role to use. Read System-Defined Roles for more information.

schema Body String Optional

Specifies the default schema to use.

Example: mySchema

warehouse Body String Optional

Specifies the warehouse that will provide resources for executing DML statements and queries.

Example: myWarehouse

Source Configuration

Amazon Redshift​

Parameters​

Examples of Different Configurations for authenticationType​

Amazon S3​

Parameters​

Examples of Different Configurations for credentialType​

Arctic​

Parameters​

Examples of Different Configurations for credentialType and azureAuthenticationType​

AWS Glue Data Catalog​

Parameters​

Examples of Different Configurations for credentialType​

Azure Storage​

Parameters​

Examples of Different Configurations for credentialsType​

Microsoft SQL Server​

Parameters​

Examples of Different Configurations for authenticationType​

MySQL​

Parameters​

Examples of Different Configurations for authenticationType​

Oracle​

Parameters​

Examples of Different Configurations for authenticationType​

PostgreSQL​

Parameters​

Examples of Different Configurations for authenticationType​

Snowflake​

Parameters​

Amazon Redshift

Parameters

Examples of Different Configurations for authenticationType

Amazon S3

Parameters

Examples of Different Configurations for credentialType

Arctic

Parameters

Examples of Different Configurations for credentialType and azureAuthenticationType

AWS Glue Data Catalog

Parameters

Examples of Different Configurations for credentialType

Azure Storage

Parameters

Examples of Different Configurations for credentialsType

Microsoft SQL Server

Parameters

Examples of Different Configurations for authenticationType

MySQL

Parameters

Examples of Different Configurations for authenticationType

Oracle

Parameters

Examples of Different Configurations for authenticationType

PostgreSQL

Parameters

Examples of Different Configurations for authenticationType

Snowflake

Parameters