Source Configuration
This page details the config payload to create each source type.
Amazon Redshift
Dremio source type: REDSHIFT. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Redshift, see Adding a Redshift Cluster as a Data Source for additional information.
"config": {
"connectionString": "jdbc:redshift://redshift.example.com:5439/database",
"authenticationType": [
"ANONYMOUS",
"MASTER",
"AWS_PROFILE"
],
"username": "myUsername",
"password": "myPassword",
"secretResourceUrl": arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy,
"awsProfile": "profileName",
"dbUser": "userName",
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"queryTimeoutSec": 0,
"propertyList": [
{
"name": "ApplicationName",
"value": "MarketingDashboard"
}
]
}
Parameters
connectionString Body String
The JDBC connection string. Read JDBC for more information.
Example: jdbc:redshift://redshift.example.com:5439/database
authenticationType Body String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed, MASTER when either credentials from a master database user or a secret resource URL is required, or AWS_PROFILE for authentication to an AWS source.
Enum: ANONYMOUS, AWS_PROFILE, MASTER
username Body String
The Amazon Redshift user name.
Example: myUsername
password Body String
The Amazon Redshift password.
Example: myPassword
secretResourceUrl Body String
For secret-based authentication, the Amazon Resource Name (ARN) for the secret. To keep the secret secure, Dremio returns the secretResourceUrl value as $DREMIO_EXISTING_VALUE$ in API responses.
Example: arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy
awsProfile Body String
The AWS profile name. If this is left blank, the default profile will be used.
Example: profileName
dbUser Body String
The name of the Amazon Redshift DbUser that will be used for authentication. If this is left blank, the default user name for your AWS IAM role will be used.
Example: userName
fetchSize Body Integer
The number of records to fetch per request. Set to 0 to enable Dremio to automatically set the size.
Example: 200
maxIdleConns Body Integer
Sets the maximum number of idle connections that you want to keep.
Example: 8
idleTimeSec Body Integer
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example: 60
queryTimeoutSec Body Integer
Sets the timeout, in seconds, for a query execution before it is canceled. if you do not want a timeout, enter 0.
Example: 0
propertyList Body Array of String
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: ["name": "ApplicationName", "value": "MarketingDashboard"]
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"secretResourceUrl": "arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy"
}
{
"authenticationType": "AWS_PROFILE",
"awsProfile": "profileName",
"dbUser": "userName"
}
Amazon S3
Dremio source type: S3. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for S3, see Adding an S3 Source for additional information.
"config": {
"credentialType": [
"PROJECT_DATA_CRED",
"ACCESS_KEY"
]
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"externalBucketList": /bucket-name/,
"secure": false,
"enableAsync": true,
"compatibilityMode": false,
"requesterPays": false,
"enableFileStatusCheck": true,
"isPartitionInferenceEnabled": false,
"rootPath": "/",
"kmsKeyARN": "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab",
"defaultCtasFormat": "ICEBERG",
"propertyList": [
{
"name": "fs.s3a.proxy.host",
"value": "proxyHost.example.com"
},
],
"whitelistedBuckets": [
"sales"
],
"isCachingEnabled": true,
"maxCacheSpacePct": 100
}
Parameters
credentialType Body String
Specifies the type of credential to enable Dremio to connect to the source. Must be either ACCESS_KEY or PROJECT_DATA_CRED.
Enum: ACCESS_KEY, PROJECT_DATA_CRED
assumedRoleARN Body String Optional
The ARN of the IAM role to assume.
Example: arn:aws:iam::123456789012:root
awsAccessKey Body String
The AWS access key.
Example: AKIAIOSFODNN7EXAMPLE
awsAccessSecret Body String
The AWS access secret.
Example: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
externalBucketList Body Array of String
Connects Dremio to one or more Amazon S3 buckets. Read Accessing a bucket using S3 for more information.
Example: ["bucket-name"]
secure Body Boolean
Option to enable a secure connection via SSL encryption between the S3 buckets and Dremio. Set to true to enable SSL encryption.
enableAsync Body Boolean
Option to enable asynchronous access, which allows cloud caching for the source to support simultaneous actions such as adding and editing. Set to true to enable asynchronous access. Read Amazon S3 > Advanced Options for more information.
compatibilityMode Body Boolean
Option to enable compatibility mode for an Amazon S3 source, which allows you to use S3-compatible storage. Set to true to enable compatibility mode.
requesterPays Body Boolean
Option to apply S3 requests to the requestor. Set to true enable this option.
enableFileStatusCheck Body Boolean
Option to enable file status check. Set to true to enable file status check.
isPartitionInferenceEnabled Body Boolean
Option to enable partition column inference. Set to true to enable partition column inference.
rootPath Body String
The root path of the S3 bucket.
Example: /
kmsKeyARN Body String
Enables you to add a server side encryption key ARN that was created in AWS Key Management Service (KMS).
Example: arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
defaultCtasFormat Body String
Enables you to choose the default format for tables you create in Dremio, either ICEBERG or PARQUET.
Enum: ICEBERG, PARQUET
propertyList Body Array of Object
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: {"name": "fs.s3a.proxy.host", "value": "proxyHost.example.com"}
whitelistedBuckets Body Array of String
List of approved S3 buckets to include in the source. Buckets entered must be valid. Misspelled or non-existent buckets will not appear in the resulting source. Omit to include all buckets that are available in the source. If you omit the whitelistedBuckets array in a PUT request, Dremio updates the source to include all available buckets. To keep existing approved buckets while making other updates, duplicate the existing whitelistedBuckets array in the PUT request.
Example: ["archive.dremio.com","logs_east-1","logs_west-1"]
isCachingEnabled Body Boolean
Option to enable local caching. When local caching is possible, set this parameter to true to enable it. Read Amazon S3 > Advanced Options for more information.
maxCacheSpacePct Body Integer
Sets the maximum percent of total available cache space to use when possible. The value, in percent, must be between 1 and 100.
Example: 100
Examples of Different Configurations for credentialType
Project data credentials{
"credentialType": "PROJECT_DATA_CRED"
}
{
"credentialType": "PROJECT_DATA_CRED",
"assumedRoleARN": "arn:aws:iam::123456789012:root"
}
{
"credentialType": "ACCESS_KEY",
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}
Arctic
Dremio source type: ARCTIC. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Arctic, see Dremio Sonar for additional information.
"config": {
"awsRootPath": "/my.s3bucket.com/path",
"propertyList": [
{
"name": "fs.s3a.proxy.host",
"value": "proxyHost.example.com"
}
],
"asyncEnabled": true,
"isCachingEnabled": true,
"maxCacheSpacePct": 100,
"defaultCtasFormat": "ICEBERG",
"storageProvider": "AWS",
"credentialType": "PROJECT_DATA_CRED",
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"arcticCatalogId": "12c7598e-8055-4326-afde-873f3ad942d0"
}
"config": {
"propertyList": [
{
"name": "propertyName",
"value": "propertyValue"
}
],
"asyncEnabled": true,
"isCachingEnabled": true,
"maxCacheSpacePct": 100,
"defaultCtasFormat": "ICEBERG",
"storageProvider": "AZURE",
"azureStorageAccount": "teamStorage",
"azureRootPath": "newCatalogStorage/folder",
"azureAuthenticationType": "ACCESS_KEY",
"azureAccessKey": "8iWvLxAFwBkFQuzkKEU8iPEdX%Zrm&ChyZfXUBMQQ*r$gCnidpRsAwPe^AjcieN!V9sJu/cefsSrGfEXAMPLE==$",
"arcticCatalogId": "5bc2f423-d46c-4b1f-a029-ca8148bc196e"
}
Parameters
storageProvider Body String
The storage provider that you are using for your Arctic catalog source.
Enum: AWS, AZURE
credentialType Body String
Type of credential for Dremio to use to connect to AWS for storage of metadata and data files for tables and views. Required if you are using Amazon S3 as the storage provider for your Arctic source.
Enum: ACCESS_KEY, PROJECT_DATA_CRED
awsAccessKey Body String Optional
The AWS access key for the Amazon S3 bucket to use for storage. Required only if you specify AWS as the storageProvider value and ACCESS_KEY as the credentialType value.
Example: AKIAIOSFODNN7EXAMPLE
awsAccessSecret Body String Optional
The AWS access secret for the Amazon S3 bucket to use for storage. Required only if you specify AWS as the storageProvider value and ACCESS_KEY as the credentialType value.
awsRootPath Body String Optional
The Amazon S3 path prefix that Dremio should use as the write location for new Iceberg metadata and data. The root path includes the name of the Amazon S3 bucket, followed by the names of any folders. Required if you are using Amazon S3 as the storage provider for your Arctic catalog source.
Example: /my.s3bucket.com/path
assumedRoleARN Body String Optional
Amazon Resource Name (ARN) for the AWS Identity and Access Management (IAM) role to assume. Include only if you specify AWS as the storageProvider value.
Example: arn:aws:iam::123456789012:root
azureStorageAccount Body String Optional
The name of the Azure Storage account to use for the Arctic catalog source. Include only if you specify AZURE as the storageProvider value.
Example: teamStorage
azureRootPath Body String Optional
The path in your Azure Storage account that Dremio should use as the write location for new Iceberg metadata and data. The root path includes the name of the Azure Storage container, followed by the names of any folders. Include only if you specify AZURE as the storageProvider value.
Example: newCatalogStorage/folder
azureAuthenticationType Body String Optional
The method you want to use to authenticate to Azure Storage: access key (ACCESS_KEY) or Microsoft Entra ID (AZURE_ACTIVE_DIRECTORY). If you choose ACCESS_KEY, you must also include the azureAccessKey parameter in your request. If you choose AZURE_ACTIVE_DIRECTORY, you must also include the azureApplicationId, azureClientSecret, and azureOAuthTokenEndpoint parameters in your request. Include only if you specify AZURE as the storageProvider value.
Enum: ACCESS_KEY, AZURE_ACTIVE_DIRECTORY
azureAccessKey Body String Optional
The shared access key for the Azure Storage account. Include only if you specify AZURE as the storageProvider value and ACCESS_KEY as the azureAuthenticationType value.
Example: abc123DEF456ghi789JKL012mno345PQR678stu901vwx234yzA567bcd890EFG123hij45EXAMPLE=
azureApplicationId Body String Optional
The ID for the application (client) in Azure. Include only if you specify AZURE as the storageProvider value and AZURE_ACTIVE_DIRECTORY as the azureAuthenticationType value.
Example: ac2e0fbf-2507-4a32-983e-9550eb5b3191
azureClientSecret Body String Optional
The Azure client secret. Include only if you specify AZURE as the storageProvider value and AZURE_ACTIVE_DIRECTORY as the azureAuthenticationType value.
Example: cb^tB3dDor.YtxywDuSvxgdRHEXAMPLE
azureOAuthTokenEndpoint Body String Optional
The OAuth 2.0 token endpoint (v1.0), including the tenant ID, that the application uses to get an access token or a refresh token. Include only if you specify AZURE as the storageProvider value and AZURE_ACTIVE_DIRECTORY as the azureAuthenticationType value.
Example: https://login.microsoftonline.com/01ca1052-ab53-4fb6-8b16-0b6f3dc73cc6/oauth2/token
propertyList Body Array of Object Optional
Connection properties for the source. Each object in propertyList includes the name of the property and the corresponding value to use for the property.
Example: [{"name": "fs.s3a.proxy.host","value": "proxyHost.example.com"}]
asyncEnabled Body Boolean
Option to enable asynchronous access, when this functionality is available in your Arctic source. This option enables cloud caching for the source to support simultaneous actions such as adding and editing. Set to true to enable asynchronous access.
isCachingEnabled Body Boolean
Option to enable local caching, when this functionality is available in your Arctic source. Set to true to enable local caching.
maxCacheSpacePct Body Integer
Sets the maximum percent of total available cache space to use when possible. The value, in percent, must be between 1 and 100.
Example: 100
defaultCtasFormat Body String
The default format for tables you create in Dremio. The available value is ICEBERG.
arcticCatalogId Body String
The unique ID of the Arctic catalog you want to use.
Example: ab12cdef-a123-b456-ab1234-cdef5678ghij
Examples of Different Configurations for credentialType and azureAuthenticationType
Authentication parameters to include for credentialTypePROJECT_DATA_CRED and storageProvider AWS
{
"credentialType": "PROJECT_DATA_CRED",
"assumedRoleARN": "arn:aws:iam::123456789012:root"
}
ACCESS_KEY and storageProvider AWS
{
"credentialType": "ACCESS_KEY",
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}
ACCESS_KEY and storageProvider AZURE
{
"azureAuthenticationType": "ACCESS_KEY",
"azureAccessKey": "8iWvLxAFwBkFQuzkKEU8iPEdX%Zrm&ChyZfXUBMQQ*r$gCnidpRsAwPe^AjcieN!V9sJu/cefsSrGfEXAMPLE=="
}
ACCESS_KEY and storageProvider AZURE
{
"azureAuthenticationType": "AZURE_ACTIVE_DIRECTORY",
"azureApplicationId": "ac2e0fbf-2507-4a32-983e-9550eb5b3191",
"azureClientSecret": "cb^tB3dDor.YtxywDuSvxgdRHEXAMPLE",
"azureOAuthTokenEndpoint": "https://login.microsoftonline.com/01ca1052-ab53-4fb6-8b16-0b6f3dc73cc6/oauth2/token"
}
AWS Glue Data Catalog
Dremio source type: AWSGLUE. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for AWS Glue, see Adding an AWS Glue Data Catalog for additional information.
"config": {
"credentialType": [
"PROJECT_DATA_CRED",
"ACCESS_KEY"
]
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"regionNameSelection": "us-east-1",
"secure": false,
"enableAsync": true,
"propertyList": [
{
"name": "hive.metastore.warehouse.dir",
"value": "s3://bucket-name/folder"
},
],
"allowedDatabases": ["sales","finance"],
"isCachingEnabled": true,
"maxCacheSpacePct": 100
}
Parameters
credentialType Body String
Specifies the type of credential to enable Dremio to connect to the source. Must be either ACCESS_KEY or PROJECT_DATA_CRED.
Enum: ACCESS_KEY, PROJECT_DATA_CRED
assumedRoleARN Body String Optional
The ARN of the IAM role to assume.
Example: arn:aws:iam::123456789012:root
awsAccessKey Body String
The AWS access key.
Example: AKIAIOSFODNN7EXAMPLE
awsAccessSecret Body String
The AWS access secret.
Example: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
regionNameSelection Body String
Sets the AWS Region where the Glue Data Catalog is located.
Example: us-east-1
secure Body Boolean
Option to enable a secure connection via SSL encryption between the Glue Data Catalog and Dremio. Set to true to enable SSL encryption.
enableAsync Body Boolean
Option to enable asynchronous access, which allows cloud caching for the source to support simultaneous actions such as adding and editing. Set to true to enable asynchronous access. Read AWS Glue Data Catalog > Advanced Options for more information.
propertyList Body Array of Object
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: [{"name": "hive.metastore.warehouse.dir", "value": "s3://bucket-name/folder"}]
allowedDatabases Body Array of String
A list of the databases that users of Dremio are allowed to access. Present the names in a comma-separated list. Databases entered must be valid. Misspelled or non-existent databases will not appear in the resulting source.
If one or more databases are added or removed from the list, the change takes effect immediately.
Example: ["sales","finance”]
isCachingEnabled Body Boolean
Option to enable local caching. When local caching is possible, set this parameter to true to enable it. Read AWS Glue Data Catalog > Advanced Options for more information.
maxCacheSpacePct Body Integer
Sets the maximum percent of total available cache space to use when possible. The value, in percent, must be between 1 and 100.
Example: 100
Examples of Different Configurations for credentialType
Project data credentials{
"credentialType": "PROJECT_DATA_CRED"
}
{
"credentialType": "PROJECT_DATA_CRED",
"assumedRoleARN": "arn:aws:iam::123456789012:root"
}
{
"credentialType": "ACCESS_KEY",
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}
Azure Storage
Dremio source type: AZURE_STORAGE. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Azure Storage, see Adding an Azure Storage Source for additional information.
"config": {
"accountKind": "STORAGE_V2",
"accountName": "testazure",
"accessKey": "l3kertpoifkegoivns93c/9AEUa+hie8XEF039/oiWrgbx2g7e+T3bk9H/4R7vua7ljA2pwDKscPZTEXAMPLE==",
"rootPath": "/",
"propertyList": [
{
"name": "dremio.azure.mode",
"value": "STORAGE_V2"
},
],
"enableSSL": true,
"enableAsync": true,
"credentialsType": "ACCESS_KEY",
"isCachingEnabled": true,
"maxCacheSpacePct": 100,
"defaultCtasFormat": "ICEBERG",
"isPartitionInferenceEnabled": false
}
Parameters
accountKind Body String
Type of the Azure storage account.
Enum: STORAGE_V1, STORAGE_V2
Example: STORAGE_V2
accountName Body String
Name of the Azure storage account.
Example: testazure
credentialsType Body String
Specifies the type of authentication to use to enable Dremio to connect to the source.
Enum: ACCESS_KEY, AZURE_ACTIVE_DIRECTORY
accessKey Body String
If using the ACCESS_KEY credentialsType, the Azure access key for the Azure Storage account. To keep the access key secure, Dremio returns the accessKey value as $DREMIO_EXISTING_VALUE$ in API responses.
Example: l3kertpoifkegoivns93c/9AEUa+hie8XEF039/oiWrgbx2g7e+T3bk9H/4R7vua7ljA2pwDKscPZTEXAMPLE==
clientId Body String
If using the AZURE_ACTIVE_DIRECTORY credentialsType, the client ID of the registered app in Azure.
Example: ba72f3e5-edc6-4aec-ad0a-c1147e81dc8d
tokenEndpoint Body String
If using the AZURE_ACTIVE_DIRECTORY credentialsType, the OAuth 2.0 token endpoint for the registered app in Azure.
Example: https://login.microsoftonline.com/2dfe8f9a-c80b-40c9-81de-7713ca393904/oauth2/token
clientSecret Body String
If using the AZURE_ACTIVE_DIRECTORY credentialsType, the client secret for the registered app in Azure. To keep the client secret secure, Dremio returns the accessKey value as $DREMIO_EXISTING_VALUE$ in API responses.
Example: pbt+a]xWAk2E*2.WmjU0wtfiLEXAMPLE
rootPath Body String Optional
Root path of the Azure Storage source. Default is /
Example: /
propertyList Body Array of Object Optional
An array of name/value pairs defining optional connection properties. Use a comma-separated list to specify multiple name/value pairs.
Example: [{"name": "dremio.azure.mode", "value": "STORAGE_V2"}]
enableSSL Body Boolean Optional
To enable a secure connection with SSL encryption between the Azure Storage source and Dremio, set to true (default). Otherwise, set to false.
Example: true
enableAsync Body Boolean Optional
To enable asynchronous access for the source and allow cloud caching to support simultaneous actions such as adding and editing, set to true (default). Otherwise, set to false.
Example: true
isCachingEnabled Body Boolean Optional
To enable local caching, set to true (default). Otherwise, set to false.
Example: true
maxCacheSpacePct Body Integer Optional
Maximum percentage of the total available cache space to use on any single executor node. Used only when isCachingEnabled is set to true. Minimum is 1. Maximum is 100 (default).
Example: 100
defaultCtasFormat Body String Optional
Default format for the tables you create in Dremio. Default is ICEBERG.
Enum: ICEBERG,PARQUET
Example: ICEBERG
isPartitionInferenceEnabled Body Boolean Optional
To enable partition column inference, set to true. Otherwise, set to false (default).
Example: false
Examples of Different Configurations for credentialsType
Azure Active Directory{
"credentialsType": "AZURE_ACTIVE_DIRECTORY",
"clientId": "ba72f3e5-edc6-4aec-ad0a-c1147e81dc8d",
"tokenEndpoint": "https://login.microsoftonline.com/2dfe8f9a-c80b-40c9-81de-7713ca393904/oauth2/token",
"clientSecret": "pbt+a]xWAk2E*2.WmjU0wtfiLEXAMPLE"
}
IBM Db2
Config attributes for connecting to an IBM Db2 Source{
"config": {
"database": "tpch",
"hostname": "172.25.1.245",
"username": "exampleuser",
"password": "$DREMIO_EXISTING_VALUE$",
"port": "50000",
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"queryTimeoutSec": 0,
"propertyList": [
{
"name": "deferPrepares",
"value": "true"
}
]
}
}
Parameters
database Body String
IBM Db2 database for Dremio to use.
Example: tpch
hostname Body String
IBM Db2 hostname.
Example: 172.25.1.245
username Body String
Username for authentication.
Example: exampleuser
password Body String
The password in plaintext. To keep the password secure, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.
Plaintext password values must be prefixed with data:, — for example, data:,<plaintext_password>. Plaintext password values that contain a colon (:) must be encoded in base64 format and prefixed with data:;base64, — for example, data:;base64,<base64_encoded_password>.
port Body String
IBM Db2 port number. Default is 50000.
Example: 50000
fetchSize Body Integer Optional
Number of records to fetch at one time. Default is 200. If set to 0, Dremio automatically decides how many records to fetch.
Example: 200
maxIdleConns Body Integer Optional
Maximum number of connections that can be idle at any given time. Default is 8.
Example: 8
idleTimeSec Body Integer Optional
Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.
Example: 60
queryTimeoutSec Body Integer Optional
Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.
Example: 0
propertyList Body Array of Object Optional
Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.
Example: [{"name": "deferPrepares","value": "true"}]
Microsoft SQL Server
Dremio source type: MSSQL. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for SQL Server, see Configuring SQL Server as a Source for additional information.
"config": {
"authenticationType": [
"ANONYMOUS",
"MASTER"
]
"username": "myUsername",
"password": "myPassword".
"hostname": "localhost",
"port": "1433",
"database": "my_database",
"useSsl": true,
"showOnlyConnectionDatabase": false,
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"enableServerVerification": true,
"hostnameOverride": "localhost",
"propertyList": [
{
"name": "applicationIntent",
"value": "ReadWrite"
}
]
}
Parameters
authenticationType Body String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed and MASTER when credentials from a master database user is required.
Enum: ANONYMOUS, MASTER
username Body String
The SQL Server user name.
Example: myUsername
password Body String
The SQL Server password.
Example: myPassword
hostname Body String
The SQL Server host name.
Example: localhost
port Body String
The SQL Server port number. The commonly used default port numbers for SQL Server are 1433 and 1434.
Example: 1433
database Body String Optional
The database name.
Example: my_database
useSsl Body Boolean
Option to enable SSL encryption. Set to true to enable this option.
showOnlyConnectionDatabase Body Boolean
Option to show only the initial database used for connecting. Set to true to enable this option.
fetchSize Body String
The number of records to fetch per request. Set to 0 to enable Dremio to automatically set the size.
Example: 200
maxIdleConns Body Integer
Sets the maximum number of idle connections that you want to keep.
Example: 8
idleTimeSec Body Integer
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example: 60
enableServerVerification Body Boolean
Option to verify the server certificate. Set to true to enable this option.
hostnameOverride Body String
This parameter can be used when the SSL/TLS server certificate contains a distinguished name. You can leave this blank if the hostname in the certificate matches the server name.
Example: localhost
propertyList Body Array of Object
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: [{"name": "applicationIntent", "value": "ReadWrite"}]
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
MySQL
Dremio source type: MYSQL. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for MySQL, see Configuring MySQL as a Source for additional information.
"config": {
"authenticationType": [
"ANONYMOUS",
"MASTER",
]
"username": "myUsername",
"password": "myPassword"
"hostname": "localhost",
"port": "3306",
"netWriteTimeout": 60,
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"propertyList": [
{
"name": "DataSource",
"value": "sales2023"
}
]
}
Parameters
authenticationType Body String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed and MASTER when credentials from a master database user is required.
Enum: ANONYMOUS, MASTER
username Body String
The MySQL user name.
Example: myUsername
password Body String
The MySQL password.
Example: myPassword
hostname Body String
The MySQL host name.
Example: localhost
port Body String
The MySQL port number. The commonly used default port number for MySQL is 3306.
Example: 3306
netWriteTimeout Body Integer
The amount of time, in seconds, to wait for data from the server before aborting the connection.
Example: 60
fetchSize Body Integer
The number of records to fetch per request. Set to 0 to enable Dremio to automatically set the size.
Example: 200
maxIdleConns Body Integer
Sets the maximum number of idle connections that you want to keep.
Example: 8
idleTimeSec Body Integer
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example: 60
propertyList Body Array of Object
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: [{"name": "DataSource", "value": "sales2023"}]
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
Oracle
Dremio source type: ORACLE. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Oracle, see Configuring Oracle as a Source for additional information.
"config": {
"authenticationType": [
"ANONYMOUS",
"MASTER"
]
"username": "myUsername",
"password": "myPassword",
"hostname": "localhost",
"port": "1521",
"instance": "sales",
"useSsl": true,
"nativeEncryption": "ACCEPTED",
"useTimezoneAsRegion": true,
"includeSynonyms": false,
"mapDateToTimestamp": true,
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"useLdap": false,
"bindDN": String,
"sslServerCertDN": String,
"propertyList": [
{
"name": "connect_timeout",
"value": "0"
}
]
}
Parameters
authenticationType Body String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed, MASTER when either credentials from a master database user or a secret resource URL is required, or KERBEROS if Kerberos is configured.
Enum: ANONYMOUS, MASTER, KERBEROS
username Body String
The Oracle user name.
Example: myUsername
password Body String
The Oracle password.
Example: myPassword
secretResourceUrl Body String
For secret-based authentication, the Amazon Resource Name (ARN) for the secret. To keep the secret secure, Dremio returns the secretResourceUrl value as $DREMIO_EXISTING_VALUE$ in API responses.
Example: arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy
useKerberos Body Boolean
Set to true if you are using Kerberos.
hostname Body String
The Oracle host name.
Example: localhost
port Body String
The Oracle port number. The commonly used default port numbers for Oracle are 1521 and 1830.
Example: 1521
instance Body String Optional
The Oracle service name.
Example: sales
useSsl Body Boolean
Option to enable SSL encryption. Set to true to enable this option.
nativeEncryption Body Boolean
Sets the Oracle native encryption. This parameter can only be used if useSsl is set to false.
Enum: REJECTED, ACCEPTED, REQUESTED, REQUIRED
useTimezoneAsRegion Body Boolean
Option to use the timezone to set the connection region. Set to true to enable this option.
includeSynonyms Body Boolean
Option to include synonyms as datasets. Set to true to enable this option.
mapDateToTimestamp Body Boolean
Option to display Oracle DATE columns as a TIMESTAMP. Set to true to enable this option.
fetchSize Body Integer
The number of records to fetch per request. Set to 0 to enable Dremio to automatically set the size.
Example: 200
maxIdleConns Body Integer
Sets the maximum number of idle connections that you want to keep.
Example: 8
idleTimeSec Body Integer
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example: 60
useLdap Body Boolean
Option to use LDAP naming services. Set to true to enable this option.
bindDN Body String
Sets the LDAP hostname resolution when the useLdap parameter is set to true. You can omit this parameter if useLdap is set to false.
sslServerCertDN Body String
Use when you have a SSL/TLS server certificate distinguished name. Otherwise, you can leave this parameter blank to disable DN match.
propertyList Body Array of Object
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: [{"name": "connect_timeout", "value": "0"}]
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"secretResourceUrl": "arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy"
}
{
"credentialType": "KERBEROS",
"username": null,
"password": null,
"useKerberos": true
}
PostgreSQL
Dremio source type: POSTGRES. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for PostgreSQL, see Configuring PostgreSQL as a Source for additional information.
"config": {
"authenticationType": [
"ANONYMOUS",
"MASTER"
]
"username": "myUsername",
"password": "myPassword",
"hostname": "localhost",
"port": "5432",
"databaseName": "sales_2023",
"useSsl": false,
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"encryptionValidationMode": "CERTIFICATE_AND_HOSTNAME_VALIDATION",
"propertyList": [
{
"name": "connect_timeout",
"value": "10&application_name=myapp"
}
]
}
Parameters
authenticationType Body String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed and MASTER when either credentials from a master database user or a secret resource URL is required.
Enum: ANONYMOUS, MASTER
username Body String
The PostgreSQL user name.
Example: myUsername
password Body String
The PostgreSQL password.
Example: myPassword
secretResourceUrl Body String
For secret-based authentication, the Amazon Resource Name (ARN) for the secret. To keep the secret secure, Dremio returns the secretResourceUrl value as $DREMIO_EXISTING_VALUE$ in API responses.
Example: arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy
hostname Body String
The PostgreSQL host name.
Example: localhost
port Body String
The PostgreSQL port number. The commonly used default port number for PostgreSQL is 5432.
Example: 5432
databaseName Body String Optional
The database name.
Example: sales_2023
useSsl Body Boolean
Option to enable SSL encryption. Set to true to enable this option.
fetchSize Body Integer
The number of records to fetch per request. Set to 0 to enable Dremio to automatically set the size.
Example: 200
maxIdleConns Body Integer
Sets the maximum number of idle connections that you want to keep.
Example: 8
idleTimeSec Body Integer
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example: 60
encryptionValidationMode Body String
When encryption is enabled, you can set the validation mode to one of the available presets.
Enum: CERTIFICATE_AND_HOSTNAME_VALIDATION, CERTIFICATE_ONLY_VALIDATION, NO_VALIDATION
propertyList Body Array of Object
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: [{"name": "connect_timeout", "value": "10&application_name=myapp"}]
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
Snowflake
Dremio source type: SNOWFLAKE. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Snowflake, see Configuring Snowflake as a Source for additional information.
"config": {
"hostname": "https://locator_id.snowflakecomputing.com",
"port": "443",
"database": "REGRESSION",
"warehouse": "CONNECTOR_25GB",
"username": "dremio",
"maxIdleConns": 8,
"idleTimeSec": 60,
"queryTimeoutSec": 0,
"fetchSize": 2000,
"authMode": "KEY_PAIR",
"privateKey": "AsB86vDRA&Ew5+jSCwn3F2LqMcmu82DcN9ByfcDzcP2GE2oJivFAS53M9s4B=hnu5Eg"
}
Parameters
hostname Body String
The Snowflake host name.
Example: https://locator_id.snowflakecomputing.com
port Body String
The Snowflake port number. The commonly used default port number for Snowflake is 443.
Example: 443
database Body String Optional
Specifies the default database to use.
Example: REGRESSION
warehouse Body String Optional
Specifies the warehouse that will provide resources for executing DML statements and queries.
Example: CONNECTOR_25GB
username Body String
The Snowflake user name.
Example: dremio
password Body String
The Snowflake password. Required if authMode is LOGIN_PASSWORD.
Example: myPassword
fetchSize Body Integer
The number of records to fetch per request. Set to 0 to enable Dremio to automatically set the size.
Example: 2000
maxIdleConns Body Integer
Sets the maximum number of idle connections that you want to keep.
Example: 8
idleTimeSec Body Integer
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example: 60
queryTimeoutSec Body Integer
Sets the timeout, in seconds, for a query execution before it is canceled. if you do not want a timeout, enter 0.
Example: 0
authMode Body String
Type of authentication for Dremio to use to connect to the source. If you do not specify an authMode, the default is LOGIN_PASSWORD.
Enum: LOGIN_PASSWORD, KEY_PAIR
Example: KEY_PAIR
privateKey Body String
The generated Snowflake private key in Privacy Enhanced Mail (PEM) format. Required if authMode is KEY_PAIR. To keep the service account key secure, Dremio returns the privateKey value as $DREMIO_EXISTING_VALUE$ in API responses.
Example: AsB86vDRA&Ew5+jSCwn3F2LqMcmu82DcN9ByfcDzcP2GE2oJivFAS53M9s4B=hnu5Eg
privateKeyPassphrase Body String
The passphrase for an encrypted private key. Required if authMode is KEY_PAIR and the privateKey value is an encrypted private key.
Example: !jV#X=%k7cG=e%G64p#ozm
Examples of Different Configurations for authMode
Login-password authentication{
"authenticationType": "LOGIN_PASSWORD",
"username": "dremio",
"password": "myPassword"
}
{
"authenticationType": "KEY_PAIR",
"username": "dremio",
"privateKey": "AsB86vDRA&Ew5+jSCwn3F2LqMcmu82DcN9ByfcDzcP2GE2oJivFAS53M9s4B=hnu5Eg",
"privateKeyPassphrase": "!jV#X=%k7cG=e%G64p#ozm"
}
Vertica
Dremio source type: VERTICA. This type determines the config parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Vertica, see Configuring Vertica as a Source for additional information.
{
"config": {
"database": "vertica",
"hostname": "172.25.0.10",
"username": "exampleuser",
"password": "$DREMIO_EXISTING_VALUE$",
"port": "5433",
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"queryTimeoutSec": 0,
"propertyList": [
{
"name": "LoginTimeout",
"value": "30"
}
]
}
}
Parameters
database Body String
Default Vertica database for Dremio to use.
Example: vertica
hostname Body String
Vertica hostname.
Example: 172.25.0.10
username Body String
Username for authentication.
Example: exampleuser
password Body String
The password in plaintext. To keep the password secure, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.
Plaintext password values must be prefixed with data:, — for example, data:,<plaintext_password>. Plaintext password values that contain a colon (:) must be encoded in base64 format and prefixed with data:;base64, — for example, data:;base64,<base64_encoded_password>.
port Body String
Vertica port number. Default is 5433.
Example: 5433
fetchSize Body Integer Optional
Number of records to fetch at one time. Default is 200. If set to 0, Dremio automatically decides how many records to fetch.
Example: 200
maxIdleConns Body Integer Optional
Maximum number of connections that can be idle at any given time. Default is 8.
Example: 8
idleTimeSec Body Integer Optional
Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.
Example: 60
queryTimeoutSec Body Integer Optional
Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.
Example: 0
propertyList Body Array of Object Optional
Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.
Example: [{"name": "LoginTimeout","value": "30&application_name=myapp"}]
Parameters of Objects in the propertyList Array
name Body String Optional
Name of the connection property.
Example: LoginTimeout
value Body String Optional
Value for the connection property.
Example: 30