Source Configuration
This page details the config
payload to create each source type.
Amazon Redshift
Dremio source type: REDSHIFT
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Redshift, see Adding a Redshift Cluster as a Data Source for additional information.
"config": {
"connectionString": "jdbc:redshift://redshift.example.com:5439/database",
"authenticationType": [
"ANONYMOUS",
"MASTER",
"AWS_PROFILE"
],
"username": "myUsername",
"password": "myPassword",
"secretResourceUrl": arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy,
"awsProfile": "profileName",
"dbUser": "userName",
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"queryTimeoutSec": 0,
"propertyList": [
{
"name": "ApplicationName",
"value": "MarketingDashboard"
}
]
}
Parameters
connectionString Body String
The JDBC connection string. Read JDBC for more information.
Example: jdbc:redshift://redshift.example.com:5439/database
authenticationType Body String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS
when no authentication is needed, MASTER
when either credentials from a master database user or a secret resource URL is required, or AWS_PROFILE
for authentication to an AWS source.
Enum: ANONYMOUS, AWS_PROFILE, MASTER
username Body String
The Amazon Redshift user name.
Example: myUsername
password Body String
The Amazon Redshift password.
Example: myPassword
secretResourceUrl Body String
For secret-based authentication, the Amazon Resource Name (ARN) for the secret. To keep the secret secure, Dremio returns the secretResourceUrl value as $DREMIO_EXISTING_VALUE$
in API responses.
Example: arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy
awsProfile Body String
The AWS profile name. If this is left blank, the default profile will be used.
Example: profileName
dbUser Body String
The name of the Amazon Redshift DbUser that will be used for authentication. If this is left blank, the default user name for your AWS IAM role will be used.
Example: userName
fetchSize Body Integer
The number of records to fetch per request. Set to 0
to enable Dremio to automatically set the size.
Example: 200
maxIdleConns Body Integer
Sets the maximum number of idle connections that you want to keep.
Example: 8
idleTimeSec Body Integer
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example: 60
queryTimeoutSec Body Integer
Sets the timeout, in seconds, for a query execution before it is canceled. if you do not want a timeout, enter 0
.
Example: 0
propertyList Body Array of String
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.", "example":
Example: ["name": "ApplicationName", "value": "MarketingDashboard"]
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"secretResourceUrl": "arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy"
}
{
"authenticationType": "AWS_PROFILE",
"awsProfile": "profileName",
"dbUser": "userName"
}
Amazon S3
Dremio source type: S3
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for S3, see Adding an S3 Source for additional information.
"config": {
"credentialType": [
"PROJECT_DATA_CRED",
"ACCESS_KEY"
]
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"externalBucketList": /bucket-name/,
"secure": false,
"enableAsync": true,
"compatibilityMode": false,
"requesterPays": false,
"enableFileStatusCheck": true,
"isPartitionInferenceEnabled": false,
"rootPath": "/",
"kmsKeyARN": "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab",
"defaultCtasFormat": "ICEBERG",
"propertyList": [
{
"name": "fs.s3a.proxy.host",
"value": "proxyHost.example.com"
},
],
"whitelistedBuckets": [
"sales"
],
"isCachingEnabled": true,
"maxCacheSpacePct": 100
}
Parameters
credentialType Body String
Specifies the type of credential to enable Dremio to connect to the source. Must be either ACCESS_KEY
or PROJECT_DATA_CRED
.
Enum: ACCESS_KEY, PROJECT_DATA_CRED
assumedRoleARN Body String Optional
The ARN of the IAM role to assume.
Example: arn:aws:iam::123456789012:root
awsAccessKey Body String
The AWS access key.
Example: AKIAIOSFODNN7EXAMPLE
awsAccessSecret Body String
The AWS access secret.
Example: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
externalBucketList Body Array of String
Connects Dremio to one or more Amazon S3 buckets. Read Accessing a bucket using S3 for more information.
Example: ["bucket-name"]
secure Body Boolean
Option to enable a secure connection via SSL encryption between the S3 buckets and Dremio. Set to true
to enable SSL encryption.
enableAsync Body Boolean
Option to enable asynchronous access, which allows cloud caching for the source to support simultaneous actions such as adding and editing. Set to true
to enable asynchronous access. Read Amazon S3 > Advanced Options for more information.
compatibilityMode Body Boolean
Option to enable compatibility mode for an Amazon S3 source, which allows you to use S3-compatible storage. Set to true
to enable compatibility mode.
requesterPays Body Boolean
Option to apply S3 requests to the requestor. Set to true
enable this option.
enableFileStatusCheck Body Boolean
Option to enable file status check. Set to true
to enable file status check.
isPartitionInferenceEnabled Body Boolean
Option to enable partition column inference. Set to true
to enable partition column inference.
rootPath Body String
The root path of the S3 bucket.
Example: /
kmsKeyARN Body String
Enables you to add a server side encryption key ARN that was created in AWS Key Management Service (KMS).
Example: arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
defaultCtasFormat Body String
Enables you to choose the default format for tables you create in Dremio, either ICEBERG
or PARQUET
.
Enum: ICEBERG, PARQUET
propertyList Body Array of Object
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: {"name": "fs.s3a.proxy.host", "value": "proxyHost.example.com"}
whitelistedBuckets Body Array of String
Add one or more approved S3 buckets.
Example: ["sales"]
isCachingEnabled Body Boolean
Option to enable local caching. When local caching is possible, set this parameter to true
to enable it. Read Amazon S3 > Advanced Options for more information.
maxCacheSpacePct Body Integer
Sets the maximum percent of total available cache space to use when possible. The value, in percent, must be between 1 and 100.
Example: 100
Examples of Different Configurations for credentialType
Project data credentials{
"credentialType": "PROJECT_DATA_CRED"
}
{
"credentialType": "PROJECT_DATA_CRED",
"assumedRoleARN": "arn:aws:iam::123456789012:root"
}
{
"credentialType": "ACCESS_KEY",
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}
Arctic
Dremio source type: ARCTIC
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Arctic, see Dremio Sonar for additional information.
"config": {
"awsRootPath": "/my.s3bucket.com/path",
"propertyList": [
{
"name": "fs.s3a.proxy.host",
"value": "proxyHost.example.com"
}
],
"asyncEnabled": true,
"isCachingEnabled": true,
"maxCacheSpacePct": 100,
"defaultCtasFormat": "ICEBERG",
"storageProvider": "AWS",
"credentialType": "PROJECT_DATA_CRED",
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"arcticCatalogId": "12c7598e-8055-4326-afde-873f3ad942d0"
}
"config": {
"propertyList": [
{
"name": "propertyName",
"value": "propertyValue"
}
],
"asyncEnabled": true,
"isCachingEnabled": true,
"maxCacheSpacePct": 100,
"defaultCtasFormat": "ICEBERG",
"storageProvider": "AZURE",
"azureStorageAccount": "teamStorage",
"azureRootPath": "newCatalogStorage/folder",
"azureAuthenticationType": "ACCESS_KEY",
"azureAccessKey": "8iWvLxAFwBkFQuzkKEU8iPEdX%Zrm&ChyZfXUBMQQ*r$gCnidpRsAwPe^AjcieN!V9sJu/cefsSrGfEXAMPLE==$",
"arcticCatalogId": "5bc2f423-d46c-4b1f-a029-ca8148bc196e"
}
Parameters
awsAccessKey Body String
The AWS access key for the Amazon S3 bucket to use for storage. Include only if you specify AWS
as the storageProvider value and ACCESS_KEY
as the credentialType value.
Example: AKIAIOSFODNN7EXAMPLE
awsRootPath Body String
The root path of the Amazon S3 bucket to use for storage. Include only if you specify AWS
as the storageProvider.
Example: /your-s3-bucket-name
propertyList Body Array of Object Optional
Connection properties for the source. Each object in propertyList includes the name of the property and the corresponding value to use for the property.
Example: [{"name": "fs.s3a.proxy.host","value": "proxyHost.example.com"}]
asyncEnabled Body Boolean
Option to enable asynchronous access, when this functionality is available in your Arctic source. This option enables cloud caching for the source to support simultaneous actions such as adding and editing. Set to true
to enable asynchronous access.
isCachingEnabled Body Boolean
Option to enable local caching, when this functionality is available in your Arctic source. Set to true
to enable local caching.
maxCacheSpacePct Body Integer
Sets the maximum percent of total available cache space to use when possible. The value, in percent, must be between 1 and 100.
Example: 100
defaultCtasFormat Body String
Enables you to choose the default format for tables you create in Dremio, either ICEBERG
or PARQUET
.
Enum: ICEBERG, PARQUET
storageProvider Body String
Specifies the storage provider to use for the Arctic catalog source.
Enum: AWS, AZURE
azureStorageAccount Body String
The name of the Azure Storage account to use for the Arctic catalog source. Include only if you specify AZURE
as the storageProvider value.
Example: teamStorage
azureRootPath Body String
The root path inside your Azure Storage account. The root path is the name of the Azure Storage container, including any folders. Include only if you specify AZURE
as the storageProvider value.
Example: newCatalogStorage/folder
azureAuthenticationType Body String
The method to use for authenticating to your Azure Storage account. Include only if you specify AZURE
as the storageProvider value.
Enum: ACCESS_KEY, AZURE_ACTIVE_DIRECTORY
azureAccessKey Body String
The shared access key for your Azure Storage account. Include only if you specify AZURE
as the storageProvider value and ACCESS_KEY
as the azureAuthenticationType value.
Example: 8iWvLxAFwBkFQuzkKEU8iPEdX%Zrm&ChyZfXUBMQQ*r$gCnidpRsAwPe^AjcieN!V9sJu/cefsSrGfEXAMPLE==
azureApplicationId Body String
The application (client) ID for your Azure Storage account. Include only if you specify AZURE
as the storageProvider value and AZURE_ACTIVE_DIRECTORY
as the azureAuthenticationType value.
Example: ac2e0fbf-2507-4a32-983e-9550eb5b3191
azureClientSecret Body String
The secret generated for the application in your Azure Storage account. Include only if you specify AZURE
as the storageProvider value and AZURE_ACTIVE_DIRECTORY
as the azureAuthenticationType value.
Example: cb^tB3dDor.YtxywDuSvxgdRHEXAMPLE
azureOAuthTokenEndpoint Body String
The OAuth 2.0 token endpoint (v1.0), including the tenant ID, that the application in your Azure Storage account uses to get an access token or a refresh token. Include only if you specify AZURE
as the storageProvider value and AZURE_ACTIVE_DIRECTORY
as the azureAuthenticationType value.
Example: https://login.microsoftonline.com/01ca1052-ab53-4fb6-8b16-0b6f3dc73cc6/oauth2/token
credentialType Body String
Specifies the type of credential to use to enable Dremio to connect to the Arctic catalog source. Include only if you specify AWS
as the storageProvider value.
Enum: ACCESS_KEY, PROJECT_DATA_CRED
assumedRoleARN Body String Optional
The Amazon Resource Name (ARN) of the IAM role to assume for the Arctic catalog source. Include only if you specify AWS
as the storageProvider value.
Example: arn:aws:iam::123456789012:root
arcticCatalogId Body String
The unique ID of the Arctic catalog you are accessing.
Example: ab12cdef-a123-b456-ab1234-cdef5678ghij
Examples of Different Configurations for credentialType and azureAuthenticationType
Authentication parameters to include for credentialType `PROJECT_DATA_CRED` and storageProvider `AWS`{
"credentialType": "PROJECT_DATA_CRED",
"assumedRoleARN": "arn:aws:iam::123456789012:root"
}
{
"credentialType": "ACCESS_KEY",
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}
{
"azureAuthenticationType": "ACCESS_KEY",
"azureAccessKey": "8iWvLxAFwBkFQuzkKEU8iPEdX%Zrm&ChyZfXUBMQQ*r$gCnidpRsAwPe^AjcieN!V9sJu/cefsSrGfEXAMPLE=="
}
{
"azureAuthenticationType": "AZURE_ACTIVE_DIRECTORY",
"azureApplicationId": "ac2e0fbf-2507-4a32-983e-9550eb5b3191",
"azureClientSecret": "cb^tB3dDor.YtxywDuSvxgdRHEXAMPLE",
"azureOAuthTokenEndpoint": "https://login.microsoftonline.com/01ca1052-ab53-4fb6-8b16-0b6f3dc73cc6/oauth2/token"
}
AWS Glue Data Catalog
Dremio source type: AWSGLUE
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for AWS Glue, see Adding an AWS Glue Data Catalog for additional information.
"config": {
"credentialType": [
"PROJECT_DATA_CRED",
"ACCESS_KEY"
]
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"regionNameSelection": "us-east-1",
"secure": false,
"enableAsync": true,
"propertyList": [
{
"name": "hive.metastore.warehouse.dir",
"value": "s3://bucket-name/folder"
},
],
"allowedDatabases": ["sales","finance"],
"isCachingEnabled": true,
"maxCacheSpacePct": 100
}
Parameters
credentialType Body String
Specifies the type of credential to enable Dremio to connect to the source. Must be either ACCESS_KEY
or PROJECT_DATA_CRED
.
Enum: ACCESS_KEY, PROJECT_DATA_CRED
assumedRoleARN Body String Optional
The ARN of the IAM role to assume.
Example: arn:aws:iam::123456789012:root
awsAccessKey Body String
The AWS access key.
Example: AKIAIOSFODNN7EXAMPLE
awsAccessSecret Body String
The AWS access secret.
Example: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
regionNameSelection Body String
Sets the AWS Region where the Glue Data Catalog is located.
Example: us-east-1
secure Body Boolean
Option to enable a secure connection via SSL encryption between the Glue Data Catalog and Dremio. Set to true
to enable SSL encryption.
enableAsync Body Boolean
Option to enable asynchronous access, which allows cloud caching for the source to support simultaneous actions such as adding and editing. Set to true
to enable asynchronous access. Read AWS Glue Data Catalog > Advanced Options for more information.
propertyList Body Array of Object
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: [{"name": "hive.metastore.warehouse.dir", "value": "s3://bucket-name/folder"}]
allowedDatabases Body Array of String
A list of the databases that users of Dremio are allowed to access. Present the names in a comma-separated list.
If one or more databases are added or removed from the list, the change takes effect immediately.
Example: ["sales","finance”]
isCachingEnabled Body Boolean
Option to enable local caching. When local caching is possible, set this parameter to true
to enable it. Read AWS Glue Data Catalog > Advanced Options for more information.
maxCacheSpacePct Body Integer
Sets the maximum percent of total available cache space to use when possible. The value, in percent, must be between 1 and 100.
Example: 100
Examples of Different Configurations for credentialType
Project data credentials{
"credentialType": "PROJECT_DATA_CRED"
}
{
"credentialType": "PROJECT_DATA_CRED",
"assumedRoleARN": "arn:aws:iam::123456789012:root"
}
{
"credentialType": "ACCESS_KEY",
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}
Azure Storage
Dremio source type: AZURE_STORAGE
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Azure Storage, see Adding an Azure Storage Source for additional information.
"config": {
"accountKind": "STORAGE_V2",
"accountName": "testazure",
"accessKey": "l3kertpoifkegoivns93c/9AEUa+hie8XEF039/oiWrgbx2g7e+T3bk9H/4R7vua7ljA2pwDKscPZTEXAMPLE==",
"rootPath": "/",
"propertyList": [
{
"name": "dremio.azure.mode",
"value": "STORAGE_V2"
},
],
"enableSSL": true,
"enableAsync": true,
"credentialsType": "ACCESS_KEY",
"isCachingEnabled": true,
"maxCacheSpacePct": 100,
"defaultCtasFormat": "ICEBERG",
"isPartitionInferenceEnabled": false
}
Parameters
accountKind Body String
Type of the Azure storage account.
Enum: STORAGE_V1, STORAGE_V2
Example: STORAGE_V2
accountName Body String
Name of the Azure storage account.
Example: testazure
credentialsType Body String
Specifies the type of authentication to use to enable Dremio to connect to the source.
Enum: ACCESS_KEY, AZURE_ACTIVE_DIRECTORY
accessKey Body String
If using the ACCESS_KEY
credentialsType, the Azure access key for the Azure Storage account. To keep the access key secure, Dremio returns the accessKey value as $DREMIO_EXISTING_VALUE$
in API responses.
Example: l3kertpoifkegoivns93c/9AEUa+hie8XEF039/oiWrgbx2g7e+T3bk9H/4R7vua7ljA2pwDKscPZTEXAMPLE==
clientId Body String
If using the AZURE_ACTIVE_DIRECTORY
credentialsType, the client ID of the registered app in Azure.
Example: ba72f3e5-edc6-4aec-ad0a-c1147e81dc8d
tokenEndpoint Body String
If using the AZURE_ACTIVE_DIRECTORY
credentialsType, the OAuth 2.0 token endpoint for the registered app in Azure.
Example: https://login.microsoftonline.com/2dfe8f9a-c80b-40c9-81de-7713ca393904/oauth2/token
clientSecret Body String
If using the AZURE_ACTIVE_DIRECTORY
credentialsType, the client secret for the registered app in Azure. To keep the client secret secure, Dremio returns the accessKey value as $DREMIO_EXISTING_VALUE$
in API responses.
Example: pbt+a]xWAk2E*2.WmjU0wtfiLEXAMPLE
rootPath Body String Optional
Root path of the Azure Storage source. Default is /
Example: /
propertyList Body Array of Object Optional
An array of name/value pairs defining optional connection properties. Use a comma-separated list to specify multiple name/value pairs.
Example: [{"name": "dremio.azure.mode", "value": "STORAGE_V2"}]
enableSSL Body Boolean Optional
To enable a secure connection with SSL encryption between the Azure Storage source and Dremio, set to true
(default). Otherwise, set to false
.
Example: true
enableAsync Body Boolean Optional
To enable asynchronous access for the source and allow cloud caching to support simultaneous actions such as adding and editing, set to true
(default). Otherwise, set to false
.
Example: true
isCachingEnabled Body Boolean Optional
To enable local caching, set to true
(default). Otherwise, set to false
.
Example: true
maxCacheSpacePct Body Integer Optional
Maximum percentage of the total available cache space to use on any single executor node. Used only when isCachingEnabled is set to true
. Minimum is 1
. Maximum is 100
(default).
Example: 100
defaultCtasFormat Body String Optional
Default format for the tables you create in Dremio. Default is ICEBERG
.
Enum: ICEBERG,PARQUET
Example: ICEBERG
isPartitionInferenceEnabled Body Boolean Optional
To enable partition column inference, set to true
. Otherwise, set to false
(default).
Example: false
Examples of Different Configurations for credentialsType
Azure Active Directory{
"credentialsType": "AZURE_ACTIVE_DIRECTORY",
"clientId": "ba72f3e5-edc6-4aec-ad0a-c1147e81dc8d",
"tokenEndpoint": "https://login.microsoftonline.com/2dfe8f9a-c80b-40c9-81de-7713ca393904/oauth2/token",
"clientSecret": "pbt+a]xWAk2E*2.WmjU0wtfiLEXAMPLE"
}
Microsoft SQL Server
Dremio source type: MSSQL
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for SQL Server, see Configuring SQL Server as a Source for additional information.
"config": {
"authenticationType": [
"ANONYMOUS",
"MASTER"
]
"username": "myUsername",
"password": "myPassword".
"hostname": "localhost",
"port": "1433",
"database": "my_database",
"useSsl": true,
"showOnlyConnectionDatabase": false,
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"enableServerVerification": true,
"hostnameOverride": "localhost",
"propertyList": [
{
"name": "applicationIntent",
"value": "ReadWrite"
}
]
}
Parameters
authenticationType Body String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed and MASTER when credentials from a master database user is required.
Enum: ANONYMOUS, MASTER
username Body String
The SQL Server user name.
Example: myUsername
password Body String
The SQL Server password.
Example: myPassword
hostname Body String
The SQL Server host name.
Example: localhost
port Body String
The SQL Server port number. The coomonly used default port numbers for SQL Server are 1433
and 1434
.
Example: 1433
database Body String Optional
The database name.
Example: my_database
useSsl Body Boolean
Option to enable SSL encryption. Set to true
to enable this option.
showOnlyConnectionDatabase Body Boolean
Option to show only the initial database used for connecting. Set to true
to enable this option.
fetchSize Body String
The number of records to fetch per request. Set to 0
to enable Dremio to automatically set the size.
Example: 200
maxIdleConns Body Integer
Sets the maximum number of idle connections that you want to keep.
Example: 8
idleTimeSec Body Integer
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example: 60
enableServerVerification Body Boolean
Option to verify the server certificate. Set to true
to enable this option.
hostnameOverride Body String
This parameter can be used when the SSL/TLS server certificate contains a distinguished name. You can leave this blank if the hostname in the certificate matches the server name.
Example: localhost
propertyList Body Array of Object
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: [{"name": "applicationIntent", "value": "ReadWrite"}]
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
MySQL
Dremio source type: MYSQL
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for MySQL, see Configuring MySQL as a Source for additional information.
"config": {
"authenticationType": [
"ANONYMOUS",
"MASTER",
]
"username": "myUsername",
"password": "myPassword"
"hostname": "localhost",
"port": "3306",
"netWriteTimeout": 60,
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"propertyList": [
{
"name": "DataSource",
"value": "sales2023"
}
]
}
Parameters
authenticationType Body String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed and MASTER when credentials from a master database user is required.
Enum: ANONYMOUS, MASTER
username Body String
The MySQL user name.
Example: myUsername
password Body String
The MySQL password.
Example: myPassword
hostname Body String
The MySQL host name.
Example: localhost
port Body String
The MySQL port number. The coomonly used default port number for MySQL is 3306
.
Example: 3306
netWriteTimeout Body Integer
The amount of time, in seconds, to wait for data from the server before aborting the connection.
Example: 60
fetchSize Body Integer
The number of records to fetch per request. Set to 0
to enable Dremio to automatically set the size.
Example: 200
maxIdleConns Body Integer
Sets the maximum number of idle connections that you want to keep.
Example: 8
idleTimeSec Body Integer
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example: 60
propertyList Body Array of Object
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: [{"name": "DataSource", "value": "sales2023"}]
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
Oracle
Dremio source type: ORACLE
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Oracle, see Configuring Oracle as a Source for additional information.
"config": {
"authenticationType": [
"ANONYMOUS",
"MASTER"
]
"username": "myUsername",
"password": "myPassword",
"hostname": "localhost",
"port": "1521",
"instance": "sales",
"useSsl": true,
"nativeEncryption": "ACCEPTED",
"useTimezoneAsRegion": true,
"includeSynonyms": false,
"mapDateToTimestamp": true,
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"useLdap": false,
"bindDN": String,
"sslServerCertDN": String,
"propertyList": [
{
"name": "connect_timeout",
"value": "0"
}
]
}
Parameters
authenticationType Body String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS
when no authentication is needed, MASTER
when either credentials from a master database user or a secret resource URL is required, or KERBEROS
if Kerberos is configured.
Enum: ANONYMOUS, MASTER, KERBEROS
username Body String
The Oracle user name.
Example: myUsername
password Body String
The Oracle password.
Example: myPassword
secretResourceUrl Body String
For secret-based authentication, the Amazon Resource Name (ARN) for the secret. To keep the secret secure, Dremio returns the secretResourceUrl value as $DREMIO_EXISTING_VALUE$
in API responses.
Example: arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy
useKerberos Body Boolean
Set to true
if you are using Kerberos.
hostname Body String
The Oracle host name.
Example: localhost
port Body String
The Oracle port number. The commonly used default port numbers for Oracle are 1521
and 1830
.
Example: 1521
instance Body String Optional
The Oracle service name.
Example: sales
useSsl Body Boolean
Option to enable SSL encryption. Set to true
to enable this option.
nativeEncryption Body Boolean
Sets the Oracle native encryption. This parameter can only be used if useSsl
is set to false
.
Enum: REJECTED, ACCEPTED, REQUESTED, REQUIRED
useTimezoneAsRegion Body Boolean
Option to use the timezone to set the connection region. Set to true
to enable this option.
includeSynonyms Body Boolean
Option to include synonyms as datasets. Set to true
to enable this option.
mapDateToTimestamp Body Boolean
Option to display Oracle DATE
columns as a TIMESTAMP
. Set to true
to enable this option.
fetchSize Body Integer
The number of records to fetch per request. Set to 0
to enable Dremio to automatically set the size.
Example: 200
maxIdleConns Body Integer
Sets the maximum number of idle connections that you want to keep.
Example: 8
idleTimeSec Body Integer
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example: 60
useLdap Body Boolean
Option to use LDAP naming services. Set to true
to enable this option.
bindDN Body String
Sets the LDAP hostname resolution when the useLdap
parameter is set to true
. You can omit this parameter if useLdap
is set to false
.
sslServerCertDN Body String
Use when you have a SSL/TLS server certificate distinguished name. Otherwise, you can leave this parameter blank to disable DN match.
propertyList Body Array of Object
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: [{"name": "connect_timeout", "value": "0"}]
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"secretResourceUrl": "arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy"
}
{
"credentialType": "KERBEROS",
"username": null,
"password": null,
"useKerberos": true
}
PostgreSQL
Dremio source type: POSTGRES
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for PostgreSQL, see Configuring PostgreSQL as a Source for additional information.
"config": {
"authenticationType": [
"ANONYMOUS",
"MASTER"
]
"username": "myUsername",
"password": "myPassword",
"hostname": "localhost",
"port": "5432",
"databaseName": "sales_2023",
"useSsl": false,
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"encryptionValidationMode": "CERTIFICATE_AND_HOSTNAME_VALIDATION",
"propertyList": [
{
"name": "connect_timeout",
"value": "10&application_name=myapp"
}
]
}
Parameters
authenticationType Body String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed and MASTER when either credentials from a master database user or a secret resource URL is required.
Enum: ANONYMOUS, MASTER
username Body String
The PostgreSQL user name.
Example: myUsername
password Body String
The PostgreSQL password.
Example: myPassword
secretResourceUrl Body String
For secret-based authentication, the Amazon Resource Name (ARN) for the secret. To keep the secret secure, Dremio returns the secretResourceUrl value as $DREMIO_EXISTING_VALUE$
in API responses.
Example: arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy
hostname Body String
The PostgreSQL host name.
Example: localhost
port Body String
The PostgreSQL port number. The coomonly used default port number for PostgreSQL is 5432
.
Example: 5432
databaseName Body String Optional
The database name.
Example: sales_2023
useSsl Body Boolean
Option to enable SSL encryption. Set to true
to enable this option.
fetchSize Body Integer
The number of records to fetch per request. Set to 0
to enable Dremio to automatically set the size.
Example: 200
maxIdleConns Body Integer
Sets the maximum number of idle connections that you want to keep.
Example: 8
idleTimeSec Body Integer
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example: 60
encryptionValidationMode Body String
When encryption is enabled, you can set the validation mode to one of the available presets.
Enum: CERTIFICATE_AND_HOSTNAME_VALIDATION, CERTIFICATE_ONLY_VALIDATION, NO_VALIDATION
propertyList Body Array of Object
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example: [{"name": "connect_timeout", "value": "10&application_name=myapp"}]
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
Snowflake
Dremio source type: SNOWFLAKE
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Snowflake, see Configuring Snowflake as a Source for additional information.
"config": {
"hostname": "https://locator_id.snowflakecomputing.com",
"username": "myUsername",
"password": "myPassword",
[ "database": "myDatabase" ],
[ "role": "PUBLIC" ],
[ "schema": "mySchema" ],
[ "warehouse": "myWarehouse" ]
}
Parameters
hostname Body String
The Snowflake host name.
Example: https://locator_id.snowflakecomputing.com
username Body String
The Snowflake user name.
Example: myUsername
password Body String
The Snowflake password.
Example: myPassword
database Body String Optional
Specifies the default database to use.
Example: myDatabase
role Body String Optional
Specifies the default access-control role to use. Read System-Defined Roles for more information.
schema Body String Optional
Specifies the default schema to use.
Example: mySchema
warehouse Body String Optional
Specifies the warehouse that will provide resources for executing DML statements and queries.
Example: myWarehouse