Configuration Attributes
This page details the config
payload to create each source type.
Amazon Redshift
Dremio source type: REDSHIFT
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Redshift, see Amazon Redshift Configuration for additional information.
"config": {
"connectionString": "jdbc:redshift://redshift.example.com:5439/database",
"authenticationType": [
"ANONYMOUS",
"MASTER",
"AWS_PROFILE"
],
"username": "myUsername",
"password": "myPassword",
"secretResourceUrl": arn:aws:secretsmanager:us-west-2:123456789012:secret:my-rds-secret-VNenFy,
"awsProfile": "profileName",
"dbUser": "userName",
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"queryTimeoutSec": 0,
"propertyList": [
{
"name": "ApplicationName",
"value": "MarketingDashboard"
}
]
}
Parameters
connectionString
String
The JDBC connection string. For more information, read JDBC.
authenticationType
String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS
when no authentication is needed, MASTER
when either credentials from a master database user or a secret resource URL is required, or AWS_PROFILE
for authentication to an AWS source.
Enum
ANONYMOUS
,
AWS_PROFILE
,
MASTER
username
String
The Amazon Redshift user name.
Example
myUsername
password
String
The Amazon Redshift password.
Example
myPassword
secretResourceUrl
String
The secret resource URL.
awsProfile
String
The AWS profile name. If this is left blank, the default profile will be used.
Example
profileName
dbUser
String
The name of the Amazon Redshift DbUser that will be used for authentication. If this is left blank, the default user name for your AWS IAM role will be used.
Example
userName
fetchSize
Number
The number of records to fetch per request. Set to 0
to enable Dremio to automatically set the size.
Example 200
maxIdleConns
Number
Sets the maximum number of idle connections that you want to keep.
Example 8
idleTimeSec
Number
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example 60
queryTimeoutSec
Number
Sets the timeout, in seconds, for a query execution before it is canceled. if you do not want a timeout, enter 0
.
propertyList
Array
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example "name": "ApplicationName", "value": "MarketingDashboard"
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"secretResourceUrl": "myPassword"
}
{
"authenticationType": "AWS_PROFILE",
"awsProfile": "profileName",
"dbUser": "userName"
}
Amazon S3
Dremio source type: S3
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for S3, see Adding an S3 Source for additional information.
"config": {
"credentialType": [
"PROJECT_DATA_CRED",
"ACCESS_KEY"
]
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"externalBucketList": /bucket-name/,
"secure": false,
"enableAsync": true,
"compatibilityMode": false,
"requesterPays": false,
"enableFileStatusCheck": true,
"isPartitionInferenceEnabled": false,
"rootPath": "/",
"kmsKeyARN": "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab",
"defaultCtasFormat": "ICEBERG",
"propertyList": [
{
"name": "fs.s3a.proxy.host",
"value": "proxyHost.example.com"
},
],
"whitelistedBuckets": "sales",
"isCachingEnabled": true,
"maxCacheSpacePct": 100
}
Parameters
credentialType
String
Specifies the type of credential to enable Dremio to connect to the source. Must be either ACCESS_KEY
or PROJECT_DATA_CRED
.
Enum
ACCESS_KEY
,
PROJECT_DATA_CRED
assumedRoleARN
String
Optional
The ARN of the IAM role to assume.
Example
arn:aws:iam::123456789012:root
awsAccessKey
String
The AWS access key.
Example
AKIAIOSFODNN7EXAMPLE
awsAccessSecret
String
The AWS access secret.
Example
wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
externalBucketList
Array
Connects Dremio to one or more Amazon S3 buckets. For more information, read Accessing a bucket using S3.
Example /bucket-name/
secure
Boolean
Option to enable a secure connection via SSL encryption between the S3 buckets and Dremio. Set to true
to enable SSL encryption.
enableAsync
Boolean
Option to enable asynchronous access, which allows cloud caching for the source to support simultaneous actions such as adding and editing. Set to true
to enable asynchronous access.
For more information, read Amazon S3 > Advanced Options.
compatibilityMode
Boolean
Option to enable compatibility mode for an Amazon S3 source, which allows you to use S3-compatible storage. Set to true
to enable compatibility mode.
requesterPays
Boolean
Option to apply S3 requests to the requestor. Set to true
enable this option.
enableFileStatusCheck
Boolean
Option to enable file status check. Set to true
to enable file status check.
isPartitionInferenceEnabled
Boolean
Option to enable partition column inference. Set to true
to enable partition column inference.
rootPath
String
The root path of the S3 bucket.
Example
/
kmsKeyARN
String
Enables you to add a server side encryption key ARN that was created in AWS Key Management Service (KMS).
Example
arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
defaultCtasFormat
String
Enables you to choose the default format for tables you create in Dremio, either ICEBERG
or PARQUET
.
Enum
ICEBERG
,
PARQUET
propertyList
Array
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example "name": "fs.s3a.proxy.host", "value": "proxyHost.example.com"
whitelistedBuckets
Array
Add one or more approved S3 buckets.
Example sales
isCachingEnabled
Boolean
Option to enable local caching. When local caching is possible, set this parameter to true
to enable it.
For more information, read Amazon S3 > Advanced Options.
maxCacheSpacePct
Number
Sets the maximum percent of total available cache space to use when possible. The value, in percent, must be between 1 and 100.
Example 100
Examples of Different Configurations for credentialType
Project data credentials{
"credentialType": "PROJECT_DATA_CRED"
}
{
"credentialType": "PROJECT_DATA_CRED",
"assumedRoleARN": "arn:aws:iam::123456789012:root"
}
{
"credentialType": "ACCESS_KEY",
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}
Arctic (Preview)
Dremio source type: ARCTIC
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Arctic, see Dremio Sonar for additional information.
"config": {
"awsRootPath": "/your-s3-bucket-name",
"asyncEnabled": true,
"isCachingEnabled": true,
"maxCacheSpacePct": 100,
"defaultCtasFormat": "ICEBERG",
"credentialType": [
"PROJECT_DATA_CRED",
"ACCESS_KEY"
]
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"arcticCatalogId": "ab12cdef-a123-b456-ab1234-cdef5678ghij"
},
Parameters
awsRootPath
String
The root path of the Amazon S3 bucket.
Example
/your-s3-bucket-name
asyncEnabled
Boolean
Option to enable asynchronous access, when this functionality is available in your Arctic source. This option enables cloud caching for the source to support simultaneous actions such as adding and editing. Set to true
to enable asynchronous access.
isCachingEnabled
Boolean
Option to enable local caching, when this functionality is available in your Arctic source. Set to true
to enable local caching.
maxCacheSpacePct
Number
Sets the maximum percent of total available cache space to use when possible. The value, in percent, must be between 1 and 100.
Example 100
defaultCtasFormat
String
Enables you to choose the default format for tables you create in Dremio, either ICEBERG
or PARQUET
.
Enum
ICEBERG
,
PARQUET
credentialType
String
Specifies the type of credential to enable Dremio to connect to the source.
Enum
ACCESS_KEY
,
PROJECT_DATA_CRED
assumedRoleARN
String
Optional
The ARN of the IAM role to assume.
Example
arn:aws:iam::123456789012:root
awsAccessKey
String
The AWS access key.
Example
AKIAIOSFODNN7EXAMPLE
awsAccessSecret
String
The AWS access secret.
Example
wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
arcticCatalogId
String
The unique ID of the Arctic catalog you are aceessing.
Example
ab12cdef-a123-b456-ab1234-cdef5678ghij
Examples of Different Configurations for credentialType
Project data credentials{
"credentialType": "PROJECT_DATA_CRED"
}
{
"credentialType": "PROJECT_DATA_CRED",
"assumedRoleARN": "arn:aws:iam::123456789012:root"
}
{
"credentialType": "ACCESS_KEY",
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}
AWS Glue Data Catalog
Dremio source type: AWSGLUE
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for AWS Glue, see Adding an AWS Glue Data Catalog for additional information.
"config": {
"credentialType": [
"PROJECT_DATA_CRED",
"ACCESS_KEY"
]
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"regionNameSelection": "us-east-1",
"secure": false,
"enableAsync": true,
"propertyList": [
{
"name": "hive.metastore.warehouse.dir",
"value": "s3://bucket-name/folder"
},
],
"isCachingEnabled": true,
"maxCacheSpacePct": 100
}
Parameters
credentialType
String
Specifies the type of credential to enable Dremio to connect to the source. Must be either ACCESS_KEY
or PROJECT_DATA_CRED
.
Enum
ACCESS_KEY
,
PROJECT_DATA_CRED
assumedRoleARN
String
Optional
The ARN of the IAM role to assume.
Example
arn:aws:iam::123456789012:root
awsAccessKey
String
The AWS access key.
Example
AKIAIOSFODNN7EXAMPLE
awsAccessSecret
String
The AWS access secret.
Example
wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
regionNameSelection
String
Sets the AWS Region where the Glue Data Catalog is located.
Example
us-east-1, eu-west-2
secure
Boolean
Option to enable a secure connection via SSL encryption between the Glue Data Catalog and Dremio. Set to true
to enable SSL encryption.
enableAsync
Boolean
Option to enable asynchronous access, which allows cloud caching for the source to support simultaneous actions such as adding and editing. Set to true
to enable asynchronous access.
For more information, read AWS Glue Data Catalog > Advanced Options.
propertyList
Array
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example "name": "hive.metastore.warehouse.dir", "value": "s3://bucket-name/folder"
isCachingEnabled
Boolean
Option to enable local caching. When local caching is possible, set this parameter to true
to enable it.
For more information, read AWS Glue Data Catalog > Advanced Options.
maxCacheSpacePct
Number
Sets the maximum percent of total available cache space to use when possible. The value, in percent, must be between 1 and 100.
Example 100
Examples of Different Configurations for credentialType
Project data credentials{
"credentialType": "PROJECT_DATA_CRED"
}
{
"credentialType": "PROJECT_DATA_CRED",
"assumedRoleARN": "arn:aws:iam::123456789012:root"
}
{
"credentialType": "ACCESS_KEY",
"assumedRoleARN": "arn:aws:iam::123456789012:root",
"awsAccessKey": "AKIAIOSFODNN7EXAMPLE",
"awsAccessSecret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}
Microsoft SQL Server
Dremio source type: MSSQL
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for SQL Server, see SQL Server Configuration for additional information.
"config": {
"authenticationType": [
"ANONYMOUS",
"MASTER"
]
"username": "myUsername",
"password": "myPassword".
"hostname": "localhost",
"port": "1433",
"database": "my_database",
"useSsl": true,
"showOnlyConnectionDatabase": false,
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"enableServerVerification": true,
"hostnameOverride": "localhost",
"propertyList": [
{
"name": "applicationIntent",
"value": "ReadWrite"
}
]
}
Parameters
authenticationType
String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed and MASTER when credentials from a master database user is required.
Enum
ANONYMOUS
,
MASTER
username
String
The SQL Server user name.
Example
myUsername
password
String
The SQL Server password.
Example
myPassword
hostname
String
The SQL Server host name.
Example
localhost
port
String
The SQL Server port number. The coomonly used default port numbers for SQL Server are 1433
and 1434
.
Example
1433
database
String
Optional
The database name.
Example
my_database
useSsl
Boolean
Option to enable SSL encryption. Set to true
to enable this option.
showOnlyConnectionDatabase
Boolean
Option to show only the initial database used for connecting. Set to true
to enable this option.
fetchSize
Number
The number of records to fetch per request. Set to 0
to enable Dremio to automatically set the size.
Example 200
maxIdleConns
Number
Sets the maximum number of idle connections that you want to keep.
Example 8
idleTimeSec
Number
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example 60
enableServerVerification
Boolean
Option to verify the server certificate. Set to true
to enable this option.
hostnameOverride
String
This parameter can be used when the SSL/TLS server certificate contains a distinguished name. You can leave this blank if the hostname in the certificate matches the server name.
Example
localhost
propertyList
Array
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example "name": "applicationIntent", "value": "ReadWrite"
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
MySQL
Dremio source type: MYSQL
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for MySQL, see MySQL Configuration for additional information.
"config": {
"authenticationType": [
"ANONYMOUS",
"MASTER",
]
"username": "myUsername",
"password": "myPassword"
"hostname": "localhost",
"port": "3306",
"netWriteTimeout": 60,
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"propertyList": [
{
"name": "DataSource",
"value": "sales2021"
}
]
}
Parameters
authenticationType
String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed and MASTER when credentials from a master database user is required.
Enum
ANONYMOUS
,
MASTER
username
String
The MySQL user name.
Example
myUsername
password
String
The MySQL password.
Example
myPassword
hostname
String
The MySQL host name.
Example
localhost
port
String
The MySQL port number. The coomonly used default port number for MySQL is 3306
Example
3306
netWriteTimeout
Number
The amount of time, in seconds, to wait for data from the server before aborting the connection.
Example 60
fetchSize
Number
The number of records to fetch per request. Set to 0
to enable Dremio to automatically set the size.
Example 200
maxIdleConns
Number
Sets the maximum number of idle connections that you want to keep.
Example 8
idleTimeSec
Number
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example 60
propertyList
Array
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example "name": "DataSource", "value": "sales2021"
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
Oracle
Dremio source type: ORACLE
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Oracle, see Oracle Configuration for additional information.
"config": {
"authenticationType": [
"ANONYMOUS",
"MASTER"
]
"username": "myUsername",
"password": "myPassword",
"hostname": "localhost",
"port": "1521",
"instance": "sales",
"useSsl": true,
"nativeEncryption": "ACCEPTED",
"useTimezoneAsRegion": true,
"includeSynonyms": false,
"mapDateToTimestamp": true,
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"useLdap": false,
"bindDN": String,
"sslServerCertDN": String,
"propertyList": [
{
"name": "connect_timeout",
"value": "0"
}
]
}
Parameters
authenticationType
String
Specifies the type of authentication to enable Dremio to connect to the source. Use `ANONYMOUS` when no authentication is needed, `MASTER` when either credentials from a master database user or a secret resource URL is required, or `KERBEROS` if Kerberos is configured.
Enum
ANONYMOUS
,
MASTER
,
KERBEROS
username
String
The Oracle user name.
Example
myUsername
password
String
The Oracle password.
Example
myPassword
secretResourceUrl
String
The secret resource URL.
useKerberos
Boolean
Set to true
if you are using Kerberos.
hostname
String
The Oracle host name.
Example
localhost
port
String
The Oracle port number. The coomonly used default port numbers for Oracle are 1521
and 1830
.
Example
1521
instance
String
Optional
The Oracle service name.
Example
sales
useSsl
Boolean
Option to enable SSL encryption. Set to true
to enable this option.
nativeEncryption
Boolean
Sets the Oracle native encryption. This parameter can only be used if useSsl
is set to false
.
Enum
REJECTED
,
ACCEPTED
,
REQUESTED
,
REQUIRED
useTimezoneAsRegion
Boolean
Option to use the timezone to set the connection region. Set to true
to enable this option.
includeSynonyms
Boolean
Option to include synonyms as datasets. Set to true
to enable this option.
mapDateToTimestamp
Boolean
Option to display Oracle DATE
columns as a TIMESTAMP
. Set to true
to enable this option.
fetchSize
Number
The number of records to fetch per request. Set to 0
to enable Dremio to automatically set the size.
Example 200
maxIdleConns
Number
Sets the maximum number of idle connections that you want to keep.
Example 8
idleTimeSec
Number
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example 60
useLdap
Boolean
Option to use LDAP naming services. Set to true
to enable this option.
bindDN
String
Sets the LDAP hostname resolution when the useLdap
parameter is set to true
. You can omit this parameter if useLdap
is set to false
.
sslServerCertDN
String
Use when you have a SSL/TLS server certificate distinguished name. Otherwise, you can leave this parameter blank to disable DN match.
propertyList
Array
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example "name": "connect_timeout", "value": "0"
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"secretResourceUrl": "String"
}
{
"credentialType": "KERBEROS",
"username": null,
"password": null,
"useKerberos": true
}
PostgreSQL
Dremio source type: POSTGRES
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for PostgreSQL, see PostgreSQL Configuration for additional information.
"config": {
"authenticationType": [
"ANONYMOUS",
"MASTER"
]
"username": "myUsername",
"password": "myPassword",
"hostname": "localhost",
"port": "5432",
"databaseName": "sales_2021",
"useSsl": false,
"fetchSize": 200,
"maxIdleConns": 8,
"idleTimeSec": 60,
"encryptionValidationMode": "CERTIFICATE_AND_HOSTNAME_VALIDATION",
"propertyList": [
{
"name": "connect_timeout",
"value": "10&application_name=myapp"
}
]
}
Parameters
authenticationType
String
Specifies the type of authentication to enable Dremio to connect to the source. Use ANONYMOUS when no authentication is needed and MASTER when either credentials from a master database user or a secret resource URL is required.
Enum
ANONYMOUS
,
MASTER
username
String
The PostgreSQL user name.
Example
myUsername
password
String
The PostgreSQL password.
Example
myPassword
secretResourceUrl
String
The secret resource URL.
hostname
String
The PostgreSQL host name.
Example
localhost
port
String
The PostgreSQL port number. The coomonly used default port number for PostgreSQL is 5432
.
Example
5432
databaseName
String
Optional
The database name.
Example
sales_2021
useSsl
Boolean
Option to enable SSL encryption. Set to true
to enable this option.
fetchSize
Number
The number of records to fetch per request. Set to 0
to enable Dremio to automatically set the size.
Example 200
maxIdleConns
Number
Sets the maximum number of idle connections that you want to keep.
Example 8
idleTimeSec
Number
Sets the idle time, in seconds, before a connection is evaluated for closure.
Example 60
encryptionValidationMode
String
When encryption is enabled, you can set the validation mode to one of the available presets.
Enum
CERTIFICATE_AND_HOSTNAME_VALIDATION
,
CERTIFICATE_ONLY_VALIDATION
,
NO_VALIDATION
propertyList
Array
An array of name/value pairs defining optional connection properties. Use a comma separated list to specify multiple name value pairs.
Example "name": "connect_timeout", "value": "10&application_name=myapp"
Examples of Different Configurations for authenticationType
No authentication required{
"authenticationType": "ANONYMOUS"
}
{
"authenticationType": "MASTER",
"username": "myUsername",
"password": "myPassword"
}
Snowflake
Dremio source type: SNOWFLAKE
. This type
determines the config
parameters that display as specified in the Source Attributes. If you are not familiar with the following parameters that are available for Snowflake, see Configuring Snowflake as a Source for additional information.
"config": {
"hostname": "https://locator_id.snowflakecomputing.com",
"username": "myUsername",
"password": "myPassword",
[ "database": "myDatabase" ],
[ "role": "PUBLIC" ],
[ "schema": "mySchema" ],
[ "warehouse": "myWarehouse" ]
}
Parameters
hostname
String
The Snowflake host name.
Example
https://locator_id.snowflakecomputing.com
username
String
The Snowflake user name.
Example
myUsername
password
String
The Snowflake password.
Example
myPassword
database
String
Optional
Specifies the default database to use.
Example
myDatabase
role
String
Optional
Specifies the default access-control role to use. For more information, read System-Defined Roles.
schema
String
Optional
Specifies the default schema to use.
Example
mySchema
warehouse
String
Optional
Specifies the warehouse that will provide resources for executing DML statements and queries.
Example
myWarehouse