Welcome to Dremio Cloud Documentation - Dremio > > > >

Source Configuration

Dremio supports a variety of source types, and the available parameters of the config object differ among the source types. This page provides examples of the config object for each supported source type and descriptions of the config object’s parameters that are available for each source type.

note:

The source object contains the same parameters for all source types except for the parameters of the config object. Read the Source page for information about the parameters all source types have in common.

Amazon Redshift

Amazon Redshift Source config Object

{
  "config": {
    "connectionString": "jdbc:redshift://testing-redshift-cluster.r57mkjay4utq.us-west-2.redshift.amazonaws.com:5439/support",
    "username": "redshift-owner",
    "password": "$DREMIO_EXISTING_VALUE$",
    "authenticationType": "MASTER",
    "propertyList": [
      {
        "name": "ApplicationName",
        "value": "testingDashboard"
      }
    ],
    "fetchSize": 200,
    "maxIdleConns": 8,
    "idleTimeSec": 60,
    "queryTimeoutSec": 0
  }
}

Amazon Redshift config Parameters

JDBC connection string for the Redshift database.

Example jdbc:redshift://testing-redshift-cluster.r57mkjay4utq.us-west-2.redshift.amazonaws.com:5439/support

Username for authentication with master credentials or a secret resource URL.

Example redshift-owner

Password for authentication with master credentials. For privacy, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.

URL for authentication with a the secret resource URL. For privacy, Dremio returns the secretResourceUrl value as $DREMIO_EXISTING_VALUE$ in API responses.

Type of authentication for Dremio to use to connect to the source.

ANONYMOUS: No authentication is needed.
MASTER: Use credentials from a master database user or use a secret resource URL.
AWS_PROFILE: Use an AWS source.

Example MASTER

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "ApplicationName", "value": "testingDashboard" } ]

Number of records to fetch at one time. Default is 200. If set to 0, Dremio automatically decides how many records to fetch.

Example 200

Maximum number of connections that can be idle at any given time. Default is 8.

Example 8

Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.

Example 60

Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.

Example 0

propertyList

Name of the connection property.

Example ApplicationName

Value for the connection property.

Example testingDashboard

Amazon S3

Amazon S3 Source config Object

{
  "config": {
    "accessKey": "EXAMPLE78HT89VS4YJEL",
    "accessSecret": "$DREMIO_EXISTING_VALUE$",
    "secure": true,
    "externalBucketList": [
      "external_bucket_1",
      "external_bucket_2"
    ],
    "propertyList": [
      {
        "name": "fs.s3a.proxy.host",
        "value": "proxyHost.example.com"
      }
    ],
    "rootPath": "/",
    "enableAsync": true,
    "compatibilityMode": false,
    "isCachingEnabled": true,
    "maxCacheSpacePct": 100,
    "whitelistedBuckets": [
      "archive.dremio.com",
      "logs_east-1",
      "logs_west-1"
    ],
    "requesterPays": false,
    "enableFileStatusCheck": true,
    "defaultCtasFormat": "ICEBERG",
    "isPartitionInferenceEnabled": false,
    "credentialType": "ACCESS_KEY"
  }
}

Amazon S3 config Parameters

AWS access key.

Example EXAMPLE78HT89VS4YJEL

AWS access secret. For privacy, Dremio returns the accessSecret value as $DREMIO_EXISTING_VALUE$ in API responses.

Amazon Resource Name (ARN) for the AWS Identity and Access Management (IAM) role to assume.

Example arn:aws:iam::594632595346:role/OrganizationAccountAccessRole

AWS profile name. If you do not provide a profile name, Dremio uses the default profile.

Example example-profile

If set to enable a secure connection with SSL encryption between the S3 bucket and Dremio, the value is true (default). Otherwise, the value is false.

Example true

List of external buckets that are not included with the specified AWS account credentials.

Example [ "external_bucket_1", "external_bucket_2" ]

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "fs.s3a.proxy.host", "value": "proxyHost.example.com" } ]

Root path of the S3 bucket. Default is /.

Example /

To enable asynchronous access for the source and allow cloud caching to support simultaneous actions such as adding and editing, set to true (default). Otherwise, set to false.

Example true

To enable the use of S3-compatible storage, set to true. Otherwise, set to false (default).

Example false

To enable local caching, set to true (default). Otherwise, set to false.

Example true

Maximum percentage of the total available cache space to use on any single executor node. Used only when isCachingEnabled is set to true. Minimum is 1. Maximum is 100 (default).

Example 100

List of approved S3 buckets to include in the source. Omit to include all buckets that are available in the source.
If you omit the whitelistedBuckets object in a PUT request, Dremio updates the source to include all available buckets. To keep existing approved buckets while making other updates, duplicate the existing whitelistedBuckets array in the PUT request.

Example [ "archive.dremio.com", "logs_east-1", "logs_west-1" ]

If the requester (instead of the bucket owner) pays the cost of the S3 request and the data downloaded from the S3 bucket, set to true. Otherwise, set to false (default).

Example false

To allow Dremio to check whether a file exists in the S3 bucket before handling errors gracefully, set to true (default). Otherwise, set to false. If no files are missing from the S3 bucket or a file's access permissions have not changed, set to false to reduce communication with the S3 bucket.

Example true

Default format for the tables you create in Dremio. Default is ICEBERG.

Enum ICEBERG , PARQUET

Example ICEBERG

To enable partition column inference, set to true. Otherwise, set to false (default). For more information, read Partition Column Inference.

Example false

Type of credential for Dremio to use to connect to the source.

Enum NONE , EC2_METADATA , ACCESS_KEY , AWS_PROFILE

Example ACCESS_KEY

propertyList

Name of the connection property.

Example fs.s3a.proxy.host

Value for the connection property.

Example proxyHost.example.com

AWS Glue Data Catalog

AWS Glue Data Catalog Source config Object

{
  "config": {
    "regionNameSelection": "US_EAST_1",
    "accessKey": "EXAMPLE78HT89VS4YJEL",
    "accessSecret": "$DREMIO_EXISTING_VALUE$",
    "secure": true,
    "enableAsync": true,
    "isCachingEnabled": true,
    "maxCacheSpacePct": 100,
    "assumedRoleARN": "arn:aws:iam::594632595346:role/OrganizationAccountAccessRole",
    "lakeFormationEnableAccessPermissions": false,
    "defaultCtasFormat": "ICEBERG",
    "propertyList": [
      {
        "name": "hive.metastore.warehouse.dir",
        "value": "s3a://example.com/buckets/glue_location"
      }
    ],
    "credentialType": "ACCESS_KEY"
  }
}

Amazon Glue Data Catalog config Parameters

AWS region where the Glue Data Catalog is located.

Example US_EAST_1

AWS access key.

Example EXAMPLE78HT89VS4YJEL

AWS access secret. For privacy, Dremio returns the accessSecret value as $DREMIO_EXISTING_VALUE$ in API responses.

AWS profile name. If you do not provide a profile name, Dremio uses the default profile.

Example example-profile

To enable a secure connection with SSL encryption between the Glue Data Catalog and Dremio, set to true (default). Otherwise, set to false.

Example true

To enable asynchronous access for the source and allow cloud caching to support simultaneous actions such as adding and editing, set to true (default). Otherwise, set to false.

Example true

To enable local caching, set to true (default). Otherwise, set to false.

Example true

Maximum percentage of the total available cache space to use on any single executor node. Used only when isCachingEnabled is set to true. Minimum is 1. Maximum is 100 (default).

Example 100

Amazon Resource Name (ARN) for the AWS Identity and Access Management (IAM) role to assume.

Example arn:aws:iam::594632595346:role/OrganizationAccountAccessRole

To enforce Lake Formation access permissions on datasets so that Dremio confirms whether users have the required permissions to perform queries, set to true. Otherwise, set to false (default).

Example false

Default format for the tables you create in Dremio. Default is ICEBERG.

Enum ICEBERG , PARQUET

Example ICEBERG

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "hive.metastore.warehouse.dir", "value": "s3a://example.com/buckets/glue_location" } ]

Type of credential for Dremio to use to connect to the source.

Enum NONE , EC2_METADATA , ACCESS_KEY , AWS_PROFILE

Example ACCESS_KEY

propertyList

Name of the connection property.

Example hive.metastore.warehouse.dir

Value for the connection property.

Example s3a://example.com/buckets/glue_location

Azure Data Lake Storage Gen1

Azure Data Lake Storage Gen1 Source config Object

{
  "config": {
    "accountName": "datalake1",
    "clientId": "98c23728-3971-4de3-a15f-61f483f51eca",
    "clientKeyRefreshUrl": "https://login.microsoftonline.com/88ba5c34-c9af-11ed-b49f-325096b39f47/oauth2/token",
    "clientKeyPassword": "$DREMIO_EXISTING_VALUE$",
    "rootPath": "/",
    "enableAsync": true,
    "isCachingEnabled": true,
    "maxCacheSpacePct": 100,
    "defaultCtasFormat": "ICEBERG",
    "propertyList": [
      {
        "name": "dfs.adls.oauth2.access.token.provider.type",
        "value": "ClientCredential"
      }
    ],
    "isPartitionInferenceEnabled": false
  }
}

Azure Data Lake Storage Gen1 config Parameters

Name for the Azure Data Lake Storage Gen1 account.

Example datalake1

Unique identifier that Azure assigned to the application when it was registered.

Example 98c23728-3971-4de3-a15f-61f483f51eca

Azure Active Directory OAuth 2.0 Token Endpoint for the registered application.

Example https://login.microsoftonline.com/88ba5c34-c9af-11ed-b49f-325096b39f47/oauth2/token

Password value for the registered application. For privacy, Dremio returns the clientKeyPassword value as $DREMIO_EXISTING_VALUE$ in API responses.

Root path of the Azure Data Lake Storage Gen1 source. Default is /.

Example /

To enable asynchronous access for the source and allow cloud caching to support simultaneous actions such as adding and editing, set to true (default). Otherwise, set to false.

Example true

To enable local caching, set to true (default). Otherwise, set to false.

Example true

Maximum percentage of the total available cache space to use on any single executor node. Used only when isCachingEnabled is set to true. Minimum is 1. Maximum is 100 (default).

Example 100

Default format for the tables you create in Dremio. Default is ICEBERG.

Enum ICEBERG , PARQUET

Example ICEBERG

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "dfs.adls.oauth2.access.token.provider.type", "value": "ClientCredential" } ]

To enable partition column inference, set to true. Otherwise, set to false (default). For more information, read Partition Column Inference.

Example false

propertyList

Name of the connection property.

Example dfs.adls.oauth2.access.token.provider.type

Value for the connection property.

Example ClientCredential

Azure Storage

Azure Storage Source config Object

{
  "config": {
    "accountKind": "STORAGE_V2",
    "accountName": "azurestoragev2accountname",
    "accessKey": "$DREMIO_EXISTING_VALUE$",
    "rootPath": "/",
    "enableSSL": true,
    "enableAsync": true,
    "credentialsType": "ACCESS_KEY",
    "isCachingEnabled": true,
    "maxCacheSpacePct": 100,
    "defaultCtasFormat": "ICEBERG",
    "propertyList": [
      {
        "name": "dremio.azure.mode",
        "value": "STORAGE_V2"
      }
    ],
    "isPartitionInferenceEnabled": false
  }
}

Azure Storage config Parameters

Type of Azure Storage account.

Enum STORAGE_V1 , STORAGE_V2

Example STORAGE_V2

Name of the Azure Storage account.

Example azurestoragev2accountname

Azure access key. For privacy, Dremio returns the accessKey value as $DREMIO_EXISTING_VALUE$ in API responses.

Azure client secret. For privacy, Dremio returns the clientSecret value as $DREMIO_EXISTING_VALUE$ in API responses.

Root path of the Azure Storage source. Default is /.

Example /

To enable a secure connection with SSL encryption between the Azure Storage source and Dremio, set to true (default). Otherwise, set to false.

Example true

To enable asynchronous access for the source and allow cloud caching to support simultaneous actions such as adding and editing, set to true (default). Otherwise, set to false.

Example true

Type of credential for Dremio to use to connect to the source.

Enum ACCESS_KEY , AZURE_ACTIVE_DIRECTORY

Example ACCESS_KEY

To enable local caching, set to true (default). Otherwise, set to false.

Example true

Maximum percentage of the total available cache space to use on any single executor node. Used only when isCachingEnabled is set to true. Minimum is 1. Maximum is 100 (default).

Example 100

Default format for the tables you create in Dremio. Default is ICEBERG.

Enum ICEBERG , PARQUET

Example ICEBERG

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "dremio.azure.mode", "value": "STORAGE_V2" } ]

To enable partition column inference, set to true. Otherwise, set to false (default). For more information, read Partition Column Inference.

Example false

propertyList

Name of the connection property.

Example dremio.azure.mode

Value for the connection property.

Example STORAGE_V2

Dremio-to-Dremio Connector

Dremio-to-Dremio Connector Source config Object

{
  "config": {
    "hostType": "DIRECT",
    "hostname": "35.219.202.153",
    "port": "31010",
    "username": "exampleuser",
    "password": "$DREMIO_EXISTING_VALUE$",
    "useSsl": false,
    "userImpersonation": false,
    "maxIdleConns": 8,
    "idleTimeSec": 60,
    "queryTimeoutSec": 0,
    "propertyList": [
      {
        "name": "disableCertificateVerification",
        "value": "false"
      }
    ]
  }
}

Dremio-to-Dremio Connector config Parameters

Connection method to use. To connect directly to a coordinator node of the cluster, DIRECT. To connect to an external ZooKeeper instance that coordinates the nodes of the cluster, ZOOKEEPER.

Enum DIRECT , ZOOKEEPER

Example DIRECT

Hostname of the coordinator node or ZooKeeper instance.

Example 35.219.202.153

Port number of the coordinator node or ZooKeeper instance. Default is 31010.

Example 31010

Username for authentication.

Example exampleuser

Password for authentication. For privacy, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.

If the source cluster is configured to use TLS for connections, set to true. Otherwise, set to false.

Example false

To allow users to run queries on the source cluster under their own user IDs rather than the user ID for the account used to authenticate, set to true. Otherwise, set to false (default). If set to true, the source cluster must be configured to allow inbound impersonation. For more information, read Inbound Impersonation.

Example false

Maximum number of connections that can be idle at any given time. Default is 8.

Example 8

Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.

Example 60

Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.

Example 0

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "disableCertificateVerification", "value": "false" } ]

propertyList

Name of the connection property.

Example disableCertificateVerification

Value for the connection property.

Example false

Elasticsearch

Elasticsearch Source config Object

{
  "config": {
    "scriptsEnabled": true,
    "showHiddenIndices": false,
    "showIdColumn": false,
    "readTimeoutMillis": 60000,
    "scrollTimeoutMillis": 300000,
    "usePainless": true,
    "scrollSize": 4000,
    "allowPushdownOnNormalizedOrAnalyzedFields": false,
    "warnOnRowCountMismatch": false,
    "encryptionValidationMode": "CERTIFICATE_AND_HOSTNAME_VALIDATION",
    "forceDoublePrecision": false,
    "hostList": [
      {
        "hostname": "172.25.0.208",
        "port": 9200
      }
    ],
    "authenticationType": "ANONYMOUS",
    "sslEnabled": false,
    "useWhitelist": false
  }
}

Elasticsearch config Parameters

If Dremio should use script pushdowns, set to true (default). Otherwise, set to false.

Example true

To show hidden indices, set to true. Otherwise, set to false (default).

Example false

To show the elasticsearch ID column, set to true. Otherwise, set to false (default).

Example false

Time to wait to read data from the source, in milliseconds. Default is 60000.

Example 60000

Time to wait for each scroll request, in milliseconds. Default is 300000.

Example 300000

To use the Painless scripting language when connecting to Elasticsearch 5.0+, set to true (default). Otherwise, set to false.

Example true

Scroll size for Dremio's Elasticsearch requests. Default is 4000. The scrollSize value must be less than or equal to the setting for index.max_result_window in Elasticsearch.

Example 4000

To enable pushdown filters and aggregations on analyzed text fields and normalized keyword fields, set to true. Otherwise, set to false (default). May produce unexpected results when enabled.

Example false

If Dremio should warn the user when a query returns fewer Elasticsearch records than expected instead of failing the query, set to true. Otherwise, set to false (default).

Example false

Method to use to validate data encryption for the source.

Enum CERTIFICATE_AND_HOSTNAME_VALIDATION , CERTIFICATE_ONLY_VALIDATION , NO_VALIDATION

Example CERTIFICATE_AND_HOSTNAME_VALIDATION

To force precision for double values, set to true. Otherwise, set to false (default).

Example false

Information about Elasticsearch hosts. Each object in the hostList includes the hostname and the corresponding port for the host.

Example [ { "hostname": "172.25.0.208", "port": 9200 } ]

Type of authentication for Dremio to use to connect to the source.

Enum ANONYMOUS , MASTER

Example ANONYMOUS

Username for authenticating with master credentials.

Example exampleuser

Password for authenticating with master credentials. For privacy, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.

To enable a secure connection with SSL encryption between Elasticsearch and Dremio, set to true. Otherwise, set to false (default).

Example false

To query only the hosts specified in the hostList, set to true. Otherwise, set to false (default).

Example false

hostList

Name of the host to use to connect to the Elasticsearch source.

Example 172.25.0.208

Port to use with the specified hostname to connect to the Elasticsearch source. Default is 9200.

Example 9200

Google Cloud Storage

Google Cloud Storage Source config Object

{
  "config": {
    "projectId": "dremio-4321",
    "authMode": "AUTO",
    "rootPath": "/",
    "asyncEnabled": true,
    "cachingEnable": true,
    "cachePercent": 70,
    "privateKeyId": "",
    "clientEmail": "",
    "clientId": "",
    "defaultCtasFormat": "ICEBERG",
    "propertyList": [
      {
        "name": "dremio.gcs.clientEmail",
        "value": "exampleuser@dremio-project.iam.gserviceaccount.com"
      }
    ],
    "isPartitionInferenceEnabled": false
  }
}

Google Cloud Storage config Parameters

Project ID for the Google Cloud Storage project.

Example dremio-4321

Type of authentication for Dremio to use to connect to the source.

Enum AUTO , SERVICE_ACCOUNT_KEYS

Example AUTO

Root path of the Google Cloud Storage source. Default is /.

Example /

To enable asynchronous access for the source and allow cloud caching to support simultaneous actions such as adding and editing, set to true (default). Otherwise, set to false.

Example true

To enable local caching, set to true (default). Otherwise, set to false.

Example true

Maximum percentage of the total available cache space to use on any single executor node. Used only when cachingEnable is set to true. Default is 70. Minimum is 1. Maximum is 100.

Example 70

Service account key ID for the Google Cloud Storage service account. Required only if authenticating with the Service Account Keys method.

Example f302b86d0247be19393bb66ea142887fc9621360

Service account key for the Google Cloud Storage service account. Required only if authenticating with the Service Account Keys method. For privacy, Dremio returns the privateKey value as $DREMIO_EXISTING_VALUE$ in API responses.

Email address associated with the Google Cloud Storage service account. Required only if authenticating with the Service Account Keys method.

Example exampleuser@dremio-project.iam.gserviceaccount.com

Client ID for the key pair. Required only if authenticating with the Service Account Keys method.

Example 243769857672272684657

Default format for the tables you create in Dremio. Default is ICEBERG.

Enum ICEBERG , PARQUET

Example ICEBERG

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "dremio.gcs.clientEmail", "value": "exampleuser@dremio-project.iam.gserviceaccount.com" } ]

To enable partition column inference, set to true. Otherwise, set to false (default). For more information, read Partition Column Inference.

Example false

propertyList

Name of the connection property.

Example dremio.gcs.clientEmail

Value for the connection property.

Example exampleuser@dremio-project.iam.gserviceaccount.com

Hadoop Distributed File System (HDFS)

HDFS Source config Object

{
  "config": {
    "hostname": "172.23.0.208",
    "port": 8020,
    "enableImpersonation": false,
    "rootPath": "/",
    "shortCircuitFlag": "SYSTEM",
    "enableAsync": true,
    "isCachingEnabled": false,
    "maxCacheSpacePct": 100,
    "defaultCtasFormat": "ICEBERG",
    "isPartitionInferenceEnabled": false,
    "impersonationUserDelegationMode": "AS_IS",
    "propertyList": [
      {
        "name": "dfs.client.socket-timeout",
        "value": "120000"
      }
    ],
    "vdsAccessDelegationEnabled": true
  }
}

HDFS config Parameters

HDFS NameNode hostname to use to connect to the HDFS source.

Example 172.23.0.208

HDFS NameNode port to use to connect to the HDFS source. Default is 8020.

Example 8020

To enable user-specific file access permissions by turning on impersonation in HDFS sources, set to true. Otherwise, set to false (default).

Example false

Root path of the HDFS source. Default is /.

Example /

Implementation status for short-circuit local reads. Default is SYSTEM.

Enum ENABLED , DISABLED , SYSTEM

Example SYSTEM

Socket path to use, if shortCircuitFlag is enabled.

Example /var/lib/hadoop-hdfs/dn-socket

To enable asynchronous access for the source and allow cloud caching to support simultaneous actions such as adding and editing, set to true (default). Otherwise, set to false.

Example true

To enable local caching, set to true. Otherwise, set to false (default).

Example false

Maximum percentage of the total available cache space to use on any single executor node. Used only when cachingEnable is set to true. Minimum is 1. Maximum is 100 (default).

Example 100

Default format for the tables you create in Dremio. Default is ICEBERG.

Enum ICEBERG , PARQUET

Example ICEBERG

To enable partition column inference, set to true. Otherwise, set to false (default). For more information, read Partition Column Inference.

Example false

Capitalization method to use for impersonation usernames. Default is AS_IS.

Enum AS_IS , LOWERCASE , UPPERCASE

Example AS_IS

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "dfs.client.socket-timeout", "value": "120000" } ]

To use the view owner as the impersonated username when enableImpersonation is enabled, set to true (default). To use the user who submits the query as the impersonated username, set to false.

Example true

propertyList

Name of the connection property.

Example dfs.client.socket-timeout

Value for the connection property.

Example 120000

Hive 2.x

Hive 2.x Source config Object

{
  "config": {
    "hostname": "172.23.0.208",
    "port": 9083,
    "enableSasl": false,
    "propertyList": [
      {
        "name": "hive.server2.enable.doAs",
        "value": "false"
      }
    ],
    "enableAsync": true,
    "isCachingEnabledForS3AndAzureStorage": true,
    "isCachingEnabledForHDFS": true,
    "maxCacheSpacePct": 100,
    "defaultCtasFormat": "ICEBERG",
    "authType": "STORAGE",
    "rangerServiceName": "",
    "rangerHostURL": "",
    "impersonationUserDelegationMode": "AS_IS",
    "vdsAccessDelegationEnabled": true
  }
}

Hive 2.x config Parameters

IP address for the Hive metastore host.

Example 172.23.0.208

Port to use with the specified hostname to connect to the Hive source. Default is 9083.

Example 9083

To enable Simple Authentication and Security Layer (SASL), set to true (default). Otherwise, set to false.

Example false

Name of the Kerberos principal identity for SASL. Required only if enableSasl is true.

Example primary/instance@REALM

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "hive.server2.enable.doAs", "value": "false" } ]

To enable asynchronous access for the source and allow cloud caching to support simultaneous actions such as adding and editing, set to true (default). Otherwise, set to false.

Example true

To enable local caching for Amazon S3 and Azure Storage, set to true (default). Otherwise, set to false.

Example true

To enable local caching for Hadoop Distributed File System (HDFS), set to true. Otherwise, set to false (default).

Example false

Maximum percentage of the total available cache space to use on any single executor node. Minimum is 1. Maximum is 100 (default).

Example 100

Default format for the tables you create in Dremio. Default is ICEBERG.

Enum ICEBERG , PARQUET

Example ICEBERG

Type of authentication for Dremio to use to connect to the source. Default is STORAGE.

Enum STORAGE , SQL , RANGER

Example STORAGE

The Ranger service name, if you are using Ranger-based authentication.

Example hivedev

The Ranger host URL, if you are using Ranger-based authentication.

Example http://example.com/6080

Capitalization method to use for impersonation usernames. Default is AS_IS.

Enum AS_IS , LOWERCASE , UPPERCASE

Example AS_IS

To use the view owner as the impersonated username when enableImpersonation is enabled, set to true (default). To use the user who submits the query as the impersonated username, set to false.

Example true

propertyList

Name of the connection property.

Example hive.server2.enable.doAs

Value for the connection property.

Example false

Hive 3.x

Hive 3.x Source config Object

{
  "config": {
    "hostname": "172.23.0.177",
    "port": 9084,
    "enableSasl": false,
    "propertyList": [
      {
        "name": "fs.s3a.aws.credentials.provider",
        "value": "org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider"
      }
    ],
    "enableAsync": true,
    "isCachingEnabledForS3AndAzureStorage": true,
    "isCachingEnabledForHDFS": false,
    "maxCacheSpacePct": 100,
    "defaultCtasFormat": "ICEBERG",
    "authType": "STORAGE",
    "rangerServiceName": "",
    "rangerHostURL": "",
    "impersonationUserDelegationMode": "AS_IS",
    "vdsAccessDelegationEnabled": true
  }
}

Hive 3.x config Parameters

IP address for the Hive metastore host.

Example 172.23.0.177

Port to use with the specified hostname to connect to the Hive source. Default is 9083.

Example 9084

To enable Simple Authentication and Security Layer (SASL), set to true (default). Otherwise, set to false.

Example false

Name of the Kerberos principal identity for SASL. Required only if enableSasl is true.

Example primary/instance@REALM

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "fs.s3a.aws.credentials.provider", "value": "org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider" } ]

To enable asynchronous access for the source and allow cloud caching to support simultaneous actions such as adding and editing, set to true (default). Otherwise, set to false.

Example true

To enable local caching for Amazon S3 and Azure Storage, set to true (default). Otherwise, set to false.

Example true

To enable local caching for Hadoop Distributed File System (HDFS), set to true. Otherwise, set to false (default).

Example false

Maximum percentage of the total available cache space to use on any single executor node. Minimum is 1. Maximum is 100 (default).

Example 100

Default format for the tables you create in Dremio. Default is ICEBERG.

Enum ICEBERG , PARQUET

Example ICEBERG

Type of authentication for Dremio to use to connect to the source. Default is STORAGE.

Enum STORAGE , SQL , RANGER

Example STORAGE

The Ranger service name, if you are using Ranger-based authentication.

Example hivedev

The Ranger host URL, if you are using Ranger-based authentication.

Example http://example.com/6080

Capitalization method to use for impersonation usernames. Default is AS_IS.

Enum AS_IS , LOWERCASE , UPPERCASE

Example AS_IS

To use the view owner as the impersonated username when enableImpersonation is enabled, set to true (default). To use the user who submits the query as the impersonated username, set to false.

Example true

propertyList

Connection property.

Example fs.s3a.aws.credentials.provider

Value for the connection property.

Example org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider

IBM Db2

IBM Db2 Source config Object

{
  "config": {
    "database": "tpch",
    "hostname": "172.25.1.245",
    "username": "exampleuser",
    "password": "$DREMIO_EXISTING_VALUE$",
    "port": "50000",
    "fetchSize": 200,
    "maxIdleConns": 8,
    "idleTimeSec": 60,
    "queryTimeoutSec": 0,
    "propertyList": [
      {
        "name": "deferPrepares",
        "value": "true"
      }
    ]
  }
}

IBM Db2 config Parameters

IBM Db2 database for Dremio to use.

Example tpch

IBM Db2 hostname.

Example 172.25.1.245

Username for authenticating with master credentials.

Example exampleuser

Password for authenticating with master credentials. For privacy, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.

IBM Db2 port number. Default is 50000.

Example 50000

Number of records to fetch at one time. Default is 200. If set to 0, Dremio automatically decides how many records to fetch.

Example 200

Maximum number of connections that can be idle at any given time. Default is 8.

Example 8

Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.

Example 60

Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.

Example 0

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "deferPrepares", "value": "true" } ]

propertyList

Name of the connection property.

Example deferPrepares

Value for the connection property.

Example true

MapR File System (MapR FS)

MapR FS Source config Object

{
  "config": {
    "clusterName": "my.cluster.com",
    "enableImpersonation": false,
    "secure": false,
    "rootPath": "/",
    "defaultCtasFormat": "ICEBERG",
    "isPartitionInferenceEnabled": false,
    "impersonationUserDelegationMode": "AS_IS",
    "vdsAccessDelegationEnabled": true
  }
}

MapR FS config Parameters

Name of the MapR FS cluster.

Example my.cluster.com

To enable user-specific file access permissions by turning on impersonation in MapR FS sources, set to true. Otherwise, set to false (default).

Example false

To enable a secure connection with SSL encryption between MapR FS and Dremio, set to true. Otherwise, set to false (default).

Example false

Root path of the MapR FS source. Default is /.

Example /

Default format for the tables you create in Dremio. Default is ICEBERG.

Enum ICEBERG , PARQUET

Example ICEBERG

To enable partition column inference, set to true. Otherwise, set to false (default). For more information, read Partition Column Inference.

Example false

Capitalization method to use for impersonation usernames. Default is AS_IS.

Enum AS_IS , LOWERCASE , UPPERCASE

Example AS_IS

To use the view owner as the impersonated username when enableImpersonation is enabled, set to true (default). To use the user who submits the query as the impersonated username, set to false.

Example true

Microsoft Azure Data Explorer

Microsoft Azure Data Explorer Source config Object

{
  "config": {
    "clusterUri": "https://prod.eastus.kusto.windows.net",
    "tenantId": "6e334262-b0c6-4c36-9faf-380e97e42964",
    "appId": "c3e4554f-514a-4912-8b75-93800f0d6c71",
    "appSecret": "$DREMIO_EXISTING_VALUE$",
    "databaseName": "datatypes",
    "maxIdleConns": 8,
    "idleTimeSec": 60,
    "queryTimeoutSec": 0
  }
}

Microsoft Azure Date Explorer config Parameters

URI for the Azure Active Directory (AD) tenant that Dremio should connect to.

Example https://prod.eastus.kusto.windows.net

Unique identifier of the Azure AD tenant.

Example 6e334262-b0c6-4c36-9faf-380e97e42964

Unique identifier of the Azure AD application Dremio should use when connecting.

Example c3e4554f-514a-4912-8b75-93800f0d6c71

Secret for the Azure AD application Dremio should use when connecting. For privacy, Dremio returns the appSecret value as $DREMIO_EXISTING_VALUE$ in API responses.

Default Azure Data Explorer database for Dremio to use.

Example datatypes

Maximum number of connections that can be idle at any given time. Default is 8.

Example 8

Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.

Example 60

Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.

Example 0

Microsoft Azure Synapse Analytics

Microsoft Azure Synapse Analytics Source config Object

{
  "config": {
    "hostname": "synapsedata.sql.azuresynapse.net",
    "port": "1433",
    "username": "exampleuser",
    "password": "$DREMIO_EXISTING_VALUE$",
    "authenticationType": "MASTER",
    "fetchSize": 200,
    "useSsl": false,
    "enableServerVerification": true,
    "maxIdleConns": 8,
    "idleTimeSec": 60,
    "queryTimeoutSec": 0,
    "database": "dedicatedpool",
    "propertyList": [
      {
        "name": "applicationIntent",
        "value": "ReadWrite"
      }
    ]
  }
}

Microsoft Azure Synapse Analytics config Parameters

Microsoft Azure Synapse Analytics hostname.

Example synapsedata.sql.azuresynapse.net

Microsoft Azure Synapse Analytics port number. Default is 1433.

Example 1433

Username for authenticating with master credentials.

Example exampleuser

Password for authenticating with master credentials. For privacy, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.

Type of authentication to use to allow Dremio to connect to the Microsoft Azure Synapse Analytics source.

ANONYMOUS: No authentication is needed.
MASTER: Use credentials from a master database user.

Example MASTER

Number of records to fetch at one time. Default is 200. If set to 0, Dremio automatically decides how many records to fetch.

Example 200

If the Microsoft Azure Synapse Analytics source is configured to use TLS for connections, set to true. Otherwise, set to false (default).

Example false

To verify the certificate for the Microsoft Azure Synapse Analytics source when connecting, set to true. Otherwise, set to false (default).

Example false

Maximum number of connections that can be idle at any given time. Default is 8.

Example 8

Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.

Example 60

Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.

Example 0

Default Microsoft Azure Synapse Analytics database for Dremio to use.

Example dedicatedpool

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "applicationIntent", "value": "ReadWrite" } ]

propertyList

Name of the connection property.

Example applicationIntent

Value for the connection property.

Example ReadWrite

Microsoft SQL Server

Microsoft SQL Server Source config Object

{
  "config": {
    "hostname": "172.25.0.10",
    "port": "1433",
    "username": "tpch",
    "password": "$DREMIO_EXISTING_VALUE$",
    "authenticationType": "MASTER",
    "fetchSize": 200,
    "useSsl": false,
    "enableServerVerification": true,
    "maxIdleConns": 8,
    "idleTimeSec": 60,
    "queryTimeoutSec": 0,
    "showOnlyConnectionDatabase": false,
    "propertyList": [
      {
        "name": "applicationIntent",
        "value": "ReadWrite"
      }
    ]
  }
}

Microsoft SQL Server config Parameters

Microsoft SQL Server hostname.

Example 172.25.0.10

Microsoft SQL Server port number. Default is 5432.

Example 1433

Username for authenticating with master credentials.

Example tpch

Password for authenticating with master credentials. For privacy, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.

Type of authentication to use to allow Dremio to connect to the Microsoft SQL Server source.

ANONYMOUS: No authentication is needed.
MASTER: Use credentials from a master database user.

Example MASTER

Number of records to fetch at one time. Default is 200. If set to 0, Dremio automatically decides how many records to fetch.

Example 200

If the Microsoft SQL Server source is configured to use TLS for connections, set to true. Otherwise, set to false (default).

Example false

To verify the certificate for the Microsoft SQL Server source when connecting, set to true. Otherwise, set to false (default).

Example false

Maximum number of connections that can be idle at any given time. Default is 8.

Example 8

Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.

Example 60

Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.

Example 0

To hide all databases that the credentialed user can access except the specified connection database, set to true. Otherwise, set to false (default).

Example false

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "applicationIntent", "value": "ReadWrite" } ]

propertyList

Name of the connection property.

Example applicationIntent

Value for the connection property.

Example ReadWrite

MongoDB

MongoDB Source config Object

{
  "config": {
    "hostList": [
      {
        "hostname": "172.23.0.208",
        "port": 27017
      }
    ],
    "useSsl": false,
    "authenticationType": "ANONYMOUS",
    "authDatabase": "admin",
    "authenticationTimeoutMillis": 2000,
    "secondaryReadsOnly": false,
    "subpartitionSize": 0,
    "sampleSize": 4095,
    "sampleMethod": "FIRST",
    "propertyList": [
      {
        "name": "maxPoolSize",
        "value": "100"
      }
    ],
    "useCaseInsensitiveFieldNames": false
  }
}

MongoDB config Parameters

Information about MongoDB hosts. Each object in the hostList includes the hostname and the corresponding port for the host.

Example [ { "hostname": "172.23.0.208", "port": 27017 } ]

To force a secure connection with SSL encryption between MongoDB and Dremio, set to true. Otherwise, set to false (default).

Example false

Type of authentication for Dremio to use to connect to the source.

Enum ANONYMOUS , MASTER

Example ANONYMOUS

Username for authenticating with master credentials.

Example exampleuser

Password for authenticating with master credentials. For privacy, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.

Name of the MongoDB database to authenticate against.

Example admin

Time to wait for authentication requests, in milliseconds. Default is 2000.

Example 2000

To disable reading from primaries, set to true. Otherwise, set to false (default). Enabling secondaryReadsOnly may degrade performance.

Example false

Number of records to read by query fragments. Use subpartitionSize to increase query parallelism. Default is 0.

Example 0

Number of records to read when sampling to determine the schema for a collection. Default is 4095. If the value is 0, the sample size is unlimited.

Example 4095

Method by which records should be read when sampling a collection to determine the schema. Default is FIRST.

Enum FIRST , LAST

Example FIRST

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "maxPoolSize", "value": "100" } ]

To treat field names as case-insensitive, set to true. Otherwise, set to false (default). If set to true, Dremio records all known variations of a field name when learning the schema and uses them when pushing an operation down to MongoDB.

Example false

hostList

Name of the host to use to connect to the MongoDB source. If MongoDB is sharded, use the mongos host. Otherwise, use the mongod host.

Example 172.23.0.208

Port to use with the specified hostname to connect to the MongoDB source. Default is 27017.

Example 27017

propertyList

Name of the connection property.

Example maxPoolSize

Value for the connection property.

Example 100

MySQL

MySQL Source config Object

{
  "config": {
    "hostname": "rdbms-linux-20231031.c.company-4321.internal",
    "port": "3306",
    "username": "exampleuser",
    "password": "$DREMIO_EXISTING_VALUE$",
    "authenticationType": "MASTER",
    "fetchSize": 200,
    "netWriteTimeout": 60,
    "maxIdleConns": 8,
    "idleTimeSec": 60,
    "propertyList": [
      {
        "name": "DataSource",
        "value": "sales2021"
      }
    ],
    "queryTimeoutSec": 0
  }
}

mySQL config Parameters

Name of the host to use to connect to the mySQL source.

Example rdbms-linux-20221031.c.dremio-4321.internal

Port to use with the specified hostname to connect to the mySQL source. Default is 3306.

Example 3306

Username for authentication with master credentials.

Example exampleuser

Password for authentication with master credentials. For privacy, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.

Type of authentication for Dremio to use to connect to the source.

Enum ANONYMOUS , MASTER

Example MASTER

Number of records to fetch at one time. Default is 200. If set to 0, Dremio automatically decides how many records to fetch.

Example 200

Length of time, in seconds, before an idle connection is eligible to be closed. Default is 60.

Example 60

Maximum number of connections that can be idle at any given time. Default is 8.

Example 8

Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.

Example 60

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "DataSource", "value": "sales2021" } ]

Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.

Example 0

propertyList

Name of the connection property.

Example DataSource

Value for the connection property.

Example sales2021

Network-attached Storage (NAS)

NAS Source config Object

{
  "config": {
    "path": "/home/dremio/",
    "defaultCtasFormat": "ICEBERG",
    "isPartitionInferenceEnabled": false
  }
}

NAS config Parameters

Path on the filesystem to use as the root for the source. The path must be accessible on all nodes.

Example /home/dremio/

Default format for the tables you create in Dremio. Default is ICEBERG.

Enum ICEBERG , PARQUET

Example ICEBERG

To enable partition column inference, set to true. Otherwise, set to false (default). For more information, read Partition Column Inference.

Example false

Oracle

Oracle Source config Object

{
  "config": {
    "hostname": "172.25.0.10",
    "port": "1521",
    "instance": "xe",
    "username": "exampleuser",
    "password": "$DREMIO_EXISTING_VALUE$",
    "authenticationType": "MASTER",
    "fetchSize": 200,
    "useSsl": false,
    "useTimezoneAsRegion": true,
    "includeSynonyms": false,
    "useLdap": false,
    "nativeEncryption": "ACCEPTED",
    "useKerberos": false,
    "maxIdleConns": 8,
    "idleTimeSec": 60,
    "mapDateToTimestamp": true,
    "queryTimeoutSec": 0,
    "propertyList": [
      {
        "name": "connect_timeout",
        "value": "0"
      }
    ]
  }
}

Oracle config Parameters

Oracle hostname.

Example 172.25.0.10

Oracle port number. Default is 1521.

Example 1521

Name of the Oracle database.

Example xe

Username for authenticating with master credentials.

Example exampleuser

Password for authenticating with master credentials. For privacy, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.

URL for authentication with a secret resource URL. For privacy, Dremio returns the secretResourceUrl value as $DREMIO_EXISTING_VALUE$ in API responses.

Type of authentication to use to allow Dremio to connect to the Oracle source.

ANONYMOUS: No authentication is needed.
MASTER: Use credentials from a master database user or use a secret resource URL.code>KERBEROS: Authenticate with Kerberos.

Example MASTER

Number of records to fetch at one time. Default is 200. If set to 0, Dremio automatically decides how many records to fetch.

Example 200

If the Oracle source is configured to use TLS for connections, set to true. Otherwise, set to false (default).

Example false

To use the timezone to set the connection region, set to true (default). Otherwise, set to false.

Example true

To include Oracle synonyms as datasets in the source, set to true. Otherwise, set to false (default).

Example false

If using Oracle native encryption for the connection, the encryption setting to use. Default is ACCEPTED.

Enum REJECTED , ACCEPTED , REQUESTED , REQUIRED

Example ACCEPTED

To use Kerberos for authentication, set to true. Otherwise, set to false (default).

Example false

Maximum number of connections that can be idle at any given time. Default is 8.

Example 8

Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.

Example 60

If the DATE column should display values in timestamp format, set to true (default). Otherwise, set to false.

Example true

Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.

Example 0

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "connect_timeout", "value": "0" } ]

propertyList

Name of the connection property.

Example connect_timeout

Value for the connection property.

Example 0

PostgreSQL

PostgreSQL Source config Object

{
  "config": {
    "hostname": "172.25.0.10",
    "port": "5432",
    "databaseName": "postgres",
    "username": "exampleuser",
    "password": "$DREMIO_EXISTING_VALUE$",
    "authenticationType": "MASTER",
    "fetchSize": 200,
    "useSsl": false,
    "encryptionValidationMode": "CERTIFICATE_AND_HOSTNAME_VALIDATION",
    "maxIdleConns": 8,
    "idleTimeSec": 60,
    "queryTimeoutSec": 0,
    "propertyList": [
      {
        "name": "connect_timeout",
        "value": "10"
      }
    ]
  }
}

PostgreSQLconfig Parameters

PostgreSQL hostname.

Example 172.25.0.10

PostgreSQL port number. Default is 5432.

Example 5432

Default PostgreSQL database for Dremio to use.

Example postgres

Username for authenticating with master credentials.

Example exampleuser

Password for authenticating with master credentials. For privacy, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.

URL for authentication with a secret resource URL. For privacy, Dremio returns the secretResourceUrl value as $DREMIO_EXISTING_VALUE$ in API responses.

Type of authentication to use to allow Dremio to connect to the PostgreSQL source.

ANONYMOUS: No authentication is needed.
MASTER: Use credentials from a master database user or use a secret resource URL.

Example MASTER

Number of records to fetch at one time. Default is 200. If set to 0, Dremio automatically decides how many records to fetch.

Example 200

If the PostgreSQL source is configured to use TLS for connections, set to true. Otherwise, set to false.

Example false

Method to use to validate data encryption for the source. The available settings map to PostgreSQL sslmode terminology as follows:

CERTIFICATE_AND_HOSTNAME_VALIDATION: verify-full
CERTIFICATE_ONLY_VALIDATION: verify-ca
NO_VALIDATION: require

Example CERTIFICATE_AND_HOSTNAME_VALIDATION

Maximum number of connections that can be idle at any given time. Default is 8.

Example 8

Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.

Example 60

Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.

Example 0

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "connect_timeout", "value": "10&application_name=myapp" } ]

propertyList

Name of the connection property.

Example connect_timeout

Value for the connection property.

Example 10&application_name=myapp

Snowflake

Snowflake Source config Object

{
  "config": {
    "schema": "myschema",
    "hostname": "https://cxazbmq-pa44061.snowflakecomputing.com",
    "database": "SNOWFLAKE_SAMPLE_DATA",
    "role": "ACCOUNTADMIN",
    "warehouse": "COMPUTE_WH",
    "username": "exampleuser1",
    "password": "$DREMIO_EXISTING_VALUE$",
    "maxIdleConns": 8,
    "idleTimeSec": 60,
    "queryTimeoutSec": 0,
    "propertyList": [
      {
        "name": "loginTimeout",
        "value": "60"
      }
    ]
  }
}

Snowflake config Parameters

Default Snowflake schema for Dremio to use for the specified database.

Example myschema

Account URL for the Snowflake source.

Example https://cxazbmq-pa44061.snowflakecomputing.com

Default Snowflake database for Dremio to use.

Example SNOWFLAKE_SAMPLE_DATA

Default access control role that Dremio should use in the Snowflake session initiated by the driver.

Example ACCOUNTADMIN

Snowflake virtual warehouse for Dremio to use.

Example COMPUTE_WH

Username for the Snowflake account.

Example exampleuser1

Password for the Snowflake account. For privacy, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.

Maximum number of connections that can be idle at any given time. Default is 8.

Example 8

Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.

Example 60

Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.

Example 0

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "loginTimeout", "value": "60" } ]

propertyList

Connection property.

Example loginTimeout

Value for the connection property.

Example 60

Teradata

Teradata Source config Object

{
  "config": {
    "hostname": "teradata.drem.io",
    "port": "1025",
    "database": "dremio_source",
    "username": "dremio",
    "password": "$DREMIO_EXISTING_VALUE$",
    "fetchSize": 200,
    "showOnlyConnectionDatabase": false,
    "useSsl": false,
    "maxIdleConns": 8,
    "idleTimeSec": 60,
    "queryTimeoutSec": 0,
    "propertyList": [
      {
        "name": "maxPoolSize",
        "value": "10"
      }
    ]
  }
}

Teradata config Parameters

Teradata hostname.

Example teradata.drem.io

Teradata port number. Default is 5432.

Example 1025

Service name of the Teradata database.

Example dremio_source

Username for authentication.

Example dremio

Password for authentication. For privacy, Dremio returns the password value as $DREMIO_EXISTING_VALUE$ in API responses.

Number of records to fetch at one time. Default is 200. If set to 0, Dremio automatically decides how many records to fetch.

Example 200

To hide all databases that the credentialed user can access except the specified connection database, set to true. Otherwise, set to false (default).

Example false

If the Teradata source is configured to use TLS for connections, set to true. Otherwise, set to false (default).

Example false

Maximum number of connections that can be idle at any given time. Default is 8.

Example 8

Maximum time that a connection can remain idle before Dremio terminates it, in seconds. Default is 60.

Example 60

Maximum time to allow for query execution, in seconds. When the query timeout expires, the connection returns to an idle state. Default is 0.

Example 0

Connection properties for the source. Each object in the propertyList includes the name of the property and the corresponding value to use for the property.

Example [ { "name": "maxPoolSize", "value": "10" } ]

propertyList

Name of the connection property.

Example maxPoolSize

Value for the connection property.

Example 10

Was this page helpful?

Glad to hear it! Thank you for your feedback. Sorry to hear that. Thank you for your feedback.