Source Types

[info] DEPRECATED
Use the Catalog API instead. The Source API will be removed in a future release.

Dremio supports a variety of source types, each with their own custom configuration.

All editions of Dremio support the following source types:

ADL, NAS, HDFS, MAPRFS, S3, MONGO, ELASTIC, ORACLE, MYSQL, MSSQL, POSTGRES, REDSHIFT, HBASE, HIVE

The Enterprise Edition adds support for:

DB2

Azure Data Lake Store

ADL

{
  "mode": "CLIENT_KEY",
  "accountName": String,
  "clientId": String,
  "clientKeyRefreshUrl": String,
  "clientKeyPassword": String
}

NAS

NAS

{
  "path": String
}

HDFS

HDFS

{
  "enableImpersonation": Boolean,
  "hostname": String,
  "port": Number,
  "rootPath": String,
  "propertyList": [
     {"name": String, "value": String},
     ...
  ]
}

MapR-FS

MAPRFS

{
  "clusterName": String,
  "enableImpersonation": Boolean,
  "secure": Boolean,
  "rootPath": String,
  "propertyList": [
     {"name": String, "value": String},
     ...
  ]
}

Amazon S3

S3

{
  "accessKey": String,
  "accessSecret": String,
  "secure": Boolean,
  "externalBucketList": [...String],
  "propertyList": [
     {"name": String, "value": String},
     ...
  ]
}

MongoDB

MONGO

{
  "username": String,
  "password": String,
  "hostList": [
     {"hostname": String, "port": Number},
     ...
  ],
  "useSsl": Boolean,
  "authenticationType": String ["ANONYMOUS", "MASTER"],
  "authDatabase": String,
  "authenticationTimeoutMillis": Number,
  "secondaryReadsOnly": Boolean,
  "subpartitionSize": Number,
  "propertyList": [
    {"name": String, "value": String},
    ...
  ]
}

Elasticsearch

ELASTIC

{
  "username": String,
  "password": String,
  "hostList": [
     {"hostname": String, "port": Number},
     ...
  ],
  "authenticationType": String ["ANONYMOUS", "MASTER"],
  "scriptsEnabled": Boolean [optional],
  "showHiddenIndices": Boolean [optional],
  "sslEnabled": Boolean [optional],
  "showIdColumn": Boolean [optional],
  "readTimeoutMillis": Number,
  "scrollTimeoutMillis": Number,
  "usePainless": Boolean [optional],
  "useWhitelist": Boolean [optional],
  "scrollSize": Number [optional]
}

Oracle

ORACLE

{
  "username": String,
  "password": String,
  "instance": String,
  "hostname": String,
  "port": String,
  "authenticationType": String ["ANONYMOUS", "MASTER"],
  "fetchSize": Number
}

MySQL

MYSQL

{
  "username": String,
  "password": String,
  "hostname": String,
  "port": String,
  "authenticationType": String ["ANONYMOUS", "MASTER"],
  "fetchSize": Number
}

Microsoft SQL Server

MSSQL

{
  "username": String,
  "password": String,
  "hostname": String,
  "port": String,
  "authenticationType": String ["ANONYMOUS", "MASTER"],
  "fetchSize": Number,
  "database": String [optional],
  "showOnlyConnectiondatabase": Boolean [optional]
}

Postgres

POSTGRES

{
  "username": String,
  "password": String,
  "hostname": String,
  "port": String,
  "authenticationType": String ["ANONYMOUS", "MASTER"],
  "fetchSize": Number,
  "databaseName": String
}

Amazon Redshift

REDSHIFT

{
  "username": String,
  "password": String,
  "authenticationType": String ["ANONYMOUS", "MASTER"],
  "fetchSize": Number,
  "connectionString": String
}

HBase

[info] As of Dremio 3.2, HBase is deprecated for existing configurations and not supported for new.

HBASE

{
  "zkQuorum": String,
  "port": Number,
  "isSizeCalcEnabled": Boolean,
  "propertyList": [
    {"name": String, "value": String},
    ...
  ]
}

Hive

HIVE

{
  "hostname": String,
  "port": String,
  "kerberosPrincipal": String,
  "enableSasl": Boolean [optional],
  "propertyList": [
    {"name": String, "value": String},
    ...
  ]
}

IBM DB2

[info] As of Dremio 3.2, IBM DB2 is deprecated for existing configurations and not supported for new.

DB2

[info] Enterprise Edition only

{
  "username": String,
  "password": String,
  "hostname": String,
  "port": String,
  "databaseName": String,
  "authenticationType": String ["ANONYMOUS", "MASTER"],
  "fetchSize": Number
}