Data Sources

Dremio supports a variety of data sources, including NoSQL databases, relational databases, Hadoop, local filesystems, and cloud storage.

[warning] IMPORTANT

Case-sensitive source data file/table names are not supported.

In Dremio, data filenames in your data source are "seen" in a case-insensitive manner. So, if you have to three (3) file names with difference cases (for example, JOE Joe, and joe), Dremio "sees" the files as having the same name. Thus, searching on Joe, JOE, or joe, can result in unanticipated data results.

Promoting Entities

When you issue queries against files or folders, the default behavior is to not auto-promote files/folders to datasets. For example, given a MAPRFS source with a Root: [folder | file] structure, if you executed select * from MAPRFS.folder.file, the query fails.

[info] Not Supported

Promoting folders with non-homogenous file types is not supported.

If you want to automatically promote files when queries are issued against the file or folder,

  1. Edit your source.
  2. Navigate to Edit Source > Metadata > Dataset Handling.
  3. Select "Automatically format files into physical datasets when users issue queries.".

[info] Previous Auto-Promote Behaviour

Prior to Dremio version 3.0, Dremio's default behavior for filesystem-based sources (HDFS, S3, NAS, etc) was to automatically auto-promote a folder or file to a dataset when you run a query on a file or folder.

For example, given a MAPRFS source with a Root: [folder | file] structure, if you executed select * from MAPRFS.folder.file, the file would automatically be promoted to a dataset.

Relational databases

Distributed Filesystems

NoSQL & Hadoop


results matching ""

    No results matching ""