Source

[info] DEPRECATED
Use the Catalog API instead. The Source API will be removed in a future release.

Represents a source in Dremio. All sources returned by the REST API will have a _type of source.

The JSON representation of a source looks like this:

{
"_type": "source" [immutable, generated by Dremio],
"id": String [immutable, generated by Dremio],
"name": String [immutable after creation],
"description": String,
"tag": String [immutable, generated by Dremio],
"type": String (Source Type) [immutable after creation],
"config": Object (Source Config),
"createdAt": String (RFC3339 date) [immutable],
"metadataPolicy": {
"authTTLMs": Number,
"datasetRefreshAfterMs": Number,
"datasetExpireAfterMs": Number,
"namesRefreshMs": Number,
"datasetUpdateMode": String ["PREFETCH", "PREFETCH_QUERIED", "INLINE"]
},
"state": {
"status": String ["good", "bad", "warn"] [immutable, generated by Dremio],
"messages": [Status Message] [immutable, generated by Dremio]
},
"accelerationRefreshPeriodMs": Number,
"accelerationGracePeriodMs": Number
}
Name Type Description
id String Source ID. Generated by Dremio, immutable.
name String Name of the source. Immutable after creation.
description String Detailed description.
tag String Identifies the instance of the source, changed each time a source is modified. Generated by Dremio, immutable.
type String Source type. Immutable after creation.
config Object Source config based on the specified source type.
createdAt String RFC3339 date (example: 2017-10-27T21:08:22.858Z) representing the creation datetime. Immutable.
metadataPolicy Metadata Policy Policies regarding updating of a source's metadata.
state Source State State of the source. Generated by Dremio, immutable.
accelerationRefreshPeriodMs Number How often to refresh reflections on data in the source.
accelerationGracePeriodMs Number How long to keep reflections on data in the source before it expires.

Metadata Policy

Policies regarding updating of a source's metadata.

{
"authTTLMs": Number,
"datasetRefreshAfterMs": Number,
"datasetExpireAfterMs": Number,
"namesRefreshMs": Number,
"datasetUpdateMode": String ["PREFETCH", "PREFETCH_QUERIED", "INLINE"]
}
Name Type Description
authTTLMs Number Authorization expiry in milliseconds.
datasetRefreshAfterMs Number How often (in milliseconds) to refresh the metadata of datasets in the source.
datasetExpireAfterMs Number How long (in miliseconds) to keep the metadata before it expires.
namesRefreshMs Number How often (in milliseconds) to discover new datasets in the source.
datasetUpdateMode String Must be either PREFETCH, PREFETCH_QUERIED, or INLINE.

PREFETCH - Dremio updates details for all datasets in a source. This mode increases query performance as less work needs to be done at query time.

PREFETCH_QUERIED - Dremio updates details for previously queried objects in a source. This mode increases query performance as less work needs to be done at query time for these datasets.

INLINE - Dremio updates details for a dataset at query time. This mode minimizes metadata queries on a source when not used, but might lead to longer planning times.

Source State

Represents the state of a source.

{
"status": String ["good", "bad", "warn"] [immutable],
"messages": [StatusMessage] [immutable]
}
Name Type Description
status String Either info, bad, or warn. Immutable
messages StatusMessage[] An array of source status messages. Immutable.

Source Status Message

Represents a status message for a source.

{
"level": String ["INFO", "WARN", "ERROR"] [immutable],
"message": String [immutable]
}
Name Type Description
status String Either INFO, WARN, or ERROR. Immutable.
message String Status message. Immutable.

results matching ""

    No results matching ""