Models
Deprecated:
Use the Catalog API instead. The Source API will be removed in a future release.
Source
Represents a source in Dremio. All sources returned by the REST API will have a _type
of source
.
The JSON representation of a source looks like this:
{
"_type": "source" [immutable, generated by Dremio],
"id": String [immutable, generated by Dremio],
"name": String [immutable after creation],
"description": String,
"tag": String [immutable, generated by Dremio],
"type": String (Source Type) [immutable after creation],
"config": Object (Source Config),
"createdAt": String (RFC3339 date) [immutable],
"metadataPolicy": {
"authTTLMs": Number,
"datasetRefreshAfterMs": Number,
"datasetExpireAfterMs": Number,
"namesRefreshMs": Number,
"datasetUpdateMode": String ["PREFETCH", "PREFETCH_QUERIED", "INLINE"]
},
"state": {
"status": String ["good", "bad", "warn"] [immutable, generated by Dremio],
"messages": [Status Message] [immutable, generated by Dremio]
},
"accelerationRefreshPeriodMs": Number,
"accelerationGracePeriodMs": Number
}
Fields
Name | Type | Description |
---|---|---|
id | String | Source ID. Generated by Dremio, immutable. |
name | String | Name of the source. Immutable after creation. |
description | String | Detailed description. |
tag | String | Identifies the instance of the source, changed each time a source is modified. Generated by Dremio, immutable. |
type | String | Source type. Immutable after creation. |
config | Object | Source config based on the specified source type. |
createdAt | String | RFC3339 date (example: 2017-10-27T21:08:22.858Z ) representing the creation datetime. Immutable. |
metadataPolicy | Metadata Policy | Policies regarding updating of a source’s metadata. |
state | Source State | State of the source. Generated by Dremio, immutable. |
accelerationRefreshPeriodMs | Number | How often to refresh reflections on data in the source. |
accelerationGracePeriodMs | Number | How long to keep reflections on data in the source before it expires. |
Metadata Policy
Policies regarding updating of a source’s metadata.
{
"authTTLMs": Number,
"datasetRefreshAfterMs": Number,
"datasetExpireAfterMs": Number,
"namesRefreshMs": Number,
"datasetUpdateMode": String ["PREFETCH", "PREFETCH_QUERIED", "INLINE"]
}
Fields
Name | Type | Description |
---|---|---|
authTTLMs | Number | Authorization expiry in milliseconds. |
datasetRefreshAfterMs | Number | How often (in milliseconds) to refresh the metadata of datasets in the source. |
datasetExpireAfterMs | Number | How long (in miliseconds) to keep the metadata before it expires. |
namesRefreshMs | Number | How often (in milliseconds) to discover new datasets in the source. |
datasetUpdateMode | String | Must be either PREFETCH , PREFETCH_QUERIED , or INLINE . |
PREFETCH
- Dremio updates details for all datasets in a source. This mode increases query performance as less work needs to be done at query time.
PREFETCH_QUERIED
- Dremio updates details for previously queried objects in a source. This mode increases query performance as less work needs to be done at query time for these datasets.
INLINE
- Dremio updates details for a dataset at query time. This mode minimizes metadata queries on a source when not used, but might lead to longer planning times.
Source State
Represents the state of a source.
{
"status": String ["good", "bad", "warn"] [immutable],
"messages": [StatusMessage] [immutable]
}
Fields
Name | Type | Description |
---|---|---|
status | String | Either info , bad , or warn . Immutable |
messages | StatusMessage[] | An array of status messages. Immutable. |
Source Status Message
Represents a status message for a source.
{
"level": String ["INFO", "WARN", "ERROR"] [immutable],
"message": String [immutable]
}
Fields
Name | Type | Description |
---|---|---|
status | String | Either INFO , WARN , or ERROR . Immutable. |
message | String | Status message. Immutable. |