Models
Deprecated:
Use the Catalog API instead. The Source API will be removed in a future release.
Source
Represents a source in Dremio. All sources returned by the REST API will have a _type of source.
The JSON representation of a source looks like this:
{
"_type": "source" [immutable, generated by Dremio],
"id": String [immutable, generated by Dremio],
"name": String [immutable after creation],
"description": String,
"tag": String [immutable, generated by Dremio],
"type": String (Source Type) [immutable after creation],
"config": Object (Source Config),
"createdAt": String (RFC3339 date) [immutable],
"metadataPolicy": {
"authTTLMs": Number,
"datasetRefreshAfterMs": Number,
"datasetExpireAfterMs": Number,
"namesRefreshMs": Number,
"datasetUpdateMode": String ["PREFETCH", "PREFETCH_QUERIED", "INLINE"]
},
"state": {
"status": String ["good", "bad", "warn"] [immutable, generated by Dremio],
"messages": [Status Message] [immutable, generated by Dremio]
},
"accelerationRefreshPeriodMs": Number,
"accelerationGracePeriodMs": Number
}
Fields
| Name | Type | Description |
|---|---|---|
| id | String | Source ID. Generated by Dremio, immutable. |
| name | String | Name of the source. Immutable after creation. |
| description | String | Detailed description. |
| tag | String | Identifies the instance of the source, changed each time a source is modified. Generated by Dremio, immutable. |
| type | String | Source type. Immutable after creation. |
| config | Object | Source config based on the specified source type. |
| createdAt | String | RFC3339 date (example: 2017-10-27T21:08:22.858Z) representing the creation datetime. Immutable. |
| metadataPolicy | Metadata Policy | Policies regarding updating of a source’s metadata. |
| state | Source State | State of the source. Generated by Dremio, immutable. |
| accelerationRefreshPeriodMs | Number | How often to refresh reflections on data in the source. |
| accelerationGracePeriodMs | Number | How long to keep reflections on data in the source before it expires. |
Metadata Policy
Policies regarding updating of a source’s metadata.
{
"authTTLMs": Number,
"datasetRefreshAfterMs": Number,
"datasetExpireAfterMs": Number,
"namesRefreshMs": Number,
"datasetUpdateMode": String ["PREFETCH", "PREFETCH_QUERIED", "INLINE"]
}
Fields
| Name | Type | Description |
|---|---|---|
| authTTLMs | Number | Authorization expiry in milliseconds. |
| datasetRefreshAfterMs | Number | How often (in milliseconds) to refresh the metadata of datasets in the source. |
| datasetExpireAfterMs | Number | How long (in miliseconds) to keep the metadata before it expires. |
| namesRefreshMs | Number | How often (in milliseconds) to discover new datasets in the source. |
| datasetUpdateMode | String | Must be either PREFETCH, PREFETCH_QUERIED, or INLINE. |
PREFETCH - Dremio updates details for all datasets in a source. This mode increases query performance as less work needs to be done at query time.
PREFETCH_QUERIED - Dremio updates details for previously queried objects in a source. This mode increases query performance as less work needs to be done at query time for these datasets.
INLINE - Dremio updates details for a dataset at query time. This mode minimizes metadata queries on a source when not used, but might lead to longer planning times.
Source State
Represents the state of a source.
{
"status": String ["good", "bad", "warn"] [immutable],
"messages": [StatusMessage] [immutable]
}
Fields
| Name | Type | Description |
|---|---|---|
| status | String | Either info, bad, or warn. Immutable |
| messages | StatusMessage[] | An array of status messages. Immutable. |
Source Status Message
Represents a status message for a source.
{
"level": String ["INFO", "WARN", "ERROR"] [immutable],
"message": String [immutable]
}
Fields
| Name | Type | Description |
|---|---|---|
| status | String | Either INFO, WARN, or ERROR. Immutable. |
| message | String | Status message. Immutable. |