Reflection

Represents a reflection in Dremio. All reflections returned by the REST API will have a _type of reflection.

The JSON representation of a reflection looks like this:

{
  "entityType": "reflection" [immutable, generated by Dremio],
  "id": String [immutable, generated by Dremio],
  "tag": String [immutable, generated by Dremio],
  "name": String,
  "enabled": Boolean,
  "createdAt": String (RFC3339 date) [immutable, generated by Dremio],
  "updatedAt": String (RFC3339 date) [immutable, generated by Dremio],
  "type": String ["AGGREGATION", "RAW"] [immutable after creation],
  "datasetId": String [immutable after creation],
  "currentSizeBytes": Number [immutable, generated by Dremio],
  "totalSizeBytes": Number [immutable, generated by Dremio],
  "status": Reflection Status [immutable, generated by Dremio],
  "dimensionFields": [Reflection Field With Granularity] [optional, only valid for AGGREGATION],
  "measureFields": [Reflection Field] [optional, only valid for AGGREGATION],
  "displayFields": [Reflection Field] [optional, only valid for RAW],
  "distributionFields": [Reflection Field] [optional],
  "partitionFields": [Reflection Field] [optional],
  "sortFields": [Reflection Field] [optional],
  "partitionDistributionStrategy": String ["CONSOLIDATED", "STRIPED"],
}

Parameter Description

Name Type Description
id String Reflection ID. Generated by Dremio, immutable.
tag String Identifies the instance of the reflection, changed each time a source is modified. Generated by Dremio, immutable.
name String Name of the reflection, required to be non empty.
enabled Boolean Whether to allow using the reflection to accelerate queries.
createdAt String RFC3339 date (example: 2017-10-27T21:08:22.858Z) representing the creation datetime. Generated by Dremio, immutable.
updatedAt String RFC3339 date (example: 2017-10-27T21:08:22.858Z) representing the last time the reflection was updated. Generated by Dremio, immutable.
type String The reflection type, must either be AGGREGATION or RAW. Immutable after creation.
datasetId String The id of the dataset the reflection is for. Immutable after creation.
currentSizeBytes Number The data size (in bytes) of the latest reflection job (if one exists). Generated by Dremio, immutable.
totalSizeBytes Number The data size (in bytes) of all reflection jobs that have not been pruned (if any exist). Generated by Dremio, immutable.
status [Reflection Status] Status of the reflection. Generated by Dremio, immutable.
dimensionFields Reflection Field With Granularity[] List of dimension fields, optional and only valid for AGGREGATION reflections.
measureFields Reflection Measure Field[] List of measure fields, optional and only valid for AGGREGATION reflections.
displayFields Reflection Field[] List of display fields, optional and only valid for RAW reflections.
sortFields Reflection Field[] List of sort fields, optional and valid for all reflections.
partitionFields Reflection Field[] List of partition fields, optional and valid for all reflections.
distributionFields Reflection Field[] List of distribution fields, optional and valid for all reflections.
partitionDistributionStrategy String Must either be CONSOLIDATED or STRIPED.

Reflection Field

{
  "name": String
}

Reflection Field With Granularity

{
  "name": String,
  "granularity": String ["NORMAL", "DATE"]
}

Reflection Measure Field

Represents a measure field.

The measureTypeList field was introduced in Dremio 2.1.0. It represents the list of statistics that will be pre-computed. The default is SUM and COUNT and the SUM value is not valid for text and date columns.

{
  "name": String,
  "measureTypeList": String[]  ['MIN', 'MAX', 'SUM', 'COUNT', 'APPROX_COUNT_DISTINCT']
}

Reflection Status

Represents the status of a reflection.

{
  "config": String ["OK", "INVALID"] [immutable, generated by Dremio],
  "refresh": String ["SCHEDULED", "RUNNING", "GIVEN_UP"] [immutable, generated by Dremio],
  "availability": String ["NONE", "INCOMPLETE", "EXPIRED", "AVAILABLE"] [immutable, generated by Dremio],
  "failureCount": Number [immutable, generated by Dremio],
  "lastRefresh": String (RFC3339 date) [immutable, generated by Dremio],
  "expiresAt": String (RFC3339 date) [immutable, generated by Dremio]
}
Name Type Description
config String State of the reflection configuration. Either OK or INVALID (the dataset schema has changed and the reflection definition is no longer applicable, for example a field that was being used no longer exists). Generated by Dremio, immutable.
refresh String Refresh state of the reflection. Either MANUAL (refresh period is set to 0 and user must manually refresh), SCHEDULED, RUNNING or GIVEN_UP (the reflection has failed to refresh multiple times and will not be reattempted). Generated by Dremio, immutable.
availablity String Whether the reflection is available to accelerate queries. Either NONE, INCOMPLETE, EXPIRED or AVAILABLE. Generated by Dremio, immutable.
failureCount Number Number of consecutive reflection creation failures. Generated by Dremio, immutable.
lastRefresh String RFC3339 date (example: 2017-10-27T21:08:22.858Z) representing the last time the reflection data was refreshed. Generated by Dremio, immutable.
expiresAt String RFC3339 date (example: 2017-10-27T21:08:22.858Z) representing when the reflection data will expire. Generated by Dremio, immutable.

Reflection Summary

Represents a summary of a reflection in Dremio. Includes a subset of reflection with some additional dataset information.

The JSON representation of a reflection summary looks like this:

{
  "_type": "reflection-summary" [immutable],
  "id": String [immutable],
  "createdAt": String (RFC3339 date) [immutable],
  "updatedAt": String (RFC3339 date) [immutable],
  "type": String ["AGGREGATION", "RAW"] [immutable],
  "name": String [immutable],
  "datasetId": String [immutable],
  "datasetPath": [String] [immutable],
  "datasetType": String ["PHYSICAL", "VIRTUAL"] [immutable],
  "currentSizeBytes": Number [immutable],
  "totalSizeBytes": Number [immutable],
  "enabled": Boolean [immutable],
  "status": Reflection Status [immutable]
}

results matching ""

    No results matching ""