Viewing Details about Reflection Refreshes, Use, Size, and More

The Reflections page lists all of the raw and aggregation reflections that are in a Dremio cluster.

To view this page, follow these steps:

1. Click in the left navbar.
2. Select Reflections in the left sidebar.

For any particular reflection, the Reflections page presents information that answers these questions:

Question	Column with the answer
What is the status of this reflection?	Name
Is this a raw or aggregation reflection?	Type
Which table or view is this reflection defined on?	Dataset
How valuable is this reflection?	Reflection Score
How can I see a list of the jobs that created and refreshed this reflection?	Refresh Job History
How many times has the query planner chosen this reflection?	Acceleration Count
How many times has the query planner considered using this reflection?	Considered Count
How many times did the query planner match a query to this reflection?	Matched Count
How do I find out how effective this reflection is?	Acceleration Count
When was this reflection last refreshed?	Last Refresh From Table
Is this reflection being refreshed now?	Refresh Status
What type of refreshes are used for this reflection?	Refresh Method
Are refreshes scheduled for this reflection, or do they need to be triggered manually?	Refresh Status
How much time did the most recent refresh of this reflection take?	Last Refresh Duration
How many records are in this reflection?	Record Count
How much storage is this reflection taking up?	Current Footprint
When does this reflection expire?	Available Until

Columns

Acceleration Count

Shows the number of times within the last 30 days that the query planner considered using a reflection defined on a dataset referenced by a query, determined the reflection could be used to satisfy the query, and chose to use the reflection to satisfy the query.

If this count is low relative to the numbers in the Considered Count and Matched Count, the reflection is not effective in reducing the execution times of queries on the dataset.

Available Until

Shows the date and time when this reflection expires, based on the refresh policy of the queried dataset.

If a reflection is set to expire soon and you want to continue using it, you can take either of these actions:

• Change the expiration setting on the table which the reflection is either directly or indirectly defined on. A reflection is indirectly defined on a table when it is defined on a view that is derived from that table. When you change the setting by using this method, the change goes into effect after the next refresh.
• Change the expiration setting on the data source where the table is located.

For the steps, see Setting the Expiration Policy for Reflections.

Considered Count

Shows the number of queries, within the last 30 days, that referenced the dataset that a reflection is defined on. Whenever a query references a dataset on which a reflection is defined, the query planner considers whether to use the reflection to help satisfy the query.

If the query planner determines that the reflection can do that (that the reflection matches the query), the query planner compares the reflection to any others that might also be defined on the same dataset.

If the query planner does not determine this, it ignores the reflection.

Reflections with high considered counts and no match counts are contributing to high logical planning times. Consider deleting them.

Reflections with a considered count of 0 should be removed. They are merely taking up storage and, during refreshes, resources on compute engines.

Current Footprint

Shows the current size, in kilobytes, of a reflection.

Dataset

Shows the name of the table or view that a reflection is defined on.

Last Refresh Duration

Shows the length of time required for the most recent refresh of a reflection.

Last Refresh From Table

Shows the date and time that the reflection data was last refreshed. If the refresh is running, failing, or disabled, the value is 12/31/1969 23:59:59.

Matched Count

Shows the number of times, within the last 30 days, that the query planner both considered a reflection for satisfying a query and determined that the reflection would in fact satisfy the query. However, the query planner might have decided to use a different reflection that also matched the query. For example, a different query plan that did not include the reflection might have had a lower cost.

This number does not show how many times the query planner used the reflection to satisfy the query. For that number, see Acceleration Count.

If the matched count is high and the accelerating count is low, the query planner is more often deciding to use a different reflection that also matches a query. In this case, consider deleting the reflection.

Name

Shows the name of the reflection and its status. The tooltip on the icon represents a combination of the status of the reflection (which you can filter on through the values in the Acceleration Status field above the list) and the value in the Refresh Status column.

Record Count

Shows the number of records in the reflection.

Reflection Score

Shows the score for a reflection on a scale of 0 (worst) to 100 (best). The score indicates the value that the reflection provides to your workloads based on the jobs that have been executed in the last 7 days. Reflection scores are calculated once each day. Factors considered in the score include the number of jobs accelerated by the reflection and the expected improvement in query run times due to the reflection.

To help you interpret the scores, the scores have the following labels:

• Good: The score is more than 75.
• Fair: The score is 25 to 75.
• Poor: The score is less than 25.
• New: The score is blank because the reflection was created within the past 24 hours.

note

If a reflection's score is listed as -, the score needs to be recalculated due to an error or an upgraded instance.

Refresh Job History

Opens a list of all of the jobs that created and refreshed a reflection.

Refresh Method

Shows which type of refresh was last used for a reflection.

• Full: All of the data in the reflection was replaced. The new data is based on the current data in the underlying dataset.
• Incremental:
  • For reflections defined on Apache Iceberg tables: Either snapshot-based incremental refresh was used (if the changes were appends only) or partition-based incremental refresh was used (if the changes included DML operations).
  • For reflections defined on Delta Lake tables: This value does not appear. Only full refreshes are supported for these reflections.
  • For reflections defined on all other tables: Data added to the underlying dataset since the last refresh of the reflection was appended to the existing data in the reflection.
• None: Incremental refreshes were selected in the settings for the table. However, Dremio has not confirmed that it is possible to refresh the reflection incrementally. Applies only to reflections that are not defined on Iceberg or Delta Lake tables.

For more information, see Refreshing Reflections.

Refresh Status

Shows one of these values:

• Manual: Refreshes are not run on a schedule, but must be triggered manually. See Manually Triggering a Refresh.
• Pending: If the reflection depends on other reflections, the refresh will begin after the refreshes of the other reflections are finished.
• Running: The reflection is currently being refreshed.
• Scheduled: Refreshes run on a schedule, but a refresh is not currently running.
• Auto: All of the reflection’s underlying tables are in Iceberg format, and the reflection automatically refreshes when new snapshots are created after an update to an underlying table, but a refresh is not currently running.
• Failed: Multiple attempts to refresh a reflection have failed. You must disable and enable the reflection to rebuild it and continue using it.

For more information, see Refreshing Reflections.

Total Footprint

Shows the current size, in kilobytes, of all of the existing materializations of the reflection. More than one materialization of a reflection can exist at the same time, so that refreshes do not interrupt running queries that are being satisfied by the reflection.

Type

Shows whether the reflection is a raw or aggregation reflection.