Skip to main content

How Reflections Work

A Reflection is Dremio's mechanism for accelerating queries through precomputed, optimized materializations. You can think of a Reflection as a highly capable materialized view: Dremio stores the precomputed result and, when a query arrives, automatically determines whether a Reflection can satisfy it and rewrites the query plan to use it. The user querying the data never has to know a Reflection exists or reference it explicitly. The acceleration is transparent.

The Query Rewrite​

When Dremio receives a query, the planner doesn't just check whether any Reflection has the same name or is directly referenced. It inspects the query's logical plan and determines whether any available Reflection could produce a correct, equivalent result. If one can, Dremio compares the cost of using it against the cost of running against the raw source data, and picks the cheaper plan.

This means a Reflection defined on a view can accelerate queries that were written against the underlying tables, or against different views built from the same tables, without any change to those queries and without anyone having to know the Reflection exists.

Two Types of Reflection​

Raw Reflections​

A raw Reflection stores a copy of the rows from its anchor (a table or view), optionally with a subset of columns, a sort order, and horizontal partitioning. It is most useful for:

  • Materializing complex or expensive views so the transformation cost is paid once
  • Converting data from slow or non-columnar formats into Dremio's optimized Iceberg storage
  • Pre-filtering data (via a view with a predicate) to enable fast needle-in-a-haystack lookups
  • Pre-joining tables so join cost is not paid at query time

A raw Reflection accelerates queries that are logically equivalent to scanning some or all of the stored rows. Column pruning, filter pushdown, and sort exploitation all apply.

Aggregate Reflections​

An aggregate Reflection is for accelerating GROUP BY queries: analytical workloads that summarize large amounts of data, such as those submitted by BI dashboards or AI agents.

Dremio materializes the aggregation at the finest grain you define, grouped by the full set of dimensions in the Reflection. This is a single base aggregate, not a cube of every possible dimension combination. Dremio does not pre-compute all rollup permutations.

For example, if your Reflection is defined with dimensions [region, product, month], Dremio stores one row per unique (region, product, month) combination, along with the measures you configured (counts, sums, min/max, and so on).

When a query asks for a coarser summary (say, grouped by [region] alone), Dremio's planner recognizes that it can satisfy the query by re-aggregating the Reflection's already-grouped rows instead of scanning the raw source data. The key insight is that the expensive part, collapsing millions or billions of raw rows into a compact grouped result, has already been done. Re-aggregating the much smaller Reflection dataset to produce a coarser rollup is fast by comparison.

This means a single aggregate Reflection defined at a fine grain covers the entire family of queries that group by any subset of its dimensions. You do not need a separate Reflection for every possible GROUP BY combination your users might write. Define one at the finest grain that makes sense for your data, and Dremio will use it to satisfy coarser rollups automatically.

Reflection Freshness and Correctness​

For Autonomous Reflections backed by Iceberg tables, Dremio can interrogate table metadata at planning time to verify that the Reflection is current. If the Reflection is stale or mid-refresh, the query falls back to scanning the raw source automatically. Manual Reflections do not have this guarantee.

For non-Iceberg sources (CSV, JSON, federated sources, and others without queryable metadata), Dremio has no metadata record to check at plan time. Freshness for these Reflections depends entirely on your refresh policy. Dremio will use the Reflection based on its last known refresh state, so keeping refresh schedules aligned with how frequently the source data changes is important.

  • Autonomous Reflections – Let Dremio analyze your query patterns and manage Reflections automatically. Recommended as the default approach for supported data formats.
  • Manual Reflections – Define and manage your own Reflections, including for data formats not supported by Autonomous Reflections.