Queue Routing
You can route refresh jobs for all reflections on a dataset (either a table or a view), all reflections in a folder, or all reflections in a space, to a specific queue.
Datasets do not inherit this setting. For example, if for Table 1 you set the queue to Queue 1, refresh jobs for reflections on views that are created on Table 1 do not run in Queue 1 by default.
For the SQL syntax, see Queue Routing in the SQL reference.
Example
Suppose that you had the following tables, views, and reflections, as shown in the first diagram:
- Table 1 has a reflection on it.
- Views 1 and 2 are derived from Table 1.
- View 1 has two reflections.
- Views 3 and 4 are derived from View 2.
- Views 3 and 4 have one reflection each.
You run these SQL commands to route the reflection-refresh jobs for Table 1, View 1, and View 3 to Queue 1. You let Dremio decide which of the two default queues for reflection-refresh jobs (Low-Cost Reflections and High-Cost Reflections) to use for View 4:
ALTER DATASET "Table 1" ROUTE REFLECTIONS TO QUEUE "Queue 1"
ALTER DATASET "View 1" ROUTE REFLECTIONS TO QUEUE "Queue 1"
ALTER DATASET "View 3" ROUTE REFLECTIONS TO QUEUE "Queue 1"
View 1 and View 3 do not inherit the setting from Table 1. If you want to run refresh jobs for the reflections on those views in Queue 1, you must run the SQL command on those views and specify Queue 1.
According to the schedule defined on the Reflection Refresh page of the settings for Table 1, the data in Table 1 is refreshed:
After the refresh of the data in Table 1 is complete, Dremio runs the reflection-refresh jobs for the reflections. The jobs for the reflections for Table 1, View 1, and View 3 run in Queue 1. The job for the reflection on View 4 runs in the default queue that Dremio selected:
