Viewing Jobs and Job Details

All jobs run in Dremio Cloud are listed on a separate page, showing the job ID, type, status, and other attributes. To navigate to the Jobs page, click the Jobs (list) icon in the side navigation bar.

Search filters and columns

By default, the Jobs page lists the jobs run within the last 30 days and the jobs are filtered by UI, External Tools job types. To change these defaults for your account, you can filter on values and manage columns directly on the Jobs page, as shown in this image:

a. Search Jobs by typing the username or job ID.

b. Start Time allows you to pick the date and time at which the job began.

c. Status represents one or more job states. For descriptions, see Job states and statuses.

d. Type includes Accelerator, Downloads, External Tools, Internal, and UI. For descriptions, see Job attributes.

e. User can be searched by typing the username or checking the box next to the username in the dropdown.

f. Manage Columns by checking the boxes next to additional columns that you want to see in the Jobs list. The grayed out checkboxes show the columns that are required by default. You can also rearrange the column order by clicking directly on a column to drag and drop.

Job attributes

Each job has the following attributes, which can appear as columns in the list of jobs:

Job states and statuses

Each job passes through a sequence of states until it is complete, though the sequence can be interrupted if a query is canceled or if there is an error during a state. In this diagram, the states that a job passes through are in white, and the possible end states are in dark gray.

This table lists the statuses that the UI lets you filter on and shows how they map to the states:

Icon	Status	State	Description
	Setup	Pending	Represents a state where the query is waiting to be scheduled on the query pool.
		Metadata Retrieval	Represents a state where metadata schema is retrieved and the SQL command is parsed.
		Planning	Represents a state where the following are done: Physical and logical planning Reflection matching Partition metadata retrieval Mapping the query to a queue based workload management rules Pick the engine associated with the query to run the query.
	Engine Start	Engine Start	Represents a state where the engine starts if it has stopped. If the engine is stopped, it takes time to restart for the executors to be active. If the engine is already started, then this state does not have a duration.
	Queued	Queued	Represents a state where a job is queued. Each queue has a limit of concurrent queries. If the queries in progress exceed the concurrency limit, the query should wait in the queue until the jobs in progress complete.
	Running	Execution Planning	Represents a state where executor nodes are selected from the chosen engine to run the query, and work is distributed to each executor.
		Running	Represents a state where executor nodes execute and complete the fragments assigned to them. Typically, most queries spend more time in this state.
		Starting	Represents a state where the query is starting up.
	Canceled	Canceled	Represents a terminal state that indicates that the query is canceled by the user or an intervention in the system.
	Completed	Completed	Represents a terminal state that indicates that the query is successfully completed.
	Failed	Failed	Represents a terminal state that indicates that the query has failed due to an error.

Viewing job details

On the Jobs page, you can also view details of a specific job. Click a job that you want to see the details of, and the Job Details page replaces the list of jobs.

Overview

The Overview tab shows the following attributes:

• Summary:

  • Status
  • Total Memory
  • CPU Used
  • Query Type
  • Start Time
  • Duration
  • Wait on Client
  • User
  • Engine
  • Input
  • Output

• Total Execution Time: The length of time for the total execution and the job state durations in the order they occur. Only the duration of the Engine Start state is in minutes and seconds. If the engine is stopped, it takes time to restart for the executors to be active. If the engine is already started, then Engine Start duration does not have a value.

• Submitted SQL: The SQL query for the selected job.

• Queried Datasets

• Scans

SQL

In the SQL tab, you can see the following details:

• Submitted SQL
• Dataset Graph

Execution

The Execution tab shows the nodes used to execute the job. On the left panel, you can sort by the runtime, total memory, and number of records for each node. The middle panel represents a visualized workflow for each phase, and the right panel provides the details for runtime, processed batches and records, peak memory, bytes sent, and aggregate statistics.

Raw Profile

Click the Raw Profile tab to open a raw profile of the job in a separate dialog, which includes a job summary, state durations, threads, resource allocation, operators, visualized plan, acceleration, and other details.

Results

To see the job results, click the Open Results link in the top-right corner of the Jobs Details page. As long as the engine that ran the job is up, the Open Results link is visible in the UI. It disappears when the engine that ran the job shuts down and is only visible for the jobs that are run through the UI.

Profile

To download the query profile, click the Download Profile button in the bottom-left corner of the Jobs Details page. The profile will help you see more granular details about the job.