Skip to main content
Version: current [25.0.x]


As an administrator, you can monitor logs, usage, system telemetry, jobs, and Dremio nodes.


Learn more about the different types and file locations for logs, and consider exporting your logs to a central location for analysis.

Default Log File Locations

By default, Dremio uses the following locations to write logs:

  • Tarball - <DREMIO_HOME>/log
  • RPM - /var/log/dremio

Log Types

Audit Logs

All user activities performed within Dremio are tracked in the audit.json file. For details, see Audit Logging.

System Logs

The following logs are enabled by default:

  • access.log - HTTP access log for the Dremio web server. This log will be generated by coordinator nodes only.
  • server.gc - Garbage collection log.
  • server.log and json/server.json - Server logs are generated in a text format (server.log) and a json format (json/server.json). An admin can disable one of these formats.
  • server.out - Log for Dremio daemon standard out.
  • metadata_refresh.log - Log for refreshing metadata.
  • tracker.json- Tracker log.

Query Logs

Query logs are located in the queries.json file. This file contains a log of completed queries; it does not include queries currently in planning or execution.

Query logging is enabled by default.


Query logs can be queried by Dremio itself or another tool for monitoring and analytics.


Query logs include the following information:

  • queryId - Unique ID of the executed query.
  • queryText- SQL query text.
  • start - Start time of the query.
  • finish - End time of the query.
  • outcome - Whether the query was completed or failed.
  • username - User that executed the query.
  • commandDescription - Type of the command. This may be a regular SQL query execution job or another SQL command.

Additional information may be found depending on your Dremio configuration.

Warning Logs

Warnings can be generated in the hive.deprecated.function.warning.log for Hive functions that have been deprecated. If you see a warning generated in this log, locate the deprecated function and replace it with a supported function. For example, you would replace NVL with COALESCE.

Retrieving Logs from Kubernetes

To retrieve logs from Kubernetes, use the container console for Amazon Elastic Container Service for Kubernetes (EKS), Azure Kubernetes Service (AKS), or Google Kubernetes Engine (GKE). You can also use the AKS container to retrieve logs for AKS.

Using the Container Console

All logs are written to the container's console (stdout) simultaneously. These logs can be monitored using either kubectl command:

Command for viewing logs using kubectl logs
kubectl logs <container-name>
Command for viewing logs using kubectl logs -f
kubectl logs -f <container-name>

You can also write logs to a file on disk in addition to stdout. Read Writing Logs to a File for details.

Using the AKS Container

Azure provides integration with AKS clusters and Azure Log Analytics to monitor container logs. This is a standard practice that puts infrastructure in place to aggregate logs from containers into a central log store to analyze them.

AKS log monitoring is useful for the following reasons:

  • Monitoring logs across lots of pods can be overwhelming.
  • When a pod (for example, a Dremio executor) crashes and restarts, only the logs from the last pod are available.
  • If a pod is crashing regularly, the logs are lost, which makes it difficult to analyze the reasons for the crash.

For more information regarding AKS, see Azure Monitor features for Kubernetes monitoring.

Enabling Log Monitoring

You can enable log monitoring when creating a AKS cluster or after the cluster has been created.

Once logging is enabled, all your container stdout and stderr logs are collected by the infrastructure for you to analyze.

  1. While creating a AKS cluster, enable container monitoring. You can use can existing Log Analytics workspace or create a new one.
  2. In an existing AKS cluster where monitoring was not enabled during creation, go to Logs on the AKS cluster and enable it.

If you want to persist logs in the PVC, follow the instructions here.

Viewing Container Logs

To view all the container logs:

  1. Go to Monitoring > Logs.
  2. Use the filter option to see the logs from the containers that you are interested in.


Monitoring usage across your cluster makes it easier to observe patterns, analyze the resources being consumed by your data platform, and understand the impact on your users.

Cluster Usage

Dremio displays the number of unique users who executed jobs on that day and the number of executed jobs.

  1. Hover over the help icon in the left navigation bar.

  2. Click on About Dremio in the menu.

  3. Click on the Cluster Usage Data tab.

Catalog Usage Enterprise

The data visualizations on the Monitor page point you to the most queried datasets, spaces, and source folders in your data catalog.

Go to Settings > Monitor to view your catalog usage. When you open the Monitor page, you are directed to the Catalog Usage tab by default where you can see the following metrics:

  • A table of the top 10 most queried datasets within the specified time range, including for each the number of linked jobs, the percentage of linked jobs in which the dataset was accelerated, and the total number of reflections defined on the dataset

  • A table of the top 10 most queried spaces and source folders within the specified time range, including for each the number of linked jobs and the top users of that space or folder


A source can be listed in the top 10 most queried spaces and source folders if the source contains a child dataset that was used in the query (for example, postgres.accounts). Queries of datasets in sub-folders (for example, s3.mybucket.iceberg_table) are classified by the sub-folder and not the source.

All datasets are assessed in the metrics on the Monitor page except for datasets in the system tables, the information schema, and home spaces.

The metrics on the Monitor page analyze only user queries. Refreshes of data reflections and metadata refreshes are excluded.

Jobs Enterprise

The data visualizations on the Monitor page show the metrics for queries executed by your cluster, including statistics about performance and utilization.

Go to Settings > Monitor > Jobs to open the Jobs tab and see an aggregate view of the following metrics for the jobs that are running on your cluster:

  • A report of today's job count and failed/canceled rate in comparison to yesterday's metrics

  • A list of the top 10 most active users within the specified time range, including the number of linked jobs for each user

  • A graph showing the total number of completed and failed jobs over time (aggregated hourly or daily)

  • A graph of all completed and failed jobs according to their queue name (aggregated hourly or daily)

  • A graph of all job states showing the percentage of time consumed for each state

  • A table of the top 10 longest running jobs within the specified time range, including the linked ID, duration, user, query type, and start time of each job

To examine all jobs and the details of specific jobs, see Viewing Jobs.

System Telemetry

Dremio exposes system telemetry metrics in Prometheus format by default. It is not necessary to configure an exporter to collect the metrics. Instead, you can specify the host and port number where metrics are exposed in the dremio.conf file and scrape the metrics with any Prometheus-compliant tool.

To specify the host and port number where metrics are exposed, add these two properties to the dremio.conf file:

  • set to the desired host address (typically or the IP address of the host where Dremio is running).
  • services.web-admin.port: set to any desired value that is greater than 1024.

For example:

Example host and port settings in dremio.conf ""
services.web-admin.port: 9090

Restart Dremio after you update the dremio.conf file to make sure your changes take effect.

Access the exported Dremio system telemetry metrics at http://<yourHost>:<yourPort>/metrics.

For more information about Prometheus metrics, read Types of Metrics in the Prometheus documentation.