Skip to main content
Version: current [26.x Preview]

26.x Release Notes

26.0.0 (April 2025) Preview​

note

This public preview release of Dremio 26.0.0 is intended exclusively for new deployments only using the latest versions of the Helm charts. Upgrades from earlier versions are not supported as part of this preview. See Terms of Service.

We welcome feedback on this public preview release. Please submit your feedback to beta@dremio.com.

What's New​

General Updates​

  • Autonomous Reflections automatically creates and manages reflections, accelerating query performance and optimizing data accessibility. This enables users to derive faster insights, scale effortlessly, and make data-driven decisions.

    DX-89558

  • Dremio now supports results cache management and eviction without having to rely on the time-to-live (TTL) policy of the cache storage. The results are stored in a distributed storage configured via dremio.conf. If you are upgrading from 25.1, you must manually remove any existing TTL policy configured. Queries accelerated by the results cache now display a lightning bolt icon and indicate a cache hit on the Job Overview page.

    DX-101298, DX-94850

  • Dremio supports the creation, modification, introspection, and deletion of Kubernetes-based engines via the Dremio console or the Engine Management API. Engines are T-shirt sized and can be set up to automatically start and stop, i.e., they can automatically start up as needed by queries and stop after an idle time without queries. Dremio allows setting guardrails and timeouts for the engine lifecycle via the Dremio console or the Engine Management API. For more details, see Managing Engines.

    DX-97061

  • Dremio clusters on Kubernetes will automatically transmit telemetry data back to Dremio's corporate endpoint. This telemetry is comprised of Prometheus metrics, which provide valuable insights into system performance and health, enabling Dremio to enhance product stability and support.

    DX-97022

  • Dremio now deploys with a built-in Iceberg catalog, powered by Apache Polaris (incubating). Learn more about this Dremio Catalog.

    DX-96982, DX-96991, DX-96989, DX-96988, DX-96907, DX-97113, DX-97955, DX-100986, DX-101757

  • You can now use clustering as a straightforward and effective alternative to partitioning. Clustering simplifies processes and helps ensure fast queries on Apache Iceberg tables.

    DX-92001

  • Parameterized prepared statements are supported for Arrow Flight SQL JDBC to prevent SQL injection and to enable users to leverage client tools that support parameterized prepared statements. This update supports SELECT statements.

    DX-10013, DX-89042

  • Dremio now supports the OAuth2.0 token exchange endpoint, which allows client applications to set up external OAuth authentication for ODBC, JDBC, Arrow Flight, and REST API requests.

    DX-95730

  • Dremio now supports a generic Iceberg REST Catalog as a source, which includes vended credentials. This would allow you to connect to Iceberg catalogs over the Iceberg REST API. This source is located within the Lakehouse Catalogs.

    DX-102149, DX-98996, DX-98713, DX-100065

  • The Polaris Catalog (Preview) source has been renamed to Snowflake Open Catalog. This rename should happen automatically after upgrading to Dremio 26+. With this source connector, you can connect and read from internal and external Snowflake Open Catalogs and write to internal Snowflake Open Catalogs. All interaction with the underlying Iceberg tables is done via vended credentials resulting is a more secure solution.

    DX-96947, DX-97271, DX-96947, DX-96951, DX-101757

  • Users can now connect to and read from Unity Catalog. All interaction with the underlying Delta tables is done via UniForm metadata. Access to the underlying storage layer is done via vended credentials.

    DX-96951

  • Dremio supports SAP HANA as a source.

    DX-93897, DX-47285

  • Dremio supports Google BigQuery as a source.

    DX-93897

APP​

  • Added a new Engines page in the Dremio console for managing engines and executors in Kubernetes deployments where users can create, set an idle time, and easily delete engines.

    DX-99357

  • The Lineage tab has been updated to give a more modern and improved user experience, allowing users to customize the layers of information shown on the lineage nodes. Lineage is supported for Dremio Catalog (powered by Polaris), Unity, and all Iceberg REST Catalogs.

    DX-96716, DX-97045, DX-97065

  • Quickly discover your data using the AI-enabled semantic search. Object names, wikis, labels, column names, and other metadata is used to serve the most relevant datasets and entities. This capability is only supported for Kubernetes deployments.

    DX-96145, DX-93754, DX-97047

SQL​

  • Added support for COPY INTO transformations with CSV input type, extended COPY INTO syntax to enable simple transformations during a data load, and added EMPTY_AS_NULL functionality with COPY INTO transformations. Note: EMPTY_AS_NULL is evaluated only on the CSV input data, not on any function result or subresult.
    DX-96995, DX-96064, DX-90263

Improvements and Issues Fixed​

General Updates​

  • Improved the performance of AWS Glue Data Catalog table reads.

    DX-96290

  • Following Microsoft's official retirement of Azure Data Lake Storage Gen1 last year (Feb 2024), Dremio has officially removed the Azure Data Lake Storage Gen1 Storage source from its product.

    DX-88437, DX-88433, DX-86550

  • Dremio supports Azure Storage container writes with the Azure Blob File System (ABFSS) driver. Iceberg metadata on Azure now defaults to the abfss:// format instead of wasbs://.

    DX-66589, DX-99125

  • You can now configure the managed identity that Dremio needs to use for Azure key vault authentication.

    DX-94964

  • Fixed an issue that could prevent async Azure reads due to a time zone issue in locations east of Greenwich Mean Time (GMT).

    DX-93739

  • The default paths.dist value in dremio.conf has been updated from the deprecated PDFs to a local filesystem path. The paths.dist value must be updated to a Dremio-supported distributed store in production or multi-node situations. Dremio will now be fully functional for single-node demo installs.

    DX-89368, DX-91559

  • Upgraded AWS SDK to version 2.18.0+ along with related dependency updates and compatibility adjustments.

    DX-60924

  • Changed the default from a 32 KB field size limit to a 16 MB row width limit, allowing users to interact with larger datasets without having to change support keys.

    DX-89764, DX-94238

  • Fixed an issue where high setup times could be seen when running queries against Iceberg tables.

    DX-100856

  • Fixed an issue where merge-on-read DML operations could fail on Apache Iceberg tables with partition transforms.

    DX-99569

  • When writing Parquet files to Iceberg tables:

    • store.parquet.block-size and dremio.iceberg.optimize.target_file_size_mb are now ignored.

    • The write.parquet.row-group-size-bytes property now controls row group and file size, with a default value of 128 MB instead of 256 MB.

    • write.metadata.metrics.max-inferred-column-defaults and write.metadata.metrics.default now control file-level metrics in Iceberg metadata. The write.metadata.metrics.max-inferred-column-defaults property is now 100 instead of unlimited.

    • The write.metadata.metrics.default mode is now truncate(16) instead of full.

    • The default value for write.parquet.page-row-limit is now 20,000 instead of unlimited.

    • The write.parquet.page-size-bytes property is now used to specify the Parquet data page size of newly created Parquet files in a table instead of the store.parquet.page-size support option and the default value has changed from 1 KB to 1 MB.

    • The write.parquet.dict-size-bytes property is now used to specify the Parquet dictionary page size of newly created Parquet files in a table instead of the store.parquet.dictionary.page-size support option. The default value has changed from 1 MB to 2 MB.

    • The write.parquet.compression-codec and write.parquet.compression-level properties are now used to specify the Parquet compression codec/level for newly created Parquet files in a table instead of the store.parquet.compression and store.parquet.compression.zstd.level support options. The default value of the codec has changed from snappy to zstd. The default value of the level is up to the codec library.

    • The compatibility.snapshot-id-inheritance.enabled property is no longer written by default but could be used for v1 Iceberg tables for backward compatibility reasons.

    • Dremio may write multiple row groups into Parquet files according to the Iceberg table properties write.target-file-size-bytes and write.parquet.row-group-size-bytes.

      DX-99417
      DX-99441
      DX-99743
      DX-98865
      DX-98550
      DX-97618
      DX-50828
      DX-101321

  • Fixed an issue where some correlated subqueries led to a "Cannot convert RexNode to equivalent Dremio expression" error message.

    DX-58073

  • Fixed an issue where updating the source caused rerunning a query to fail with a "Source X was updated, and the given configuration has older version (current: y, given: z)" error message.

    DX-69486

  • Fixed an issue that could cause an "Unable to find the reference field" error during query planning due to Common Subexpression Elimination (CSE).

    DX-90660

  • Fixed an issue where a query with window functions used with non-nullable decimal columns fails with a "Schema does not match expected schema" error message.

    DX-102023

  • Fixed a slow resource leak on the coordinator that can slow down queries and fill the heap memory for idle Workload Management (WLM) queues that are tied to engines. The leak is proportional to the number of idling WLM queues and may fix itself when a query is submitted to those queues.

    DX-88418

  • Fixed an issue where queries using reflections failed or referenced stale materializations.

    DX-100896

  • Reflection manager now starts to collect the duration for every refresh job:

    • Fixed the miscalculation of the lastRefreshDuration field in the system materializations table
    • Enhanced the reflection refresh retry policy with a dynamic interval where long-running refresh jobs and a small number of maximum retries are better handled
      DX-101134

  • Fixed an issue where we could fail to write a generated reflection score due to concurrent modification exception.

    DX-100983

  • Enhanced default raw reflection matching to include query tree hashes so that more SQL queries can match without direct view references.

    DX-64652

  • Fixed an issue where query phase timeouts would not impact reflection matching.

    DX-98080

  • Fixed an issue where, in rare cases, the retry policy for reflection refreshes would not exponentially back off.

    DX-99314

  • Reduced the jobs metadata size by no longer persisting the reflection matching plans into REFRESH REFLECTION jobs.

    DX-98022

  • Fixed an issue where REFLECTION REFRESH jobs could fail for reflections involving joins in the query plan if field-based incremental refresh was configured on the underlying datasets. These reflection refreshes will now succeed using full refreshes.

    DX-97085

  • Enabled reflections for cases when there is an equivalent row and column access control (RCAC) filter on all children of a view that is the union of two or more datasets.

    DX-95563

  • Fixed an issue that could cause garbage collection logs to not be accessible.

    DX-98240

  • Fixed a NullPointerException that could occur during a metadata refresh due to closing a filesystem object already evicted from the cache.

    DX-88625

  • Reduced noise in the logs generated when reading data from Parquet files by changing the log level of certain messages from WARNING to DEBUG.

    DX-101258

  • Changed automated backup not to reuse a backup destination folder. On every backup, a new folder with the pattern <distStore>/backups/dremio_backup_<timestamp>-<dremio_version> is now created.

    DX-95898

  • Incompatible runtime filters are now ignored so as to avoid an UnsupportedOperationException while setting up Parquet readers.

    DX-90910

  • Updated Apache Arrow to 18.1.

    DX-96942

  • Fixed an issue that could occur when a routing rule with a query_label() condition would not work as expected for queries triggered using Arrow Flight client applications.

    DX-99808

  • When the results cache is enabled, certain ODBC and JDBC clients might receive an error when running a query involving complex types if the results for that query had been previously cached by a newer ODBC or JDBC driver.

    DX-100448

  • You can now update Sample Source and SampleDB source settings with the necessary permissions.

    DX-94810

  • Improved the planning time in the filter pushdown phase for a query with too many joins.

    DX-96044

  • Reduced coordinator memory footprint due to interim query telemetry updates, thereby improving reliability. There will be a slight increase (~2MB per query) in transient disk space as these interim updates are now stored on disk. This additional storage space will be freed once the query finishes.

    DX-90141

  • Fixed an issue that could cause higher resource utilization on the coordinators due to the incorrect scheduling of a background task that deletes old jobs.

    DX-95235

  • Updated the Snowflake driver to 3.19.0 to resolve SNOW-928973.

    DX-92315

  • When creating a new PAT, Dremio will read the initial value from the auth.personal-access-token.max_lifetime_days support key.

    DX-99097

  • Updated the following libraries to address potential security issues:

    • Aircompressor from 0.10 to 0.27 [CVE-2024-36114]
      DX-96853
    • Apache Avro library [CVE-2024-47561]
      DX-99132
    • Apache HBase to version 2.6.0 [CVE-2023-44487]
      DX-96283
    • AsyncHttpClient to 3.0.1 [CVE-2024-53990]
      DX-98578
    • Avro from 1.11.3 to 1.11.4 [CVE-2024-47561]
      DX-96442
    • Elasticsearch client library to 8.14.2 and removed support of clusters older than Elasticsearch version 7
      DX-100797
    • commons-configuration to version 2.11.0 [CVE-2024-29131]
      DX-96821
    • jaeger-client library version 1.8.1 [CVE-2020-13949]
      DX-96096

  • Disabled all TRACE web requests, which now return 405(NOT_ALLOWED) error codes.

    DX-97052

  • Fixed an issue where a duplicated table schema could be written to its metadata file.

    DX-97502

  • Fixed a filter pushdown issue that could cause a query to run slowly or return incorrect results.

    DX-97512, DX-97880

  • Fixed an issue with a permanent query slot loss in multi-coordinator setups that could gradually lose query concurrency slots on Workload Management (WLM) queues at very high loads when a client application pushes more queries to a particular WLM queue than the configured concurrency, causing the client application to time out and close its connection. Before the fix, restarting the coordinators was the only way to fix this issue.

    DX-98355

  • Removed legacy support for pseudo-distributed file system (PDFS) nodes and non-Iceberg reflection storage.

    DX-60382

  • Warnings about schema being out of date are no longer visible for views in Dremio.

    DX-101973

  • Fixed a defect where we could get wrong data in a reflection when an extremely large amount of data files are present.

    DX-101552

  • Join keys over 128 KB are now handled correctly.

    DX-99636

  • Removed support for MapR Hadoop distributions.

    DX-99456, DX-99455

  • Fixed an issue where excessive memory usage may occur when executing queries containing UNION ALLs or when executing highly parallelized queries.

    DX-97947

  • Adjusted metadata collection methods to reduce the load of queries to a MongoDB instance.

    DX-85654

  • Removed the following support keys:

    • reflection.manager.skip_drop_table_job_for_incremental_refresh

      DX-90480

    • reflection.manager.refresh_schedule_policy.enabled

      DX-92633

    • exec.operator.aggregate.vectorize.use_spilling_operator

      DX-96210

    • dremio.jobs.new.ui

      DX-97516

  • Increased the default heap memory on executor nodes from 8 GB to 12 GB when memory is more than 100 GB to avoid query failures due to running out of heap.

    DX-96441

  • Updated Hadoop version used with Azure paths to use Dremio's Hadoop 3.3.6 version.

    DX-96228

  • Fixed an array index out of bound issue with query planning for views that do not project ORDER BY columns.

    DX-99095

  • The maximum number of max reflection refresh retry attempts is no longer user configurable.

    DX-98535

  • Reduced the size of query profiles by removing serialized logical plans. Improves planner performance and reduces query profile footprint.

    DX-98491

  • Improved user experience by minimizing the time during which no recommendations are available.

    DX-98299

APP​

  • Implemented new look and feel for logging in to the Dremio console.

    DX-101440, DX-101647

  • Fixed an issue where a benign error would be logged with a message like "FileSystem is closed!" when shutting down Dremio.

    DX-91951

  • Users with MODIFY privileges can now access source settings in the Dremio console without getting stuck in a permanent loading state.

    DX-96544

  • Deprecated and removed previous job pages.

    DX-97516

  • Acceleration profile now shows full stack trace of the error when verbose profile is enabled.

    DX-101247

  • Removed Acceleration Settings on the Reflections page.

    DX-98300

  • Icons indicating that a column is Partitioned / Sorted are now visible in the Details panel and Details tab on the Datasets page. Also, the same icons will be visible in the Data panel without having to hover on a given column.

    DX-99302

  • Fixed an issue when using a backslash in the search box for searching for reflections on the Reflections page.

    DX-95530

  • Fixed an issue where a reflection could have an empty reflection score on the Reflections page.

    DX-100774, DX-99208, DX-99040, DX-99129

  • The Recommended Joins tab in the SQL Runner is deprecated. Starting Dremio 26.0.0, all joins need to be created manually.

    DX-101102, DX-96716

  • Switching between query tabs in the SQL Runner will no longer cause the duration timer to reset.

    DX-99577

  • The script lookup in the All/Mine tabs on the Scripts panel in the SQL Runner was previously case-sensitive. Now, the script lookup is case insensitive.

    DX-48612

  • Navigating to a dataset with dots in the name on the History tab of the Datasets page will now work as expected. History is no longer shown in the SQL Runner for views or scripts.

    DX-98406, DX-98252, DX-98332

  • Fixed an issue that could prevent users from being able to run or preview a query in the SQL Runner after viewing the History tab for the query on the Datasets page.

    DX-96139

  • Fixed an issue where clicking on a script in the search results on the Datasets page does not open that script in the SQL Runner.

    DX-98620

  • Navigating to the wiki of a dataset from the SQL Runner will no longer cause (edited) to appear next to the dataset name.

    DX-96470

  • Tables in the wiki view were previously cut off. You should now be able to see the full width of the table.

    DX-97221

  • Added a new Rerun query on download option to the Preferences page in the Dremio console. When enabled, Dremio reruns the query before downloading the results.

    DX-101308

SQL​

  • The SHOW TBLPROPERTIES SQL command will now return the format version for Iceberg tables.

    DX-87471

  • The vacuum log vacuum.json file will now capture detailed information about scanning and deletion through VACUUM TABLE and VACUUM CATALOG SQL commands.

    DX-99408

  • Fixed an issue that could cause VACCUM CATALOG to use synchronous readers for metadata JSON files, which might cause memory leak issues. Now VACUUM CATALOG uses asynchronous readers instead.

    DX-98605

  • Addressed an issue with REGEXP_COL_LIKE in certain cases producing incorrect results.

    DX-97524

  • EXECUTE statements can now utilize the plan cache entry created by the prepared statements.

    DX-100780

  • Improved performance of the varchar IN filter evaluation.

    DX-92584

  • Fixed an issue preventing access control policies to be added to Iceberg REST catalog tables using ALTER TABLE command.

    DX-100489

  • Queries that use TABLESAMPLE now throw a meaningful error message because they are not supported in Dremio.

    DX-48838

  • Fixed an issue that could cause a query with a LIMIT clause to not be accelerated due to miscalculated row adjustments.

    DX-96246

  • Improved implicit casting for some operands. As an example, SUBSTRING can cast integer input as characters, so SELECT SUBSTRING(42, 1) successfully returns 42.

    DX-89554

  • Fixed an issue in some cases that could prevent runtime filtering if the CONVERT_FROM SQL function was used in the query.

    DX-93179

  • Fixed an issue that could cause an OutOfMemoryException (OOM) to be ignored in a HASH_JOIN operator, which is now explicitly thrown to prevent unexpected behavior with the HASH_JOIN.

    DX-94029

  • Fixed an issue in some cases where filters on the date column with COALESCE were preventing the partition filter from being pushed down.

    DX-97665

  • Fixed an issue where decorrelating a subquery with an EXISTS statement and an empty GROUP BY clause could result in incorrect data.

    DX-96652, DX-96946

  • Fixed an issue with zero milliseconds that could occur when using the CURRENT_TIME function.

    99210

  • Fixed an issue where a schema error could be thrown if a user ran an INSERT SQL command with a subset of the columns in an Iceberg table.

    DX-92442

  • Fixed a nested user-defined function issue where a UDF was unable to call other UDFs.

    DX-95612

  • Fixed a issue that could occur when you request reflection recommendations for a specific job and the query you want to accelerate contains a subquery.

    DX-97274

  • Resolved an issue for SELECT queries when using the LAG SQL function with an OFFSET parameter that could cause incorrect results.

    DX-99536

  • Fixed an issue for SELECT queries when using LIMIT and OFFSET for a value greater than the maximum value for a signed integer. Now LIMIT and OFFSET cannot exceed the maximum integer value.

    DX-97963, DX-18632

  • You can now use columns with a VARCHAR data type in a SELECT subquery of a SELECT statement.

    DX-83189

  • Fixed the PARSE_URL SQL function to handle URLs longer than 256 characters.

    DX-100691

  • Fixed an issue that could occur with class compilation when using a literal value in a window function.

    DX-100466

  • Fixed a stack overflow issue with queries containing a long list of constants in a WHERE IN clause.

    DX-98414

  • Queries now scan only the required fields from complex structure type columns to prevent errors in query execution.

    DX-93643

  • Added an explicit exception for the unsupported IGNORE NULLS option in window functions.

    DX-100400

  • Fixed a bug where queries using ROW types may fail with the error "DremioSqlValidator is a dummy SqlValidator to adapt to upstream API changes".

    DX-100331

  • Fixed an issue that could cause a CompileException error when using a large number of columns in an ORDER BY clause.

    DX-99689

  • OPTIMIZE TABLE now will honor Iceberg table properties like write.target-file-size-bytes. Also, the default has changed to 512 MB to align with Iceberg's default and increase Dremio's interoperability with other engines. In addition, OPTIMIZE TABLE will honor the following table properties:

    • dremio.iceberg.optimize.minimal_file_size_mb
    • dremio.iceberg.optimize.maximal_file_size_mb
    • dremio.iceberg.optimize.minimal_input_files
DX-99413, DX-100951

Breaking Changes​

  • Enforced reflection count guardrail so new reflections cannot be created if there are already 500 or more active reflections.

    DX-94627

  • Because the Arrow caching feature for reflections has been deprecated, any remaining references to Arrow cache for reflections have been removed, such as in the sys.reflections system table and APIs.

    DX-53451

  • Deprecated COMPACTION and LOAD MATERIALIZATION for reflection jobs because they optimize non-Iceberg materialization and Dremio now supports only Iceberg materialization. The sys.materializations table now only shows REFRESH REFLECTION jobs. COMPACT and LOAD are no longer considered reserved keywords for SQL queries.

    DX-64438

Known Issues​

  • Dremio Catalog does not inherit role-based access control (RBAC) privileges that have been granted at the system level. Instead, you need to grant RBAC privileges directly on the Dremio Catalog.

    DX-101932

  • You cannot drop tables and views through the Dremio console for Dremio Catalog, Snowflake Open Catalog, and Iceberg REST Catalog sources. Instead, you must use the DROP TABLE or DROP VIEW SQL commands.

    DX-102380

  • If a user only has the USAGE and SELECT privileges on a table and tries to run an INSERT operation on that table, Dremio currently returns a false positive error message that says the table has been updated, but nothing actually happens.

    DX-102429

  • S3-compatible storage does not support credential vending as an authentication method. You must use master storage credentials to authenticate access to S3-compatible storage.

    DX-101841