23.x Release Notes
Releases are listed in reverse order, starting with the latest release of Dremio 23.x.
23.2.4 (January 2024) Enterprise
What's New
- Added the option to force parallelism in write queries by using a round-robin exchange before the writer, even when input is only single-threaded. Enable this option by enabling the support key
planner.writer.round_robin
.DX-64388
Issues Fixed
-
Changing the size of an existing EC2 engine in Dremio's AWS Edition no longer resets the engine type.
DX-54590 -
Split assignment for tables in Delta Lake format no longer result in a NullPointerException.
DX-60247 -
Fixed an issue that was causing the error
Gandiva code generation is handled during build
for CASE WHEN queries that contain nested Java/Gandiva functions.DX-62402 -
Row-level runtime filtering is disabled for reflection refresh jobs so that views no longer return incorrect results due to an incorrect match to a single Starflake reflection.
DX-68144 -
Impersonation users who do not have access to a remote table cannot view local versions of the remote table that are created by reflections.
DX-69553 -
Cancelled queries now immediately stop reading data from data sources.
DX-69759 -
Fixed an issue with node endpoint checks that could cause coordinators to restart.
DX-83120 -
C3 system stats for storage_plugins, mount_points, datasets, and objects were missing due to an internal error. They have been enabled again.
DX-83172 -
To increase coordinator stability, plan cache size is decreased to 1k (from 10k) and time duration is decreased to 8 hours (from 10 hours).
DX-83190 -
Fixed an issue that could cause a
BSONRecordReader
crash for MongoDB data sources.DX-83263 -
Dremio now reads indices with date fields formatted in either Joda-Time or java.time without the use of a prefix.
DX-83727 -
Resolved a path travel security issue that bypassed folder-level role-based access control (RBAC).
DX-84516 -
Updated snappy-java to 1.10.5 to address a potential security issue [CVE-2023-43642].
DX-84755 -
LIMIT
queries can now be executed in parallel rather than using only one thread for execution.DX-83289 -
Removed an errant dependency check that prevented some engines from starting or scaling replicas.
DX-84081
23.2.3 (October 2023) Enterprise
What's New
-
When run by
PUBLIC
users, the data returned by theapiv2/user/<userid>
internal API is limited to only information that is required to search users and assign privileges.DX-82605 -
In this release, the
NOT IN
clause is supported with correlated subqueries.DX-57298
Issues Fixed
-
In some cases, if a deployment had a large number of sources, the SQL Runner was considerably unresponsive when loading the page or typing in the editor.
DX-72618 -
Removed stack trace information from REST API payload JSON parsing error message.
DX-71937 -
In some cases, running
ALTER TABLE <table_path> FORGET METADATA
against a view could result in the view being deleted instead of the command failing with an error.DX-68202 -
Dremio queries in some Tableau executors would start failing after the access token had expired and been renewed.
DX-68133 -
Top-level
CASE
statements intended to return a boolean were not being rewritten correctly, resulting in an error for some SQL Server queries.DX-67527 -
Fixed an issue in C3 recovery logic that was causing C3 to be disabled at startup on some nodes.
DX-67436 -
Incorrect dates were returned when passing a date that was prior to the start of the Gregorian calendar using
TO_DATE
and referencing a data source.DX-66663 -
Fixed an issue that was causing increased startup time for the Jobs service when the jobs history table was very large.
DX-64792 -
Fixed the following issues with acceleration information in job profiles when the plan cache was used: acceleration information was missing for a prepared query, plan cache usage was missing for a prepared query, acceleration information was missing when the query was not accelerated but reflections were considered, canonicalized user query alternatives were missing, and matching hints were missing for reflections that were only considered.
DX-64636 -
Fixed an issue that was causing certain queries to fail when using hash joins, leading to an unexpected restart of an executor.
DX-63029 -
Some queries that contained large
IN
conditions were failing with a stack overflow error.DX-62755 -
In Dremio versions later than v20, casting
+Infinity
was returning an error.DX-61115 -
Fixed a number of issues that were affecting proper handling of inferred partition columns, specifically
FOR PARTITIONS (...)
was not working for inferred partition columns.DX-60565 -
Deleting a space or folder that contained a user-defined function was resulting in an error.
DX-59151 -
Fixed an issue with the default Jobs results cleanup path that was resulting in disk space issues and unexpected restarts on some cluster nodes.
DX-41291
23.2.2 (July 2023) Enterprise
What's New
- Added support for full outer joins that resolve to a
true
join condition.DX-59222
Issues Fixed
-
Top-level
CASE
statements intended to return a boolean were not being rewritten correctly, resulting in an error for some SQL Server queries.DX-67527 -
Fixed an issue that was causing invalid SQL comparison syntaxes in SQL Server queries if nested
CASE
statements were encountered.DX-67183 -
The Dremio vectorized reader could not read invalid Parquet files where a dictionary page offset was used if no dictionary page was present.
DX-64608 -
Fixed a single-stream processing issue and improved execution times when using
UNION ALL
withplanner.unionall_distribute_all_children
set to "true".DX-67138 -
When unlimited splits were enabled, partition pruning was failing due to complex filter conditions that were unable to transform a query to its normalized form using CNF.
DX-60604 -
Fixed query concurrency issues that could lead to "Selected table has no columns" errors.
DX-59967 -
Dremio was generating unnecessary exchanges with multiple unions, and changes have been made to set the proper parallelization width on JDBC operators and reduce the number of exchanges.
DX-55300 -
When unlimited splits were enabled and running incremental metadata refreshes on a file-based table, running subsequent raw reflections would fail with a
DATA_READ
error.DX-51394 -
Fixed an issue that could result in duplicate column names being written by the planner when an expression in the project included a field named
*
.DX-47931
23.2.1 (July 2023) Enterprise
What's New
-
The
COL_LIKE
SQL function has been updated to improve performance.DX-66586 -
This release includes some changes to improve logical planning performance and query planning times in certain scenarios.
DX-66473 -
For Azure Blob Storage and Data Lake Gen 2 sources, you can now enable checksum-based verification to ensure data integrity during network transfers. To enable this option, set the
store.azure.checksum
support key totrue
.DX-63026
Issues Fixed
-
Privilege changes for entities via SQL were not being captured in the audit log.
DX-66643 -
Made an update to ensure that custom (whitelabel) logos will be left-aligned on the Dremio login page.
DX-65948 -
GRANT
commands on catalog entities were failing withRole/User <ID> not found
if existing user or role grantees were no longer present in the system.DX-65364 -
Adding a
CAST
to an Oracle index column was leading to a missed partition key and resulting in an expensive and slow query.DX-64979 -
Fixed an issue that was causing an exception in the BRIDGE_FILE_READER_RECEIVER SQL operator for some queries.
DX-64687 -
If a Hive source connection was slow, Dremio was repeatedly pinging the source to add databases.
DX-64495 -
Some queries that included multiple levels of nested fields were failing.
DX-64106 -
For some queries on Oracle sources, an interval of time was being processed incorrectly, resulting in the following error:
(full) year must be between -4713 and +9999
DX-63742 -
If a corrupted or malformed checkpoint file was encountered during metadata refresh, queries would fail with a
Metadata read Failed
error.DX-62568 -
Queries that utilized external reflections were not being logged in
queries.json
.DX-61654 -
Fixed an issue that was causing
Failed to decode column
anduncompressed_page_size
errors during reflection refresh.DX-61622 -
When parsing CSV, Dremio now allows multi-character strings to be used as field delimiter, quote, quote escape, and comment. Previously, only single characters were supported for these.
DX-60794 -
Fixed query concurrency issues that could lead to "Selected table has no columns" errors.
DX-59967 -
In some cases, Dremio nodes could reboot unexpectedly due to queries that contained deeply nested functions.
DX-57773 -
Resolved a frequent internal hash collision in the C3 eviction path that disabled the cloud columnar cache (C3), potentially causing performance degradation.
DX-66251
23.2.0 (June 2023) Enterprise
Issues Fixed
-
Added more security around DML permission checks to ensure that users can access data only according to their privileges.
DX-64746 -
Improved permission validation around view-based query execution.
DX-64688 -
In this release, the plan cache is user-specific for increased security, and it will be utilized when the same query is executed by the same user.
DX-63531
23.1.50 (April 2023) Enterprise
Issues Fixed
-
In some cases, the JVM's direct memory allocator was triggering garbage collection when there was sustained and high usage of direct memory, which was causing performance issues.
DX50135 -
Container probes have been updated to support Kubernetes versions greater than 1.20. Dremio V2 helm charts did not have
timeoutSeconds
configured for readiness probes and were failing if the check took more than one second.DX-49122 -
When using the
ui.whitelabel.url
support key to apply a custom logo in Dremio, the logo was not being displayed in the side navigation bar.DX-64491 -
Pushdowns were not working for
UUID
data types on a PostgresSQL source. This change maps PostgresSQL'sUUID
type to Dremio'sVARCHAR
. Comparison operators (=, >, <, <=, >=, !=) between aUUID
and aUUID
and between aUUID
and aVARCHAR
will now be pushed down.DX-62978 -
At times, using the
CONCAT
function with non-string operands would yield results that were truncated incorrectly.DX-61914 -
Expression operator names were being used as intermediate field names. In some queries, the multiplication operator
*
was later treated as aSELECT *
statement, which was causing physical planning to fail to find the field.DX-61572 -
When querying
INFORMATION_SCHEMA
tables as a non-admin user from JDBC/ODBC applications, the query was taking much longer than when performed by an admin user.DX-61168 -
When unlimited splits were enabled, partition pruning was failing due to complex filter conditions that were unable to transform a query to its normalized form using CNF.
DX-60604 -
If a view used in a raw reflection contained the
CONVERT_FROM()
function, trying to access the view would result in a planning error.DX-59542 -
In some cases, removing and adding privileges for a user on a space was failing with a "Failed to create SPACE with: Role not found" error.
DX-59527 -
Fixed some issues that were causing performance issues with the
REGEXP_LIKE
SQL function.DX-59182 -
Some queries were performing poorly if the query contained an
ORDER BY
clause.DX-58777 -
A Zookeeper class was missing from the JDBC jar in some earlier releases, resulting in a ClassNotFound exception.
DX-56584 -
When using the
GROUP BY
expression with an aliased column of the same name as the original column, a validation error was indicating that the column was not being grouped.DX-48015 -
Reflections that had zero rows were not available for substitution.
DX-20318
Known Issues
- In some cases, if a deployment has a large number of sources, the SQL Runner can be considerably unresponsive when loading the page or typing in the editor. DX-72618