15.0.0 Release Notes (Dremio March 2021)
Preview: Query Plan Caching
Dremio 15.0.0 improves query performance by providing an in-memory cache for the query planner. Dremio stores query plans in the cache so that it doesn’t need to reproduce the query plan the next time it encounters that query. Contact Dremio Customer Support to enable the feature.
Do not use preview features in a production environment.
- Dremio 15.0.0 automatically purges materializations on disk that do not have an entry in
- Dremio 15.0.0 provides a KVStore report API that returns files that help Support determine if individual stores contain incorrect entries. See KVStore Report API for more information.
- Dremio 15.0.0 provides an API to automatically invalidate entries in the LDAP cache for specified Dremio users. See Invalidate LDAP User Cache API for more information.
- Dremio 15.0.0 supports queries that contain non-ISO-8859-1 literals, such as ‘€’. See Configuring Dremio for Queries with non ISO-8859-1 Literals for instructions.
- Dremio supportsRuntime Filtering on partitioned columns for the Delta Lake data format.
Fixed Issues in 15.0.0
CVE-2020-25649: Vulnerability reported against jackson-databind-2.10.2.jar
Dremio is not affected by this vulnerability. However, we upgraded to
jackson-databind-2.11.4.jar out of caution.
CVE-2020-13956: Vulnerability reported against org.apache.httpcomponents.httpclient version 4.5.5
Dremio is unlikely to be affected by this vulnerability. However, we upgraded to
httpclient 4.5.13 out of caution.
Dremio doesn’t clean up spill directories when Dremio crashes or is shutdown
Fixed by implementing a management module for temporary folders.
Dremio may not automatically clean up spill directories and their files from prior releases due to changes in the underlying directory structure. See Cleaning up Spill Directores for more information.
Queries with a
JOIN statement run slowly
Resolved by pushing down join conditions on literals below the
JOIN statement to avoid NestedLoopJoin.
Slow queries with the Oracle Connector when casting numeric columns
Resolved by omitting added casts in filters for decimal types.
Creating an external relational database source overwrites Spaces in Dremio with the same name
Fixed by cleaning up the plugin state in case of persistent source failures in the KVStore.
Query returns error: Source ‘LAMP SQL SERVER’ returned error ‘Arithmetic overflow error converting numeric to data type varchar
Resolved SQL Server numeric overflow when converting to VARCHAR.
Query returns error: ‘GRIP SQL Server’ returned error ‘The column ‘Business_Date’ was specified multiple times
Fixed by rewriting aliases to be unique.
Dremio errors in AWS CloudTrail log: “nosuchkey”
Resolved by adding an interface implementation for S3 FS and using the AWS SDK for health check status.
Dremio improperly pushes down Window functions into a
Fixed by pushing down the
SELECT statement into a subquery and referencing the WINDOW function column in the outer
Dremio returns an error when querying MS SQL sources with predicates based on
BOOLEAN data types
Resolved by detecting and preventing pushdowns of bit columns treated as boolean expressions.
Dremio cancels query: SCHEMA_CHANGE ERROR: New schema found. Please reattempt the query. Multiple attempts may be necessary to fully learn the schema
Fixed by moving operations related to the Delta Lake data format out of the scope of the following system properties:
Same query using
FULL OUTER JOIN SQL statement returns different number of results
Resolved by not applying runtime filtering to
FULL OUTER JOIN SQL statements.
15.2.0 Release Notes (Enterprise Edition Only)
JDBC Driver 15.2.0
- Dremio 15.2.0 provides an updated version of the Dremio JDBC driver.
Dremio 15.1.0 provides a support key,
exec.gandiva.target_host_cpu, to enable and disable CPU-specific instructions in generatedGandiva code. The key is enabled by default. Dremio administrators should disable this support key to prevent Gandiva from generating CPU-specific instructions, such as AVX and AVX512. See Support Keys for instructions on setting support keys.
Fixed Issues in 15.2.0
Queries in Dremio 14.3.0 take longer when runtime filtering is enabled
Fixed by revising the scan of the fact table to include any number of
JOIN conditions and choosing only records that pass all filters.
Dremio fails during query using Gandiva-based execution
Resolved by addressing a Gandiva crash in select virtual machines.
Dremio JDBC driver returns incorrect date and time for date and time-formatted columns
Fixed by adjusting datetime values to use the local timezone rather than UTC.
Coordinator node crashes while promoting a Delta Lake dataset
Resolved by improving the promotion flow for Delta Lake datasets. The new flow does not list all files in the folder for Delta Lake datasets.
15.4.0 Release Notes (Enterprise Edition Only)
Fixed Issues in 15.4.0
Dremio cancels all queries after reaching the timeout limit
Fixed by ensuring that the Columnar Cloud Cache releases locks after threads encounter exceptions.
15.5.0 Release Notes (Dremio April 2021)
Fixed Issues in 15.5.0
When querying a MongoDB data source, any
undefined values included in a dataset caused errors.
Resolved by converting
undefined values to
All datasets on an Azure data source intermittently disappear from the Dremio interface due to the Azure REST API returning new attributes in its response to listing containers. Dremio is unable to deserialise the response. Resolved by ignoring unknown attributes from the response.
15.7.0 Release Notes (Dremio June 2021)
Fixed Issues in 15.7.0
When performing numerous concurrent queries, an unused reference to a query profile was causing excessive heap usage in the coordinator.
Resolved by removing unused references.
15.8.0 Release Notes (Enterprise Edition Only)
Fixed Issues in 15.8.0
The client connection was closed abruptly, canceling a query while outstanding messages for a coordinator node had not yet been acknowledged. This prevented any future query messages to the same coordinator from being acknowledged.
All messages are acknowledged by the coordinator, even in the case of a query failing.
15.9.0 Release Notes (Enterprise Edition Only, August 2021)
Fixed Issues in 15.9.0
Dremio was reading beyond the end of an organization’s Parquet files, causing memory issues.
If Dremio reaches the end of a Parquet file and attempts to read any further, an exception is thrown to stop the service.