15.0.0 Release Notes (Dremio March 2021)

What’s New

Preview: Query Plan Caching

Dremio 15.0.0 improves query performance by providing an in-memory cache for the query planner. Dremio stores query plans in the cache so that it doesn’t need to reproduce the query plan the next time it encounters that query. Contact Dremio Customer Support to enable the feature.

Note:
Do not use preview features in a production environment.

Other Enhancements

  • Dremio 15.0.0 automatically purges materializations on disk that do not have an entry in sys.reflections.
  • Dremio 15.0.0 provides a KVStore report API that returns files that help Support determine if individual stores contain incorrect entries. See KVStore Report API for more information.
  • Dremio 15.0.0 provides an API to automatically invalidate entries in the LDAP cache for specified Dremio users. See Invalidate LDAP User Cache API for more information.

Fixed Issues in 15.0.0

CVE-2020-25649: Vulnerability reported against jackson-databind-2.10.2.jar
Dremio is not affected by this vulnerability. However, we upgraded to jackson-databind-2.11.4.jar out of caution.

CVE-2020-13956: Vulnerability reported against org.apache.httpcomponents.httpclient version 4.5.5
Dremio is unlikely to be affected by this vulnerability. However, we upgraded to httpclient 4.5.13 out of caution.

Dremio doesn’t clean up spill directories when Dremio crashes or is shutdown
Fixed by implementing a management module for temporary folders.
Note:
Dremio may not automatically clean up spill directories and their files from prior releases due to changes in the underlying directory structure. See Cleaning up Spill Directores for more information.

Queries with a JOIN statement run slowly
Resolved by pushing down join conditions on literals below the JOIN statement to avoid NestedLoopJoin.

Slow queries with the Oracle Connector when casting numeric columns
Resolved by omitting added casts in filters for decimal types.

Creating an external relational database source overwrites Spaces in Dremio with the same name
Fixed by cleaning up the plugin state in case of persistent source failures in the KVStore.

Query returns error: Source ‘LAMP SQL SERVER’ returned error ‘Arithmetic overflow error converting numeric to data type varchar
Resolved SQL Server numeric overflow when converting to VARCHAR.

Query returns error: ‘GRIP SQL Server’ returned error ‘The column ‘Business_Date’ was specified multiple times
Fixed by rewriting aliases to be unique.

Dremio errors in AWS CloudTrail log: “nosuchkey”
Resolved by adding an interface implementation for S3 FS and using the AWS SDK for health check status.

Dremio improperly pushes down Window functions into a WHERE clause
Fixed by pushing down the SELECT statement into a subquery and referencing the WINDOW function column in the outer WHERE clause.

Dremio returns an error when querying MS SQL sources with predicates based on BOOLEAN data types
Resolved by detecting and preventing pushdowns of bit columns treated as boolean expressions.

Dremio cancels query: SCHEMA_CHANGE ERROR: New schema found. Please reattempt the query. Multiple attempts may be necessary to fully learn the schema
Fixed by moving operations related to the Delta Lake data format out of the scope of the following system properties: store.json.all_text_mode and store.json.read_numbers_as_double.

Same query using FULL OUTER JOIN SQL statement returns different number of results
Resolved by not applying runtime filtering to FULL OUTER JOIN SQL statements.

15.2.0 Release Notes (Enterprise Edition Only)

What’s New

JDBC Driver 15.2.0

Other Enhancements

Dremio 15.1.0 provides a support key, exec.gandiva.target_host_cpu, to enable and disable CPU-specific instructions in generated Gandiva code. The key is enabled by default. Dremio administrators should disable this support key to prevent Gandiva from generating CPU-specific instructions, such as AVX and AVX512. See Support Keys for instructions on setting support keys.

Fixed Issues in 15.2.0

Queries in Dremio 14.3.0 take longer when runtime filtering is enabled
Fixed by revising the scan of the fact table to include any number of JOIN conditions and choosing only records that pass all filters.

Dremio fails during query using Gandiva-based execution
Resolved by addressing a Gandiva crash in select virtual machines.

Dremio JDBC driver returns incorrect date and time for date and time-formatted columns
Fixed by adjusting datetime values to use the local timezone rather than UTC.

Coordinator node crashes while promoting a Delta Lake dataset
Resolved by improving the promotion flow for Delta Lake datasets. The new flow does not list all files in the folder for Delta Lake datasets.

15.4.0 Release Notes (Enterprise Edition Only)

Fixed Issues in 15.4.0

Dremio cancels all queries after reaching the timeout limit
Fixed by ensuring that the Columnar Cloud Cache releases locks after threads encounter exceptions.