12.0.0 (Dremio December 2020) Release Notes

What’s New

Arrow Flight Server GA

The Arrow Flight server endpoint in Dremio 12.0.0 is GA. In addition, the endpoint now supports Arrow Flight 2.0.0 and a new authentication mode, enabled by default. Dremio administrators may reconfigure Dremio’s Flight Server endpoint to use the legacy authentication mode of Arrow Flight 1.0.0+ client applications. See 12.0.0 Upgrade Notes for instructions.

Note:
Dremio recommends waiting for production use of the Arrow Flight server endpoint until Apache Arrow releases Arrow 3 in January 2021.

YARN Query Support Bundle

Dremio 12.0.0 provides additional support for YARN-based Dremio deployments. Any Dremio user with support access and permission to view a job can download a query support bundle by clicking the Download Query Support Bundle button on the Job Details page.

Cloudera Data Platform 7.1

Dremio 12.0.0 supports deployments on Cloudera Data Platform 7.1.

Vectorized Reader for Parquet-formatted Datasets

Dremio provides a vectorized reader for simple and complex types in Parquet-formatted datasets. The feature is enabled by default.

Multiple Pushdown Filters for Parquet-formatted Datasets

Dremio supports pushing down multiple filters into Parquet-formatted datasets. The feature is enabled by default.

Changed Behavior

• The default expiration time for Flight client user session is reduced from 30 hours to 5 minutes.

• Dremio 12.0.0 removes the dataset votes feature (EE only) for Data Reflections.

Other Enhancements

• Dremio supports Kerberos when connecting to Oracle data sources.

• Dremio provides a support key, flight.session.expiration.minutes, to customize the expiration time for Flight client user sessions.

• Gandiva supports the castFloat function.

• Improved error message when an Arrow Flight 1.0.0 client application queries a dataset that returns a union data type.

• Dremio improves query performance by reusing opened HDFS files that contain multiple row groups.

• Dremio displays the number of unique users who executed jobs on that day and the number of executed jobs on the About Dremio modal. The number of unique users may include an internal Dremio user.

Known Issues in 12.0.0

Dremio provides limited support for insert_only tables in Hive.

Fixed Issues in 12.0.0

Transitive Filter pushdowns fail with an extra expression on a join key
Resolved by removed the CompositeFilterJoinRule, extending and modifying calcite’s JoinPushTransitivePredicatesRule, and added support in FindSimpleFilters to simplify filter conditions with IS NOT DISTINCT FROM operator.

Arrow flight client fails with IllegalArgumentException: Could not load buffers for field test: Union
Fixed by invalidating Dremio results store for UNION jobs after upgrading Dremio.

castINT on a literal causes executor node crash
Resolved linking issue on castInt/float in Gandiva-based execution.

12.1.0 Release Notes

Fixed Issues in 12.1.0

Dremio may return incorrect query results when applying the transtiive-filter-join optimization to an equality condition with an implicit cast
Resolved by simplifying filter conditions on mismatched types.

12.2.0 Release Notes (Enterprise Edition Only)

Other Enhancements

Dremio improves query performance by using HashJoin rather than NestedLoopJoin.

Fixed Issues in 12.2.0

Dremio throws a UnsupportedOperationException while creating a virtual dataset
Resolved error when pushing down predicates.

Dremio displays the following error after upgrading to Dremio 12.0.0: Some virtual datasets are out of date and need to be manually updated
Fixed by correcting the RelRecordType generated by the Kryo deserializer.

Some filters are not pushed down after upgrading to Dremio 12.0.0
Resolved by pushing down all filters.

Query fails with a planning error after upgrading to Dremio 12.0.0
Fixed issues with complex data types.