4.5 Release Notes

What's New

AWS Edition

• Streamlined user experience with Dremio Projects with AWS CloudFormation.

• Support for starting and stopping coordinator nodes.

Other Enhancements

• Tableau SDK Connector with single sign-on.

• AWS Redshift GA.

• MongoDB data sources support MongoDB 4.0.

• Jobs progress through the following states for each query:

  • Pending
  • Metadata Retrieval
  • Planning
  • Queued
  • Engine Start
  • Execution Planning
  • Starting
  • Running

  Jobs will finish with one of the following states:

  • Completed
  • Canceled
  • Failed

• Efficient retrieval of metadata from large catalogs.

Changed Behavior

MongoDB Data Source

Dremio no longer pushes down aggregation pipelines to MongoDB.

System Requirement

Dremio requires a new ephemeral network port for inter-node communications in addition to port 45678. To assign a static port number to the conduit port, assign a value to services.conduit.port in the Dremio configuration file, dremio.conf. If TLS is enabled, Dremio applies the same configuration to communications of the new conduit port. To use a different configuration or to enable TLS for only the new conduit port, specify all values for services.conduit.ssl in dremio.conf.

Fixed Issues in 4.5.0

Dremio 4.2.x and 4.3.x unable to access data nodes on Hadoop clusters prior to 3.0
Fixed by updating the version of the Hadoop client.

JDBC Connector taking too long to retrieve query results
Fixed by adding an END_OF_STREAM_MESSAGE message that signals when a queue is empty.

Queries do not complete after refreshing the metadata for a table in Hive
Fixed by removing a race condition.

Slow sorts in some circumstances when micro-spilling is disabled and batch sizes are especially small
Addressed by resizing record batches during spilling despite uneven incoming batches.

IllegalStateException: Inconsistent row count error while joining two Hive data sets using complex data in Parquet format
Fixed by handling non-existing Parquet columns.

Executor nodes in AWS Edition don't automatically terminate if the AWS API is unavailable
Added capability to terminate executor nodes using Linux commands.

TOTAL_SPILLED_DATA_SIZE is 0 after an external sort with micro-spilling enabled
Fixed by updating the metric for total spill size.

JDBC driver leaves connection open with Looker data sources
Fixed by more robustly cleaning up resources after a connection failure.

Standby coordinator node consumes more memory than allocated
Fixed memory error in standby coordinator node.

Job API sometimes returns fewer records than specified with the LIMIT statement
Fixed a bug that occurs when the offset is equal to or greater than the batch size.

Schema learning fails with 'New schema found. Please re-attempt the query' error message
Fixed by only pushing down projections that are simple column references.